Plant disease and insect pest identification method based on multispectral image acquisition system

Abstract

The invention relates to the technical field of plant disease and insect pest detection, and discloses a plant disease and insect pest identification method based on a multispectral image acquisition system. Specifically, multispectral image data of a plant sample is obtained through a multispectral image acquisition system, geometric calibration is performed on the multispectral image data, noise and atmospheric influence in the multispectral image data are eliminated, and target multispectral image data are obtained. An improved competitive self-adaptive reweighted sampling method is used for extracting key characteristic wave bands related to diseases from target multispectral data, and the characteristic selection efficiency is remarkably improved. And finally, feature extraction and classification decision are performed on the key feature wavebands by using a multi-modal classification model, so that the model interpretability and classification precision can be enhanced, and the method has significant advantages in early disease detection and can effectively identify early disease symptoms.

Description

Technical Field

[0001] This application relates to the technical field of plant pest and disease detection, and particularly to a method for identifying plant pests and diseases based on a multispectral image acquisition system. Background Art

[0002] Plant pests and diseases are one of the main threats in agricultural production. Traditional disease identification methods mainly rely on manual visual judgment, which has problems such as strong subjectivity, low accuracy, and difficulty in early detection of diseases. With the development of spectral imaging technology, multispectral cameras can simultaneously obtain the spatial and spectral information of plants, providing a new technical means for the automated detection of plant diseases.

[0003] Currently, some studies have used spectral imaging technology for plant disease detection. For example, using time-series hyperspectral and multi-task learning systems to predict early diseases of rice, using spectral imaging technology to detect plant leaf diseases, and using spectral information to monitor the health of crops and the spread of diseases.

[0004] Existing plant disease detection methods mainly rely on hyperspectral imaging technology and combine traditional classification algorithms (such as support vector machines, random forests, etc.) for disease identification. However, such methods do not extract features sufficiently, and cannot extract key features closely related to diseases from a large amount of spectral data, resulting in limited classification accuracy and difficulty in meeting actual requirements, especially performing poorly in early disease detection and the identification of complex disease types. Summary of the Invention

[0005] This application provides a method for identifying plant pests and diseases based on a multispectral image acquisition system to solve the problem in the prior art that the existing plant disease detection methods have limited classification accuracy and are difficult to meet actual requirements, especially performing poorly in early disease detection and the identification of complex disease types.

[0006] To solve the above technical problems, this application discloses a method for identifying plant pests and diseases based on a multispectral image acquisition system, and the method includes:

[0007] Obtain multispectral image data of a plant sample through a multispectral image acquisition system;

[0008] Perform geometric calibration on the multispectral image data, and eliminate noise and atmospheric effects in the multispectral image data to obtain target multispectral image data;

[0009] Extract key feature bands related to diseases from the target multispectral data by using an improved competitive adaptive reweighted sampling method; wherein, the improved competitive adaptive reweighted sampling method includes a Monte Carlo sampling method based on information entropy weighting and a band screening method based on Bayesian optimization;

[0010] Feature extraction and classification decision are performed on the key feature bands using a multi-modal classification model to obtain a disease classification result; among them, the multi-modal classification model includes a maximum likelihood probability classifier and a depthwise separable convolutional neural network.

[0011] Preferably, after feature extraction and classification decision are performed on the key feature bands using the multi-modal classification model and the prediction result is output, the method further includes:

[0012] Optimizing the multi-modal classification model through a transfer learning strategy and fine-tuning the parameters of the multi-modal classification model using a cross-crop disease dataset;

[0013] Among them, the transfer learning strategy adopts cross-domain adversarial training, and the domain adaptation loss function is used to reduce the difference in feature distributions of different crop disease datasets.

[0014] Preferably, an improved competitive adaptive reweighted sampling method is used to extract key feature bands related to diseases from the target multispectral data, including:

[0015] Using the Monte Carlo sampling method to sample the target multispectral data and screening out a sample subset with significant differences between spectral bands; among them, in each sampling, information entropy weights are used to weight each spectral band in the target multispectral data.

[0016] Iteratively screening and removing redundant features from the sampled sample subset to obtain a target sample subset; among them, in each iterative screening, the iteration round threshold for determining the screening termination condition is dynamically adjusted, and it is determined whether to terminate the redundant feature screening by judging whether the root mean square error of cross-validation or the band redundancy index of the sample subset reaches the screening termination condition.

[0017] Based on the classification contribution degree of each band, K bands are screened out from the target sample subset as key feature bands; where K is determined by Bayesian optimization.

[0018] Preferably, feature extraction and classification decision are performed on the key feature bands using the multi-modal classification model to obtain a disease classification result, including:

[0019] Using the maximum likelihood probability classifier to perform statistical distribution on the key feature bands to obtain a class probability map;

[0020] Taking the class probability map as the attention-guided input of the depthwise separable convolutional neural network, extracting spatial-spectral fusion features from the key feature bands, and obtaining a disease classification result through classification decision.

[0021] Preferably, the depthwise separable convolutional neural network includes a spectral feature compression layer and a spatial convolutional residual module;

[0022] Taking the class probability map as the attention-guided input of the depthwise separable convolutional neural network, extracting the spatial-spectral fusion features from the key feature bands, and obtaining the disease classification results through classification decision-making, including:

[0023] Under the guidance of the class probability map, extracting the spectral features and spatial features of the key feature bands through the spectral feature compression layer and the spatial convolutional residual module respectively;

[0024] Fusing the spectral features and spatial features through a bidirectional feature fusion mechanism to obtain spatial-spectral fusion features;

[0025] Based on the spatial-spectral fusion features, making a classification decision to obtain the disease classification results.

[0026] Preferably, under the guidance of the class probability map, extracting the spectral features and spatial features of the key feature bands through the spectral feature compression layer and the spatial convolutional residual module respectively, including:

[0027] In the spectral feature compression layer, taking the class probability map as the channel attention weight to weight the spectral channels of the key feature bands to obtain spectral features.

[0028] In the spatial convolutional residual module, taking the class probability map as the spatial attention mask to extract features from the key feature bands to obtain spatial features.

[0029] Preferably, geometrically calibrating the multispectral image data and eliminating the noise and atmospheric effects in the multispectral image data to obtain the target multispectral image data, including:

[0030] Geometrically calibrating the multispectral image to obtain a calibrated image;

[0031] Smoothing the calibrated image by the Savitzky-Golay convolutional smoothing method to eliminate the noise in the calibrated image and obtain a denoised image;

[0032] Performing atmospheric correction on the denoised data by the FLAASH algorithm to obtain the target multispectral image data.

[0033] Preferably, the geometric calibration includes spatial distortion correction based on a checkerboard calibration plate and pixel-level registration between multispectral channels.

[0034] Preferably, the multispectral image acquisition system includes:

[0035] A multi-band LED light source array for dynamically switching the illumination mode according to a preset band;

[0036] An electric rotating stage for adjusting the placement angle of the plant sample;

[0037] A high-resolution multispectral camera for collecting multispectral image data of plant samples;

[0038] An embedded control module for synchronously controlling the operation of the high-resolution multispectral camera and the LED light source array.

[0039] In this application, multispectral image data of plant samples is obtained through a multispectral image acquisition system, geometrically calibrated, and the noise and atmospheric effects in the multispectral image data are eliminated to obtain target multispectral image data. Then, an improved competitive adaptive reweighted sampling method is used to extract key feature bands related to diseases from the target multispectral data. This sampling method combines information entropy weighting and Bayesian optimization, significantly improving the feature selection efficiency. Finally, a multimodal classification model is used to perform feature extraction and classification decision on the key feature bands. The multimodal classification model adopts the cooperative mechanism of a maximum likelihood probability classifier and a depthwise separable convolutional neural network, and uses probability priors to guide deep learning feature extraction, which can enhance the model interpretability and classification accuracy. Based on the above method, significant advantages can be shown in early disease detection, and the initial symptoms of diseases can be effectively identified.

[0040] Additional aspects and advantages of this application will be given in the following description section, which will become apparent from the following description or be understood through the practice of this application. Description of the Drawings

[0041] The above and / or additional aspects and advantages of this application will become apparent and easy to understand from the following description of the embodiments in conjunction with the drawings, where:

[0042] Figure 1 is a flowchart of a method for identifying plant diseases and pests based on a multispectral image acquisition system provided by an embodiment of this application;

[0043] Figure 2 is a schematic diagram of a multispectral image acquisition system provided by an embodiment of this application;

[0044] Figure 3 is a flowchart of data preprocessing provided by an embodiment of this application;

[0045] Figure 4 is a flowchart of extracting key feature bands provided by an embodiment of this application. Detailed Description of the Embodiments

[0046] The embodiments of this application will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain this application and should not be construed as limiting this application.

[0047] Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of this application means the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their combinations. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more related listed items.

[0048] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the art to which this invention belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.

[0049] The solution provided by the embodiments of this application can be executed by any electronic device, such as a terminal device or a server. Among them, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication methods, and this application does not make any restrictions here. For the technical problems existing in the prior art, the method for identifying plant diseases and pests based on a multi-spectral image acquisition system provided by this application aims to solve at least one of the technical problems of the prior art.

[0050] The technical solution of this application and how the technical solution of this application solves the above technical problems will be described in detail below with specific embodiments. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below in conjunction with the accompanying drawings.

[0051] The embodiments of this application provide a possible implementation manner, such as Figure 1As shown, a flowchart of a method for identifying plant diseases and pests based on a multispectral image acquisition system is provided. This solution can be executed by any electronic device. Optionally, it can be executed on the server side or the terminal device.

[0052] As Figure 1 shown in the figure, the method may include the following steps:

[0053] Step 101, obtain multispectral image data of a plant sample through a multispectral image acquisition system.

[0054] In the embodiments of the present application, the collected multispectral image data covers the visible light to near-infrared spectral range. Optionally, the near-infrared spectral range is the region of 780 - 1000nm.

[0055] Step 102, perform geometric calibration on the multispectral image data, and eliminate the noise and atmospheric effects in the multispectral image data to obtain target multispectral image data.

[0056] Geometric calibration can ensure that the pixel coordinates of the multispectral image data are consistent with the actual geographical coordinates. Eliminating the noise and atmospheric effects in the multispectral image data can improve the signal-to-noise ratio of the multispectral image data and reduce the interference of non-target factors on the model.

[0057] Step 103, use an improved competitive adaptive reweighted sampling method to extract key feature bands related to diseases from the target multispectral data;

[0058] Among them, the improved competitive adaptive reweighted sampling method includes a Monte Carlo sampling method based on information entropy weighting and a band screening method based on Bayesian optimization, which can extract key features related to diseases from a large amount of spectral data and improve the classification accuracy.

[0059] Step 104, use a multimodal classification model to perform feature extraction and classification decision on the key feature bands to obtain a disease classification result.

[0060] Among them, the multimodal classification model includes a maximum likelihood probability classifier and a depthwise separable convolutional neural network (DS-CNN). The maximum likelihood classifier is used to generate a prior probability distribution, and the DS-CNN is used to extract spatial-spectral joint features and perform classification decision under the guidance of the prior probability distribution, making the multimodal classification model more efficient and accurate, capable of handling complex disease types, and capable of effectively identifying diseases in the early stage.

[0061] The method in the embodiments of the present application is based on multispectral imaging technology and machine learning algorithms for disease detection, realizing the automated detection of plant diseases and pests, reducing the dependence on manual visual judgment, and enabling timely prevention and control measures to be taken.

[0062] In the embodiments of the present application, multispectral image data of a plant sample is obtained through a multispectral image acquisition system, the multispectral image data is geometrically calibrated, and noise and atmospheric effects in the multispectral image data are eliminated to obtain target multispectral image data. Subsequently, an improved competitive adaptive reweighted sampling method is used to extract key feature bands related to diseases from the target multispectral data. This sampling method combines information entropy weighting and Bayesian optimization, significantly improving the feature selection efficiency. Finally, a multimodal classification model is used to perform feature extraction and classification decision on the key feature bands. The multimodal classification model adopts a collaborative mechanism of a maximum likelihood probability classifier and a depthwise separable convolutional neural network, and uses probability priors to guide deep learning feature extraction, which can enhance the model interpretability and classification accuracy. Based on the above method, significant advantages can be shown in early disease detection, and the initial symptoms of diseases can be effectively identified.

[0063] In an alternative embodiment, as Figure 2 shown, the multispectral image acquisition system includes:

[0064] A multi-band LED light source array for dynamically switching the lighting mode according to a preset band;

[0065] An electric rotating stage for adjusting the placement angle of the plant sample;

[0066] A high-resolution multispectral camera for collecting multispectral image data of the plant sample;

[0067] An embedded control module for synchronously controlling the operation of the high-resolution multispectral camera and the LED light source array.

[0068] Among them, the multispectral image data of the plant sample collected by the high-resolution multispectral camera contains rich spatial information and spectral information, providing a reliable data basis for disease identification. The multispectral image acquisition system further includes a tripod for supporting the high-resolution multispectral camera.

[0069] In an alternative embodiment, geometrically calibrating the multispectral image data and eliminating noise and atmospheric effects in the multispectral image data to obtain target multispectral image data includes:

[0070] Geometrically calibrating the multispectral image to obtain a calibrated image;

[0071] Smoothing the calibrated image by the Savitzky-Golay (SG) convolution smoothing method to eliminate noise in the calibrated image and obtain a denoised image;

[0072] Performing atmospheric correction on the denoised data by the FLAASH algorithm to obtain the target multispectral image data.

[0073] Among them, geometric calibration includes spatial distortion correction based on a checkerboard calibration plate and pixel-level registration between multi-spectral channels.

[0074] Data preprocessing is a crucial step in ensuring data quality and analysis accuracy. In the embodiments of this application, ENVI software is used to preprocess the collected multi-spectral image data and perform extended analysis in combination with the WHU-Hi dataset (Unmanned Aerial Vehicle Hyperspectral Images of Wuhan University) database. The WHU-Hi dataset database was collected and released by the RSIDEA research team of Wuhan University and provides a benchmark reference for the precise classification of crops and the research on hyperspectral image classification.

[0075] During the process of multi-spectral image data collection, due to factors such as external stray light, background reflection, and the relatively narrow sampling bandwidth of high-resolution multi-spectral cameras, problems such as noise and baseline drift may occur in the multi-spectral image data, affecting the subsequent processing effect. To improve the signal-to-noise ratio of the multi-spectral image data, reduce the interference of non-target factors on the model, and ensure the consistency between the WHU-Hi database and the sample data, the data preprocessing process in this article is as Figure 3 shown.

[0076] 1.1. Geometric calibration: It is a process of precisely correcting the spatial position of an image, aiming to ensure that the pixel coordinates of the multi-spectral image data are consistent with the actual geographical coordinates. During multi-spectral imaging, due to factors such as changes in viewing angle, sensor characteristic limitations, and terrain undulation, errors may occur in the spatial structure of the image. The purpose of geometric calibration is to eliminate these errors, thereby ensuring the spatial consistency and accuracy of the image data, enabling it to be effectively compared with other geographical information data such as the WHU-Hi database. In multi-source data fusion analysis, geometric calibration is particularly important to ensure that images from multiple time phases or different data sources have a strict spatial correspondence relationship in the same coordinate system, thereby improving the accuracy of subsequent feature extraction and classification analysis. The calculation formulas are as follows:

[0077] x′ = a0 + a1x + a2y + a3x 2 + a4xy + a5y 2 +...

[0078] y′ = b0 + b1x + b2y + b3x 2 + b4xy + b5y 2 +...

[0079] 1.2. Savitzky-Golay Convolution Smoothing Method (SG): The Savitzky-Golay (SG) convolution smoothing method is a polynomial smoothing algorithm based on the least squares principle. By fitting a low-order polynomial to a series of adjacent data points and replacing the original data points with the polynomial values, the smoothing effect is achieved. Using the Savitzky-Golay convolution smoothing method, the signal is smoothed by local polynomial fitting. While maintaining the overall shape and trend of the signal, this method can effectively suppress the interference of high-frequency components, thereby improving the signal-to-noise ratio of the data. The SG convolution formula is as follows:

[0080]

[0081] where y' i is the smoothed data point, y i+j is the (i + j)-th data point in the original data, and c j is the smoothing coefficient. The smoothing coefficient c j is obtained by fitting a low-order polynomial within the window and is used to retain low-frequency signals while suppressing high-frequency noise.

[0082] 1.3. FLAASH (Fast Line-of-sight Atmospheric Analysis of Hypercubes) Algorithm: An adaptive atmospheric parameter inversion model is adopted, and the real-time environmental sensor data is combined to optimize the calibration accuracy.

[0083] Regarding the possible atmospheric effects in the acquisition environment, the FLAASH algorithm is selected for atmospheric correction to eliminate the interference of atmospheric scattering and absorption on the spectrum, ensuring that the obtained hyperspectral image data can truly reflect the characteristics of plant diseases. Considering that plant diseases have significant characteristics in the visible light band (400 - 700 nm), the data is cropped to this band range to remove redundant information, and the consistency of the spectral values of each band is ensured through normalization processing, providing a balanced data basis for subsequent feature extraction and analysis. In addition, to improve the diversity of the dataset and the generalization performance of the model, similar plant species and disease feature samples in the WHU-Hi database are combined for analysis. The representativeness of the disease features is verified through feature matching, and the disease samples in the database are merged with the experimental data to construct a more comprehensive hyperspectral dataset to support model training. The atmospheric radiation correction formula is as follows:

[0084] L = L0 + ρ·T·E

[0085] Where: L is the radiance observed by the sensor, L0 is the atmospheric path radiation (caused by atmospheric scattering), ρ is the target reflectivity (reflecting the reflection characteristics of the object surface), T is the atmospheric transmittance (i.e., the penetration effect of the atmosphere on radiation), and E is the solar irradiance reaching the earth's surface. The normalization formula is as follows:

[0086]

[0087] Where: ρ is the original reflectivity, ρ' is the normalized reflectivity, min(ρ) and max(ρ) are the minimum and maximum reflectivity values in the band, respectively.

[0088] Finally, the spectral data is normalized to ensure the consistency of spectral values in each band.

[0089] In the embodiment of the present application, geometric calibration, SG (Savitzky-Golay) convolution smoothing, and FLAASH atmospheric correction are performed on the collected multi-spectral images to remove noise and baseline drift.

[0090] In an alternative embodiment, as Figure 4 shown, an improved competitive adaptive reweighted sampling method is used to extract key feature bands related to diseases from the target multi-spectral data, including:

[0091] Using the Monte Carlo sampling method to sample the target multi-spectral data, and screening out a sample subset with significant differences between spectral bands; among them, in each sampling, information entropy weights are used to weight each spectral band in the target multi-spectral data;

[0092] Iteratively screening and removing redundant features from the sampled sample subset to obtain the target sample subset; among them, in each iterative screening, the iteration round threshold for determining the screening termination condition is dynamically adjusted, and it is determined whether to terminate the redundant feature screening by judging whether the root mean square error of cross-validation or the band redundancy index of the sample subset reaches the screening termination condition;

[0093] Based on the classification contribution degree of each band, K bands are screened out from the target sample subset as key feature bands; where K is determined by Bayesian optimization.

[0094] In the embodiment of the present application, the method of Monte Carlo weighted sampling based on information entropy weights is as follows:

[0095] 2.1. Discretization of band data

[0096] For each spectral band X in the target multi-spectral data b (with a dimension of n×1, where n is the number of samples) is discretized and divided into m equal-width intervals. The frequency n of samples in each interval is counted i, calculate its probability distribution:

[0097]

[0098] 2.2. Calculate the information entropy

[0099] Based on the discretized probability distribution, calculate the information entropy H of band b b :

[0100]

[0101] The larger the information entropy value, the more complex the data distribution of the band, and the richer the discriminant features (such as reflectance anomalies caused by diseases) it contains.

[0102] 2.3. Normalize to weights

[0103] Normalize the information entropy of all bands to the probability weight w b , ensuring that the sum of weights is 1:

[0104]

[0105] where B is the total number of bands. The weight w b directly determines the probability that band b is selected in Monte Carlo sampling, and high-entropy bands (such as disease-sensitive bands) obtain higher weights.

[0106] 2.4. Monte Carlo weighted sampling

[0107] In each round of sampling, according to the weight w b perform weighted random selection on the bands to generate a candidate subset. For example, if the entropy value of a certain band in the range of 550 - 600 nm is significantly higher than other regions, its weight increases, and the sampling probability increases, so it is more likely to be retained.

[0108] In the embodiments of the present application, the difference between spectral bands can be judged to be significant through a difference threshold, that is, if the difference between the entropy value of a certain spectral band and the entropy values of other spectral bands is higher than the difference threshold, it is considered significant. The difference threshold can be adjusted according to actual needs.

[0109] After obtaining the sample subset by Monte Carlo sampling, redundant features are iteratively screened from the sample subset. Among them, in each iterative screening, the iterative round threshold is dynamically adjusted, and the iterative round threshold is the judgment criterion for the iterative termination condition. The basis for its dynamic adjustment is the comprehensive evaluation of the following two indicators:

[0110] Root mean square error of cross-validation (RMSE-CV): reflecting the classification performance of the current feature subset, the lower the error, the higher the feature effectiveness;

[0111] Band redundancy index: By calculating the correlation coefficient or information overlap between bands, the redundancy degree of the feature subset is measured.

[0112] During the iterative screening process, the changing trends of the above two indicators are monitored in real time. When one of the following conditions is met, the redundant feature screening is terminated:

[0113] The decreasing amplitude of RMSE-CV is less than the preset threshold (e.g., the decreasing amplitude ≤ 1% for 3 consecutive iterations);

[0114] The band redundancy index exceeds the preset redundancy tolerance threshold (e.g., the average correlation coefficient between bands ≥ 0.8).

[0115] By dynamically adjusting the trigger values of these thresholds (e.g., adaptively relaxing or tightening according to historical iterative data), overfitting or underfitting is avoided.

[0116] In the embodiments of this application, the screening of redundant features is terminated by combining RMSE-CV and the band redundancy index, and the specific implementation logic is as follows:

[0117] Index quantification:

[0118] RMSE-CV calculates the classification error of the current feature subset through k-fold cross-validation;

[0119] The band redundancy index is quantified by the trace of the covariance matrix between bands or mutual information.

[0120] Termination condition design:

[0121] Define the comprehensive scoring function: Score = α·RMSE-CV + β·Redundancy, where α and β are weight coefficients (e.g., α = 0.7, β = 0.3);

[0122] Calculate the Score value in each iteration. If the Score does not decrease significantly (e.g., the change rate ≤ 2%) for N consecutive iterations (e.g., N = 5), the screening is terminated.

[0123] Dynamic optimization mechanism:

[0124] According to the changing trend of Score during the iteration process, dynamically adjust the weight allocation of α and β (e.g., increase the β weight when the redundancy increases);

[0125] Automatically search for the optimal combination of termination condition parameters through the Bayesian optimization algorithm.

[0126] In the embodiments of this application, through the joint optimization of dual indicators and dynamic parameter adjustment, both the classification performance of the feature subset (RMSE-CV constraint) and the computational burden introduced by redundant features (redundancy constraint) are ensured. By introducing the redundancy index and the adaptive threshold mechanism, the robustness of feature selection is significantly improved.

[0127] In an optional embodiment, a multi-modal classification model is used to extract features and make classification decisions on key feature bands, and a disease classification result is obtained, including:

[0128] The maximum likelihood probability classifier is used to perform statistical distribution on the key feature bands to obtain a class probability map;

[0129] The class probability map is used as the attention-guided input of the depthwise separable convolutional neural network to extract spatio-spectral fusion features from the key feature bands, and a disease classification result is obtained through classification decision-making.

[0130] In the embodiment of the present application, the specific implementation manner for the maximum likelihood probability classifier to obtain the class probability map is as follows:

[0131] Assume that the i-th class follows a multivariate Gaussian distribution, and its probability density function is:

[0132]

[0133] where x is the input spectral feature vector; d is the feature dimension (such as 20 CARS screening bands); μ is the mean vector, representing the central position of each class (healthy, downy mildew, brown spot) in the spectral feature space; ∑ is the covariance matrix, used to describe the distribution shape of the spectral features of each class and the correlation between bands; P(ω I i) is the prior probability, that is, the occurrence frequency of each class in the training data, reflecting the statistical distribution of the disease.

[0134] Through the above probability density function, the class probability map can be calculated and obtained.

[0135] In an optional embodiment, the depthwise separable convolutional neural network includes a spectral feature compression layer and a spatial convolutional residual module;

[0136] Taking the class probability map as the attention-guided input of the depthwise separable convolutional neural network to extract spatio-spectral fusion features from the key feature bands, and a disease classification result is obtained through classification decision-making, including:

[0137] Under the guidance of the class probability map, the spectral features and spatial features of the key feature bands are respectively extracted through the spectral feature compression layer and the spatial convolutional residual module;

[0138] The spectral features and spatial features are fused through a bidirectional feature fusion mechanism to obtain spatio-spectral fusion features;

[0139] Based on the spatio-spectral fusion features, classification decision-making is performed to obtain a disease classification result.

[0140] In an alternative embodiment, guided by the class probability map, spectral features and spatial features of the key feature bands are respectively extracted through a spectral feature compression layer and a spatial convolutional residual module, including:

[0141] In the spectral feature compression layer, the class probability map is used as the channel attention weight to weight the spectral channels of the key feature bands, obtaining spectral features.

[0142] In the spatial convolutional residual module, the class probability map is used as the spatial attention mask to extract features from the key feature bands, obtaining spatial features.

[0143] In the embodiment of the present application, the class probability map, as the attention-guided input of the depthwise separable convolutional neural network, can help the network better capture the important features in the input data and improve the performance of the classification task.

[0144] In the embodiment of the present application, the spectral feature compression layer uses convolutional operations and pooling operations to extract spectral features. The convolutional operations can capture local features in the key feature bands through different convolutional kernels, while the pooling operations can further reduce the dimension of the data while retaining the most important information.

[0145] The spatial convolutional residual module extracts spatial features through convolutional layers and pooling layers. The convolutional layers capture local features in the key feature bands through convolutional kernels, while the pooling layers are used to reduce the dimension of the data and extract higher-level features. In addition, residual connections are also introduced into the module to help the network better learn features and prevent the problem of gradient vanishing or explosion.

[0146] In the bidirectional feature fusion mechanism, the spectral features and spatial features are merged together and further processed through a series of fusion operations to obtain spatial-spectral fusion features.

[0147] In an alternative embodiment, after using the multi-modal classification model to extract features and make classification decisions for the key feature bands and outputting the prediction results, the method further includes:

[0148] Optimizing the multi-modal classification model through a transfer learning strategy and fine-tuning the parameters of the multi-modal classification model using a cross-crop disease dataset;

[0149] Among them, the transfer learning strategy adopts cross-domain adversarial training to narrow the difference in feature distributions of different crop disease datasets through a domain adaptation loss function.

[0150] The optimization method of the multi-modal classification model is specifically as follows:

[0151] 3.1. Design of the feature extractor of the multi-modal classification model:

[0152] Deep separable convolutional neural network (DS-CNN): Extract the spatial features of the key feature bands.

[0153] Maximum likelihood probability classifier: Use 1D CNN or LSTM to process the key feature bands to obtain spectral features.

[0154] Bidirectional feature fusion mechanism: Fuse the spatial features and spectral features into the spatial-spectral fusion feature h through concatenation, weighted summation, or attention mechanism (such as cross-modal Transformer).

[0155] 3.2. Determine the domain adversarial network structure

[0156] 1) Gradient reversal layer (GRL):

[0157] Insert GRL between the feature extractor (G f ) and the domain discriminator (G a ).

[0158] Forward propagation: Normally transmit the spatial-spectral fusion feature h.

[0159] Backward propagation: Multiply the gradient of the domain discriminator by the negative coefficient -λ to force the feature extractor to generate features that confuse the domain discriminator.

[0160] 2) Domain discriminator design:

[0161] It is composed of fully connected layers. The input is the spatial-spectral fusion feature h, and the output is the domain classification probability (source domain vs. target domain).

[0162] Objective: Minimize the domain classification loss to align the feature distributions.

[0163] 3.3. Loss function design

[0164] 1) Classification loss (source domain supervision):

[0165]

[0166] Where G c is the classifier, N s is the number of source domain samples, and C is the number of disease categories.

[0167] 2) Domain adaptation loss (adversarial training):

[0168]

[0169] d i represents the sample domain label (source domain is 1, target domain is 0), and N t is the number of target domain samples.

[0170] 3) Total loss:

[0171] L total = L cls + λL domain

[0172] λ is the dynamic weight (which can gradually increase from 0 as the number of training epochs increases).

[0173] 3.4. Adversarial Training Process

[0174] 1) Pre-training stage:

[0175] Pre-train the multi-modal classification model on the source domain dataset (only optimize L cls ).

[0176] 2) Cross-domain fine-tuning stage:

[0177] a. Input: Mix and input the source domain (with labels) and target domain (without labels) data.

[0178] b. Forward propagation: Calculate the joint feature h and the classification / domain discrimination results.

[0179] c. Backward propagation:

[0180] Fix the domain discriminator G d , update the feature extractor G f and the classifier G c (minimize L total ).

[0181] Fix G f , update G d (only optimize L domain ).

[0182] d. Alternating optimization: Repeat the above steps until convergence.

[0183] 3.5. Key Implementation Details

[0184] 1) Dynamic weight λ: Adopt a progressive adjustment strategy (e.g., p is the training progress ratio).

[0185] 2) Modal alignment: If the domain differences between modalities are unbalanced, domain discriminators can be designed separately for each modality.

[0186] 3) Data augmentation: Augment the target domain data (e.g., random cropping, noise injection) to improve generalization.

[0187] 3.6. Effect Evaluation

[0188] 1) Domain difference measurement: Calculate the feature distribution distance between the source domain and the target domain through MMD or CORAL.

[0189] 2) Target domain performance: Test the classification accuracy on a small amount of labeled data in the target domain to verify the transfer effect.

[0190] In the embodiments of the present application, through transfer learning and domain adaptation strategies, the generalization problem of cross-crop disease identification can be solved, and the dependence on labeled data can be reduced.

[0191] The above description is only a preferred embodiment of the present application and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present application.

Claims

1. A method for identifying plant diseases and pests based on a multispectral image acquisition system, characterized in that, The method includes: Obtaining multispectral image data of a plant sample through a multispectral image acquisition system; Performing geometric calibration on the multispectral image data, and eliminating noise and atmospheric influence in the multispectral image data to obtain target multispectral image data; Extracting key feature bands related to diseases from the target multispectral data by using an improved competitive adaptive reweighted sampling method; wherein, the improved competitive adaptive reweighted sampling method includes a Monte Carlo sampling method based on information entropy weighting and a band screening method based on Bayesian optimization; Performing feature extraction and classification decision on the key feature bands by using a multimodal classification model to obtain a disease classification result; wherein, the multimodal classification model includes a maximum likelihood probability classifier and a depthwise separable convolutional neural network.

2. The method for identifying plant diseases and insect pests based on a multi-spectral image acquisition system according to claim 1, wherein After performing feature extraction and classification decision on the key feature bands by using the multimodal classification model and outputting a prediction result, the method further includes: Optimizing the multimodal classification model through a transfer learning strategy, and fine-tuning the parameters of the multimodal classification model by using a cross-crop disease dataset; Wherein, the transfer learning strategy adopts cross-domain adversarial training, and reduces the feature distribution difference between different crop disease datasets through a domain adaptation loss function.

3. The method for identifying plant diseases and pests based on a multi-spectral image acquisition system according to claim 1, wherein, The extracting of the key feature bands related to diseases from the target multispectral data by using the improved competitive adaptive reweighted sampling method includes: Sampling the target multispectral data by using the Monte Carlo sampling method, and screening out a sample subset with significant differences between spectral bands; wherein, in each sampling, information entropy weights are used to weight each spectral band in the target multispectral data; Iteratively screening and removing redundant features from the sampled sample subset to obtain a target sample subset; wherein, in each iterative screening, the iteration round threshold for determining the screening termination condition is dynamically adjusted, and it is determined whether to terminate the redundant feature screening by judging whether the root mean square error of cross-validation or the band redundancy index of the sample subset reaches the screening termination condition; Based on the classification contribution degree of each band, screening out K bands from the target sample subset as the key feature bands; wherein, K is determined by Bayesian optimization.

4. The method for identifying plant diseases and pests based on a multi-spectral image acquisition system according to claim 1, wherein, The performing of feature extraction and classification decision on the key feature bands by using the multimodal classification model to obtain a disease classification result includes: Performing statistical distribution on the key feature bands by using the maximum likelihood probability classifier to obtain a class probability map; Using the class probability map as the attention-guided input of the depthwise separable convolutional neural network, extracting spatio-spectral fusion features from the key feature bands, and obtaining the disease classification result through classification decision.

5. The method for identifying plant diseases and insect pests based on a multi-spectral image acquisition system according to claim 4, wherein, The depthwise separable convolutional neural network includes a spectral feature compression layer and a spatial convolutional residual module; Using the class probability map as the attention-guided input of the depthwise separable convolutional neural network, extracting spatio-spectral fusion features from the key feature bands, and obtaining the disease classification result through classification decision, includes: Under the guidance of the class probability map, the spectral features and spatial features of the key feature bands are respectively extracted through the spectral feature compression layer and the spatial convolutional residual module; The spectral features and spatial features are fused through a bidirectional feature fusion mechanism to obtain the spatial-spectral fusion features; Based on the spatial-spectral fusion features, a classification decision is made to obtain the disease classification result.

6. The method for identifying plant diseases and pests based on a multi-spectral image acquisition system according to claim 5, wherein Under the guidance of the class probability map, the spectral features and spatial features of the key feature bands are respectively extracted through the spectral feature compression layer and the spatial convolutional residual module, including: In the spectral feature compression layer, the class probability map is used as the channel attention weight to weight the spectral channels of the key feature bands to obtain the spectral features. In the spatial convolutional residual module, the class probability map is used as the spatial attention mask to extract features from the key feature bands to obtain the spatial features.

7. The method for identifying plant diseases and pests based on a multi-spectral image acquisition system according to claim 1, characterized in that, The multi-spectral image data is geometrically calibrated, and the noise and atmospheric effects in the multi-spectral image data are eliminated to obtain the target multi-spectral image data, including: The multi-spectral image is geometrically calibrated to obtain a calibrated image; The calibrated image is smoothed by the Savitzky-Golay convolutional smoothing method to eliminate the noise in the calibrated image and obtain a denoised image; The denoised data is atmospherically corrected by the FLAASH algorithm to obtain the target multi-spectral image data.

8. The method for identifying plant diseases and pests based on a multi-spectral image acquisition system according to claim 1, characterized in that, The geometric calibration includes spatial distortion correction based on a checkerboard calibration plate and pixel-level registration between multi-spectral channels.

9. The plant disease and pest identification method based on the multi-spectral image acquisition system according to claim 1, characterized in that The multi-spectral image acquisition system includes: A multi-band LED light source array for dynamically switching the illumination mode according to a preset band; An electric rotating stage for adjusting the placement angle of the plant sample; A high-resolution multi-spectral camera for collecting multi-spectral image data of the plant sample; An embedded control module for synchronously controlling the operation of the high-resolution multi-spectral camera and the LED light source array.

Images