Agricultural product disease automatic identification method and system based on local-global collaborative network

Abstract

The application discloses a kind of local global collaborative network-based agricultural product disease automatic identification method and system, first, agricultural product sample hyperspectral image is collected, and data preprocessing is carried out;Then the hyperspectral image after preprocessing is input into local global collaborative network to carry out the automatic identification of agricultural product disease;Output result is the probability value of each category, select the category label of maximum output probability as the classification to which the current image to be predicted belongs.The hyperspectral image of tea leaf blade obtained by the application using hyperspectral imaging technology has more wave bands and higher spectral resolution, can more accurately reflect the disease condition of tea leaf blade, and improve the accuracy of disease identification.In addition, using local global collaborative network model, the characteristics of tea disease can be automatically extracted, thereby realizing the automatic identification of tea disease.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of agricultural product safety detection, and relates to an agricultural product disease automatic identification method and system, in particular to an agricultural product disease automatic identification method and system based on hyperspectral technology and local-global collaborative network, which can be applied to automatic identification of tea diseases. BACKGROUND

[0002] At present, traditional tea tree disease monitoring methods mainly include artificial observation method, pathogenic bacteria detection method, lesion diagnosis method and the like. Among them, the artificial observation method is the most commonly used method, which is to observe the tea tree leaves, fruits, branches and the like, find the lesion, lesion morphology, color and other characteristics of the tea tree disease, and make judgment and diagnosis [1]. However, this method needs professional personnel to observe and judge, and the error is large and not accurate enough. The pathogenic bacteria detection method is to separate, culture and identify the pathogenic bacteria of the tea tree disease to determine the cause and pathogenic bacteria species of the tea tree disease [2]. This method can improve the accuracy and sensitivity of monitoring, but needs professional technology and equipment support, and has high cost. The lesion diagnosis method is to observe and analyze the morphology, anatomy and histology of the lesion of the tea tree disease to determine the cause and pathogenic bacteria species of the tea tree disease [3]. This method has the advantages of being simple, fast and economical, but needs professional personnel to observe and judge. Therefore, it is necessary to develop more scientific and efficient monitoring methods to timely find the tea tree disease, realize automatic diagnosis and early warning of the tea tree disease, and provide strong guarantee for the healthy growth and high yield and high quality of the tea tree.

[0003] Dai Q L, Xu Y P, Lin Q J, et al. Distribution and endophytic characteristics of endophytic Colletotrichum gloeosporioides in tea plant [J]. Forest Science, 2008, 44(5): 84-89.

[0004] Guo C, Gao X B, He L, et al. Application research progress of biological control of tea tree diseases and insect pests [J]. Guangdong Agricultural Sciences, 2014, 41(6): 105-109.

[0005] Wang Z K, Tan W Z, Zhang K C, et al. Identification of the pathogen of tea leaf clouded and blight and its biological characteristics [J]. Henan Agricultural Sciences, 2008, 4: 67-70. SUMMARY

[0006] In order to solve the problems of large error, high cost and need for expert experience in the prior art, the application provides an agricultural product disease automatic identification method and system based on hyperspectral technology and local-global collaborative network.

[0007] The technical scheme of the method of the application is as follows: an agricultural product disease automatic identification method based on local-global collaborative network, comprising the following steps:

[0008] Step 1: collect the hyperspectral image of the agricultural product sample and perform data preprocessing;

[0009] Step 2: input the preprocessed hyperspectral image into the local-global collaborative network for automatic identification of agricultural product diseases.

[0010] The local-global collaborative network comprises a shallow local network branch, a deep global network branch, a context-aware feature fusion module, and a classifier.

[0011] The shallow local network branch comprises two convolutional layers arranged in series, a normalization layer is added before the first and second convolutional layers, and an activation layer is added after each of them.

[0012] The deep global network branch comprises four convolutional layers and a global attention mechanism layer arranged in series, the first and second layers are convolutional layers, the third layer is a global attention mechanism layer, and the fourth and fifth layers are convolutional layers; a normalization layer is added before each of the first, second, fourth, and fifth convolutional layers, and an activation layer is added after each of them.

[0013] The context-aware feature fusion module uses bilinear interpolation to upsample the features output by the deep global network branch, so that the features of the shallow local network branch and the deep global network branch are of the same scale, and then connects the two features according to the channel dimension, and then passes through a convolutional layer to obtain the fused features.

[0014] The classifier is arranged after the context-aware feature fusion module and comprises a fully connected layer and a Softmax activation function, which is used to map the features to different categories after feature extraction and output the prediction probability of each category.

[0015] Preferably, in step 1, the preprocessing step comprises denoising and normalization; median filtering, Gaussian filtering, or wavelet transform is used to remove noise in the hyperspectral image, and linear normalization, maximum-minimum value normalization, or z-score normalization method is used to normalize the hyperspectral data, so that all bands have the same data range and eliminate the influence of spectral differences between different bands caused by different response values.

[0016] Preferably, in step 2, the shallow local network branch, the first layer is a convolutional layer with a convolution kernel size of 3 and a step size of 1, and the second layer is a convolutional layer with a convolution kernel size of 3 and a step size of 1.

[0017] The deep global network branch, the first layer is a convolutional layer with a convolution kernel size of 3 and a step size of 2; the second layer is a convolutional layer with a convolution kernel size of 3 and a step size of 1; the third layer is a global attention mechanism layer, which is used to construct three feature maps Q, K and V from the output feature F of the second layer convolutional layer through a convolution kernel with a size of 1; the feature map Q and the feature map K are subjected to matrix multiplication and softmax operation to obtain an attention matrix A, and then A is multiplied with the feature map V to obtain a weight value The weight value is added to the feature map F and then output; the fourth layer is a convolutional layer with a convolution kernel size of 1 and a step size of 1; the fifth layer is a convolutional layer with a convolution kernel size of 1 and a step size of 1.

[0018] The context-aware feature fusion module obtains a fusion feature through a convolutional layer with a convolution kernel size of 3 and a step size of 1.

[0019] Preferably, the local-global collaborative network for automatic identification of agricultural diseases in step 2 is a trained local-global collaborative network for automatic identification of agricultural diseases; the training process includes the following sub-steps:

[0020] Step S1: Collecting a plurality of healthy agricultural product samples and diseased agricultural product samples, and obtaining hyperspectral data by using an imaging spectrometer;

[0021] Step S2: Hyperspectral image data preprocessing;

[0022] Step S3: Making an agricultural disease identification data set and dividing it into a training set and a test set;

[0023] Step S4: Training the model on the training set and constantly optimizing the model parameters using the back propagation algorithm to obtain a trained model.

[0024] Preferably, in step S2, the preprocessing step includes denoising and normalization; median filtering, Gaussian filtering or wavelet transform is used to remove noise in the hyperspectral image, and linear normalization, maximum-minimum value normalization or z-score normalization method is used to normalize the hyperspectral data, so that all wavebands have the same data range, and the influence of spectral differences between different wavebands caused by different response values is eliminated.

[0025] Preferably, in step S3, the agricultural disease identification data set includes image samples covering different agricultural diseases and corresponding labels, and the data set is divided into a training set and a test set according to the proportion of different diseases.

[0026] Preferably, in step S4, the training process includes the following steps:

[0027] (1) Forward propagation: input the image in the training set into the model, extract features through the convolutional layer and the pooling layer, and then pass through the fully connected layer to obtain the classifier output; finally, the output result is converted into the probability of each category using the Softmax function;

[0028] (2) Calculate loss: use the cross-entropy function to calculate the error of the model, and adjust the model parameters according to the loss size;

[0029] (3) Back propagation: use the back propagation algorithm to calculate the gradient, and update the model weight according to the gradient; the process of back propagation is from the loss, and the parameters of each layer are inversely transmitted to calculate the influence degree of each parameter on the loss;

[0030] (4) Parameter update: update the model parameters according to the gradient descent algorithm to minimize the loss of the model;

[0031] (5) Repeat the above steps until the model reaches the preset training number or reaches the preset accuracy.

[0032] The technical scheme adopted by the system of the application is: an agricultural product disease automatic identification system based on a local-global collaborative network, comprising:

[0033] One or more processors;

[0034] A storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the local-global collaborative network-based agricultural product disease automatic identification method.

[0035] The advantage of the application is that the hyperspectral image of agricultural products obtained by using hyperspectral imaging technology has more wavebands and higher spectral resolution, which can more accurately reflect the disease condition of agricultural products and improve the accuracy of disease identification. In addition, the local-global collaborative network model can automatically extract the local features and global features of the agricultural product disease hyperspectral image, and adaptively fuse them, while retaining the local spatial detail information and global context information of the agricultural product disease hyperspectral image, thereby realizing automatic and accurate identification of agricultural product diseases. BRIEF DESCRIPTION OF DRAWINGS

[0036] The technical scheme of the present application is further illustrated by the following examples and specific embodiments. In addition, some drawings are also used in the process of explaining the technical scheme. For those skilled in the art, other drawings and the intention of the present application can also be obtained without creative labor.

[0037] Figure 1 The method flowchart of the embodiment of the present application;

[0038] Figure 2 A local-global collaborative network structure diagram for an embodiment of the present application;

[0039] Figure 3 A local-global collaborative network training flowchart provided by an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to facilitate those skilled in the art to understand and implement the present application, the present application will be further described in detail below in conjunction with the drawings and embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0041] The present embodiment takes tea pesticide residue detection as an example to further illustrate the present application. Please see Figure 1 The present embodiment provides a tea disease automatic identification method based on a local-global collaborative network, comprising the following steps:

[0042] Step 1: Collecting a tea sample hyperspectral image and performing data preprocessing;

[0043] In an embodiment, the preprocessing step includes denoising and normalization; using median filtering, Gaussian filtering, wavelet transform and other technologies to remove noise in the hyperspectral image, and using linear normalization, maximum and minimum value normalization, z-score normalization and other methods to normalize the hyperspectral data, so that all bands have the same data range, and eliminate the influence of spectral differences between different bands due to different response values.

[0044] Step 2: Inputting the preprocessed hyperspectral image into the agricultural product disease automatic identification local-global collaborative network to automatically identify tea diseases;

[0045] Please see Figure 2 The agricultural product disease automatic identification local-global collaborative network of the present embodiment comprises a shallow local network branch, a deep global network branch, a context-aware feature fusion module and a classifier.

[0046] The shallow local network branch of the present embodiment comprises two convolutional layers arranged in series, the first layer is a convolutional layer with a convolution kernel size of 3 and a step size of 1; the second layer is a convolutional layer with a convolution kernel size of 3 and a step size of 1; a normalization layer is added before the first and second convolutional layers, and an activation layer is added after the first and second convolutional layers;

[0047] The deep global network branch of the embodiment comprises four convolutional layers arranged in series and a global attention mechanism layer, the first layer is a convolutional layer with a convolution kernel size of 3 and a step size of 2; the second layer is a convolutional layer with a convolution kernel size of 3 and a step size of 1; the third layer is a global attention mechanism layer, which is used to construct three feature maps Q, K and V from the output feature F of the second convolutional layer through a convolution kernel with a size of 1; the feature map Q and the feature map K are subjected to matrix multiplication and softmax operation to obtain an attention matrix A, and then A is multiplied by the feature map V to obtain a weight value The weight value is added to the feature map F and then output; the fourth layer is a convolutional layer with a convolution kernel size of 1 and a step size of 1; the fifth layer is a convolutional layer with a convolution kernel size of 1 and a step size of 1. A normalization layer is added before each of the first layer, the second layer, the fourth layer and the fifth layer, and an activation layer is added after each of the first layer, the second layer, the fourth layer and the fifth layer;

[0048] The context-aware feature fusion module of the embodiment uses bilinear interpolation to upsample the features output by the deep global network branch, so that the features of the shallow local network branch and the deep global network branch are of the same scale, and the two features are connected in the channel dimension, and then a convolutional layer with a convolution kernel size of 3 and a step size of 1 is used to obtain the fused features.

[0049] The classifier of the embodiment comprises a fully connected layer and a Softmax activation function, which is used to map the features to different categories after feature extraction, and output the prediction probability of each category.

[0050] The output result is the probability value of each category, and the category label with the maximum output probability is selected as the classification to which the current image to be predicted belongs.

[0051] See Figure 3 The automatic recognition local-global collaborative network of agricultural diseases of the embodiment is a trained automatic recognition local-global collaborative network of agricultural diseases; the training process comprises the following sub-steps:

[0052] Step S1: Collect healthy tea leaf samples and diseased tea leaf samples, and use an imaging spectrometer to obtain hyperspectral data.

[0053] In one embodiment, healthy tea leaf samples and diseased tea leaf samples are collected in an experimental tea garden, and a thermos box is used to transport them to the laboratory. A Micro-Hyperspec lightweight hyperspectral imaging platform is used to collect hyperspectral data, and the spectral range is 400-2500nm.

[0054] Step S2: Hyperspectral image data preprocessing.

[0055] In an embodiment, the preprocessing step includes denoising and normalization. Wavelet transform is used to remove noise in the hyperspectral image, and the maximum-minimum value normalization method is used to normalize the hyperspectral data, so that all bands have the same data range, and the spectral difference between different bands caused by different response values is eliminated.

[0056] Step S3: Making a tea disease identification dataset, and dividing a training set and a test set.

[0057] In an embodiment, the tea disease identification dataset made collects hyperspectral images of tea samples of anthracnose, brown spot and branch blight, and considers many factors such as different leaf parts, different disease courses and different lighting conditions. For each collected picture, its disease category is labeled. The dataset is divided into a training set and a test set according to the proportion of different diseases.

[0058] Step S4: Training the model on the training set, and continuously optimizing the model parameters by using the back propagation algorithm.

[0059] The local-global collaborative network model in this embodiment is specially designed for tea disease identification. It includes a shallow local network branch, a deep global network branch, a context-aware feature fusion module and a classifier.

[0060] The shallow local network branch in this embodiment includes two convolutional layers arranged in series. The first layer is a convolutional layer with a kernel size of 3 and a step size of 1. The second layer is a convolutional layer with a kernel size of 3 and a step size of 1. Batch normalization layers (BN) are added before the first and second convolutional layers, and activation functions ReLU are added after the first and second convolutional layers. The convolutional block can effectively learn and extract local features of the image, and gradually form more abstract feature representations through multiple stacking methods.

[0061] The deep global network branch in this embodiment includes four convolutional layers arranged in series and a global attention mechanism layer. The first layer is a convolutional layer with a kernel size of 3 and a step size of 2. The second layer is a convolutional layer with a kernel size of 3 and a step size of 1. The third layer is a global attention mechanism layer, which is used to construct three feature maps Q, K and V from the output features F of the second convolutional layer through a convolutional kernel with a size of 1. The feature map Q and the feature map K are subjected to matrix multiplication and softmax operation to obtain an attention matrix A, and then A is multiplied by the feature map V to obtain a weight value The weight value is added to the feature map F and then output. The fourth layer is a convolutional layer with a kernel size of 1 and a step size of 1. The fifth layer is a convolutional layer with a kernel size of 1 and a step size of 1. Normalization layers are added before the first, second, fourth and fifth convolutional layers, and activation layers are added after the first, second, fourth and fifth convolutional layers.

[0062] In this embodiment, the context-aware feature fusion module uses bilinear interpolation to upsample the features output by the deep global network branch, so that the features of the shallow local network branch and the deep global network branch are of the same scale, and the two features are connected according to the channel dimension, and then a convolution layer with a kernel size of 3 and a step size of 1 is used to obtain the fused features.

[0063] In this embodiment, the classifier includes a fully connected layer and a Softmax activation function, which is used to map the features to different categories after feature extraction and output the prediction probability of each category.

[0064] In one embodiment, the loss of the model is calculated using the cross-entropy function, and the gradient is calculated using the backpropagation algorithm, and the model parameters are updated according to the descent method to minimize the loss of the model. The gradient descent algorithm updates the model parameter value by a certain step size according to the direction of the current gradient. The training process can be divided into the following steps:

[0065] Forward propagation: input the image in the training set into the model, extract features through the convolution layer and the pooling layer, and then pass through the fully connected layer to obtain the classifier output. Finally, the output result is converted into the probability of each category using the Softmax function.

[0066] Calculate the loss: use the cross-entropy function to calculate the error (loss) of the model, and adjust the model parameters according to the loss size. A larger loss means that there is a larger difference between the model's predicted result and the actual label. The error of the model is calculated using the cross-entropy loss function, and the model weights are updated using the backpropagation algorithm.

[0067] Backpropagation: use the backpropagation algorithm to calculate the gradient, and update the model weights according to the gradient. The process of backpropagation starts from the loss and propagates back to the parameters of each layer to calculate the degree of influence of each parameter on the loss. Calculate the degree of influence of each parameter on the loss and pass it back to update the model parameters.

[0068] Parameter update: update the model parameters according to the gradient descent algorithm to minimize the loss of the model. The gradient descent algorithm updates the model parameter value by a certain step size according to the direction of the current gradient.

[0069] Repeat the above steps: repeat the above steps until the model reaches the preset number of training rounds or reaches a certain accuracy.

[0070] In this embodiment, the trained model parameters are loaded, and the tea disease image to be predicted is input into the local-global collaborative network model for prediction, and the output result is the probability value of each category. The category label with the maximum output probability is selected as the classification to which the current image to be predicted belongs.

[0071] The tea leaf hyperspectral image obtained by using the hyperspectral imaging technology has more wave bands and higher spectral resolution, can more accurately reflect the disease condition of the tea leaf, and improves the accuracy of disease identification.

[0072] The application is not only suitable for tea leaves, but also can be applied to automatic identification of diseases of other leaf crops, and has a wide application range.

[0073] It should be understood that the above description of the preferred embodiments is relatively detailed, and should not be considered as a limitation on the scope of patent protection of the present application. Those skilled in the art can make substitutions or modifications without departing from the scope of protection of the present application, and all fall within the scope of protection of the present application. The scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for automatic identification of crop diseases based on a local-global collaborative network, characterized in that, The method comprises the following steps: Step 1: collecting a hyperspectral image of the agricultural product sample and performing data preprocessing; Step 2: inputting the preprocessed hyperspectral image into a local-global collaborative network for automatic identification of the agricultural product disease; The local-global collaborative network comprises a shallow local network branch, a deep global network branch, a context-aware feature fusion module and a classifier; The shallow local network branch comprises two convolutional layers arranged in series, a normalization layer is added before the first and second convolutional layers, and an activation layer is added after the first and second convolutional layers; The deep global network branch comprises four convolutional layers arranged in series and a global attention mechanism layer, the first and second layers are convolutional layers, the third layer is a global attention mechanism layer, and the fourth and fifth layers are convolutional layers; a normalization layer is added before the first, second, fourth and fifth convolutional layers, and an activation layer is added after the first, second, fourth and fifth convolutional layers; The context-aware feature fusion module uses bilinear interpolation to upsample the features output by the deep global network branch, so that the features of the shallow local network branch and the deep global network branch are of the same scale, and then connects the two features in the channel dimension, and then obtains the fused features through a convolutional layer; The classifier is arranged after the context-aware feature fusion module and comprises a fully connected layer and a Softmax activation function, which is used to map the features to different categories after feature extraction and output the prediction probability of each category; The first layer of the shallow local network branch is a convolutional layer with a kernel size of 3 and a step of 1; the second layer is a convolutional layer with a kernel size of 3 and a step of 1; The deep global network branch, the first layer is a convolutional layer with a convolution kernel size of 3 and a step size of 2; the second layer is a convolutional layer with a convolution kernel size of 3 and a step size of 1; the third layer is a global attention mechanism layer, used for constructing three feature maps Q, K and V through a convolution kernel with a size of 1 on the output feature F of the second layer convolutional layer; the feature map Q and the feature map K are subjected to matrix multiplication and softmax operation to obtain an attention matrix A, and then A is multiplied with the feature map V to obtain a weight value The weight value is added to the feature map F to output; the fourth layer is a convolutional layer with a convolution kernel size of 1 and a step size of 1; the fifth layer is a convolutional layer with a convolution kernel size of 1 and a step size of 1; The context-aware feature fusion module obtains the fused features through a convolutional layer with a kernel size of 3 and a step of 1; The agricultural product disease automatic identification local-global collaborative network is a trained agricultural product disease automatic identification local-global collaborative network; The training process comprises the following sub-steps: Step S1: collecting a plurality of healthy agricultural product samples and disease agricultural product samples, and obtaining hyperspectral data using an imaging spectrometer; Step S2: hyperspectral image data preprocessing; Step S3: preparing an agricultural product disease identification dataset and dividing the training set and the test set; Step S4: training the model on the training set, continuously optimizing the model parameters using the back propagation algorithm, and obtaining the trained model.

2. The method of claim 1, wherein the method comprises: In step 1, the preprocessing step comprises denoising and normalization; The median filter, Gaussian filter or wavelet transform is used to remove noise in the hyperspectral image, and the linear normalization, maximum-minimum value normalization or z-score normalization method is used to normalize the hyperspectral data, so that all bands have the same data range, and the influence of spectral differences caused by different response values between different bands is eliminated.

3. The method of claim 1, wherein the method comprises: In step S2, the preprocessing step comprises denoising and normalization; The median filter, Gaussian filter or wavelet transform is used to remove the noise in the hyperspectral image, and the linear normalization, maximum-minimum value normalization or z-score normalization method is used to normalize the hyperspectral data, so that all bands have the same data range, and the spectral difference caused by the different response values between different bands is eliminated.

4. The method of claim 1, wherein the method comprises: In step S3, the prepared agricultural product disease identification dataset includes image samples covering different agricultural product diseases and corresponding labels, and the dataset is divided into a training set and a test set according to the proportion of different diseases.

5. The method of claim 1, wherein the method comprises: In step S4, the training process includes the following steps: (1) Forward propagation: input the image in the training set into the model, perform feature extraction through the shallow local network branch and the deep global network branch, then perform feature fusion using the context-aware feature fusion module, and finally pass through the fully connected layer to obtain the classifier output, and use the Softmax function to convert the output result into the probability of each category; (2) Loss calculation: use the cross-entropy function to calculate the error of the model, and adjust the model parameters according to the loss size; (3) Back propagation: use the back propagation algorithm to calculate the gradient, and update the model weights according to the gradient; the process of back propagation is to start from the loss and inversely pass through the parameters of each layer to calculate the influence degree of each parameter on the loss; (4) Parameter update: update the model parameters according to the gradient descent algorithm to minimize the loss of the model; (5) Repeat the above steps until the model reaches the preset number of training rounds or reaches the preset accuracy.

6. An automatic agricultural product disease identification system based on a local-global collaborative network, characterized in that, Comprise: One or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the local-global collaborative network-based automatic identification method for agricultural product diseases according to any one of claims 1-5.

Images