Crop leaf disease identification system and method

Abstract

The invention discloses a crop leaf disease identification system and method. The system comprises a data processing terminal and a data acquisition device, the data acquisition device comprises a moving mechanism, a mechanical arm and an image acquisition module, the mechanical arm is arranged at the top of the moving mechanism, and the image acquisition module is arranged at the top of the mechanical arm. The image acquisition module comprises an annular support and a driving mechanism used for driving the annular support to rotate, a plurality of cameras are arranged on the annular support, a controller and a wireless communication module are arranged on the moving mechanism, and the controller is electrically connected with the moving mechanism, the mechanical arm, the cameras, the driving mechanism and the wireless communication module. And the wireless communication module is wirelessly connected with the data processing terminal through a wireless network. According to the method, the crop leaf diseases can be quickly and accurately identified.

Description

Technical Field

[0001] The present invention relates to the technical field of crop disease detection, and in particular to a crop leaf disease recognition system and method. Background Art

[0002] Traditionally, crop leaf disease identification is performed by experienced experts. This identification method is inefficient, labor-intensive, and cannot make scientific and accurate diagnosis of diseases in real time. With the widespread application of image processing technology and machine vision technology in the agricultural field, automatic identification of crop leaf diseases has become a trend in the development of intelligent agriculture.

[0003] At present, most methods for identifying crop leaf diseases involve manually collecting diseased leaves, taking photos in the laboratory, and using image recognition technology to identify leaf diseases. This is cumbersome and time-consuming. In addition, crop leaf diseases have diverse morphologies, large intra-class differences, and high inter-class similarities. Existing image recognition technology for identifying crop leaf diseases has a low recognition accuracy rate. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a crop leaf disease identification system and method, which can quickly and accurately identify crop leaf diseases.

[0005] In order to solve the above problems, the present invention adopts the following technical solutions:

[0006] A crop leaf disease identification system of the present invention comprises a data processing terminal and a data acquisition device, wherein the data acquisition device comprises a mobile mechanism, a mechanical arm and an image acquisition module, wherein the mechanical arm is arranged on the top of the mobile mechanism, and the image acquisition module is arranged on the top of the mechanical arm, and the image acquisition module comprises an annular bracket and a driving mechanism for driving the annular bracket to rotate, wherein a plurality of cameras are arranged on the annular bracket, and a controller and a wireless communication module are arranged on the mobile mechanism, wherein the controller is electrically connected to the mobile mechanism, the mechanical arm, the camera, the driving mechanism and the wireless communication module respectively, and the wireless communication module is wirelessly connected to the data processing terminal via a wireless network.

[0007] In this solution, when it is necessary to detect leaf diseases of a certain crop, the mobile mechanism moves to the leaves of the crop to be tested, the robotic arm drives the image acquisition module to move to the position facing the leaves of the crop to be tested, and the annular bracket rotates, so that each camera is rotated to the position facing the leaves of the crop to be tested in turn. When the camera rotates to the position facing the leaves of the crop to be tested, the camera collects the image of the leaves of the crop to be tested. The controller sends the images of the leaves of the crop to be tested collected by all cameras to the data processing terminal. The data processing terminal analyzes the images of the leaves of the crop to be tested collected by all cameras and outputs the recognition results. This solution can automatically complete the identification of crop leaf diseases without the need for manual collection of diseased leaves. It is simple to operate and saves time and effort.

[0008] Preferably, the driving mechanism includes a support shaft, the support shaft is connected to the robotic arm, the annular bracket is arranged on the outside of the support shaft and is coaxial with the support shaft, a rotating sleeve coaxial with the support shaft is provided on the outside of the support shaft, the annular bracket is connected to the rotating sleeve through a connecting rod, and a rotating mechanism is provided on the support shaft for driving the rotating sleeve to rotate around the support shaft.

[0009] Preferably, the spacing between adjacent cameras on the annular bracket is equal.

[0010] A crop leaf disease identification method of the present invention is used in the above-mentioned crop leaf disease identification system, comprising the following steps:

[0011] S1: The mobile mechanism moves to the position of the crop leaves to be measured, the mechanical arm drives the image acquisition module to move to the position facing the crop leaves to be measured, and the annular bracket rotates to rotate each camera to the position facing the crop leaves to be measured in turn. When the camera rotates to the position facing the crop leaves to be measured, the camera collects the image of the crop leaves to be measured;

[0012] S2: The controller sends the images of the crop leaves to be tested collected by all cameras to the data processing terminal;

[0013] S3: The data processing terminal averages the images of the crop leaves to be tested collected by all cameras to obtain a mean image;

[0014] S4: Input the mean image into the crop leaf disease recognition model, and the crop leaf disease recognition model outputs the recognition result.

[0015] In the process of camera shooting images, some non-fixed noises often appear. These noises are not related to the spatial domain and are basically additive noises. Therefore, multiple cameras are used to shoot the images of the leaves of the crops to be tested respectively, and then the images of the leaves of the crops to be tested are averaged to eliminate the noise contained in the images.

[0016] Preferably, the crop leaf disease recognition model is obtained by the following method:

[0017] M1: Collect crop leaf disease images, manually label the crop leaf disease images, establish a crop leaf disease image dataset, and divide the crop leaf disease image dataset into a training set and a test set;

[0018] M2: Perform image enhancement on the labels in the training set;

[0019] M3: Preprocessing of crop leaf disease image dataset;

[0020] M4: Add the multi-scale context aggregation module between the encoder and decoder of the Unet network, replace the up-sampled transposed convolution of the Unet network with bilinear interpolation, and obtain the feature extraction neural network model;

[0021] M5: Construct a loss function to evaluate the gap between the true value and the predicted value;

[0022] M6: Use the training set with online data enhancement to train the feature extraction neural network model of the migration parameter weights, and use the trained feature extraction neural network model as the crop leaf disease recognition model;

[0023] M7: Use the test set to test the crop leaf disease recognition model.

[0024] Preferably, in step M1, the crop leaf disease images are collected, the crop leaf disease images are manually labeled, and the method for establishing a crop leaf disease image dataset is as follows:

[0025] Collect original images of different types of crop leaf diseases, use image annotation tools to manually label the diseased areas on the collected original images, and obtain a crop leaf disease image dataset;

[0026] The labeling method is to draw the diseased area on the original image and assign semantic labels to it.

[0027] Preferably, the method for performing image enhancement on the label in step M2 is as follows:

[0028] Convert the original label image into a grayscale image to obtain the grayscale image matrix I gray ;

[0029] Generate grayscale image matrix I gray A noise matrix of uniform size, where each value in the noise matrix is ​​a random value drawn from a standard normal distribution;

[0030] Multiply the noise matrix by the weight η to obtain the weighted noise matrix;

[0031] The grayscale image matrix I gray Add it to the weighted noise matrix to get the grayscale image matrix I after adding noise noisy ;

[0032] Use edge detection algorithm to analyze the grayscale image matrix I noisy Perform edge detection to obtain a mask;

[0033] The mask is inverted and applied to the original image of the label to obtain the image-enhanced label.

[0034] Preferably, the feature extraction neural network model includes an encoder, a decoder, and a multi-scale context aggregation module, the encoder includes a conv1 layer, a conv2 layer, a conv3 layer, a conv4 layer, and a conv5 layer arranged in sequence, the decoder includes an up_concat4 layer, an up_concat3 layer, an up_concat2 layer, an up_concat1 layer, and a final layer arranged in sequence, and the multi-scale context aggregation module includes an atrous_conv1 layer, an atrous_conv1 layer, an atrous_conv1 layer, an atrous_conv1 layer, a global_pooling layer, and a Concat layer.

[0035] Preferably, the atrous_conv1 layer is used to generate a 32×32×512 feature map through a 512-channel convolution with a convolution kernel size of 1×1, and then use a global context module to perform feature fusion and enhancement to output a 32×32×512 feature map;

[0036] The atrous_conv2 layer is used to generate a 32×32×512 feature map of 32×32×512 output by the conv5 layer through a dilated convolution with a convolution kernel size of 3×3, a dilation rate of 6, and a channel number of 512, and then use the global context module to perform feature fusion and enhancement to output a 32×32×512 feature map;

[0037] The atrous_conv3 layer is used to generate a 32×32×512 feature map of 32×32×512 output by the conv5 layer through a dilated convolution with a convolution kernel size of 3×3, a dilation rate of 12, and a channel number of 512, and then use the global context module to perform feature fusion and enhancement, and output a 32×32×512 feature map;

[0038] The atrous_conv4 layer is used to generate a 32×32×512 feature map of 32×32×512 output by the conv5 layer through a dilated convolution with a convolution kernel size of 3×3, a dilation rate of 18, and a channel number of 512, and then use the global context module to perform feature fusion and enhancement, and output a 32×32×512 feature map;

[0039] The global_pooling layer is used to perform global average pooling on the 32×32×512 feature map output by the conv5 layer, reducing the feature map size from 32×32×512 to 1×1×512, and then using a 1×1 convolution kernel to perform 512-channel convolution, and finally using the ReLU activation function to enhance the nonlinear feature expression and output a 1×1×512 global pooling feature map;

[0040] The Concat layer is used to superimpose the output feature maps of the atrous_conv1 layer, atrous_conv2 layer, atrous_conv3 layer, atrous_conv4 layer, and global_pooling layer in the depth dimension to generate a comprehensive feature map of 32×32×2560, and use a convolution with a convolution kernel size of 1×1 to convolve the 32×32×2560 comprehensive feature map to reduce the number of channels to 512. Finally, the ReLU activation function is used to enhance the nonlinear feature expression, and finally a feature map of 32×32×512 is output to the up_concat4 layer.

[0041] Preferably, the up_concat4 layer is used to upsample the 32×32×512 feature map output by the Concat layer through bilinear interpolation to obtain a 64×64×512 feature map, merge the 64×64×512 feature map with the 64×64×512 feature map output by the conv4 layer to obtain a 64×64×1024 feature map, and the 64×64×1024 feature map is convolved twice with a convolution kernel size of 3×3 and a channel number of 512, and then further refined by a ReLU activation function, and finally output a 64×64×512 feature map;

[0042] The up_concat3 layer is used to upsample the 64×64×512 feature map output by the up_concat4 layer through bilinear interpolation to obtain a 128×128×512 feature map, merge the 128×128×512 feature map with the 128×128×256 feature map output by the conv3 layer to obtain a 128×128×768 feature map, and perform two convolutions with a kernel size of 3×3 and a channel number of 256 on the 128×128×768 feature map, and further refine the features through the ReLU activation function, and finally output a 128×128×256 feature map;

[0043] The up_concat2 layer is used to upsample the 128×128×256 feature map output by the up_concat3 layer through bilinear interpolation to obtain a 256×256×256 feature map, merge the 256×256×256 feature map with the 256×256×128 feature map output by the conv2 layer to obtain a 256×256×384 feature map, and perform two convolutions with a kernel size of 3×3 and a channel number of 128 on the 256×256×384 feature map, and further refine the features through a ReLU activation function, and finally output a 256×256×128 feature map;

[0044] The up_concat1 layer is used to upsample the 256×256×128 feature map output by the up_concat2 layer through bilinear interpolation to obtain a 512×512×128 feature map, merge the 512×512×128 feature map with the 512×512×64 feature map output by the conv1 layer to obtain a 512×512×192 feature map, and perform two convolutions with a kernel size of 3×3 and a channel number of 64 on the 512×512×192 feature map, and further refine the features through the ReLU activation function, and finally output a 512×512×64 feature map;

[0045] The final layer is used to perform convolution with a kernel size of 1×1 on the 512×512×64 feature map output by the up_concat1 layer, and processes it using K output channels to map the 64-channel features of each pixel to the predicted probabilities of K categories.

[0046] Preferably, the step M5 comprises the following steps:

[0047] Constructing the cross entropy loss function L CE :

[0048]

[0049] Constructing Dice loss function LDice :

[0050]

[0051] Constructing the hybrid loss function L total :

[0052] L total =α*L Dice +β*L CE ,

[0053] Where N is the number of pixels, D is the number of categories, and T i，c is the true label of the i-th pixel in class c, is the predicted probability of the i-th pixel in the c-th class, α and β are weight coefficients;

[0054] The mixed loss function L total As a loss function used to evaluate the gap between the true value and the predicted value.

[0055] Preferably, the method for training the feature extraction neural network model of the migration parameter weights using the online data enhanced training set in step S6 is as follows:

[0056] M1: Download the model parameter weights of the Unet network pre-trained in the public image dataset Imagenet, and perform transfer learning on the feature extraction neural network model;

[0057] M2: The training set is used to train the feature extraction neural network model after transfer learning. The following operations are performed on the training set before each iterative training process: the image samples in the training set are uniformly randomly cropped and resized to images of 512×512 pixels; the image samples are uniformly randomly horizontally flipped with a probability of 0.5; the image samples are uniformly randomly occluded with a probability of 0.2 and an aspect ratio ranging from 2 / 100 to 1 / 3.

[0058] The beneficial effects of the present invention are as follows: (1) It can quickly and accurately identify crop leaf diseases. (2) By enhancing the label image, the manually annotated label is closer to the real leaf disease area; by adding a multi-scale context aggregation module between the encoder and the decoder of the Unet network to obtain the multi-scale context feature information of the image, the complex leaf disease features can be better captured and processed; the outline of small disease spots can be better segmented by the constructed hybrid loss function; the purpose of stably identifying crop leaf diseases is achieved, the recognition accuracy is improved, and the robustness is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a structural schematic diagram of an embodiment;

[0060] Figure 2 It is a structural diagram of the feature extraction neural network model.

[0061] In the figure: 1. Mobile mechanism, 2. Robotic arm, 3. Ring bracket, 4. Camera, 5. Support shaft, 6. Rotating sleeve, 7. Connecting rod, 8. Rotating mechanism, 9. Encoder, 10. Decoder, 11. Multi-scale context aggregation module. DETAILED DESCRIPTION

[0062] The technical solution of the present invention is further specifically described below through embodiments and in conjunction with the accompanying drawings.

[0063] Embodiment: This embodiment is a crop leaf disease identification system, such as Figure 1 As shown, it includes a data processing terminal and a data acquisition device, the data acquisition device includes a mobile mechanism 1, a mechanical arm 2 and an image acquisition module, the mechanical arm is arranged on the top of the mobile mechanism 1, the image acquisition module is arranged on the top of the mechanical arm 2, the image acquisition module includes an annular bracket 3 and a driving mechanism for driving the annular bracket 3 to rotate, eight cameras 4 are arranged on the annular bracket 3, the mobile mechanism 1 is provided with a controller and a wireless communication module, the controller is electrically connected to the mobile mechanism 1, the mechanical arm 2, the camera 4, the driving mechanism, and the wireless communication module, and the wireless communication module is wirelessly connected to the data processing terminal via a wireless network.

[0064] The driving mechanism includes a support shaft 5, which is connected to the robot arm 2, an annular bracket 3 is arranged outside the support shaft 5 and coaxial with the support shaft 5, a rotating sleeve 6 coaxial with the support shaft 5 is arranged outside the support shaft 5, the annular bracket 3 is connected to the rotating sleeve 6 through eight connecting rods 7, and a rotating mechanism 8 for driving the rotating sleeve 6 to rotate around the support shaft 5 is arranged on the support 5. The spacing between adjacent cameras 4 on the annular bracket 3 is equal.

[0065] In this solution, when it is necessary to detect leaf diseases of a certain crop, the mobile mechanism moves to the leaves of the crop to be tested, the robotic arm drives the image acquisition module to move to the position facing the leaves of the crop to be tested, and the annular bracket rotates, so that each camera is rotated to the position facing the leaves of the crop to be tested in turn. When the camera rotates to the position facing the leaves of the crop to be tested, the camera collects the image of the leaves of the crop to be tested. The controller sends the images of the leaves of the crop to be tested collected by all cameras to the data processing terminal. The data processing terminal analyzes the images of the leaves of the crop to be tested collected by all cameras and outputs the recognition results. This solution can automatically complete the identification of crop leaf diseases without the need for manual collection of diseased leaves. It is simple to operate and saves time and effort.

[0066] A crop leaf disease identification method of this embodiment is used in the above-mentioned crop leaf disease identification system, and includes the following steps:

[0067] S1: The mobile mechanism moves to the position of the crop leaves to be measured, the mechanical arm drives the image acquisition module to move to the position facing the crop leaves to be measured, and the annular bracket rotates to rotate each camera to the position facing the crop leaves to be measured in turn. When the camera rotates to the position facing the crop leaves to be measured, the camera collects the image of the crop leaves to be measured;

[0068] S2: The controller sends the images of the crop leaves to be tested collected by all cameras to the data processing terminal;

[0069] S3: The data processing terminal averages the images of the crop leaves to be tested collected by all cameras to obtain a mean image;

[0070] S4: Input the mean image into the crop leaf disease recognition model, and the crop leaf disease recognition model outputs the recognition result.

[0071] Each camera moves to the same position to shoot the crop leaves. In the process of camera shooting images, some non-fixed noise often appears. This noise is not related to the spatial domain and is basically additive noise. Therefore, multiple cameras are used to shoot the images of the crop leaves to be tested, and then the images of the crop leaves to be tested are averaged to eliminate the noise contained in the image.

[0072] The crop leaf disease recognition model is obtained by the following method:

[0073] M1: Collect crop leaf disease images, manually label the crop leaf disease images, establish a crop leaf disease image dataset, and divide the crop leaf disease image dataset into a training set and a test set. The ratio of the training set to the test set is 9:1;

[0074] M2: Perform image enhancement on the labels in the training set;

[0075] M3: Preprocessing of crop leaf disease image dataset;

[0076] M4: Add the multi-scale context aggregation module between the encoder and decoder of the Unet network, replace the up-sampled transposed convolution of the Unet network with bilinear interpolation, and obtain the feature extraction neural network model;

[0077] M5: Construct a loss function to evaluate the gap between the true value and the predicted value;

[0078] M6: Use the training set with online data enhancement to train the feature extraction neural network model of the migration parameter weights, and use the trained feature extraction neural network model as the crop leaf disease recognition model;

[0079] M7: Use the test set to test the crop leaf disease recognition model.

[0080] In step M1, crop leaf disease images are collected, and the crop leaf disease images are manually labeled to establish a crop leaf disease image dataset as follows:

[0081] Collect original images of different types of crop leaf diseases, use the image annotation tool Labelme to manually label the diseased areas on the collected original images, and obtain a crop leaf disease image dataset;

[0082] The labeling method is: draw the diseased area on the original image and assign it a semantic label. After the labeling is completed, save the labeling result as a JSON file, and convert the JSON file into a binary label image through code.

[0083] The method for image enhancement of the label in step M2 is as follows:

[0084] Convert the original label image into a grayscale image to obtain the grayscale image matrix I gray ;

[0085]

[0086] Among them, I ij represents the pixel value at position (i, j) of the grayscale image matrix,

[0087] Generate grayscale image matrix I gray A noise matrix N of uniform size, where each value in the noise matrix N is a random value drawn from a standard normal distribution;

[0088]

[0089] Among them, N ij is a random value drawn from a standard normal distribution,

[0090] Multiply the noise matrix N by the weight η to obtain the weighted noise matrix G, G = η*N;

[0091] The grayscale image matrix I gray Add it to the weighted noise matrix G to get the grayscale image matrix I after adding noise noisy , I noisy =I gray +η*N;

[0092] Use the Canny edge detection algorithm to detect the grayscale image matrix I noisy Perform edge detection to obtain a mask;

[0093] E=Canny(Inoisv , T1, T2),

[0094] Among them, E is the binary edge detection result matrix, with a shape of H×W; I noisy is a grayscale image matrix with a shape of H×W; T1 is the low threshold in the Canny edge detection algorithm; T2 is the high threshold in the Canny edge detection algorithm;

[0095] The mask is inverted and applied to the original image of the label to obtain the image-enhanced label.

[0096] By adding noise, uncertainty is introduced near the label contour, causing the contour to change slightly. This scheme generates an array of the same size as the input label by adding a random value generated by a normal distribution to each pixel of the grayscale image. Each value in the array is randomly drawn from a standard normal distribution. η is a coefficient that controls the intensity of the noise. The η value is multiplied by the random noise generated by the normal distribution and then added to the original grayscale image. The coefficient η is allowed to adjust the amount of added noise, thereby controlling the degree of noise impact on the image. Choosing a suitable η value can both enhance the edge of the image contour and maintain the natural quality of the image.

[0097] Step M3 includes the following steps: converting all images in the crop leaf disease image dataset into JPG format, then compressing and resizing all images to 512×512 pixel images and converting them into a tensor format that can be input into the model.

[0098] Feature extraction neural network models, such as Figure 2 As shown, it includes an encoder 9, a decoder 10, and a multi-scale context aggregation module 11 (GC_ASPP module). The encoder 9 includes a conv1 layer, a conv2 layer, a conv3 layer, a conv4 layer, and a conv5 layer arranged in sequence. The decoder 10 includes an up_concat4 layer, an up_concat3 layer, an up_concat2 layer, an up_concat1 layer, and a final layer arranged in sequence. The multi-scale context aggregation module 11 includes an atrous_conv1 layer, an atrous_conv1 layer, an atrous_conv1 layer, an atrous_conv1 layer, a global_pooling layer, and a Concat layer.

[0099] The encoder performs 4 times of 2-fold downsampling, and each downsampling reduces the size of the input feature map to half of its original size. The image is input into the network, and the backbone network extracts features from the original image to obtain feature maps of four different scales (1 / 2, 1 / 4, 1 / 8, 1 / 16). Then the deepest features are sent to the multi-scale context aggregation module.

[0100] Define X (l) represents the input feature map of the lth layer, W (l) represents the convolution kernel weight matrix of the lth layer, ReLU is the activation function, MaxPool2d is the 2×2 maximum pooling operation, and the formula for the downsampling part can be expressed as:

[0101]

[0102] in, and are the weight matrices for the first and second convolution operations in layer l, respectively.

[0103] Define the feature map input to the multi-scale context aggregation module as X, and the dimension of the feature map is (N, 512, H, W), where N is the batch size, C is the number of channels, and H and W are the height and width, respectively. When the feature map is input to the multi-scale context aggregation module, C = 512.

[0104] The atrous_conv1 layer is used to generate a 32×32×512 feature map of the 32×32×512 feature map output by the conv5 layer through a 512-channel convolution with a convolution kernel size of 1×1, and then use the global context module to perform feature fusion and enhancement, and output a 32×32×512 feature map. The global context module includes a global average pooling layer and two linear transformation layers.

[0105] The feature map passes through the atrous_conv1 layer:

[0106]

[0107] in, is the weight matrix of the 1×1 convolution, is the weight of the fully connected layer in the global context block, σ is the Sigmoid function, and ⊙ represents element-wise multiplication.

[0108] The atrous_conv2 layer is used to generate a 32×32×512 feature map of 32×32×512 output by the conv5 layer through a dilated convolution with a convolution kernel size of 3×3, a dilation rate of 6, and a channel number of 512. The global context module is then used for feature fusion and enhancement to output a 32×32×512 feature map.

[0109] The feature map passes through the atrous_conv2 layer:

[0110]

[0111] in, is the weight matrix of a 3×3 convolution with dilation rate 6, are the fully connected layer weights in the global context block.

[0112] The atrous_conv3 layer is used to generate a 32×32×512 feature map of 32×32×512 output by the conv5 layer through a dilated convolution with a convolution kernel size of 3×3, a dilation rate of 12, and a channel number of 512. The global context module is then used for feature fusion and enhancement to output a 32×32×512 feature map.

[0113] The feature map passes through the atrous_conv3 layer:

[0114]

[0115] in, is the weight matrix of a 3×3 convolution with dilation rate 12, are the fully connected layer weights in the global context block.

[0116] The atrous_conv4 layer is used to generate a 32×32×512 feature map of 32×32×512 output by the conv5 layer through a dilated convolution with a convolution kernel size of 3×3, a dilation rate of 1 8, and a channel number of 512. Then, the global context module is used for feature fusion and enhancement, and a 32×32×512 feature map is output.

[0117] The feature map passes through the atrous_conv4 layer:

[0118]

[0119] in, is the weight matrix of a 3×3 convolution with dilation rate 1 8, are the fully connected layer weights in the global context block.

[0120] The global_pooling layer is used to perform global average pooling on the 32×32×512 feature map output by the conv5 layer, reducing the feature map size from 32×32×512 to 1×1×512, and then using a 1×1 convolution kernel to perform 512-channel convolution. Finally, the ReLU activation function is used to enhance the nonlinear feature expression and output a 1×1×512 global pooling feature map.

[0121] The feature map passes through the global_pooling layer:

[0122]

[0123] in, is the weight matrix of the 1×1 convolution.

[0124] The Concat layer is used to superimpose the output feature maps of the atrous_conv1 layer, atrous_conv2 layer, atrous_conv3 layer, atrous_conv4 layer, and global_pooling layer in the depth dimension to generate a comprehensive feature map of 32×32×2560. The 32×32×2560 comprehensive feature map is convolved with a convolution kernel size of 1×1 to reduce the number of channels to 512. Finally, the ReLU activation function is used to enhance the nonlinear feature expression, and finally a feature map of 32×32×512 is output to the up_concat4 layer.

[0125] The atrous_conv1 layer, atrous_conv2 layer, atrous_conv3 layer, atrous_conv4 layer, and global_pooling layer are respectively recorded as the 1st, 2nd, 3rd, 4th, and 5th branches of the multi-scale context aggregation module;

[0126] Concatenate the outputs of these five branches along the channel dimension:

[0127] X concat =Concat(X1,X2,X3,X4,X5),

[0128] At this time, X concat The dimension is (N, 2560, H, W). Finally, it passes through a 1×1 convolutional layer and a ReLU activation function:

[0129]

[0130] in, It is the weight matrix of the 1×1 convolution, which reduces the number of channels from 2560 to 512. The dimension of the final output feature map Y is (N, 512, H, W).

[0131] Assume that the expansion rate of the i-th branch of the multi-scale context aggregation module is d i , then the formula can be expressed as:

[0132]

[0133] The multi-scale context module combines the spatial pyramid pooling and the global context module. It contains four multi-scale context branches and one global pooling branch. Finally, the feature maps of the five branches are merged. In the multi-scale context branch, four different atrous convolution layers capture different scale information through convolution kernels of different sizes and different dilation rates. The global information of the feature map is obtained through global average pooling. The global features are mapped to the required output space through the fully connected layer and nonlinear activation function, and the weights are generated through the Sigmoid activation function for weighted fusion with the original features. The important information in the input features is enhanced and the unimportant information is suppressed, thereby improving the feature representation ability. The ReLU activation function is used to activate the features of the positive part and suppress the features of the negative part, thereby increasing the expressiveness and robustness of the model. Through the splicing operation, the feature information of different scales is integrated into a feature map, thereby enriching the feature representation and enhancing the expressiveness of the multi-scale and global context features.

[0134] The up_concat4 layer is used to upsample the 32×32×512 feature map output by the Concat layer through bilinear interpolation to obtain a 64×64×512 feature map, and merge the 64×64×512 feature map with the 64×64×512 feature map output by the conv4 layer to obtain a 64×64×1024 feature map. The 64×64×1024 feature map is convolved twice with a kernel size of 3×3 and a channel number of 512, and then the features are further refined through the ReLU activation function, and finally a 64×64×512 feature map is output.

[0135] The up_concat3 layer is used to upsample the 64×64×512 feature map output by the up_concat4 layer through bilinear interpolation to obtain a 128×128×512 feature map, and merge the 128×128×512 feature map with the 128×128×256 feature map output by the conv3 layer to obtain a 128×128×768 feature map. The 128×128×768 feature map is convolved twice with a kernel size of 3×3 and a channel number of 256, and then the features are further refined through the ReLU activation function, and finally a 128×128×256 feature map is output.

[0136] The up_concat2 layer is used to upsample the 128×128×256 feature map output by the up_concat3 layer through bilinear interpolation to obtain a 256×256×256 feature map, and merge the 256×256×256 feature map with the 256×256×128 feature map output by the conv2 layer to obtain a 256×256×384 feature map. The 256×256×384 feature map is convolved twice with a kernel size of 3×3 and a channel number of 128, and then the features are further refined through the ReLU activation function, and finally a 256×256×128 feature map is output.

[0137] The up_concat1 layer is used to upsample the 256×256×128 feature map output by the up_concat2 layer through bilinear interpolation to obtain a 512×512×128 feature map, and merge the 512×512×128 feature map with the 512×512×64 feature map output by the conv1 layer to obtain a 512×512×192 feature map. The 512×512×192 feature map is convolved twice with a kernel size of 3×3 and 64 channels, and then the ReLU activation function is used to further refine the features, and finally a 512×512×64 feature map is output.

[0138] The final layer is used to perform a convolution with a kernel size of 1×1 on the 512×512×64 feature map output by the up_concat1 layer, using K output channels for processing, and mapping the 64-channel features of each pixel to the predicted probabilities of K categories.

[0139] After the feature map passes through the multi-scale context aggregation module, it enters the decoder. The decoder gradually restores the spatial resolution of the feature map through upsampling and convolution operations. For each upsampling block, the main operations include upsampling, feature map concatenation, and two convolution operations. is the upsampled feature map of the lth layer, Represents the feature map of the jump connection from the corresponding layer of the encoder. The output feature map Y(l) of the lth layer can be expressed as:

[0140]

[0141] The unified formula for the entire upsampling process is:

[0142] (1) Upsampling operation:

[0143]

[0144] (2) Feature map concatenation and convolution operation:

[0145]

[0146] in, and It is the weight matrix of the l-th layer convolution operation, Upsample represents the bilinear interpolation upsampling operation, Concat represents the feature map concatenation operation in the channel dimension, and scale factor represents the multiple of the feature map size expansion.

[0147] Step M5 includes the following steps:

[0148] Constructing the cross entropy loss function L CE :

[0149]

[0150] Constructing Dice loss function L Dice :

[0151]

[0152] Constructing the hybrid loss function L total :

[0153] L total =α*L Dice +β*L CE ,

[0154] Where N is the number of pixels, D is the number of categories, and T i，c is the true label of the i-th pixel in class c, is the predicted probability of the i-th pixel in the c-th class, α and β are weight coefficients;

[0155] The mixed loss function L total As a loss function used to evaluate the gap between the true value and the predicted value.

[0156] The method for training the feature extraction neural network model of the migration parameter weights using the online data enhanced training set in step M6 is as follows:

[0157] N1: Download the model parameter weights of the Unet network pre-trained in the public image dataset Imagenet, and perform transfer learning on the feature extraction neural network model;

[0158] N2: The training set is used to train the feature extraction neural network model after transfer learning. Before each iteration (batch) training process, the following operations are performed on the training set: the image samples in the training set are uniformly randomly cropped and resized to images of 512×512 pixels; the image samples are uniformly randomly horizontally flipped with a probability of 0.5; the image samples are uniformly randomly occluded with a probability of 0.2 and an aspect ratio ranging from 2 / 100 to 1 / 3, and the image samples are uniformly converted into a tensor format that can be input into the model.

[0159] The optimization strategy for training the feature extraction neural network model uses the Adam method, with the initial learning rate set to 0.000l and the number of training epochs set to 150.

[0160] After each epoch, the test set is used to verify and evaluate the current model. After the training is completed, the accuracy, average intersection-over-union ratio and other data of each epoch are obtained. The epoch with the best effect is selected to obtain the feature extraction neural network model as the crop leaf disease recognition model.

[0161] To train the feature extraction neural network model, the network is built and trained using the Pycharm development environment under the PyTorch deep learning framework.

[0162] The image data in the training set is normalized when input into the feature extraction neural network model, and the image samples are normalized using the mean and standard deviation of the Imagenet dataset.

[0163] The trained feature extraction neural network model is saved as a .pth file, which contains the model structure and parameter weights.

[0164] This method enhances the label image to make the manually annotated label closer to the actual leaf disease area; by adding a multi-scale context aggregation module between the encoder and decoder of the Unet network, the multi-scale context feature information of the image is obtained to better capture and process the complex leaf disease features; the constructed hybrid loss function can better segment the outline of small spots; it achieves the purpose of stably identifying crop leaf disease images, improves the segmentation accuracy, and has good robustness.

Claims

1. A crop leaf disease identification system, characterized in that: The invention comprises a data processing terminal and a data acquisition device, wherein the data acquisition device comprises a mobile mechanism (1), a mechanical arm (2) and an image acquisition module, wherein the mechanical arm is arranged on the top of the mobile mechanism (1), and the image acquisition module is arranged on the top of the mechanical arm (2). The image acquisition module comprises an annular bracket (3) and a driving mechanism for driving the annular bracket (3) to rotate, wherein a plurality of cameras (4) are arranged on the annular bracket (3), and a controller and a wireless communication module are arranged on the mobile mechanism (1). The controller is electrically connected to the mobile mechanism (1), the mechanical arm (2), the camera (4), the driving mechanism and the wireless communication module respectively, and the wireless communication module is wirelessly connected to the data processing terminal via a wireless network.

2. A crop leaf disease identification system according to claim 1, characterized in that: The driving mechanism comprises a support shaft (5), the support shaft (5) is connected to the mechanical arm (2), the annular bracket (3) is arranged outside the support shaft (5) and is coaxial with the support shaft (5), a rotating sleeve (6) coaxial with the support shaft (5) is arranged outside the support shaft (5), the annular bracket (3) is connected to the rotating sleeve (6) via a connecting rod (7), and a rotating mechanism (8) for driving the rotating sleeve (6) to rotate around the support shaft (5) is arranged on the support shaft (5).

3. A crop leaf disease identification system according to claim 1, characterized in that: The spacing between adjacent cameras (3) on the annular bracket (3) is equal.

4. A method for identifying crop leaf diseases, used in a crop leaf disease identification system according to claim 1, characterized in that: The following steps are involved: S1: The mobile mechanism moves to the position of the crop leaves to be measured, the mechanical arm drives the image acquisition module to move to the position facing the crop leaves to be measured, and the annular bracket rotates to rotate each camera to the position facing the crop leaves to be measured in turn. When the camera rotates to the position facing the crop leaves to be measured, the camera collects the image of the crop leaves to be measured; S2: The controller sends the images of the crop leaves to be tested collected by all cameras to the data processing terminal; S3: The data processing terminal averages the images of the crop leaves to be tested collected by all cameras to obtain a mean image; S4: Input the mean image into the crop leaf disease recognition model, and the crop leaf disease recognition model outputs the recognition result.

5. A method for identifying crop leaf diseases according to claim 4, characterized in that: The crop leaf disease recognition model is obtained by the following method: M1: Collect crop leaf disease images, manually label the crop leaf disease images, establish a crop leaf disease image dataset, and divide the crop leaf disease image dataset into a training set and a test set; M2: Perform image enhancement on the labels in the training set; M3: Preprocessing of crop leaf disease image dataset; M4: Add the multi-scale context aggregation module between the encoder and decoder of the Unet network, replace the up-sampled transposed convolution of the Unet network with bilinear interpolation, and obtain the feature extraction neural network model; M5: Construct a loss function to evaluate the gap between the true value and the predicted value; M6: Use the training set with online data enhancement to train the feature extraction neural network model of the migration parameter weights, and use the trained feature extraction neural network model as the crop leaf disease recognition model; M7: Use the test set to test the crop leaf disease recognition model.

6. A method for identifying crop leaf diseases according to claim 5, characterized in that: In step M1, the method of collecting crop leaf disease images, manually labeling the crop leaf disease images, and establishing a crop leaf disease image dataset is as follows: Collect original images of different types of crop leaf diseases, use image annotation tools to manually label the diseased areas on the collected original images, and obtain a crop leaf disease image dataset; The labeling method is to draw the diseased area on the original image and assign semantic labels to it.

7. A method for identifying crop leaf diseases according to claim 5, characterized in that: The method for performing image enhancement on the label in step M2 is as follows: Convert the original label image into a grayscale image to obtain the grayscale image matrix I gray ; Generate grayscale image matrix I gray A noise matrix of uniform size, where each value in the noise matrix is ​​a random value drawn from a standard normal distribution; Multiply the noise matrix by the weight η to obtain the weighted noise matrix; The grayscale image matrix I gray Add it to the weighted noise matrix to get the grayscale image matrix I after adding noise noisy ; Use edge detection algorithm to analyze the grayscale image matrix I noisy Perform edge detection to obtain a mask; The mask is inverted and applied to the original image of the label to obtain the image-enhanced label.

8. A method for identifying crop leaf diseases according to claim 5, characterized in that: The feature extraction neural network model includes an encoder, a decoder, and a multi-scale context aggregation module. The encoder includes a conv1 layer, a conv2 layer, a conv3 layer, a conv4 layer, and a conv5 layer arranged in sequence. The decoder includes an up_concat4 layer, an up_concat3 layer, an up_concat2 layer, an up_concat1 layer, and a final layer arranged in sequence. The multi-scale context aggregation module includes an atrous_conv1 layer, an atrous_conv1 layer, an atrous_conv1 layer, an atrous_conv1 layer, a global_pooling layer, and a Concat layer.

9. A method for identifying crop leaf diseases according to claim 5, characterized in that: The step M5 comprises the following steps: Constructing the cross entropy loss function L CE : Constructing Dice loss function L Dice : Constructing the hybrid loss function L total : L total =α*L Dice +β*L CE , Where N is the number of pixels, D is the number of categories, and T i，c is the true label of the i-th pixel in class c, is the predicted probability of the i-th pixel in the c-th class, α and β are weight coefficients; the mixed loss function L total As a loss function used to evaluate the gap between the true value and the predicted value.

Images