Crop leaf disease quantification method based on deep learning

Abstract

The invention discloses a crop leaf disease quantification method based on deep learning, belongs to the technical field of leaf disease detection, and solves the problems that time and labor are wasted and errors are large when crop leaf disease degree is manually observed and measured in the prior art. According to the method, the RGB image and the depth map are simultaneously input through the RDA U-Net model, so that interference of other leaves is eliminated, the leaf closest to the lens is conveniently found, the disease degree of the crop leaves is automatically quantified by using the deep learning technology, the efficiency and accuracy of disease degree detection are improved, subjective errors of manual observation and measurement are reduced, and the detection accuracy is improved. The method is suitable for large-scale farmland environment.

Description

Technical Field

[0001] The present invention relates to the technical field of leaf disease detection, and in particular to a crop leaf disease quantification method based on deep learning. Background Art

[0002] The degree of leaf disease of crops directly affects the growth and final yield of crops, and is a key indicator in crop health monitoring and management. The leaf lesion area is an important indicator to measure the severity of the disease. Accurately obtaining the degree of leaf disease has important theoretical significance and practical application value for understanding the disease transmission mechanism, formulating prevention and control measures, and increasing crop yields.

[0003] At present, the acquisition of the degree of crop leaf disease mainly relies on manual observation and measurement. This method is not only inefficient, especially time-consuming and laborious when dealing with a large number of leaves in a field environment, but also easily affected by human factors, resulting in subjective errors in the measurement results. Summary of the invention

[0004] In response to the above-mentioned problems in the prior art, the present invention provides a crop leaf disease quantification method based on deep learning, which solves the problem that the existing manual observation and measurement of the degree of crop leaf disease is time-consuming, labor-intensive and has large errors.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for quantifying crop leaf diseases based on deep learning is provided, comprising the steps of:

[0007] S1. Obtain training images and target images of crop leaves. Use a mobile phone or other camera equipment to capture images of crop leaves in a field environment, ensuring that leaf images under different lighting, angles, and backgrounds are covered, and set a higher shutter speed to avoid image blur, and adjust the image resolution to ensure the clarity of details.

[0008] S2, find the leaf closest to the camera in the training image, and use the Labelme annotation tool to annotate the leaf contour and disease contour of the leaf;

[0009] S3, dividing the annotated training images into training set, validation set and test set according to a certain ratio, and obtaining the corresponding depth maps of the training set, validation set and test set through the depth estimation model;

[0010] S4, inputting the original RGB image and depth map in the training set into the RDA U-Net model, and training the RDA U-Net model with the leaf area closest to the lens and the diseased spot area on the leaf as outputs. The RDA U-Net model is a training model in which a deep encoder is added to the U-Net model;

[0011] S5. Optimize the RDA U-Net model on the validation set and the test set to obtain the optimal RDA U-Net model;

[0012] S6, inputting the target image into the optimal RDA U-Net model, segmenting to obtain the leaf area closest to the lens in the target image and the diseased spot area on the leaf;

[0013] S7. The proportion of the diseased spot area in the target image is obtained by dividing the number of pixels in the diseased spot area in step S6 by the number of pixels in the leaf area.

[0014] In this scheme, considering the complex environment of the field, there are inevitably other leaves in the background, which will affect the calculation. Based on this, the depth information of the image is introduced. By simultaneously inputting the RGB image and the depth map into the RDA U-Net model, the interference of other leaves and spots in the background is eliminated, and only the leaves closest to the lens are calculated and evaluated. This scheme uses deep learning technology to automatically quantify the disease degree of crop leaves, improves the efficiency and accuracy of disease degree detection, reduces the subjective errors of manual observation and measurement, and is suitable for large-scale farmland environments.

[0015] Furthermore, the number of channels of the depth encoder is half of the number of channels of the RGB image encoder in the U-Net model. The depth map is a single-channel grayscale image, and each pixel represents the depth value of the corresponding point in the scene. The amount of information in the depth map is relatively small, so using fewer channels can reduce the complexity of the model without significantly affecting the extraction of depth information.

[0016] Furthermore, the method of segmenting the leaf area closest to the lens and the diseased spot area on the leaf by the RDA U-Net model includes:

[0017] S4.1. Extract features from the input depth map and original RGB image through the depth encoder and RGB image encoder respectively to generate a feature map F of the depth map. d And the feature map F of the original RGB image rgb ; Extracting features from RGB images and depth maps respectively can make full use of two different types of information.

[0018] S4.2, after the RGB image encoder and the depth encoder complete the last downsampling of the original RGB image and the depth map respectively, F rgb and F d Splice in the channel dimension to get the fused feature map F f , F f =concat(F rgb , F d); By concatenating the two feature maps, the feature information of the RGB image and the depth map can be effectively combined to form a comprehensive feature representation.

[0019] S4.3. Generate attention weight W through channel attention mechanism a , for the fusion feature map F f The features of each channel in are weighted: F′ f =F f ×W a ; Among them, W a It is an attention weight vector generated by global average pooling, fully connected layer and sigmoid function. The channel attention mechanism can help the model automatically learn which channel features are more important and thus weight these features. This helps to highlight key features, suppress noise and irrelevant information, and improve the robustness of the model and segmentation accuracy.

[0020] S4.4. Use 1x1 convolution operation to reduce the fusion feature map F′ f The number of channels is used to generate the feature map F after dimensionality reduction r : F r =Conv1x1(F′ f ); 1x1 convolution operation can reduce the number of channels of feature maps, reduce the computational complexity of the model, and improve processing speed. At the same time, through dimensionality reduction operation, redundant information can be removed, the most important features can be retained, and the efficiency and performance of the model can be improved.

[0021] S4.5. Reduce the dimension of the feature map F r The image is input into the decoder, and up-sampled and reconstructed through the decoder, and finally segmented to obtain the leaf area closest to the lens and the diseased area on the leaf.

[0022] Furthermore, the training process of the RDA U-Net model includes data augmentation of the original RGB images in the training set, including random cropping, random flipping in the horizontal and vertical directions, random angle rotation, random scaling, and random changes in brightness, contrast, saturation, and hue. Data augmentation increases the diversity of training data, improves the generalization ability of the model and its adaptability to different environmental conditions, and helps improve the accuracy and stability of the model.

[0023] Furthermore, the optimizer of the RDA U-Net model is the Adam optimizer.

[0024] Furthermore, the loss function L of the RDA U-Net model is:

[0025] L=λ1·CrossEntropyLoss+λ2·FocalLoss(p t )

[0026] FocalLoss(p t )=-α(1-p t )γlog(p t )

[0027] Among them, λ1 and λ2 are weight coefficients, CrossEntropyLoss is the cross entropy loss, FocalLoss(p t ) is the focal loss, pt is the prediction probability, α is the balance factor, and γ is the focusing parameter. By combining cross entropy loss and focal loss, the model pays more attention to samples that are difficult to classify, and improves the recognition accuracy of the model for the lesion area, especially when the lesion area is small.

[0028] Furthermore, the training process of the RDA U-Net model adopts the cosine annealing learning rate adjustment strategy to adjust the learning rate; the cosine annealing learning rate adjustment strategy can quickly reduce the learning rate in the early stage of training, and then gradually reduce the fluctuation of the learning rate, which helps the model to converge more stably to the global optimal or near-global optimal solution.

[0029] The cosine annealing learning rate adjustment strategy is:

[0030]

[0031] Among them, η t is the current training cycle T cur The learning rate, η min and η max are the minimum and maximum learning rates, T cur is the current training cycle, T max is the maximum training cycle.

[0032] Furthermore, the optimization method of the RDA U-Net model is as follows: input the validation set into the trained RDA U-Net model to obtain the total loss and mIOU index of the RDA U-Net model, and according to the total loss and mIOU index, continuously adjust the hyperparameters of the RDA U-Net model to obtain multiple RDA U-Net models, and select the optimal RDA U-Net model through the test set.

[0033] Furthermore, the ratio of the training set, validation set, and test set is 8:1:1. This ratio ensures that the training set has enough data to train the model, and that the validation set and test set can effectively evaluate the performance of the model, thus balancing the training and evaluation requirements of the model.

[0034] The present invention discloses a crop leaf disease quantification method based on deep learning, which has the following beneficial effects:

[0035] The present invention simultaneously inputs RGB images and depth maps through the RDA U-Net model, thereby eliminating interference from other leaves and facilitating finding the leaves closest to the lens. It also uses deep learning technology to automatically quantify the disease extent of crop leaves, thereby improving the efficiency and accuracy of disease extent detection, reducing the subjective errors of manual observation and measurement, and is suitable for large-scale farmland environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Schematic diagram of a deep learning-based crop leaf disease quantification method. DETAILED DESCRIPTION

[0037] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0038] refer to Figure 1 This embodiment provides a crop leaf disease quantification method based on deep learning, comprising the steps of:

[0039] S1. Obtain training images and target images of crop leaves. Use a mobile phone or other camera equipment to capture images of crop leaves in a field environment, ensuring that leaf images under different lighting, angles, and backgrounds are covered, and set a higher shutter speed to avoid image blur, and adjust the image resolution to ensure the clarity of details.

[0040] S2. Find the leaf closest to the camera in the training image, and use the Labelme annotation tool to annotate the leaf contour and disease contour of the leaf.

[0041] S3. The labeled training images are divided into training set, validation set and test set according to a certain ratio, and the corresponding depth maps of the training set, validation set and test set are obtained through the depth estimation model; in this embodiment, the ratio of the training set, validation set and test set is 8:1:1. Such a division ratio ensures that the training set has enough data to train the model, and ensures that the validation set and test set can effectively evaluate the performance of the model, balancing the training and evaluation requirements of the model.

[0042] S4. Input the original RGB image and depth map in the training set into the RDA U-Net model, and train the RDA U-Net model with the leaf area closest to the lens and the diseased area on the leaf as output. The RDA U-Net model is a training model in which a deep encoder is added to the U-Net model; the number of channels of the deep encoder is half of the number of channels of the RGB image encoder in the U-Net model.

[0043] The RDA U-Net model inputs the RGB image and the depth map at the same time, thereby eliminating the interference of other leaves and facilitating the finding of the leaf closest to the lens.

[0044] Specifically, the method of segmenting the leaf area closest to the camera and the diseased spot area on the leaf by the RDA U-Net model includes:

[0045] S4.1. Extract features from the input depth map and original RGB image through the depth encoder and RGB image encoder respectively to generate a feature map F of the depth map. d And the feature map F of the original RGB image rgb ; Extracting features from RGB images and depth maps respectively can make full use of two different types of information.

[0046] S4.2, after the RGB image encoder and the depth encoder complete the last downsampling of the original RGB image and the depth map respectively, F rgb and F d Splice in the channel dimension to get the fused feature map F f , F f =concat(F rgb , F d ); By concatenating the two feature maps, the feature information of the RGB image and the depth map can be effectively combined to form a comprehensive feature representation.

[0047] S4.3. Generate attention weight W through channel attention mechanism a , for the fusion feature map F f The features of each channel in are weighted: F′ f =F f ×W a ; Among them, W a It is an attention weight vector generated by global average pooling, fully connected layer and sigmoid function. The channel attention mechanism can help the model automatically learn which channel features are more important and thus weight these features. This helps to highlight key features, suppress noise and irrelevant information, and improve the robustness of the model and segmentation accuracy.

[0048] S4.4. Use 1x1 convolution operation to reduce the fusion feature map F′ f The number of channels is used to generate the feature map F after dimensionality reduction r : F r =Conv1x1(F′ f ); 1x1 convolution operation can reduce the number of channels of feature maps, reduce the computational complexity of the model, and improve processing speed. At the same time, through dimensionality reduction operation, redundant information can be removed, the most important features can be retained, and the efficiency and performance of the model can be improved.

[0049] S4.5. Reduce the dimension of the feature map F r The image is input into the decoder, and up-sampled and reconstructed through the decoder, and finally segmented to obtain the leaf area closest to the lens and the diseased area on the leaf.

[0050] S5. Train the RDA U-Net model and optimize the RDA U-Net model through the validation set and the test set to obtain the optimal RDAU-Net model.

[0051] S6. Input the target image into the optimal RDA U-Net model to segment the leaf area closest to the lens in the target image and the diseased spot area on the leaf.

[0052] Specifically, the optimization method of the RDA U-Net model is as follows: input the validation set into the trained RDA U-Net model to obtain the total loss and mIOU index of the RDAU-Net model; according to the total loss and mIOU index, the hyperparameters of the RDAU-Net model are continuously adjusted to obtain multiple RDAU-Net models; and the optimal RDA U-Net model is selected through the test set.

[0053] S7. The proportion of the diseased spot area in the target image is obtained by dividing the number of pixels in the diseased spot area in step S6 by the number of pixels in the leaf area.

[0054] As a further solution of this embodiment, the training process of the RDA U-Net model includes data enhancement of the original RGB images in the training set, and the data enhancement includes random cropping, random flipping in the horizontal and vertical directions, rotation at random angles, random scaling, and random changes in brightness, contrast, saturation, and hue. Data enhancement increases the diversity of training data, improves the generalization ability of the model and its adaptability to different environmental conditions, and helps to improve the accuracy and stability of the model.

[0055] The optimizer of the RDAU-Net model is the Adam optimizer, and the loss function L of the RDAU-Net model is:

[0056] L=λ1·CrossEntropyLoss+λ2·FocalLoss(p t )

[0057] FocalLoss(p t )=-α(1-p t )γlog(p t )

[0058] Among them, λ1 and λ2 are weight coefficients, CrossEntropyLoss is the cross entropy loss, FocalLoss(p t ) is the focal loss, pt is the prediction probability, α is the balance factor, and γ is the focusing parameter. By combining cross entropy loss and focal loss, the model pays more attention to samples that are difficult to classify, and improves the recognition accuracy of the model for the lesion area, especially when the lesion area is small.

[0059] The training process of the RDA U-Net model uses the cosine annealing learning rate adjustment strategy to adjust the learning rate; the cosine annealing learning rate adjustment strategy can quickly reduce the learning rate in the early stage of training, and then gradually reduce the fluctuation of the learning rate, which helps the model converge more stably to the global optimal or near-global optimal solution.

[0060] The cosine annealing learning rate adjustment strategy is:

[0061]

[0062] Among them, η t is the current training cycle T cur The learning rate, η min and η max are the minimum and maximum learning rates, T cur is the current training cycle, T max is the maximum training cycle.

[0063] In summary, the beneficial effects of this solution are:

[0064] The RGB image and depth map are simultaneously input through the RDA U-Net model to eliminate interference from other leaves, making it easier to find the leaves closest to the lens. Deep learning technology is used to automatically quantify the disease extent of crop leaves, which improves the efficiency and accuracy of disease extent detection and reduces the subjective errors of manual observation and measurement. It is suitable for large-scale farmland environments.

[0065] Although the specific implementation of the invention is described in detail in conjunction with the drawings, it should not be understood as limiting the scope of protection of this patent. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still fall within the scope of protection of this patent.

Claims

1. A crop leaf disease quantification method based on deep learning, characterized in that: Includes steps: S1, obtaining training images and target images of crop leaves; S2, find the leaf closest to the camera in the training image, and use the Labelme annotation tool to annotate the leaf contour and disease contour of the leaf; S3, dividing the annotated training images into training set, validation set and test set according to a certain ratio, and obtaining the corresponding depth maps of the training set, validation set and test set through the depth estimation model; S4, inputting the original RGB image and depth map in the training set into the RDA U-Net model, and training the RDA U-Net model with the leaf area closest to the lens and the diseased spot area on the leaf as outputs. The RDA U-Net model is a training model in which a deep encoder is added to the U-Net model; S5. Optimize the RDA U-Net model on the validation set and the test set to obtain the optimal RDA U-Net model; S6, inputting the target image into the optimal RDA U-Net model, segmenting to obtain the leaf area closest to the lens in the target image and the diseased spot area on the leaf; S7. The proportion of the diseased spot area in the target image is obtained by dividing the number of pixels in the diseased spot area in step S6 by the number of pixels in the leaf area.

2. The leaf disease degree quantification method according to claim 1, characterized in that: The number of channels of the depth encoder is half of that of the RGB image encoder in the U-Net model.

3. The leaf disease degree quantification method according to claim 1, characterized in that: The method of segmenting the leaf area closest to the camera and the diseased area on the leaf using the RDA U-Net model includes: S4.

1. Extract features from the input depth map and original RGB image through the depth encoder and RGB image encoder respectively to generate a feature map F of the depth map. d And the feature map F of the original RGB image rgb ; S4.2, after the RGB image encoder and the depth encoder complete the last downsampling of the original RGB image and the depth map respectively, F rgb and F d Splice in the channel dimension to get the fused feature map F f , F f =concat(F rgb ,F d ); S4.

3. Generate attention weight W through channel attention mechanism a , for the fusion feature map F f The features of each channel in are weighted: F f ′ =F f ×W a ; Among them, W a is the attention weight vector generated by global average pooling, fully connected layer and Sigmoid function. S4.

4. Use 1x1 convolution operation to reduce the fusion feature map F f ′ The number of channels is used to generate the feature map F after dimensionality reduction r :F r =Conv1x1(F f ′ ); S4.

5. Reduce the dimension of the feature map F r The image is input into the decoder, and up-sampled and reconstructed through the decoder, and finally segmented to obtain the leaf area closest to the lens and the diseased area on the leaf.

4. The method for quantifying the degree of leaf disease according to claim 1, characterized in that: The training process of the RDA U-Net model includes data augmentation of the original RGB images in the training set, which includes random cropping, random flipping in the horizontal and vertical directions, random rotation at random angles, random scaling, and random changes in brightness, contrast, saturation, and hue.

5. The method for quantifying the degree of leaf disease according to claim 1, characterized in that: The optimizer of the RDA U-Net model is the Adam optimizer.

6. The method for quantifying the degree of crop leaf disease based on deep learning according to claim 1, characterized in that: The loss function L of the RDA U-Net model is: L=λ1·CrossEntropyLoss+λ2·FocalLoss(p t ) FocalLoss(p t )=-α(1-p t ) γ log(p t ) Among them, λ1 and λ2 are weight coefficients, CrossEntropyLoss is the cross entropy loss, FocalLoss(p t ) is the focal loss, p t is the prediction probability, α is the balance factor, and γ is the focusing parameter.

7. The leaf disease degree quantification method according to claim 1, characterized in that: The training process of the RDA U-Net model uses the cosine annealing learning rate adjustment strategy to adjust the learning rate; The cosine annealing learning rate adjustment strategy is: Among them, η t is the current training cycle T cur The learning rate, η min and η max are the minimum and maximum learning rates, T cur is the current training cycle, T max is the maximum training cycle.

8. The method for quantifying the degree of leaf disease according to claim 1, characterized in that: The optimization method of the RDA U-Net model is as follows: input the validation set into the trained RDA U-Net model to obtain the total loss and mIOU indicators of the RDA U-Net model, and continuously adjust the hyperparameters of the RDA U-Net model according to the total loss and mIOU indicators to obtain multiple RDA U-Net models, and select the optimal RDA U-Net model through the test set.

9. The method for quantifying the degree of leaf disease according to claim 1, characterized in that: The ratio of training set, validation set and test set is 8:1:1.

Images