Crop disease quantification method based on unmanned aerial vehicle image

Abstract

The invention discloses a crop disease quantification method based on an unmanned aerial vehicle image, and belongs to the technical field of crop disease identification, and the method comprises the steps: obtaining a leaf image through an unmanned aerial vehicle, and carrying out the preprocessing, and obtaining a leaf image data set; constructing a crop scab segmentation model based on UNet improvement, and performing training and evaluation optimization by using the leaf image data set to obtain an optimal crop scab segmentation model; and processing the unmanned aerial vehicle leaf image by using the optimal crop scab segmentation model to obtain a scab region, and calculating to obtain an area proportion of the scab region to complete quantification of the leaf disease degree. According to the method, UNet is improved through the feature extraction layer, the snake-shaped convolution layer and the cross attention layer, tiny and irregular disease spots in the unmanned aerial vehicle leaf image can be effectively segmented, the accuracy of disease spot region segmentation and the disease spot detection efficiency are improved, personal errors are reduced, and more scientific and accurate data support is provided for agricultural management.

Description

Technical Field

[0001] The present invention belongs to the technical field of crop disease identification, and in particular relates to a crop disease quantification method based on unmanned aerial vehicle images. Background Art

[0002] The degree of leaf disease of crops directly affects the health status and final yield of crops, and is one of the important agronomic traits in agricultural production management. The area of ​​lesions on leaves is an important parameter for measuring the impact of diseases. Accurately obtaining this disease information has important theoretical significance and practical application value for understanding the transmission mechanism of diseases, formulating effective prevention and control strategies, and improving crop yields. At present, the detection and quantification of leaf disease degree mainly rely on manual visual evaluation and scoring, which usually requires technicians to complete through on-site observation and manual recording. This manual method is not only inefficient, time-consuming and labor-intensive, especially in the case of large-scale planting, but also easily affected by human factors, resulting in subjective errors in the measurement results. UAVs can quickly cover large planting areas and obtain high-resolution images, while deep learning models can automatically extract and analyze disease information from images to achieve accurate quantification of disease degree. This crop disease quantification method based on drone images can not only significantly improve detection efficiency, but also reduce human errors, providing more scientific data support for precision agricultural management. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides a crop disease quantification method based on drone images, which solves the problems of low efficiency and poor accuracy in crop leaf disease recognition.

[0004] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a crop disease quantification method based on drone images, comprising the following steps:

[0005] S1: Acquire leaf images from multiple angles and under different lighting conditions using drones;

[0006] S2: preprocessing leaf images at multiple angles and under different lighting conditions to obtain a leaf image dataset;

[0007] S3: Build a crop lesion segmentation model based on UNet improvement and use leaf image dataset for training to obtain a trained crop lesion segmentation model;

[0008] S4: Based on the trained crop lesion segmentation model, the leaf image dataset is used to perform index evaluation to obtain the optimal crop lesion segmentation model;

[0009] S5: Use the optimal crop disease spot segmentation model to process the drone leaf images that need to be quantified, obtain the disease spot area, and calculate the area ratio of the disease spot area to complete the quantification of the degree of crop disease.

[0010] The beneficial effects of the present invention are as follows: by improving UNet and using it for crop disease spot image segmentation, the present invention can accurately and quickly segment the disease spot area from the drone leaf image, and quantify the degree of leaf disease spots by area calculation. By integrating the feature extraction layer, the serpentine convolution layer and the cross attention layer into the UNet jump connection, the small and irregular disease spots in the drone leaf image can be effectively segmented, the accuracy of disease spot area segmentation can be improved, and the accurate quantification of the disease degree can be achieved, which significantly improves the detection efficiency, reduces human errors, and provides more scientific data support for precision agricultural management.

[0011] Further: the preprocessing of the leaf image specifically includes:

[0012] Segment leaf images at multiple angles and under different lighting conditions into small leaf sub-images;

[0013] enhancing the image quality of the leaf sub-image;

[0014] Perform diversified processing on the leaf sub-images to obtain various leaf sub-images;

[0015] Use image annotation tools to annotate the crop area of ​​the leaf sub-image and the diseased area on the leaf;

[0016] The leaf sub-images are divided according to the set ratio to obtain a leaf image dataset, including: a training set, a validation set and a test set.

[0017] The beneficial effect of the above further scheme is that by preprocessing the leaf images, more diverse and higher quality leaf images can be obtained, thereby improving the generalization ability of the crop disease spot segmentation model and the accuracy of image segmentation.

[0018] Furthermore: the improved crop disease spot segmentation model based on UNet specifically includes: a downsampling layer, a bottleneck layer, an upsampling layer, a jump connection module, a feature fusion layer and an output layer connected in sequence, and the jump connection module includes a feature extraction layer, a snake convolution layer and a cross attention layer connected in sequence.

[0019] The beneficial effect of the above further scheme is that by adding feature extraction layers, snake convolution layers, and cross attention layers to the original UNet, tiny, irregularly shaped lesions in drone leaf images can be effectively segmented, thereby improving the lesion segmentation and recognition performance of the crop lesion segmentation model.

[0020] Further: the leaf image dataset is used for training to obtain a trained crop disease spot segmentation model, and the specific steps are as follows:

[0021] A1: Input the training set in the leaf image dataset into the downsampling layer, bottleneck layer and upsampling layer for processing to obtain the downsampling feature map and upsampling feature map respectively;

[0022] A2: Input the downsampled feature map to the feature extraction layer for micro-feature extraction to obtain a detailed feature map of the leaf;

[0023] A3: Input the leaf detailed feature map into the snake convolution layer for processing to obtain the leaf irregular feature map;

[0024] A4: Flatten the downsampled feature map and the leaf irregular feature map, and use the cross-attention layer to perform feature interaction to obtain the cross-attention spatial feature map;

[0025] A5: Reshape the cross-attention spatial feature map to obtain the cross-attention spatial feature map of the original spatial dimension, and fuse it with the upsampled feature map using the feature fusion layer to obtain the leaf fusion feature map;

[0026] A6: Input the leaf fusion feature map to the output layer, and output the identified leaf spot area;

[0027] A7: According to the leaf image data set, the leaf disease spot area and the actual leaf disease spot area are identified, the loss value of the loss function of the crop disease spot segmentation model is calculated, and it is determined whether the loss value reaches the convergence condition. If so, the trained crop disease spot segmentation model is obtained. Otherwise, the parameters of the crop disease spot segmentation model are adjusted and optimized, and then return to A1.

[0028] The beneficial effect of the above further scheme is: by training the crop lesion segmentation model, the crop lesion segmentation model's ability to segment lesions in drone leaf images can be improved, the characteristics of tiny and irregular lesions can be extracted, and the accuracy of lesion segmentation quantification can be improved.

[0029] Further: the expression of the blade detailed characteristic diagram is as follows:

[0030] F conv =Conv 3×3 (F down )

[0031] Among them, F conv is the detailed feature map of the leaf, Conv 3×3 is a 3×3 padded basic convolution, F down is the downsampled feature map.

[0032] The beneficial effect of the above further solution is that more detailed information in the feature map can be extracted through the 3×3 basic convolution with padding.

[0033] Further: the expression of the blade irregularity characteristic graph is as follows:

[0034] F snake =SnakeConv(F conv )

[0035] Among them, F snake is the irregular feature map of leaves, SnakeConv is the snake convolution, F conv Detailed feature diagram of the leaf.

[0036] The beneficial effect of the above further solution is that through snake convolution, small and complex details in the feature map can be extracted.

[0037] Further: The expression of the cross-attention spatial feature map is as follows:

[0038]

[0039] Q=f snake W Q

[0040] K=f down W K

[0041] V=f down W V

[0042]

[0043] Among them, Attention(Q, K, V) is the cross attention feature map, softmax() is the softmax function, Q is the query matrix, K is the key matrix, V is the value matrix, T is the transposed symbol, and f snake is the flattened irregular feature map of the blade, f down is the flattened downsampled feature map, W Q , W K and W V are all weight matrices, Flatten is a flattening operation, F snake is the irregular characteristic diagram of the blade, F down is the downsampled feature map, H, W, C′, C″ and d V All dimensions, d k is the key value, is the set of real numbers.

[0044] The beneficial effect of the above further scheme is that the leaf fusion feature map and the down-sampling feature map are fused through the cross attention layer, which can optimize the learning process of the jump connection and improve the lesion segmentation accuracy of the crop lesion segmentation model.

[0045] Further: the expression of the leaf fusion feature map is as follows:

[0046]

[0047] Among them, F skip is the leaf fusion feature map, F up is the upsampled feature map, is the feature fusion operation of element-by-element addition, F attention is the cross-attention feature map of the original spatial dimension, Reshape is the reshaping operation, Q is the query matrix, K is the key matrix, V is the value matrix, H, W and d V All dimensions, is the set of real numbers.

[0048] The beneficial effect of the above further scheme is that by fusing feature maps at different levels, the crop disease spot segmentation model can enhance its ability to recognize disease spots of different scales, thereby improving the robustness and adaptability of the model.

[0049] Furthermore: the loss function of the crop disease spot segmentation model includes a cross entropy loss function and a focal loss function, and its expression is as follows:

[0050]

[0051] Among them, L CE is the cross entropy loss function, c is the category, C is the number of categories, y c is the true label of category c, p c is the probability that the prediction result of the crop lesion segmentation model belongs to category c, L FL is the focal loss function, α is the weight factor, and γ is the adjustment factor.

[0052] The beneficial effect of the above further scheme is: by setting the cross entropy loss function and the focal loss function as the loss function of the crop lesion segmentation model, a better performance evaluation can be performed on image segmentation, so that the crop lesion segmentation model focuses on areas that are more difficult to segment. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Flowchart of a crop disease quantification method based on drone imagery. DETAILED DESCRIPTION

[0054] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0055] like Figure 1 As shown, a crop disease quantification method based on drone images includes the following steps:

[0056] S1: Acquire leaf images from multiple angles and under different lighting conditions using drones;

[0057] S2: preprocessing leaf images at multiple angles and under different lighting conditions to obtain a leaf image dataset;

[0058] S3: Build a crop lesion segmentation model based on UNet improvement and use leaf image dataset for training to obtain a trained crop lesion segmentation model;

[0059] S4: Based on the trained crop lesion segmentation model, the leaf image dataset is used to perform index evaluation to obtain the optimal crop lesion segmentation model;

[0060] S5: Use the optimal crop disease spot segmentation model to process the drone leaf images that need to be quantified, obtain the disease spot area, and calculate the area ratio of the disease spot area to complete the quantification of the degree of crop disease.

[0061] In S1, drones equipped with high-resolution cameras are used to fly above crop fields to collect images of large crop areas, ensuring that leaf images are obtained from multiple angles and under different lighting conditions, thereby enhancing data diversity and model robustness.

[0062] In S2, the leaf image is preprocessed, which specifically includes:

[0063] Segment leaf images at multiple angles and under different lighting conditions into small leaf sub-images;

[0064] enhancing the image quality of the leaf sub-image;

[0065] The leaf sub-images are processed in a diversified manner to obtain various leaf sub-images, wherein the diversified processing includes: hue adjustment, saturation adjustment, color disturbance of hue, random rotation, random scaling, and random flipping in horizontal and vertical directions;

[0066] Use image annotation tools to annotate the leaf sub-images, including the crop area and the diseased area on the leaf;

[0067] The leaf sub-images are divided according to the set ratio to obtain a leaf image dataset, including: training set, validation set and test set. During the division process, the distribution of each subset is kept balanced to cover different shooting conditions and disease types.

[0068] In S3, the crop disease spot segmentation model improved based on UNet specifically includes: incorporating dynamic snake convolution into the jump connection of UNet, which can capture tiny and complex details that may be ignored by traditional convolution operations; adding a feature extraction layer before each jump connection, which can capture more detailed information in the feature map, thereby improving the segmentation effect of the crop disease spot segmentation model on leaf disease spots and improving the accuracy of the final leaf disease identification; using a cross-attention layer to fuse the features extracted by the snake convolution with the features downsampled by the encoder in UNet to obtain better feature fusion results and enhance the disease spot segmentation capability of the crop disease spot segmentation model.

[0069] The structure of the crop disease spot segmentation model is a sequentially connected downsampling layer, bottleneck layer, upsampling layer, skip connection module, feature fusion layer and output layer. The skip connection module includes a sequentially connected feature extraction layer, a snake convolution layer and a cross attention layer. The training set in the leaf image dataset is used for training to obtain a trained crop disease spot segmentation model. The specific steps are as follows:

[0070] A1: Input the training set in the leaf image dataset into the downsampling layer, bottleneck layer and upsampling layer for processing to obtain the downsampling feature map and upsampling feature map respectively.

[0071] A2: The downsampled feature map is input into the feature extraction layer for micro-feature extraction to obtain a detailed feature map of the leaf. A 3×3 basic convolution with padding can be used as the feature extraction layer to extract micro-features, thereby increasing the number of channels and capturing more detailed information. The expression of the detailed feature map of the leaf is as follows:

[0072] F conv =Conv 3×3 (F down )

[0073] Among them, F conv is the detailed feature map of the leaf, Conv 3×3 is a 3×3 padded basic convolution, F down is the down-sampled feature map, and at the same time, And C′>C, H, W and C′ are all dimensions, is the set of real numbers.

[0074] A3: Input the leaf detailed feature map into the snake convolution layer for processing to obtain the leaf irregular feature map. The expression of the leaf irregular feature map is as follows:

[0075] F snake =SnakeConv(F conv )

[0076] Among them, F snake is the irregular feature map of leaves, SnakeConv is the snake convolution, F conv is the detailed characteristic diagram of the blade. At the same time, F snake ∈R H×W×C′ .

[0077] A4: Flatten the downsampled feature map and the leaf irregular feature map, and use the cross attention layer to perform feature interaction to obtain the cross attention space feature map. The specific process is as follows:

[0078] Flatten the feature map obtained after the snake convolution, and flatten the downsampled feature map corresponding to the jump connection, the expression is as follows:

[0079]

[0080] Among them, f snake is the flattened irregular feature map of the blade, f down is the flattened downsampled feature map, Flatten() is the flattening operation, F snake is the irregular characteristic diagram of the blade, F down is the downsampled feature map, is the set of real numbers, C″ is the dimension;

[0081] Through the cross attention layer, the flattened leaf irregular feature map and the flattened downsampled feature map are fused to obtain the cross attention spatial feature map, which is expressed as follows:

[0082]

[0083] Q=f snake W Q

[0084] K=f down W K

[0085] V=f down W V

[0086] Among them, Attention(Q, K, V) is the cross attention space feature map, softmax() is the softmax function, Q is the query matrix, K is the key matrix, V is the value matrix, T is the transposed symbol, and W Q , W K and W V is the weight matrix, dk is the key value, and d v is the key value.

[0087] A5: Reshape the cross-attention spatial feature map to obtain the cross-attention spatial feature map of the original spatial dimension, and use the feature fusion layer to fuse it with the upsampled feature map to obtain the leaf fusion feature map. The specific process is as follows:

[0088] The feature map output by the cross attention layer is reshaped back to the original spatial dimension to obtain the cross attention spatial feature map of the original spatial dimension, and fused with the upsampled feature map to obtain the leaf fusion feature map, which is expressed as follows

[0089]

[0090] Among them, F skip is the leaf fusion feature map, F up is the upsampled feature map, is the feature fusion operation of element-by-element addition, F attention is the cross-attention spatial feature map of the original spatial dimension, and Reshape is the reshaping operation.

[0091] A6: Input the leaf fusion feature map to the output layer, and output the identified leaf spot area;

[0092] A7: According to the leaf image data set, the leaf lesion area and the actual leaf lesion area are identified, the loss value of the loss function of the crop lesion segmentation model is calculated, and it is determined whether the loss value reaches the convergence condition. If so, a trained crop lesion segmentation model is obtained. Otherwise, the parameters of the crop lesion segmentation model are adjusted and optimized, and the return is made to A1. The convergence condition can be set to when the loss value and the mIOU of the crop lesion segmentation model no longer decrease, indicating that the crop lesion segmentation model has converged.

[0093] In one embodiment of the present invention, a combination of a cross entropy loss function and a focal loss function is used as the loss function of a crop lesion segmentation model, and an Adam optimizer is used to efficiently optimize the parameters of the crop lesion segmentation model through adaptive learning rate and momentum adjustment, thereby accelerating model convergence and improving training stability. The expressions of the cross entropy loss function and the focal loss function are as follows:

[0094]

[0095] Among them, L CE is the cross entropy loss function, c is the category, C is the number of categories, y c is the true label of category c, p c is the probability that the prediction result of the crop lesion segmentation model belongs to category c, L FLis the focal loss function, α is the weight factor, and γ is the adjustment factor.

[0096] In S4, according to the trained crop lesion segmentation model, the validation set in the leaf image dataset is used to perform indicator evaluation to obtain the optimal crop lesion segmentation model; wherein, the total loss of the crop lesion segmentation model is calculated using the validation set, and the segmentation performance of the crop lesion segmentation model is evaluated in combination with the MIoU indicator, and the parameters of the crop lesion segmentation model are optimized according to the evaluation results to select the optimal crop lesion segmentation model.

[0097] In S5, the drone leaf image that needs to be quantified is processed using the optimal crop disease spot segmentation model to obtain the disease spot area, and the area ratio of the disease spot area is calculated to complete the quantification of the degree of leaf disease; in one embodiment of the present invention, for the drone leaf image that needs to be quantified, the mask of the crop area is extracted, and the number of pixels in the crop area is calculated, and then the mask and the drone leaf image are intersected to extract the crop area, which can exclude the complex background and avoid interference with the disease spot segmentation.

[0098] The beneficial effects of the present invention are as follows: by improving UNet and using it for crop disease spot image segmentation, the present invention can accurately and quickly segment the disease spot area from the drone leaf image, and quantify the degree of leaf disease spots by area calculation. By integrating the feature extraction layer, the serpentine convolution layer and the cross attention layer into the UNet jump connection, the small and irregular disease spots in the drone leaf image can be effectively segmented, the accuracy of disease spot area segmentation can be improved, and the accurate quantification of the disease degree can be achieved, which significantly improves the detection efficiency, reduces human errors, and provides more scientific data support for precision agricultural management.

Claims

1. A method for quantifying crop diseases based on drone images, characterized in that: The following steps are involved: S1: Acquire leaf images from multiple angles and under different lighting conditions using drones; S2: preprocessing leaf images at multiple angles and under different lighting conditions to obtain a leaf image dataset; S3: Build a crop lesion segmentation model based on UNet improvement and use leaf image dataset for training to obtain a trained crop lesion segmentation model; S4: Based on the trained crop lesion segmentation model, the leaf image dataset is used to perform index evaluation to obtain the optimal crop lesion segmentation model; S5: Use the optimal crop disease spot segmentation model to process the drone leaf images that need to be quantified, obtain the disease spot area, and calculate the area ratio of the disease spot area to complete the quantification of the degree of crop disease.

2. The crop disease quantification method based on drone images according to claim 1, characterized in that: The preprocessing of the leaf image specifically includes: Segment leaf images at multiple angles and under different lighting conditions into small leaf sub-images; enhancing the image quality of the leaf sub-image; Perform diversified processing on the leaf sub-images to obtain various leaf sub-images; Use image annotation tools to annotate the crop area of ​​the leaf sub-image and the diseased area on the leaf; The leaf sub-images are divided according to the set ratio to obtain a leaf image dataset, including: a training set, a validation set and a test set.

3. The crop disease quantification method based on drone images according to claim 1, characterized in that: The improved crop disease spot segmentation model based on UNet specifically includes: a downsampling layer, a bottleneck layer, an upsampling layer, a jump connection module, a feature fusion layer and an output layer connected in sequence, and the jump connection module includes a feature extraction layer, a snake convolution layer and a cross attention layer connected in sequence.

4. The crop disease quantification method based on drone images according to claim 3 is characterized in that: The leaf image dataset is used for training to obtain a trained crop disease spot segmentation model, and the specific steps are as follows: A1: Input the training set in the leaf image dataset into the downsampling layer, bottleneck layer and upsampling layer for processing to obtain the downsampling feature map and upsampling feature map respectively; A2: Input the downsampled feature map to the feature extraction layer for micro-feature extraction to obtain a detailed feature map of the leaf; A3: Input the leaf detailed feature map into the snake convolution layer for processing to obtain the leaf irregular feature map; A4: Flatten the downsampled feature map and the leaf irregular feature map, and use the cross-attention layer to perform feature interaction to obtain the cross-attention spatial feature map; A5: Reshape the cross-attention spatial feature map to obtain the cross-attention spatial feature map of the original spatial dimension, and fuse it with the upsampled feature map using the feature fusion layer to obtain the leaf fusion feature map; A6: Input the leaf fusion feature map to the output layer, and output the identified leaf spot area; A7: According to the leaf image data set, the leaf disease spot area and the actual leaf disease spot area are identified, the loss value of the loss function of the crop disease spot segmentation model is calculated, and it is determined whether the loss value reaches the convergence condition. If so, the trained crop disease spot segmentation model is obtained. Otherwise, the parameters of the crop disease spot segmentation model are adjusted and optimized, and then return to A1.

5. The crop disease quantification method based on drone images according to claim 4 is characterized in that: The expression of the blade detailed characteristic diagram is as follows: F conv =Conv 3×3 (F down ) Among them, F conv is the detailed feature map of the leaf, Conv 3×3 is a 3×3 padded basic convolution, F down is the downsampled feature map.

6. The crop disease quantification method based on drone images according to claim 4, characterized in that: The expression of the blade irregularity characteristic graph is as follows: F snake =SnakeConv(F conv ) Among them, F snake is the irregular feature map of leaves, SnakeConv is the snake convolution, F conv Detailed feature diagram of the leaf.

7. The crop disease quantification method based on drone images according to claim 4, characterized in that: The expression of the cross-attention spatial feature map is as follows: Q=f snake W Q K=f down W K V=f down W V Among them, Attention(Q,K,V) is the cross attention feature map, softmax() is the softmax function, Q is the query matrix, K is the key matrix, V is the value matrix, T is the transposed symbol, and f snake is the flattened irregular feature map of the blade, f down is the flattened downsampled feature map, W Q , W K and W V are all weight matrices, Flatten is a flattening operation, F snake is the irregular characteristic diagram of the blade, F down is the downsampled feature map, H, W, C′, C″ and d V All dimensions, d k is the key value, is the set of real numbers.

8. The crop disease quantification method based on drone images according to claim 4, characterized in that: The expression of the leaf fusion feature map is as follows: Among them, F skip is the leaf fusion feature map, F up is the upsampled feature map, is the feature fusion operation of element-by-element addition, F attention is the cross-attention feature map of the original spatial dimension, Reshape is the reshaping operation, Q is the query matrix, K is the key matrix, V is the value matrix, H, W and d V All dimensions, is the set of real numbers.

9. The crop disease quantification method based on drone images according to claim 4, characterized in that: The loss function of the crop disease spot segmentation model includes a cross entropy loss function and a focal loss function, and its expression is as follows: Among them, L CE is the cross entropy loss function, c is the category, C is the number of categories, y c is the true label of category c, p c is the probability that the prediction result of the crop lesion segmentation model belongs to category c, L FL is the focal loss function, α is the weight factor, and γ is the adjustment factor.

Images