A rice disease identification method and system based on image processing

Abstract

The application is a rice disease identification method and system based on image processing, relating to the technical field of crop disease identification. Firstly, the OTSU segmentation method is used to segment the actual shooting paddy field leaf cluster image, and the foreground rice leaf is obtained. Then, the edge detection algorithm based on HED is used to detect the edge of the foreground leaf image, and the edge image is obtained. The edge image and the OTSU segmentation mask image are subjected to XOR operation, the edge contour information is superimposed, and the morphological operation is used to extract the complete and independent single leaf image. Finally, the single rice leaf is input into the DenseNet classification network for rice disease identification. The method can identify the rice leaf cluster disease with high precision, and the workload is less without labeling instance segmentation dataset, in view of the complex background of the rice leaf cluster image in the natural scene and the poor segmentation effect of the single rice leaf.

Description

Technical Field

[0001] This invention relates to the field of crop disease identification technology, specifically a method and system for identifying rice diseases based on image processing. Background Technology

[0002] Rice is one of my country's main food crops. Pests and diseases in rice cultivation severely restrict the healthy and sustainable development of the rice industry. Therefore, timely identification and control are crucial for ensuring a healthy rice growing environment and achieving stable and high yields. Traditional manual identification relies primarily on human observation, which is subjective and requires in-depth field observation, making it time-consuming and labor-intensive. Instrumental identification suffers from low automation, and statistical methods have geographical and temporal limitations, failing to meet the real-time needs of rice disease identification. In recent years, with the rapid development of deep learning technology, applying deep learning to crop disease identification in complex and varied field environments, including diseased leaves obscuring the diseased plant and varying target sizes, has become a research hotspot. There is an urgent need to provide an accurate and low-cost intelligent rice disease identification method to promote the development of rice disease identification technology, thereby protecting the legitimate rights and interests of rice farmers and ensuring safe rice production.

[0003] This invention addresses the issue of low disease identification accuracy in rice paddy field identification due to complex backgrounds of leaf clusters in natural scenes, and the significant workload and inaccurate segmentation of instance segmentation algorithms. To improve the accuracy and efficiency of rice disease identification in field fields, an image processing-based method and system for rice disease identification are proposed. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention aims to provide a rice disease identification method and system based on image processing. The method first obtains a single leaf with a simple background through leaf edge detection and leaf segmentation, and then performs single-leaf disease classification. Compared with traditional expert identification methods, this method has higher accuracy and efficiency in disease identification. Moreover, when faced with new crop types or new diseases of the current rice crop, it is only necessary to rebuild the leaf classification dataset, rather than creating a new instance segmentation dataset that requires a lot of annotation work.

[0005] This application achieves the above-mentioned effects through the following technical solution: a method for identifying rice diseases based on image processing, the method comprising the following steps:

[0006] S1 captures images of rice paddy leaves in a natural scene (Image1);

[0007] S2 uses the OTSU thresholding method to remove complex backgrounds from field rice leaf cluster images and outputs a foreground rice leaf cluster mask image Image2_mask;

[0008] S3 performs morphological hole filling on the foreground rice leaf cluster mask image Image2_mask obtained by OTSU segmentation in step S2 to obtain the hole-filled OTSU segmentation mask image Image2_fillhole_mask. The segmentation mask image Image2_fillhole_mask is then superimposed on the field rice leaf cluster image Image1 to output the foreground rice leaf cluster image Image2.

[0009] S4 uses an HED-based edge detection network to detect the edges of the foreground rice leaf image Image2 and outputs the foreground rice leaf edge image Image3.

[0010] S5 performs a logical XOR operation on the edge image Image3 and the OTSU segmentation mask image Image2_fillhole_mask after filling the hole, superimposes the blade outer contour position information, and outputs the XORed image Image4.

[0011] S6 uses morphological operations to extract the single leaf mask image Image5 from the XORed image Image4, and then overlays the single leaf mask image Image5 with the field rice leaf cluster image Image1 to output the single leaf image Image6.

[0012] S7 inputs the single leaf image Image6 into the trained DenseNet disease classification and recognition model to identify the type of disease on the rice leaf.

[0013] Furthermore, S2 specifically includes:

[0014] S21 divides the field rice leaf image into two parts, background and rice leaf, according to the distribution law of gray value, and uses a threshold T to separate the target object pixels from the background pixels.

[0015] Count the number of pixels with each gray level in the image of paddy field rice leaves. Let ni be the number of pixels with gray level i in the image. Then we have n0, n1, ..., n 255 The probability that a pixel in the image has a gray level of i is:

[0016] S22 determines the optimal threshold T and calculates the foreground rice leaf image at the optimal threshold T.

[0017] Furthermore, S22 specifically includes:

[0018] Iterate through all gray levels to find the optimal segmentation threshold T, at which point the inter-class variance between the foreground and background is maximized; the formula for calculating the optimal segmentation threshold is T = max[w0(T)·(u0(T)-u)]. 2 +w1(T)·(u1(T)-u)2 ], where T is the optimal segmentation threshold; w0 is the proportion of the background; u0 is the mean of the background; w1 is the proportion of the target; u1 is the mean of the target; and u is the mean of the entire image.

[0019] Furthermore, S4 specifically includes:

[0020] S41 constructs an HED edge detection network. The HED edge detection network removes the fully connected layers from the VGG16 network and retains only the convolutional pooling layers. After multiple convolutions and max pooling, the input foreground rice leaf image Image2 is added to the last layer of each convolution to obtain 5 outer outputs. Transposed convolution and deconvolution operations are used to expand it to the input size, and these 5 outer outputs are fused to output the foreground rice leaf edge image Image3.

[0021] S42 Input the foreground rice leaf image Image2, use the HED edge detection network model to detect the Image2 image, obtain the leaf edge probability map probability_image, set the pixel value of the edge probability ≥ 0.5 to 255, set the pixel value of the edge probability < 0.5 to 0, and obtain the foreground rice leaf edge image Image3.

[0022] Furthermore, S7 specifically includes:

[0023] S71 acquires images of brown spot disease, rice blast, Donggelu disease, bacterial blight, and healthy single rice leaves against a simple white background from publicly available data (Image6_open), and performs data augmentation on the acquired images to construct a rice leaf classification dataset (Image6_dataset).

[0024] S72 divides the rice leaf classification image dataset Image6_dataset into a training set and a test set, and trains the DenseNet leaf disease classification network based on the weights of the DenseNet121 pre-trained model using the training set.

[0025] After S73 reaches the set number of training steps, it ends the training and saves the corresponding model training weight results. Then, it uses a confusion matrix on the test set to determine the performance index of the disease classification model.

[0026] S74 identifies and outputs a single leaf image (Image6) based on the trained DenseNet leaf disease classification network model to determine whether the leaf is healthy and the type of disease.

[0027] In another aspect, this application also provides a rice disease identification system based on image processing, the system comprising:

[0028] Image acquisition module, the image acquisition module is used to acquire an image of rice foliage in a natural scene Image1;

[0029] The leaf cluster segmentation module is used to remove the complex background in the field rice leaf cluster image using the OTSU threshold segmentation method, and outputs the foreground rice leaf cluster mask image Image2_mask and the foreground rice leaf cluster image Image2.

[0030] An edge detection module is used to detect the edges of the foreground rice leaf image Image2 using a HED-based edge detection network, and outputs the foreground rice leaf edge image Image3.

[0031] The leaf segmentation module performs a logical XOR operation on the edge image Image3 and the OTSU segmentation mask image Image2_fillhole_mask after filling the hole, superimposes the leaf outer contour position information, and outputs the XORed image Image4; it then uses morphological operations to extract the single leaf mask image Image5 from the XORed image Image4, and superimposes the single leaf mask image Image5 with the paddy field rice leaf cluster image Image1 to output the single leaf image Image6.

[0032] The leaf selection module uses a drop-down box to select the segmented single rice leaf image. Since the rice leaves are dense in the rice clump in the field, the leaf segmentation module obtains multiple single rice leaf images, and it is necessary to select the single rice leaf image of the disease to be identified by using the drop-down box.

[0033] The disease identification module is used to identify the type of disease on a single leaf image (Image6) using a trained DenseNet disease classification and identification model.

[0034] The innovation of this invention lies in:

[0035] 1. Automatic diagnosis and identification of diseases in paddy rice crops has been achieved.

[0036] 2. An integrated framework for identifying rice leaf diseases in natural scenarios was constructed.

[0037] 3. Compared with the instance-based leaf segmentation model, the leaf segmentation model that combines edge detection does not require labeled datasets, thus reducing the workload.

[0038] 4. Since the edges of plant leaves are similar, when faced with new crop types or new diseases of rice, it is only necessary to rebuild the dataset of leaf classification, which is more efficient.

[0039] 5. Compared with traditional expert identification of rice diseases, it has higher identification accuracy and speed. Attached Figure Description

[0040] Figure 1 This is a flowchart of the method of the present invention;

[0041] Figure 2 This is a diagram of the HED edge detection network model. Detailed Implementation

[0042] Rice plant leaf edges have unique characteristics compared to the edges of ordinary objects. For example, the image gradient changes at the veins and leaf edges. In this case, the veins can interfere with the judgment of leaf edges, and the observable shape of an occluded leaf differs significantly from its actual phenotype, which affects the model's learning of leaf shape features. Since the data is collected outdoors, light intensity, airflow, and image shooting angle all affect image quality, thus impacting edge detection. Existing segmentation models cannot achieve high accuracy when segmenting dense rice leaves. Based on these considerations, this invention proposes a method to first obtain single leaves with simple backgrounds through leaf edge detection and segmentation, and then classify diseases on those single leaves. This invention expands, splits, recombines, and reuses image segmentation, edge detection, and image classification models. While the specific technical solutions share many similarities with the original technologies, there are also many different improvements, which are hereby explicitly stated. The following examples further illustrate this invention.

[0043] Example 1

[0044] This application provides a method for rice disease identification based on image processing, the flowchart of which is shown below. Figure 1 As shown, the overall implementation path includes the following steps:

[0045] S1 captures images of rice paddy leaves in a natural scene (Image1);

[0046] S2 uses the OTSU thresholding method to remove complex backgrounds from field rice leaf cluster images and outputs a foreground rice leaf cluster mask image Image2_mask;

[0047] S3 performs morphological hole filling on the foreground rice leaf cluster mask image Image2_mask obtained by OTSU segmentation in step S2 to obtain the hole-filled OTSU segmentation mask image Image2_fillhole_mask. The segmentation mask image Image2_fillhole_mask is then superimposed on the field rice leaf cluster image Image1 to output the foreground rice leaf cluster image Image2.

[0048] S4 uses an HED-based edge detection network to detect the edges of the foreground rice leaf image Image2 and outputs the foreground rice leaf edge image Image3.

[0049] S5 performs a logical XOR operation on the edge image Image3 and the OTSU segmentation mask image Image2_fillhole_mask after filling the hole, superimposes the blade outer contour position information, and outputs the XORed image Image4.

[0050] S6 uses morphological operations to extract the single leaf mask image Image5 from the XORed image Image4, and then overlays the single leaf mask image Image5 with the field rice leaf cluster image Image1 to output the single leaf image Image6.

[0051] S7 inputs the single leaf image Image6 into the trained DenseNet disease classification and recognition model to identify the type of disease on the rice leaf.

[0052] Based on the above, S8 analyzes the types of diseases affecting rice leaf clusters in the field and realizes automated rice disease diagnosis.

[0053] Firstly, step S2 of the above scheme provides a foreground rice leaf segmentation method based on OTSU thresholding to extract foreground rice leaf images in natural scenes. The OTSU algorithm divides the image into foreground and background parts according to the grayscale characteristics of the image. Variance can be regarded as a measure of uniformity of grayscale distribution. Therefore, the larger the inter-class variance between the foreground and the background, the greater the difference between the two parts of the image. When part of the foreground is misclassified as background or part of the background is misclassified as foreground, the difference between the two parts will decrease. The segmentation method with the largest inter-class variance has the lowest probability of positional misclassification and is also known as the maximum inter-class variance method.

[0054] S2 includes the following:

[0055] The S21 OTSU threshold segmentation algorithm divides the image into two parts, background and rice leaf cluster, according to the distribution pattern of gray values. It uses a threshold T to separate the target object pixels from the background pixels. The gray value with the largest variance between the background and the rice leaf cluster is the optimal segmentation threshold.

[0056] S22 Count the number of pixels of each gray level in a field rice leaf cluster image. Let ni be the number of pixels with gray level i in the image, then we have n0, n1, ..., n 255 The probability that a pixel in the image has a gray level of i is: And there are

[0057] S23 iterates through all gray levels to find the optimal segmentation threshold T, at which point the inter-class variance between the foreground and background is maximized. The formula for calculating the optimal segmentation threshold is T = max[w0(T)·(u0(T)-u)]. 2 +w1(T)·(u1(T)-u) 2 ], where T is the optimal segmentation threshold; w0 is the proportion of the background; u0 is the mean of the background; w1 is the proportion of the target; u1 is the mean of the target; and u is the mean of the entire image.

[0058] Secondly, step S4 of the above scheme provides a rice leaf cluster edge detection method based on the HED model, which extracts images of the salient outer edges of rice leaves. The network structure diagram of the HED model is shown below. Figure 2 As shown, HED is a deep learning edge extraction algorithm that uses nested edge detection. Based on deep learning, HED edge detection uses convolutional neural networks for feature extraction, selection, and classification. HED can predict edges using the entire image and can perform multi-scale, multi-layer feature learning. The HED algorithm significantly outperforms the Canny algorithm in denoising and can automatically learn the edge image features of the target leaf, producing clear and distinct leaf edge features.

[0059] S4 includes the following:

[0060] The S41 network is used to build the HED edge detection network. HED is an improvement on the VGG network and is a deep learning edge extraction algorithm with nested edge detection. HED adds 5 side output layers to the VGG network, which output from the last convolutional layer of the VGG16 network. However, the size of each outer output is not consistent. Therefore, each outer output is deconvolved in the network to expand it to the size of the input, and finally the outputs of these 5 layers are fused.

[0061] The S42 HED network model training employs the concept of transfer learning, using VGG for initialization, loading VGG network initialization parameters, and fine-tuning them. To prevent getting stuck in local minima during training, an adaptive learning rate is introduced, dynamically adjusting the learning rate.

[0062] S43 takes a foreground image of rice leaves as input, uses a general edge detection HED network model to detect the image, and obtains a leaf edge probability map. Pixels with an edge probability ≥ 0.5 are set to 255, and pixels with an edge probability < 0.5 are set to 0, thus obtaining a significant outer edge image of the leaf.

[0063] Thirdly, step S7 of the above scheme provides a method for classifying and identifying rice leaf diseases based on the DenseNet model, which identifies whether the rice is healthy and the types of diseases.

[0064] S7 includes the following:

[0065] S71 acquires images of brown spot disease, rice blast, Donggelu disease, bacterial blight, and healthy single rice leaves against a simple white background from a public dataset, and performs data augmentation on the acquired images to construct a rice leaf classification dataset;

[0066] S72 divides the rice leaf classification image dataset into a training set and a test set in a ratio of 8:2. Based on the training set, the DenseNet leaf disease classification network is trained using the weights of the DenseNet121 pre-trained model.

[0067] After S73 reaches the set number of training steps, it ends the training and saves the corresponding model training weight results. Then, it uses a confusion matrix on the test set to judge the performance index of the disease classification model. The recognition accuracy can reach more than 90%.

[0068] S74 uses a trained DenseNet model to identify whether rice leaf images are healthy and what types of diseases they may have.

[0069] Example 2

[0070] This application also provides a rice disease identification system based on image processing, the system comprising:

[0071] Image acquisition module, the image acquisition module is used to acquire an image of rice foliage in a natural scene Image1;

[0072] The leaf cluster segmentation module is used to remove the complex background in the field rice leaf cluster image using the OTSU threshold segmentation method, and outputs the foreground rice leaf cluster mask image Image2_mask and the foreground rice leaf cluster image Image2.

[0073] An edge detection module is used to detect the edges of the foreground rice leaf image Image2 using a HED-based edge detection network, and outputs the foreground rice leaf edge image Image3.

[0074] The leaf segmentation module performs a logical XOR operation on the edge image Image3 and the OTSU segmentation mask image Image2_fillhole_mask after filling the hole, superimposes the leaf outer contour position information, and outputs the XORed image Image4; it then uses morphological operations to extract the single leaf mask image Image5 from the XORed image Image4, and superimposes the single leaf mask image Image5 with the paddy field rice leaf cluster image Image1 to output the single leaf image Image6.

[0075] The leaf selection module uses a drop-down box to select the segmented single rice leaf image. Since the rice leaves are dense in the rice clump in the field, the leaf segmentation module obtains multiple single rice leaf images, and it is necessary to select the single rice leaf image of the disease to be identified by using the drop-down box.

[0076] The disease identification module is used to identify the type of disease on a single leaf image (Image6) using a trained DenseNet disease classification and identification model.

[0077] This application presents a method and system for rice disease identification based on image processing. This method combines threshold segmentation, edge detection, image processing, and a classification network to construct an integrated framework for automatic diagnosis of leaf cluster diseases in paddy fields. For actual field images of rice leaf clusters, the method first uses the OTSU threshold segmentation model to remove complex backgrounds and segment the foreground rice leaf clusters. Then, the HED edge detection model is used to detect the significant outer edges of the foreground rice leaf clusters. Next, the edge detection image and the foreground rice leaf cluster image are combined using XOR logic operations and morphological operations to obtain single-leaf images after leaf segmentation. Finally, the single rice leaf is input into a disease classification and identification network, which outputs whether the leaf is healthy and the accurate rice disease category. Compared to traditional single-stage identification and classification networks, the method proposed in this invention, which first obtains single leaves with simple backgrounds through leaf edge detection and leaf segmentation, and then performs single-leaf disease classification, has higher accuracy and efficiency in disease identification than traditional expert identification methods. Furthermore, when facing new crop types or new diseases of the current rice crop, it only requires rebuilding the leaf classification dataset, rather than creating a new instance segmentation dataset that requires extensive annotation work.

[0078] The above descriptions are merely preferred embodiments of this application, and this application is not limited to the above embodiments. It is understood that other improvements and variations that can be directly derived or conceived by those skilled in the art without departing from the spirit and concept of this application should be considered to be included within the protection scope of this application.

Claims

1. A method for identifying rice diseases based on image processing, characterized in that, The method includes the following steps: S1 captures images of rice paddy leaves in a natural scene (Image1); S2 uses the OTSU thresholding method to remove complex backgrounds from field rice leaf cluster images and outputs a foreground rice leaf cluster mask image Image2_mask; S3 performs morphological hole filling on the foreground rice leaf mask image Image2_mask obtained from OTSU segmentation in step S2 to obtain the hole-filled OTSU segmentation mask image Image2_fillhole_mask. The segmentation mask image Image2_fillhole_mask is then superimposed on the field rice leaf image Image1 to output the foreground rice leaf image Image2. S4 uses an HED-based edge detection network to detect the edges of the foreground rice leaf image Image2 and outputs the foreground rice leaf edge image Image3. S5 performs a logical XOR operation on the edge image Image3 and the OTSU segmentation mask image Image2_fillhole_mask after filling the hole, superimposes the blade outer contour position information, and outputs the XORed image Image4. S6 uses morphological operations to extract the single leaf mask image Image5 from the XORed image Image4, and then overlays the single leaf mask image Image5 with the field rice leaf cluster image Image1 to output the single leaf image Image6. S7 inputs the single leaf image Image6 into the trained DenseNet disease classification and recognition model to identify the type of disease on the rice leaf.

2. The rice disease identification method based on image processing according to claim 1, characterized in that, Specifically, S2 is: S21 divides the field rice leaf image into two parts, background and rice leaf, according to the distribution law of gray value, and uses a threshold T to separate the target object pixels from the background pixels. Count the number of pixels with each gray level in the image of paddy field rice leaves. Let ni be the number of pixels with gray level i in the image. Then we have n0, n1, ..., n 255 The probability that a pixel in the image has a gray level of i is: S22 determines the optimal threshold T and calculates the foreground rice leaf image at the optimal threshold T.

3. The rice disease identification method based on image processing according to claim 2, characterized in that, Specifically, S22 is: Iterate through all gray levels to find the optimal segmentation threshold T, at which point the inter-class variance between the foreground and background is maximized; the formula for calculating the optimal segmentation threshold is T = max[w0(T)·(u0(T)-u)]. 2 +w1(T)·(u1(T)-u) 2 ], where T is the optimal segmentation threshold; w0 is the proportion of the background; u0 is the mean of the background; w1 is the proportion of the target; u1 is the mean of the target; and u is the mean of the entire image.

4. The rice disease identification method based on image processing according to claim 1, characterized in that, Specifically, S4 is: S41 constructs an HED edge detection network. The HED edge detection network removes the fully connected layers from the VGG16 network and retains only the convolutional pooling layers. After multiple convolutions and max pooling, the input foreground rice leaf image Image2 is added to the last layer of each convolution to obtain 5 outer outputs. Transposed convolution and deconvolution operations are used to expand it to the input size, and these 5 outer outputs are fused to output the foreground rice leaf edge image Image3. S42 Input the foreground rice leaf image Image2, use the HED edge detection network model to detect the Image2 image, obtain the leaf edge probability map probability_image, set the pixel value of the edge probability ≥ 0.5 to 255, set the pixel value of the edge probability < 0.5 to 0, and obtain the foreground rice leaf edge image Image3.

5. The rice disease identification method based on image processing according to claim 1, characterized in that, Specifically, S7 is: S71 acquires images of brown spot disease, rice blast, Donggelu disease, bacterial blight, and healthy single rice leaves against a simple white background from publicly available data (Image6_open), and performs data augmentation on the acquired images to construct a rice leaf classification dataset (Image6_dataset). S72 divides the rice leaf classification image dataset Image6_dataset into a training set and a test set, and trains the DenseNet leaf disease classification network based on the weights of the DenseNet121 pre-trained model using the training set. After S73 reaches the set number of training steps, it ends the training and saves the corresponding model training weight results. Then, it uses a confusion matrix on the test set to determine the performance index of the disease classification model. S74 identifies and outputs a single leaf image (Image6) based on the trained DenseNet leaf disease classification network model to determine whether the leaf is healthy and the type of disease.

6. A rice disease identification system based on image processing, characterized in that, The system includes: Image acquisition module, the image acquisition module is used to acquire an image of rice paddy leaves in a natural scene Image1; The leaf cluster segmentation module is used to remove the complex background in the field rice leaf cluster image using the OTSU threshold segmentation method, and outputs the foreground rice leaf cluster mask image Image2_mask and the foreground rice leaf cluster image Image2. An edge detection module is used to detect the edges of the foreground rice leaf image Image2 using a HED-based edge detection network, and outputs the foreground rice leaf edge image Image3. The leaf segmentation module performs a logical XOR operation on the edge image Image3 and the OTSU segmentation mask image Image2_fillhole_mask after filling the hole, superimposes the leaf outer contour position information, and outputs the XORed image Image4; it then uses morphological operations to extract the single leaf mask image Image5 from the XORed image Image4, and superimposes the single leaf mask image Image5 with the paddy field rice leaf cluster image Image1 to output the single leaf image Image6. The leaf selection module uses a drop-down box to select the segmented single rice leaf image. Since the rice leaves are dense in the rice clump in the field, the leaf segmentation module obtains multiple single rice leaf images, and it is necessary to select the single rice leaf image of the disease to be identified by using the drop-down box. The disease identification module is used to identify the type of disease on a single leaf image (Image6) using a trained DenseNet disease classification and identification model.

Images