A plant leaf disease detection and identification method

Abstract

The application discloses a kind of plant leaf disease identification detection method in the technical field of crop disease identification, which includes image segmentation, data enhancement and image recognition.The specific steps are as follows:1) collect original plant leaves for labeling, establish original dataset, train to obtain Unet semantic segmentation model, make background removal dataset, and train to obtain DCGAN data enhancement model, make data enhancement dataset, form a graph classification model, and analyze the plant leaf disease classification through the model.The advantages of the present application compared with the prior art are that the MobileNetV3-Large image recognition network trained by the database obtained by integrating image segmentation and data enhancement can greatly improve the recognition accuracy of the network.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of crop disease identification, and particularly relates to a plant leaf disease detection and identification method. BACKGROUND

[0002] With the development of deep learning, plant disease identification using computer vision is the only way for intelligent agriculture. The main problems in current plant leaf disease identification research are as follows: first, the research object often selects plant leaves in a simple background environment, or the background is laboratory or the main body of the leaves is cut by artificial, but the problem of plant leaf disease identification in a complex environment has not been solved; second, it depends on a large number of sample images, but due to the suddenness and seasonality of plant leaf diseases, it is difficult to collect a small number of disease images, and when the total recognition accuracy is affected by insufficient disease images, the disease is often excluded from the research range by default, resulting in the problems of plant leaf disease multi-classification and data scarcity which are difficult to solve. SUMMARY

[0003] To achieve the above purpose, the application provides a plant leaf disease detection and identification method, and the scheme is as follows:

[0004] S1: data acquisition, collecting original images of plant leaves with different diseases through an image acquisition device;

[0005] S2: labeling disease leaf images and making VOC format semantic segmentation data set;

[0006] S3: training Unet model using the semantic segmentation data set obtained in S2 and saving;

[0007] S4: using the Unet semantic segmentation model trained in S3 to segment the original image of the plant leaf to obtain a segmented image of the disease leaf and the background, and making a background removed data set;

[0008] S5: training DCGAN model using the background removed data set, and saving the best training result as the final data enhancement model;

[0009] S6: using the final DCGAN data enhancement model to generate virtual images of various plant leaf diseases and saving, and making a disease leaf image classification data set;

[0010] S7: training MobileNetV3-Large image classification model using the disease leaf classification data set constructed in S6;

[0011] S8: using the S3 trained Unet semantic segmentation model and the S7 trained MobileNetV3-Large image classification model to identify the original image of the plant disease leaf not involved in the training.

[0012] Compared with the prior art, the technical scheme of the application has the following advantages:

[0013] 1. Compared with the method of taking plant leaves with a simple background in the past, the method of extracting the plant body using the semantic segmentation network proposed in the application is lower in cost and simpler in operation.

[0014] 2. The method of using DCGAN to enhance the data of diseased plant leaves proposed in the application solves the problem of data scarcity, and does not need to collect data back and forth multiple times and in large quantities as in the past, saving a lot of labor cost, time cost and economic cost.

[0015] 3. The database obtained by integrating the data after data enhancement is used to train the MobileNetV3-Large image recognition network, which can greatly improve the accuracy of recognition.

[0016] Further, the S1 collecting original images of plant leaves with different diseases specifically comprises: keeping the shooting direction perpendicular to the leaf to be shot as much as possible, the lens and the leaf have different spacings, and the main purpose is to shoot clear diseased plant leaves, the equipment is stabilized before shooting, and finally saved as JPG format.

[0017] Further, the S2 labeling the diseased leaf image and making a VOC format semantic segmentation dataset specifically comprises: using Labelme labeling software to label the diseased plant leaf part in the original image, which is divided into two types of targets, the diseased plant leaf part is the first type, and the background is the 0th type; the generated labeling file is arranged into the format of VOC dataset, the labeling information is arranged into json format, and the original dataset is established.

[0018] Further, the S3 training the Unet model using the semantic segmentation dataset and saving specifically comprises: inputting the original dataset into the semantic segmentation model Unet network, setting appropriate BatchSize, epoch and LearnRate parameters, training the Unet model, and saving the model weights of different iteration times; respectively using the saved models after training in the test set, selecting the best segmentation effect model as the final Unet segmentation model.

[0019] Furthermore, S5, which uses a background-removed dataset to train a DCGAN model and saves the best training result as the final data augmentation model, specifically includes: inputting the created diseased leaf segmentation dataset into the DCGAN network, setting appropriate BatchSize, epoch, and LearnRate parameters, and then training the model; saving the model weights for different iterations; testing the trained and saved models respectively, and selecting the model with the best generation effect as the final DCGAN data augmentation model.

[0020] Furthermore, step S6 uses the final DCGAN data augmentation model to generate and save virtual images of various plant leaf diseases, creating a disease leaf image classification dataset. Specifically, this includes: using the selected DCGAN data augmentation model to generate virtual images of various leaf diseases, and saving the better virtual images; the better virtual images, along with the background-removed dataset, are divided into corresponding folders according to disease category to create a data augmentation dataset; the dataset is divided into a training set and a validation set at a 2:1 ratio, and the original dataset that did not participate in image segmentation and data augmentation is used as the test set for subsequent plant leaf disease image classification.

[0021] Furthermore, step S7, which uses the constructed diseased leaf classification dataset to train the MobileNetV3-Large image classification model, specifically includes: importing the divided training and validation sets into the MobileNetV3-Large image classification model, setting appropriate LearnRate, BatchSize, and epoch parameters, training multiple times, and saving the training results; testing the trained and saved model on the test set, and selecting the model with the highest classification accuracy as the final image classification model.

[0022] Furthermore, step S8 uses the trained Unet semantic segmentation model and MobileNetV3-Large image classification model to identify and apply the original images of plant diseased leaves that were not trained. Specifically, this includes: collecting the plant leaves to be identified; inputting the original images of the plant diseased leaves that were not trained into the final Unet segmentation model to obtain segmented images; inputting the segmented images of the diseased leaves into the final MobileNetV3-Large image classification model to obtain the classification and identification results for each diseased image, and counting the number of results for each class; comparing the results with those obtained manually to evaluate the identification method; if the accuracy of disease identification does not meet expectations, adjusting the dataset and retraining the corresponding model; if it meets expectations, then using it for actual detection. Attached Figure Description

[0023] Figure 1This is a flowchart of a method for detecting and identifying plant leaf diseases.

[0024] Figure 2 These are example images of leaves from seven different plant species, showing leaves with different diseases and healthy plant leaves.

[0025] Figure 3 This is an example diagram of the Unet semantic segmentation network structure.

[0026] Figure 4 This is an example image of the segmentation results from the Unet semantic segmentation network.

[0027] Figure 5 This is an example image of plant leaf diseases generated by DCGAN.

[0028] Figure 6 This is an example diagram of the MobileNetV3-Large network architecture.

[0029] Figure 7 This is an example image of the training MobileNetV3-Large classification confusion matrix results. Detailed Implementation

[0030] The present invention will now be described in further detail with reference to the accompanying drawings, taking sugarcane leaf diseases as an example.

[0031] Combined with appendix Figures 1-7 A method for detecting and identifying plant leaf diseases, such as Figure 1 As shown, the method includes the following steps:

[0032] S1: During image acquisition, three mobile phones were used: OnePlus 8T, iPhone 6, and Vivo S9. The default camera settings were used, and the shooting direction was kept as perpendicular as possible to the leaf being photographed. The distance between the lens and the leaf varied to ensure clear images of the plant leaves. Since the classification criterion is the difference in leaf lesions, it was necessary to check for any non-leaf-related obstructions, such as dirt, flying insects, or light spots caused by uneven lighting. The acquired images had varying resolutions and were finally saved as a dataset in JPG format. (e.g.) Figure 2 (as shown);

[0033] S2: Using LabelMe 3.16.7 software, the main plant leaf region in each sample image was manually labeled to generate a mask image. The labeled files (Labels) containing category and region information were used as the standard for training, validating, and testing the semantic segmentation model. There were two target classes: the diseased plant leaves were classified as Class 1, and the background was classified as Class 0 by default. The generated labeled files were formatted into the VOC dataset format, and the labeled information was formatted into JSON. The original dataset was constructed with a training set to validation set ratio of 9:1.

[0034] S3: Input the original dataset into the Unet semantic segmentation model, setting the input image size to 512×512×3; set the number of freeze training iterations (Freeze_Epoch) to 50; set the batch size (Freeze_batch_size) to 8; and set the initial learning rate (Freeze_learning rate) to 1×10^6. -4 The number of iterations for unfreezing training (UnFreeze_Epoch) is set to 50; the batch size for unfreezing training (Unfreeze_batch_size) is set to 4; and the initial learning rate for the unfreezing phase (Unfreeze_learning rate) is set to 1×10⁻⁶. -6 Save the model weights for different iterations; test each trained and saved model on the test set, and select the model with the best segmentation performance as the semantic segmentation model (e.g., ...). Figure 3 (as shown);

[0035] S4: Input the original image set into the trained semantic segmentation model to obtain segmented images of the background and plant leaf diseases (e.g., Figure 4 (As shown), create a dataset with background removed;

[0036] S5: Train the DCGAN network on the background-removed dataset using the Adam optimizer. The generator's learning rate is 0.0001, the discriminator's learning rate is 0.00005, the epoch is set to 50,000, the batch size is 64, and the input image size is set to 64×64×3. Train the model; save the model weights for different iterations; test the trained and saved models, and select the model with the best generation performance as the final DCGAN data augmentation model.

[0037] S6: Using the final DCGAN data augmentation model, generate virtual images of plant leaf diseases, and save the better virtual images. These better virtual images, along with the background-removed dataset, are categorized into corresponding folders according to disease type to create a data augmentation dataset (e.g., ...). Figure 5(As shown); the dataset is divided into a training set and a validation set in a 2:1 ratio. The original dataset, which was not used for image segmentation and data augmentation, is used as the test set for subsequent image classification of plant leaf diseases.

[0038] S7: Divide the data augmentation dataset into a training set and a validation set in a 2:1 ratio, and use the original dataset that was not involved in image segmentation and data augmentation as the test set; import the training set and the validation set data into the MobileNetV3-Large image classification model (e.g., ...). Figure 6 As shown), the initial learning rate was set to 0.0001, the batch size to 16, and the epochs to 50. The model was trained multiple times, and the training results were saved. Each trained and saved model was then tested on the test set, and the model with the highest classification accuracy (e.g., ...) was selected. Figure 7 (As shown) serves as the final image classification model;

[0039] S8: Collect the plant leaves to be identified. Input the original images of the diseased leaves that were not used in the training into the final Unet segmentation model to obtain segmented images. Input the segmented images of the diseased leaves into the final MobileNetV3-Large image classification model to obtain the classification and recognition results for each diseased image, and count the number of results for each class. Compare with the results of manual recognition to evaluate the recognition method. If the accuracy of disease recognition does not meet expectations, adjust the dataset and retrain the corresponding model. If the mIOU value after segmentation is greater than 80% and the diseased region is completely segmented, it means that the accuracy of the segmentation model is high and can be used for actual detection. If the mIOU value is less than 80%, it is considered that the error is large and the segmentation model needs to be retrained. If, after classification, the classification accuracy of each type of diseased leaf calculated by the confusion matrix is ​​higher than 85%, the model is considered to have good classification performance and can be used for actual detection. If the classification accuracy of most diseased leaves is lower than 85%, the classification model is considered invalid and the parameters need to be readjusted to train the model. If only a small number of diseased leaf categories have a classification accuracy lower than 85%, the model is considered to have low sensitivity to that type of disease, and the image datasets of diseased leaves for these categories need to be enhanced and cleaned accordingly (such as performing transformations such as flipping, scaling, and translating diseased leaf images, and removing some partially damaged images of leaves), and then the classification model should be retrained using the modified dataset.

[0040] If it meets expectations, it will be used for actual testing.

[0041] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A method for detecting and identifying plant leaf diseases, characterized in that: Includes the following steps: S1: Data acquisition, acquiring original images of plant leaves suffering from different diseases using image acquisition equipment; S2: Annotate images of diseased leaves and create a semantic segmentation dataset in VOC format; S3: Train the Unet model using the semantic segmentation dataset obtained in S2 and save it; S4: Use the Unet semantic segmentation model trained in S3 to segment the original images of plant leaves, obtain segmented images of diseased leaves and background, and create a background-removed dataset. S5: Train the DCGAN model using the background-removed dataset, and save the best training results as the final data augmentation model; S6: Using the final DCGAN data augmentation model, generate and save virtual images of plant leaf diseases to create a diseased leaf image classification dataset. S7: Train the MobileNetV3-Large image classification model using the diseased leaf classification dataset constructed in S6; S8: Use the Unet semantic segmentation model trained in S3 and the MobileNetV3-Large image classification model trained in S7 to identify and apply the original images of plant diseased leaves that were not trained.

2. The method for detecting and identifying plant leaf diseases according to claim 1, characterized in that: The process of collecting original images of plant leaves with different diseases according to S1 includes: keeping the shooting direction as perpendicular as possible to the leaf to be photographed, with varying distances between the lens and the leaf, aiming to capture clear images of the diseased leaves, stabilizing the equipment before taking the picture, and finally saving it in JPG format.

3. The method for detecting and identifying plant leaf diseases according to claim 1, characterized in that: The process of annotating diseased leaf images according to S2 and creating a semantic segmentation dataset in VOC format specifically includes: using Labelme annotation software to annotate the diseased plant leaf parts in the original images, which are divided into two target categories: the diseased plant leaf parts are category 1, and the background is category 0; organizing the generated annotation files into the VOC dataset format, organizing the annotation information into JSON format, and establishing the original dataset.

4. The method for detecting and identifying plant leaf diseases according to claim 3, characterized in that: The process of training and saving the Unet model using the semantic segmentation dataset as described in S3 includes: inputting the original dataset into the Unet semantic segmentation model network, setting appropriate BatchSize, epoch, and LearnRate parameters, training the Unet model, and saving the model weights for different iterations; testing the trained and saved models on the test set, and selecting the model with the best segmentation performance as the final Unet segmentation model.

5. The method for detecting and identifying plant leaf diseases according to claim 1, characterized in that: According to S5, training the DCGAN model using the background-removed dataset and saving the best training result as the final data augmentation model specifically includes: inputting the created diseased leaf segmentation dataset into the DCGAN network, setting the BatchSize, epoch, and LearnRate parameters, and then training the model; saving the model weights for different iterations; testing the trained and saved models respectively, and selecting the model with the best generation effect as the final DCGAN data augmentation model.

6. The method for detecting and identifying plant leaf diseases according to claim 1, characterized in that: According to S6, the final DCGAN data augmentation model is used to generate and save virtual images of various plant leaf diseases. The specific steps for creating a disease leaf image classification dataset include: using the selected DCGAN data augmentation model to generate virtual images of various plant leaf diseases, and saving the better virtual images; the better virtual images, along with the background-removed dataset, are divided into corresponding folders according to disease categories to create a data augmentation dataset; the dataset is divided into a training set and a validation set at a ratio of 2:1, and the original dataset that did not participate in image segmentation and data augmentation is used as the test set for subsequent image classification of plant leaf diseases.

7. The method for detecting and identifying plant leaf diseases according to claim 6, characterized in that: The training of the MobileNetV3-Large image classification model using the constructed diseased leaf classification dataset according to S7 specifically includes: importing the divided training and validation sets into the MobileNetV3-Large image classification model, setting appropriate LearnRate, BatchSize, and epoch parameters, training multiple times and saving the training results; testing the trained and saved model on the test set, and selecting the model with the highest classification accuracy as the final image classification model.

8. The method for detecting and identifying plant leaf diseases according to claim 1, characterized in that: According to S8, the trained Unet semantic segmentation model and MobileNetV3-Large image classification model are used to identify the original images of plant diseased leaves that were not trained. Specifically, this includes: collecting the plant leaves to be identified; inputting the original images of the plant diseased leaves that were not trained into the final Unet segmentation model to obtain segmented images; inputting the segmented images of the diseased leaves into the final MobileNetV3-Large image classification model to obtain the classification and recognition results for each diseased image, and counting the number of results for each class; comparing the results with those obtained manually to evaluate the recognition method. If the accuracy of disease identification does not meet expectations, the dataset is adjusted and the corresponding model is retrained. If the accuracy meets expectations, the model is used for actual detection.

Images