Plant leaf disease detection method based on improved MobileNetV3

Abstract

The invention discloses a plant leaf disease detection method based on improved MobileNetV3, and relates to the technical field of plant disease detection, and the method comprises the following steps: 1) obtaining and marking a plant leaf disease image used for model training, and dividing the image into a training set and a test set according to a ratio of 4: 1; 2) constructing a plant disease detection model; 3) pre-training the model by using the plant species data, reducing the learning and training cost, and accelerating the model convergence speed; 4) training the model by using the plant disease sample set; 5) identifying a plant leaf image by using the trained plant leaf disease detection model to obtain a detection result; and 6) deploying the model on a mobile platform to obtain the portable plant disease detection equipment. According to the method, the MobileNetV3 network model is improved, the precision of the model is further improved on the premise that the parameter quantity and the training time of the model are not obviously influenced, and the model can realize rapid and accurate plant disease detection on mobile equipment due to less parameter quantity and higher precision.

Description

Technical Field

[0001] The present invention relates to the field of automated detection and prevention of plant diseases, and in particular to a plant leaf disease detection method based on improved MobileNetV3. Background Art

[0002] Early prevention and control of plant diseases can recover some agricultural economic losses, improve the output and quality of agricultural production and food safety. The diagnosis and prevention of plant diseases and pests is one of the important links in the food production process.

[0003] The traditional method of identifying and diagnosing plant diseases and insect pests is to complete the identification through visual observation by plant protection experts or people with planting experience. This method is limited by human experience and subjective thinking, and is prone to bias, leading to misdiagnosis. This process is time-consuming and labor-intensive, and there is usually a lack of such experts in some underdeveloped or remote areas.

[0004] At present, automatic diagnosis of plant diseases mainly relies on computer vision (CV). The main popular methods in this field can be roughly divided into machine learning-based methods and deep learning-based methods. Traditional machine learning methods are highly subjective, heavily dependent on manual feature selection, time-consuming and inefficient. Most of the current advanced deep CNN models for crop disease diagnosis have too many parameters and high computational costs, making them difficult to deploy on low-cost edge devices. In the agricultural field, the use of complex laboratory equipment with GPUs is disadvantageous in application and promotion, and growers cannot afford the additional costs brought by complex equipment.

[0005] Therefore, a lightweight, fast and accurate automatic plant disease identification system is very important for the immediate diagnosis of crop diseases. It can recover some agricultural production losses, improve the output and quality of agricultural production, and further promote the promotion of automatic diagnosis methods for crop diseases. Summary of the invention

[0006] In order to solve the defects of the above-mentioned existing plant disease diagnosis methods, the present invention provides a plant leaf disease detection method based on improved MobileNetV3, which can realize fast and accurate diagnosis of plant diseases with fewer parameters and deploy it on edge devices.

[0007] In order to solve the above problems, the technical solution of the present invention comprises the following steps:

[0008] S1. Collect plant leaf disease images, annotate and clean the data set, divide the sample images into training set and test set at a ratio of 4:1, and use image preprocessing methods to enhance the training set.

[0009] S2. Build a plant leaf disease detection model, use the improved MobileNetV3 as the feature extraction network of the model, modify the output of the fully connected layer according to the number of sample set labels, and set the model's initial learning rate, training cycle, batchsize and other hyperparameters.

[0010] S3. Pre-training a plant leaf disease detection model using a data set containing various plant images, thereby obtaining a pre-trained plant leaf disease detection model.

[0011] S4. Use the training set of step S1 to fine-tune the pre-trained model of step S3, thereby obtaining a plant leaf disease detection model.

[0012] S5. Input the image to be detected into the plant disease detection model to obtain the leaf disease classification result and realize automatic detection.

[0013] S6. Deploy the plant leaf disease detection model obtained in step S4 onto a mobile platform, and use the mobile device to achieve rapid and accurate detection of plant leaf diseases.

[0014] Furthermore, the method of labeling, cleaning and preprocessing the data set in step S1 is specifically as follows:

[0015] S1.1. Collect plant pathology sample images and store them in folders with corresponding pathology names, and remove unqualified sample images (afterimages, blur, occlusions, etc.). Then, randomly divide the sample images into training sets and test sets at a ratio of 4:1, and create a Json document to store the pathology name and the corresponding label number.

[0016] S1.2. Use translation, flipping, rotation, scaling and other methods to expand the training data of the training set sample images in step S1.1 to enhance the generalization of the model.

[0017] Furthermore, the improvement method of MobileNetV3 in step S2 is:

[0018] S2.1. Insert SkipBlock into the feature extraction network of MobileNetV3, and transfer the low-level features extracted by the underlying network to the high-level network through skip connections, so as to further enrich the input features of the high-level network without significantly affecting the number of model parameters.

[0019] S2.2. Use the improved whale optimization algorithm to automatically adjust the hidden layer size in SkipBlock and automatically determine the most suitable number of neurons to better transmit low-level features.

[0020] S2.3. Modify the loss function and replace the traditional cross entropy loss function with a weighted bias loss function. This can further reduce the bias impact of the compact network caused by feature differences during training, and cooperate with the skip connection of SkipBlock to further improve the overall performance of the network.

[0021] Furthermore, the specific structure of the SkipBlock includes convolution blocks, BN normalization, and adaptive pooling.

[0022] Furthermore, the number of inserted SkipBlocks is 2, which respectively connect the first bneck and the sixth bneck and the first bneck and the tenth bneck, and transmit the low-level features extracted from one bneck to the input channels of the sixth and tenth bnecks respectively.

[0023] Furthermore, the improved method of the whale optimization algorithm is to use the Lévy flight strategy to update the individual position again after each position update, so as to further improve the global optimization ability of the algorithm. The mathematical formula is as follows:

[0024]

[0025] in is the individual's current position, is the updated position, the value range of β is (0,2), here β=1.5, and Each component u d and v d It follows the following normal distribution:

[0026]

[0027]

[0027] Furthermore, the traditional cross entropy loss calculation formula is:

[0028]

[0029] Among them, N represents the number of samples, k represents the number of categories, and p ij Represents the probability that the i-th sample belongs to category j.

[0030] The calculation formula of the deviation loss function is:

[0031]

[0032] z(v i )=exp(v i *α)-β (6)

[0033] Among them, α and β are adjustable contribution parameters, generally α=0.3, β=0.3, v i is the scaled variance of the output features.

[0034] Furthermore, the pre-training method in step S3 is to use about 300,000 plant images of 1,081 categories in total to train the initial plant leaf disease detection model to obtain a pre-trained model.

[0035] Furthermore, the fine-tuning method in step S4 is specifically to modify the output of the fully connected layer to the number of types in the current data set and reduce the learning rate to 0.0001.

[0036] Furthermore, in step S5, the automatic detection method is to input the image to be detected into the trained plant disease detection system, and the system will give the disease category name of the image, the accuracy of the disease and corresponding treatment suggestions, and the treatment suggestions come from the locally established text library.

[0037] Furthermore, in step S6, the model deployment method on the mobile platform is to use the mobile_optimizer in the Pytorch environment to convert the trained model file into a pt format that can support Android devices, and then use the pytorch_android_lite toolkit in the Android development environment to develop a plant disease identification system application.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The present invention provides a plant leaf disease detection method based on improved MobileNetV3. This method adopts MobileNetV3 as the feature extraction network and embeds jump blocks into the network, so as to further improve the detection accuracy of the lightweight model without significantly increasing the number of model parameters.

[0040] 2. The present invention improves the traditional whale optimization algorithm to enhance the global optimization capability of the algorithm, and is used to optimize the number of hidden layer neurons in the skip block in the model, thereby further improving the performance upper limit of the model.

[0041] 3. The present invention uses the bias loss function to replace the traditional cross entropy loss function. The bias loss function can optimize the error in the feature learning process and improve the performance of the lightweight model. In addition, the proposed model is pre-trained on the plant classification task dataset, which can further reduce the impact of overfitting and improve the accuracy of the disease recognition task compared to pre-training on the ImageNet dataset.

[0042] 4. The present invention provides a plant leaf disease detection method based on improved MobileNetV3, which has few model parameters and high precision. The model parameter volume is about 2.5M, the model size is about 10MB, and it can achieve low-latency and high-precision plant disease detection on Android devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is an implementation flow chart of a plant leaf disease detection method based on improved MobileNetV3 of the present invention.

[0044] Figure 2 This is a structural diagram of the plant leaf disease detection model of the present invention.

[0045] Figure 3 This is an application effect diagram of the present invention on an Android device. DETAILED DESCRIPTION

[0046] In order to understand the present invention more clearly, specific embodiments are given below in conjunction with the accompanying drawings of the present invention. It should be noted that the embodiments of the present invention are only used to further explain the present invention, and the present invention can also be implemented in other different ways, and the protection scope of the present invention is not limited by the following embodiments.

[0047] The present invention provides a plant leaf disease detection method based on improved MobileNetV3, such as Figure 1 As shown, the method comprises the following steps:

[0048] S1. Collect plant leaf disease images, annotate and clean the data set, divide the sample images into training set and test set at a ratio of 4:1, and use image preprocessing methods to enhance the training set.

[0049] S2. Build a plant leaf disease detection model, use the improved MobileNetV3 as the feature extraction network of the model, modify the output of the fully connected layer according to the number of sample set labels, and set the model's initial learning rate, training cycle, batchsize and other hyperparameters.

[0050] S3. Pre-training a plant leaf disease detection model using a data set containing various plant images, thereby obtaining a pre-trained plant leaf disease detection model.

[0051] S4. Use the training set of step S1 to fine-tune the pre-trained model of step S3, thereby obtaining a plant leaf disease detection model.

[0052] S5. Input the image to be detected into the plant disease detection model to obtain the leaf disease classification result and realize automatic detection.

[0053] S6. Deploy the plant leaf disease detection model obtained in step S4 onto a mobile platform, and use the mobile device to achieve rapid and accurate detection of plant leaf diseases.

[0054] The specific method of labeling, cleaning and preprocessing the data set in step S1 is:

[0055] S1.1. Collect plant pathology sample images and store them in folders with corresponding pathology names, and remove unqualified sample images (afterimages, blur, occlusions, etc.). Then, randomly divide the sample images into training sets and test sets at a ratio of 4:1, and create a Json document to store the pathology name and the corresponding label number.

[0056] S1.2. Use translation, flipping, rotation, scaling and other methods to expand the training data of the training set sample images in step S1.1 to enhance the generalization of the model.

[0057] The improvement method of MobileNetV3 in step S2 is:

[0058] S2.1. Insert SkipBlock into the feature extraction network of MobileNetV3, and transfer the low-level features extracted by the underlying network to the high-level network through skip connections, so as to further enrich the input features of the high-level network without significantly affecting the number of model parameters.

[0059] S2.2. Use the improved whale optimization algorithm to automatically adjust the hidden layer size in SkipBlock and automatically determine the most suitable number of neurons to better transmit low-level features.

[0060] S2.3. Modify the loss function and replace the traditional cross entropy loss function with a weighted bias loss function. This can further reduce the bias impact of the compact network caused by feature differences during training, and cooperate with the skip connection of SkipBlock to further improve the overall performance of the network.

[0061] The specific structure of SkipBlock in step S2.1 includes convolution block, BN normalization, and adaptive pooling.

[0062] The number of SkipBlocks inserted in step S2.1 is 2, which connect the first bneck with the sixth bneck and the first bneck with the tenth bneck respectively, and pass the low-level features extracted from one bneck to the input channels of the sixth and tenth bnecks respectively.

[0063] The improvement method of the traditional whale optimization algorithm in step S2.2 is to use the Lévy flight strategy to update the individual position again after each position update, further improving the global optimization ability of the algorithm. The mathematical formula is as follows:

[0064]

[0065] in is the individual's current position, is the updated position, the value range of β is (0,2), here β=1.5, and Each component u d and v d It follows the following normal distribution:

[0066]

[0067]

[0068] The traditional cross entropy loss calculation formula in step S2.3 is:

[0069]

[0070] Among them, N represents the number of samples, k represents the number of categories, and p ij Represents the probability that the i-th sample belongs to category j.

[0071] The calculation formula of the deviation loss function is:

[0072]

[0073] z(v i )=exp(v i *α)-β (6)

[0074] Among them, α and β are adjustable contribution parameters, generally α=0.3, β=0.3, v i is the scaled variance of the output features.

[0075] The pre-training method in step S3 is to use about 300,000 plant images of 1,081 categories in total to train the initial plant leaf disease detection model to obtain a pre-trained model.

[0076] The fine-tuning method in step S4 is specifically to modify the output of the fully connected layer to the number of types in the current data set and reduce the learning rate to 0.0001.

[0077] In step S5, the automatic detection method is to input the image to be detected into the trained plant disease detection system, and the system will give the disease category name of the image, the accuracy of the disease and the corresponding treatment suggestions, and the treatment suggestions come from the locally established text library.

[0078] In step S6, the model deployment method on the mobile platform is to use the mobile_optimizer in the Pytorch environment to convert the trained model file into a pt format that can support Android devices, and then use the pytorch_android_lite toolkit in the Android development environment to develop a plant disease identification system application.

[0079] In addition to the above embodiments, the present invention has other implementation modes. Any equivalent replacement or transformation shall fall within the protection scope of the appended claims of the present invention.

Claims

1. A plant leaf disease detection method based on improved MobileNetV3, characterized in that: The following steps are involved: S1. Collect plant leaf disease images, annotate and clean the dataset, divide the sample images into training and test sets at a ratio of 4:1, and enhance the training set using image preprocessing methods. S2. Build a plant leaf disease detection model, use the improved MobileNetV3 as the model's feature extraction network, modify the output of the fully connected layer according to the number of sample set labels, and set the model's initial learning rate, training cycle, batch size and other hyperparameters. S3. Pre-training a plant leaf disease detection model using a data set containing various plant images, thereby obtaining a pre-trained plant leaf disease detection model. S4. Use the training set of step S1 to fine-tune the pre-trained model of step S3, thereby obtaining a plant leaf disease detection model. S5. Input the image to be detected into the plant disease detection model to obtain the leaf disease classification result and realize automatic detection. S6. Deploy the plant leaf disease detection model obtained in step S4 onto a mobile platform, and use the mobile device to achieve rapid and accurate detection of plant leaf diseases.

2. The plant leaf disease detection method based on improved MobileNetV3 according to claim 1, characterized in that: The method for labeling, cleaning and preprocessing the data set in step S1 is specifically as follows: S1.

1. Collect plant pathology sample images and store them in folders with corresponding pathology names. Remove unqualified sample images (afterimages, blur, occlusions, etc.). Then, randomly divide the sample images into training and test sets at a ratio of 4:

1. Create a Json document to store the pathology name and the corresponding label number. S1.

2. Use translation, flipping, rotation, scaling and other methods to expand the training data of the training set sample images in step S1.1 to enhance the generalization of the model.

3. The plant leaf disease detection method based on improved MobileNetV3 according to claim 1, characterized in that: The improvement method of MobileNetV3 in step S2 is: S2.

1. Insert SkipBlock into the feature extraction network of MobileNetV3, and pass the low-level features extracted by the underlying network to the high-level network through skip connections, further enriching the input features of the high-level network without significantly affecting the number of model parameters. S2.

2. Use the improved whale optimization algorithm to automatically adjust the hidden layer size in SkipBlock and automatically determine the most suitable number of neurons to better transmit low-level features. S2.

3. Modify the loss function and replace the traditional cross-entropy loss function with a weighted bias loss function. This can further reduce the bias caused by feature differences during the training of the compact network, and cooperate with the skip connection of SkipBlock to further improve the overall performance of the network.

4. The plant leaf disease detection method based on improved MobileNetV3 according to claim 3, characterized in that: The specific structure of SkipBlock in step S2.1 includes convolution block, BN normalization, and adaptive pooling.

5. The plant leaf disease detection method based on improved MobileNetV3 according to claim 3, characterized in that: The number of SkipBlocks inserted in step S2.1 is 2, which connect the first bneck with the sixth bneck and the first bneck with the tenth bneck respectively, and pass the low-level features extracted from one bneck to the input channels of the sixth and tenth bnecks respectively.

6. The plant leaf disease detection method based on improved MobileNetV3 according to claim 3, characterized in that: The improvement method of the traditional whale optimization algorithm in step S2.2 is to use the Lévy flight strategy to update the position of the individual again after each position update, further improving the global optimization ability of the algorithm. The mathematical formula is as follows: in is the individual's current location, is the updated position, the value range of β is (0,2), here β=1.5, and Each component u d and v d Obey the following normal distribution:

7. The plant leaf disease detection method based on improved MobileNetV3 according to claim 3, characterized in that: The traditional cross entropy loss calculation formula in step S2.3 is: Among them, N represents the number of samples, k represents the number of categories, and p ij Represents the probability that the i-th sample belongs to category j. The calculation formula of the deviation loss function is: from(in i )=exp(v i *α)-β (6) Among them, α and β are adjustable contribution parameters, generally α=0.3, β=0.3, v i is the scaled variance of the output features.

8. The plant leaf disease detection method based on improved MobileNetV3 according to claim 1, characterized in that: The pre-training method in step S3 is to use approximately 300,000 plant images totaling 1,081 types to train the initial plant leaf disease detection model to obtain a pre-trained model.

9. The plant leaf disease detection method based on improved MobileNetV3 according to claim 1, characterized in that: The fine-tuning method in step S4 is specifically to modify the output of the fully connected layer to the number of types in the current data set and reduce the learning rate to 0.0001.

10. The plant leaf disease detection method based on improved MobileNetV3 according to claim 1, characterized in that: The automatic detection method in step S5 is to input the image to be detected into the trained plant disease detection system, and the system will give the disease category name of the image, the accuracy of the disease and the corresponding treatment suggestions. The treatment suggestions come from the locally established text library.

11. The plant leaf disease detection method based on improved MobileNetV3 according to claim 1, characterized in that: The model deployment method on the mobile platform in step S6 is to use the mobile_optimizer in the Pytorch environment to convert the trained model file into a pt format that can support Android devices, and then use the pytorch_android_lite toolkit in the Android development environment to develop a plant disease identification system application.

Images