A Wheat Rust Disease Recognition Method Based on Transfer Learning and Sharpness-Aware Minimization

Abstract

The present invention provides a wheat rust disease recognition method based on transfer learning and sharpness-aware minimization. 1. It is characterized by the following steps: S1: Obtain wheat rust disease pictures in different disease periods, and classify the wheat rust disease pictures according to the severity of the disease; S2: Preprocess and perform data augmentation on the wheat rust disease pictures to obtain a wheat rust disease training set; S3: Use the ImageNet large-scale picture dataset to pre-train a lightweight model to obtain a pre-trained model, and save the pre-trained model parameters; S4: On the basis of the pre-trained model, use the wheat rust disease training set to train the pre-trained model, adopt a label smoothing loss function, and combine the sharpness-aware minimization method to fine-tune the pre-trained model to improve the generalization ability of the pre-trained model. By introducing the transfer learning method, the present invention effectively improves the recognition accuracy of the lightweight model, and at the same time, combining the sharpness-aware minimization method can prevent the risk of model overfitting.

Description

Technical Field

[0001] The present invention relates to the field of crop disease classification, and specifically relates to a wheat rust disease recognition method based on transfer learning and sharpness-aware minimization. Background Art

[0002] Wheat is one of the most important food crops, ranking second in total output among all food crops. The yield of wheat is affected by pests and diseases, and wheat rust is one of the more common diseases. When the disease occurs, it generally causes a 5-15% reduction in wheat yield, and in severe cases, it can reach more than 50%.

[0003] The traditional prevention and control of wheat rust mainly relies on experienced farmers to detect diseased wheat at the early stage of the disease occurrence and treat it. This requires a high level of knowledge of disease prevention and control for farmers, and it is difficult to ensure the timeliness of disease detection. At present, with the development of technologies such as automation and artificial intelligence, using drone cruising combined with a deep vision model to achieve the detection and recognition of wheat rust is a feasible solution. However, due to hardware limitations of the equipment, relatively lightweight models usually need to be loaded on the drone. Compared with traditional deep models, lightweight models require less storage space and lower computing power, but they also lead to a loss of accuracy and possible overfitting problems. Summary of the Invention

[0004] Aiming at the deficiencies existing in the prior art, the purpose of the present invention is to provide a wheat rust disease recognition method based on transfer learning and sharpness-aware minimization. The present invention effectively improves the recognition accuracy of the lightweight model by introducing the transfer learning method, and at the same time combines the sharpness-aware minimization method to prevent the risk of model overfitting.

[0005] To solve the above technical problems, the present invention is achieved through the following technical solutions:

[0006] A wheat rust disease recognition method based on transfer learning and sharpness-aware minimization, characterized in that it includes the following steps:

[0007] S1: Obtain wheat rust disease pictures at different disease stages, and classify the wheat rust disease pictures according to the severity of the disease;

[0008] S2: First preprocess and perform data augmentation on the wheat rust disease pictures to obtain a wheat rust disease training set;

[0009] S3: Use the ImageNet large-scale picture dataset to pre-train the lightweight model to obtain a pre-trained model, and save the pre-trained model parameters;

[0010] S4: Based on the pre-trained model, use the wheat rust training set to train the pre-trained model. Adopt the label smoothing loss function and combine the Sharpness-Aware Minimization (SAM) method to fine-tune the pre-trained model to improve the generalization ability of the pre-trained model;

[0011] S5: Install a vision module consisting of an embedded platform and a high-definition camera on the drone. Transplant the model trained in step S4 to the embedded platform, and then realize the real-time detection and recognition of wheat rust.

[0012] Furthermore: In step S2, the preprocessing method for wheat rust pictures is as follows: According to the aspect ratio distribution histogram of the sizes of the collected wheat rust pictures, select the aspect ratio that is close to most of the collected wheat rust pictures, and accordingly stipulate the standard size of the input pictures of the model; The data augmentation method for wheat rust pictures is: Perform geometric transformation and color transformation on the wheat rust pictures to generate a large number of new training pictures, and obtain the wheat rust training set.

[0013] Furthermore: In step S3, the lightweight model adopts a lightweight convolutional neural network model.

[0014] Furthermore: In step S4, use the wheat rust training set to train the pre-trained model in two stages: In the first stage, freeze the feature extraction layer in the pre-trained model and fine-tune the classifier parameters of the pre-trained model; In the second stage, combine the sharpness-aware minimization method to adjust the global parameters of the pre-trained model.

[0015] Furthermore: In step S4, define the wheat rust training set as The parameters of the lightweight model are w, and the loss function is l(w, x, y);

[0016] where x i is the picture sample, and y i is the class label corresponding to the sample;

[0017] The calculation formula for the loss of the wheat rust training set is:

[0018]

[0019] The optimization objective of the sharpness-aware minimization method is:

[0020]

[0021]

[0022] where ρ is a hyperparameter, is the maximum value of the loss within the parameter space limited by ρ, represents sharpness, and λ is the regularization coefficient;

[0023] The formula of the label smoothing loss function is as follows:

[0024]

[0025] Among them, is the smoothed label, y k is the One-hot label, K is the number of classes, and α is a hyperparameter.

[0026] 8. Further: The hyperparameters ρ and α are selected to obtain the optimal parameter values by using the grid search method, and the search space of the parameters is [0, 1] 2 ; The adjustment of the hyperparameters of the learning rate and momentum term of the lightweight model mainly considers the similarity between the wheat rust training set and the large ImageNet image dataset;

[0027] The principle of hyperparameter setting is as follows: If the similarity between the wheat rust training set and the large ImageNet image dataset is high, set a smaller learning rate and momentum term; if the similarity between the wheat rust training set and the large ImageNet image dataset is low, set a larger learning rate, and the momentum term is set to the empirical value 0.9;

[0028] The EMD (Earth Mover’s Distance) method is used to measure the similarity between the wheat rust training set and the large ImageNet image dataset. The large ImageNet image dataset is defined as S, and the similarity calculation formula is as follows:

[0029]

[0030] sim(S,T)=e -0.01d(S,T) (6)

[0031] Among them, d i,j =||g(s i )-g(t j )||, g(·) is the feature extractor, g(s i ) represents the mean value of the features of the i-th class of images in the ImageNet dataset, g(t j ) represents the mean value of the features of the j-th class of images in the wheat rust training set, M and K respectively represent the number of classes of S and T, and f i,j represents the optimal flow for solving the EMD problem.

[0032] Further: In step S5, the drone flies according to a pre-set route, altitude, and speed to perform a cruise mission. During the process, wheat pictures are continuously taken, and the wheat pictures are processed and analyzed in real time on the embedded platform. Combining with the pre-trained model that has been trained in step S4, the wheat rust situation in the pictures is identified. When it is identified that there is a wheat rust occurrence, the location of the drone and the disease identification information are uploaded, and the disease pictures are transmitted.

[0033] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0034] By introducing the transfer learning method, the present invention effectively improves the recognition accuracy of the lightweight model. At the same time, combining with the sharpness-aware minimization method can prevent the risk of model overfitting. The lightweight model trained by the method proposed in the present invention can be applied on the embedded platform. Cooperating with the drone, it can detect wheat rust in real time and stably. The method proposed in the present invention has a low cost, good real-time performance and stability, and has high practical value. Description of the Drawings

[0035] Figure 1 It is a flowchart of an embodiment of the present invention;

[0036] Figure 2 It is a schematic diagram of picture preprocessing of an embodiment of the present invention;

[0037] Figure 3 It is a schematic diagram of the training process of an embodiment of the present invention;

[0038] Figure 4 It is a display of the wheat rust recognition effect of an embodiment of the present invention. Detailed Embodiments

[0039] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the following describes the preferred implementation solutions of the present invention with reference to specific embodiments. However, it should be understood that the drawings are only for illustrative purposes and cannot be construed as a limitation of the present invention; for better illustration of this embodiment, some components in the drawings will be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted. The positional relationships described in the drawings are only for illustrative purposes and cannot be construed as a limitation of the present invention.

[0040] The following further illustrates the present invention with reference to the drawings and embodiments, but it is not used as a basis for limiting the present invention.

[0041] Referring to Figure 1 , a wheat rust recognition method based on transfer learning and sharpness-aware minimization includes the following steps:

[0042] S1: Obtain wheat rust disease pictures at different disease stages, and classify the wheat rust disease pictures according to the severity of the disease;

[0043] S2: First, preprocess and perform data augmentation on the wheat rust disease pictures to obtain a wheat rust disease training set;

[0044] S3: Use the ImageNet large-scale picture dataset to pre-train the lightweight model to obtain a pre-trained model, and save the parameters of the pre-trained model;

[0045] S4: On the basis of the pre-trained model, use the wheat rust disease training set to train the pre-trained model. Adopt a label smoothing loss function and combine the Sharpness-Aware Minimization (SAM) method to fine-tune the pre-trained model to improve the generalization ability of the pre-trained model;

[0046] S5: Install a vision module consisting of an embedded platform and a high-definition camera on the drone, transplant the model trained in step S4 to the embedded platform, and then realize the real-time detection and recognition of wheat rust disease.

[0047] In the above step S1, the pictures of wheat rust disease can be collected by a camera, a mobile phone or a drone; in the above step S1, the disease severity of wheat rust disease is divided into six levels from low to high, namely I, II, …, VI. Level I is a picture of healthy wheat, and level VI is a picture of wheat with the most severe disease.

[0048] In the above step S2, the preprocessing method for wheat rust disease pictures is: according to the histogram of the aspect ratio distribution of the sizes of the collected wheat rust disease pictures, select the aspect ratio that is close to most of the collected wheat rust disease pictures, and accordingly specify the standard size of the model input pictures, such as Figure 2 shown. In this embodiment, the peak position of the aspect ratio distribution of the pictures is around the value 7. For the convenience of calculation, the integer 7 is taken as the standard aspect ratio of the pictures, and before the pictures are input into the model, they are standardized into pictures with a size of 420x60; the data augmentation method for wheat rust disease pictures is: perform geometric transformation and color transformation on the wheat rust disease pictures to generate a large number of new training pictures, and obtain a wheat rust disease training set.

[0049] Geometric transformation can adopt transformation means such as horizontal flipping, vertical flipping, and random rotation, and color transformation can adopt transformation means such as brightness adjustment, contrast adjustment, and saturation adjustment.

[0050] In step S3, the lightweight model adopts a lightweight convolutional neural network model. The lightweight convolutional neural network model can be classic models such as SqueezeNetV1.1, ShuffleNetV2, MobileNetV3, and Xception. The lightweight model is not limited to a specific model among the above, and can also be other models in the lightweight convolutional neural network model. In this embodiment, Xception is used as the benchmark model in this embodiment, pre-trained using the ImageNet dataset, and after training is completed, the number of output neurons is modified to be the same as the number of categories of wheat rust disease pictures.

[0051] In step S4, the pre-trained model is trained in two stages using the wheat rust disease training set: in the first stage, the feature extraction layer in the pre-trained model is frozen, and the classifier parameters of the pre-trained model are fine-tuned; in the second stage, combined with the sharpness-aware minimization method, the global parameters of the pre-trained model are adjusted. As Figure 3 shown, in this embodiment, in the first stage, the feature extraction layer in the pre-trained model is frozen, and the Adam gradient descent method is used to fine-tune the classifier; in the second stage, on the basis of the Adam gradient descent method, the sharpness-aware minimization method is combined to adjust the global parameters of the pre-trained model, and continuous training and iteration are performed until the pre-trained model converges to obtain the final wheat rust disease recognition model.

[0052] In step S4, the wheat rust disease training set is defined as the lightweight model parameters are w, and the loss function is l(w, x, y);

[0053] where, x i is the picture sample, and y i is the category label corresponding to the sample;

[0054] The calculation formula for the loss of the wheat rust disease training set is:

[0055]

[0056] The optimization objective of the sharpness-aware minimization method is:

[0057]

[0058]

[0059] where ρ is a hyperparameter, is the maximum value of the loss within the parameter space defined by ρ, represents sharpness, and λ is the regularization coefficient;

[0060] The formula for the label smoothing loss function is as follows:

[0061]

[0062] Among them, is the smoothed label, y k is the One-hot label, K is the number of categories, and α is a hyperparameter.

[0063] 9. The optimal parameter values of the hyperparameters ρ and α are selected by the grid search method, and the search space of the parameters is [0, 1] 2 ; The adjustment of the hyperparameters of the learning rate and momentum term of the lightweight model mainly considers the similarity between the wheat rust training set and the large ImageNet image dataset;

[0064] The principle of hyperparameter setting is as follows: When the similarity between the wheat rust training set and the large ImageNet image dataset is high, set a smaller learning rate and momentum term; when the similarity between the wheat rust training set and the large ImageNet image dataset is low, set a larger learning rate, and the momentum term is set to the empirical value 0.9;

[0065] The EMD (Earth Mover’s Distance) method is used to measure the similarity between the wheat rust training set and the large ImageNet image dataset. The large ImageNet image dataset is defined as S, and the similarity calculation formula is as follows:

[0066]

[0067] sim(S, T) = e -0.01d(S,T) (6)

[0068] Among them, d i,j = ||g(s i ) - g(t j )||, g(·) is the feature extractor, g(s i ) represents the mean of the features of the i-th category of pictures in the ImageNet dataset, g(t j ) represents the mean of the features of the j-th category of pictures in the wheat rust training set, M and K respectively represent the number of categories of S and T, and f i,j represents the optimal flow for solving the EMD problem.

[0069] In the step S5, the unmanned aerial vehicle flies according to the pre-set route, altitude, and speed, performs the cruise task, continuously takes pictures of wheat during the process, and the wheat pictures are processed and analyzed in real time on the embedded platform. Combining with the pre-trained model trained in the step S4, the wheat rust situation in the pictures is identified. When it is identified that there is a wheat rust occurrence, the location of the unmanned aerial vehicle and the disease identification information are uploaded, and the disease pictures are transmitted.

[0070] As shown in Table 1, the recognition accuracy of the model in this embodiment for the wheat rust test dataset reached 94.73%. Compared with other lightweight models SqueezeNetV1.1, ShuffleNetV2, MobileNetV3-Small, MobileNetV3-Large, and Xception directly trained using the wheat rust training set, the model in this embodiment trained by the method proposed in the present invention achieved the best results and was significantly better than the other models. As Figure 4 shown is the confusion matrix corresponding to the recognition results of the model in this embodiment. It can be seen from the figure that most of the misrecognized samples are at adjacent levels to the correct samples, indicating that the model can well recognize the disease severity of wheat rust. However, the smaller the difference between samples at adjacent levels, the greater the difficulty of recognition. At the same time, when the disease severity of wheat rust tends to be severe, the number of misrecognized samples by the model increases, indicating that it is more difficult to distinguish samples at adjacent levels when the disease is severe. If the classification errors at adjacent levels are not counted, then the number of misrecognized samples is only 5, and the recognition accuracy is as high as 99.83%.

[0071]

[0072] Table 1

[0073] Based on the description and drawings of the present invention, those skilled in the art can easily manufacture or use a wheat rust recognition method based on transfer learning and sharpness-aware minimization of the present invention and can produce the positive effects recorded in the present invention.

[0074] The above are only preferred embodiments of the present invention and do not impose any form of limitation on the present invention. Any simple modification or equivalent change made to the above embodiments based on the technical essence of the present invention falls within the protection scope of the present invention.

Claims

1. A wheat rust disease recognition method based on transfer learning and sharpness-aware minimization, characterized in that: It includes the following steps: S1: Obtain wheat rust disease pictures in different disease periods, and classify the wheat rust disease pictures according to the severity of the disease; S2: Preprocess and perform data augmentation on the wheat rust disease pictures to obtain a wheat rust disease training set; S3: Use the ImageNet large-scale picture dataset to pre-train a lightweight model to obtain a pre-trained model, and save the parameters of the pre-trained model; S4: Based on the pre-trained model, use the wheat rust disease training set to train the pre-trained model. Adopt a label smoothing loss function, and combine the Sharpness-Aware Minimization (SAM) method to fine-tune the pre-trained model to improve the generalization ability of the pre-trained model; S5: Install a vision module composed of an embedded platform and a high-definition camera on the drone, transplant the model trained in step S4 to the embedded platform, and then realize the real-time detection and recognition of wheat rust disease; In step S4, the wheat rust training set is defined as The lightweight model parameters are w, the loss function is l(w, x, y), and x i is the image sample, and y i is the class label corresponding to the sample; The calculation formula for the loss of the wheat rust disease training set is: The optimization objective of the sharpness-aware minimization method is: where ρ is a hyperparameter, is the maximum value of the loss within the parameter space defined by ρ, represents sharpness, and λ is the regularization coefficient; The formula for the label smoothing loss function is as follows: Among them, is the smoothed label, and y k is the One-hot label, K is the number of classes, and α is a hyperparameter.

2. The wheat rust disease recognition method based on transfer learning and sharpness-aware minimization according to claim 1, wherein: In step S2, the preprocessing method for the wheat rust disease pictures is: according to the aspect ratio distribution histogram of the sizes of the collected wheat rust disease pictures, select the aspect ratio that is close to most of the collected wheat rust disease pictures, and accordingly specify the standard size of the input pictures of the model; the data augmentation method for the wheat rust disease pictures is: perform geometric transformation and color transformation on the wheat rust disease pictures to generate a large number of new training pictures, and obtain a wheat rust disease training set.

3. A wheat rust disease recognition method based on transfer learning and sharpness-aware minimization according to claim 1, characterized in that: In step S3, the lightweight model adopts a lightweight convolutional neural network model.

4. A wheat rust disease recognition method based on transfer learning and sharpness-aware minimization according to claim 1, characterized in that: In step S4, the pre-trained model is trained in two stages using the wheat rust disease training set: in the first stage, the feature extraction layer in the pre-trained model is frozen, and the classifier parameters of the pre-trained model are fine-tuned; in the second stage, combined with the sharpness-aware minimization method, the global parameters of the pre-trained model are adjusted.

5. A wheat rust disease recognition method based on transfer learning and sharpness-aware minimization according to claim 1, characterized in that: The optimal parameter values of the hyperparameters ρ and α are selected by the grid search method, and the search space of the parameters is [0, 1]. 2 ; The adjustment of the hyperparameters of the learning rate and momentum term of the lightweight model mainly considers the similarity between the wheat rust training set and the large ImageNet image dataset; The principle for setting hyperparameters is as follows: when the similarity between the wheat rust disease training set and the ImageNet large-scale picture dataset is high, set a small learning rate and momentum term; when the similarity between the wheat rust disease training set and the ImageNet large-scale picture dataset is low, set a large learning rate, and set the momentum term to the empirical value 0.9; Use the EMD (Earth Mover’s Distance) method to measure the similarity between the wheat rust disease training set and the ImageNet large-scale picture dataset. The ImageNet large-scale picture dataset is defined as S, and the similarity calculation formula is as follows: sim(S,T) = e -0.01d(S,T) (6) where d i,j = ||g(s i ) - g(t j )||, g(·) is a feature extractor, g(s i ) represents the mean of the features of the i-th class of images in the ImageNet dataset, g(t j ) represents the mean of the features of the j-th class of images in the wheat rust disease training set, M and K represent the number of classes of S and T respectively, and f i,j represents the optimal flow for solving the EMD problem.

6. The wheat rust disease recognition method based on transfer learning and sharpness-aware minimization according to claim 1, wherein: In step S5, the drone flies according to the pre-set route, altitude, and speed to perform a cruising task. During the process, wheat pictures are continuously taken. The wheat pictures are processed and analyzed in real time on the embedded platform. Combined with the pre-trained model trained in step S4, the wheat rust disease situation in the pictures is identified. When it is identified that there is a wheat rust disease occurrence, the location of the drone and the disease identification information are uploaded, and the disease pictures are transmitted.

Images