Plant disease identification method and system based on unsupervised category incremental learning, and storage medium

Abstract

The invention discloses a plant disease identification method and system based on unsupervised category incremental learning, and a storage medium, relates to the technical field of plant disease identification, and aims to solve the problems that an existing deep learning model cannot meet the application requirements of real-time updating and optimization of the model, the timeliness is poor, and the efficiency is high. And the problem of failure of the class incremental learning strategy when the new class data is not labeled by the label exists. Comprising the following steps: S1, dividing a data set into category sets of a plurality of stages according to categories; s2, performing interpolation between the image of the current task and the random image uniformly sampled and selected from the cache region of the old category data by using a hybrid playback algorithm to generate a synthetic sample; s3, constructing a SimSiam model which comprises two identical neural network branches and shared parameters of the two branches, constructing a loss function based on the negative cosine similarity of the two network branches, and training the SimSiam model in multiple stages; s4, evaluating the performance of the model based on a KNN algorithm; and S5, classifying the plant diseases by adopting the trained SimSiam model.

Description

Technical Field

[0001] The present invention relates to the technical field of plant disease identification, and in particular to a plant disease identification method, system and storage medium based on unsupervised category incremental learning. Background Art

[0002] When plant leaves are infected with diseases, they usually form diverse and complex lesions. Therefore, plant disease detection based on visible light images usually uses leaf disease images as detection samples to identify and diagnose diseases. However, in the actual application of plant disease diagnosis, new types of diagnosis requirements are sometimes faced.

[0003] In the practical application of traditional deep learning models, the annotation of plant disease images requires professionals, so most plant disease images are unlabeled. If there is no label annotation in the new category data, the traditional category incremental learning strategy will fail, so an unsupervised category incremental learning method for plant disease image classification is needed.

[0004] At present, in the field of plant disease detection, there are few studies on unsupervised category incremental learning, and no system has been formed. Most of the existing methods use clustering-based algorithms to generate pseudo labels for newly added categories. This algorithm requires that the number of newly added categories for each task remains unchanged during the entire incremental learning process, and requires that all newly added data belong to new categories. The traditional deep learning model update method is to retrain the model using old category data together with new category data. This update method cannot meet the application requirements of real-time update and optimization, and has poor timeliness. If only new category data is used to update model parameters, there will be a problem of catastrophic forgetting, that is, the model completely forgets the previously learned plant disease characteristics. Therefore, there is an urgent need for a category incremental learning strategy for plant disease detection that enables the model to retain old knowledge while absorbing new knowledge. Summary of the invention

[0005] The technical problems to be solved by the present invention are:

[0006] Existing deep learning models cannot meet the application requirements of real-time updating and optimization of models, have poor timeliness, and there is a problem that the category incremental learning strategy will fail when there is no label annotation in the new category data.

[0007] The present invention adopts the following technical solutions to solve the above technical problems:

[0008] The present invention provides a plant disease identification method based on unsupervised category incremental learning, comprising the following steps:

[0009] S1. Collect plant disease image datasets and divide the datasets into multiple stages of category sets by category.

[0010] S2, using a hybrid replay algorithm to interpolate between the image of the current task and a random image uniformly sampled and selected from the cache of old category data to generate a synthetic sample;

[0011] S3. Construct a SimSiam model, which includes two identical neural network branches, the two branches share parameters, and each branch includes a feature extraction module, a projection head, and a prediction head; input two randomly enhanced views of the synthetic sample into the SimSiam model for training, construct a loss function based on the negative cosine similarity of the two network branches, and train the SimSiam model in multiple stages;

[0012] S4. Randomly select a part of the data from the training set as the memory data set, input the memory data set and the test set into the SimSiam network to extract features, calculate the distance between the image features of the memory data set and the image features of each test set, find the K samples with the closest distance, generate the corresponding feature labels, and evaluate the model performance;

[0013] S5. Use the trained SimSiam model to classify plant diseases.

[0014] Furthermore, in S1, the data set is divided into category sets of multiple stages by category. Specifically, the data set is equally divided into multiple incremental tasks by category.

[0015] Furthermore, in S1, the data set is divided into multiple stage category sets by category. Specifically, half of the total number of categories is used as the initial tasks, and the remaining categories are equally divided into multiple tasks.

[0016] Furthermore, the generation process of synthetic samples in S2 is:

[0017]

[0018] Where λ is the interpolation coefficient, is a synthetic sample, xj, M ~M represents the sample selected from the buffer M using uniform sampling, and xi,t represents the current task.

[0019] Furthermore, the method for obtaining the two randomly enhanced views of the synthetic sample in S3 is to adopt two different data enhancement methods, including randomly adopting rotation, noise addition, scaling or flipping methods to obtain two data enhanced images.

[0020] Furthermore, in the model training process in S3, a stop gradient mechanism is introduced, that is, the gradient is calculated for only one branch during the training process, while the stop gradient mechanism is used for the other branch without parameter update. The constructed loss function is:

[0021]

[0022] in, is the cosine similarity, and The features are obtained by mapping the prediction heads for the two branches respectively. and They are the branch vectors obtained by the extraction module for the two branches respectively.

[0023] Furthermore, in S4, the distance between the image feature of the memory data set and each image feature of the test set is calculated, including calculating the Euclidean distance, Manhattan distance, Chebyshev distance or Mahalanobis distance between the image feature of the memory data set and each image feature of the test set.

[0024] The present invention provides a plant disease identification system based on unsupervised category incremental learning. The system has a program module corresponding to the steps of the method described in any one of the above technical solutions, and executes the steps in the above plant disease identification method based on unsupervised category incremental learning during operation.

[0025] The present invention provides a computer-readable storage medium, which stores a computer program. The computer program is configured to implement the steps in the plant disease identification method based on unsupervised category incremental learning described in any one of the above technical solutions when called by a processor.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The present invention discloses a plant disease recognition method based on unsupervised category incremental learning. Based on collected plant disease leaf images, the SimSiam model is used to extract image features of plant diseases in an unsupervised scenario, thereby solving the problem of time-consuming and costly labeling of a large number of disease images. A hybrid replay algorithm is combined to generate synthetic samples of new category data by random interpolation, so that the model continuously adapts to unlabeled new category data during training, realizes autonomous continuous updating and optimization of the plant disease detection model, and meets the diagnosis requirements for newly added disease categories; and can further improve the accuracy and robustness of classification, and reduce classification errors caused by data changes.

[0028] The SimSiam model of the present invention simplifies the framework of contrastive learning, reduces the reliance on large-scale negative sample comparison or complex data enhancement strategies, and learns the feature representation of the image by maximizing the similarity between different enhanced views of the same image when negative samples are not applicable, thereby achieving efficient and accurate unsupervised image feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 These are images from part of the PlantVillage dataset;

[0030] Figure 2 is a flow chart of a plant disease identification method based on unsupervised category incremental learning in an embodiment of the present invention;

[0031] Figure 3 Schematic diagram of a hybrid playback process in an embodiment of the present invention;

[0032] Figure 4 1 is an overall workflow diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, exemplary implementations or embodiments of the present invention will be described below in conjunction with the accompanying drawings. Obviously, the described implementations or embodiments are only implementations or embodiments of a part of the present invention, not all of them. Based on the implementations or embodiments of the present invention, all other implementations or embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.

[0034] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0035] Combination Figure 2 and Figure 4 As shown, the present invention provides a plant disease identification method based on unsupervised category incremental learning, comprising the following steps:

[0036] S1. Collect plant disease image datasets and divide the datasets into multiple stages of category sets by category.

[0037] S2, using a hybrid replay algorithm to interpolate between the image of the current task and a random image uniformly sampled and selected from the cache of old category data to generate a synthetic sample;

[0038] S3. Construct a SimSiam model, which includes two identical neural network branches, the two branches share parameters, and each branch includes a feature extraction module, a projection head, and a prediction head; input two randomly enhanced views of the synthetic sample into the SimSiam model for training, construct a loss function based on the negative cosine similarity of the two network branches, and train the SimSiam model in multiple stages;

[0039] S4. Randomly select a part of the data from the training set as the memory data set, input the memory data set and the test set into the SimSiam network to extract features, calculate the distance between the image features of the memory data set and the image features of each test set, find the K samples with the closest distance, generate the corresponding feature labels, and evaluate the model performance;

[0040] S5. Use the trained SimSiam model to classify plant diseases.

[0041] The goal of the class incremental learning of the present invention is to train the model in multiple stages to learn new categories in new tasks. Specifically, given a data set with a category set C, it is divided into T incremental tasks using class incremental learning. Each incremental task contains a category set Ci (1≤i≤T), and there is no overlap between the categories of the incremental tasks. The predicted categories of the model at any stage 1≤t≤T are all the learned categories Each category set Ci (1≤i≤T) is not annotated with a label.

[0042] SimSiam models include:

[0043] (1) Siamese network structure: It contains two identical neural network branches, which share parameters and process images with different enhancements respectively.

[0044] (2) Feature extraction module (ResNet18): The residual network ResNet18 is used as the feature extraction network.

[0045] (3) Projection head (Projection MLP): The projection head is a multi-layer perceptron MLP structure. The output of each branch passes through a projection head to map the features to a lower-dimensional space.

[0046] (4) Prediction head (Prediction MLP): The SimSiam model is different from other contrastive learning methods in that an additional prediction head is added in one of the branches. The role of the prediction head is to help the network better learn meaningful representations.

[0047] (5) Stop-Gradient mechanism: During the training process, SimSiam only calculates the gradient for one branch, and uses the "stop-gradient" technique for the other branch, that is, it does not update its parameters. The "stop-gradient" mechanism is believed to prevent the network from collapsing and ensure that the network can learn stable representations.

[0048] Preferably, the data set is divided into category sets of multiple stages by category. Specifically, the data set is equally divided into T incremental tasks by category.

[0049] Preferably, the data set is divided into multiple-stage category sets by category. Specifically, half of the total number of categories is used as the initial tasks, and the remaining categories are equally divided into T tasks.

[0050] Preferably, in S2, the image of the current task (xi,t)∈Ct is interpolated with a random image uniformly sampled and selected from the cache of the old category data to generate a synthetic sample like Figure 3 As shown, specifically:

[0051]

[0052] Where λ is the interpolation coefficient, is a synthetic sample, xj, M ~M represents the sample selected from the buffer M using uniform sampling, and xi,t represents the current task. Synthetic Samples It includes information about both new category data and old category data, allowing the model to continuously adapt to new tasks and new unlabeled data during training.

[0053] Preferably, the method for obtaining the two randomly enhanced views of the synthetic sample in S3 is to use two different data enhancement methods, including randomly using random rotation, noise addition, scaling or flipping to obtain two data enhanced images. and

[0054] Preferably, during the model training process in S3, the synthetic samples Two data augmented images and The two branches of the SimSiam network are input respectively, and then pass through the same encoder, i.e. the ResNet18 network of the feature extraction module, to obtain two feature vectors. Through ResNet18 feature extraction, a vector with large receptive field and small dimension is obtained. After two MLP mappings of the projection head and the prediction head, we get Same reason get The stop gradient mechanism is introduced, that is, only one branch is calculated for the gradient during training, and the stop gradient mechanism is used for the other branch without updating the parameters to prevent the network from collapsing and ensure that the network can learn stable representations. The SimSiam network is trained by maximizing the similarity of the outputs of the two branches, and the loss function constructed is:

[0055]

[0056] in, is the cosine similarity, and The features are obtained by mapping the prediction heads for the two branches respectively. and They are the branch vectors obtained by the extraction module for the two branches respectively.

[0057] Preferably, calculating the distance between the image feature of the memory data set and each image feature of the test set in S4 includes calculating the Euclidean distance, Manhattan distance, Chebyshev distance or Mahalanobis distance between the image feature of the memory data set and each image feature of the test set. These distance metrics are used to measure the degree of difference between different feature dimensions.

[0058] In this implementation, based on the KNN algorithm, a part of the data is randomly selected from the training set as the memory data set, and the memory data set and the test set are input into the SimSiam network to extract features. For the image features P(x1, x2, ..., x n ), and calculate its correlation with the image features Q(y1,y2,...,y n )’s Euclidean distance d(P,Q):

[0059]

[0060] where x1,x2,...,x n and y1,y2,...,y n is a feature vector in n-dimensional space; find the K samples with the closest distance and generate the corresponding feature labels to complete the evaluation of model performance.

[0061] Example 1

[0062] The dataset constructed in this embodiment comes from PlantVillage. The PlantVillage dataset contains 14 crops, 38 categories, and 54,305 images. Considering the design of subsequent category incremental learning tasks, 30 plant disease categories were selected from the PlantVillage dataset, referred to as PlantVillage30, as shown in Table 1 and Figure 1 shown.

[0063] Table 1

[0064]

[0065]

[0066] In this embodiment, the incremental learning divides the PlantVillage30 dataset into five tasks, each with six categories, and learns the model in sequence. When the model completes the training of task t (1≤t≤5), the predicted category of the model at any stage 1≤t≤T is all the learned categories. The model uses the KNN algorithm to evaluate the performance of the trained model on the validation set corresponding to task t. The validation set contains all the learned categories The validation sample.

[0067] The performance evaluation index of the model in this embodiment is mainly divided into two parts: accuracy and forgetting rate. The accuracy is evaluated by using the average accuracy and performance degradation rate, and the forgetting rate is evaluated by using the average forgetting rate, as shown in Table 2.

[0068] Table 2

[0069]

[0070] After the training of the five tasks, the average accuracy of the model was 75.36%, and the average forgetting rate was 6.87%, indicating that the invention has a strong learning ability for new disease categories and can retain the knowledge of old disease categories.

[0071] (1) Promote the development of smart agriculture. The present invention promotes the development of smart agriculture. By real-time monitoring and identification of plant diseases, farmers can manage crops more accurately and improve the yield and quality of crops.

[0072] (2) Improve agricultural production efficiency. The present invention can automatically identify and classify plant diseases, reduce the need for manual intervention, and improve agricultural production efficiency. Farmers can understand the disease situation more quickly, take timely measures to prevent and control it, reduce the impact of diseases on crop growth, and ensure the healthy growth of crops.

[0073] (3) Reduce agricultural production costs. Unsupervised category incremental learning methods reduce the cost of data annotation and improve the accuracy of disease identification, thereby reducing the cost of disease and pest control in agricultural production. This helps farmers improve economic benefits and increase income.

[0074] The plant disease identification method (algorithm) based on unsupervised category incremental learning proposed in the present invention is the underlying technical core of the present invention, and various products can be derived based on the algorithm.

[0075] Based on the method proposed in the present invention, a plant disease identification system based on unsupervised category incremental learning is developed using a programming language. The system has program modules corresponding to the steps of the above-mentioned technical solution, and executes the steps in the above-mentioned plant disease identification method based on unsupervised category incremental learning during operation.

[0076] The computer program of the developed system (software) is stored on a computer-readable storage medium, and the computer program is configured to implement the steps of the above-mentioned plant disease identification method based on unsupervised category incremental learning when called by a processor, that is, the present invention is materialized on a carrier to become a computer program product.

[0077] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0078] The computer programs (also referred to as programs, software, software applications, or codes) of the present invention include machine instructions for programmable processors, and these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or device (e.g., disk, optical disk, memory, programmable logic device PLD) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0079] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A plant disease identification method based on unsupervised category incremental learning, characterized in that: The steps include: S1. Collect plant disease image datasets and divide the datasets into multiple stages of category sets by category. S2, using a hybrid replay algorithm to interpolate between the image of the current task and a random image uniformly sampled and selected from the cache of old category data to generate a synthetic sample; S3. Construct a SimSiam model, which includes two identical neural network branches, the two branches share parameters, and each branch includes a feature extraction module, a projection head, and a prediction head; input two randomly enhanced views of the synthetic sample into the SimSiam model for training, construct a loss function based on the negative cosine similarity of the two network branches, and train the SimSiam model in multiple stages; S4. Randomly select a part of the data from the training set as the memory data set, input the memory data set and the test set into the SimSiam network to extract features, calculate the distance between the image features of the memory data set and the image features of each test set, find the K samples with the closest distance, generate the corresponding feature labels, and evaluate the model performance; S5. Use the trained SimSiam model to classify plant diseases.

2. The plant disease identification method based on unsupervised category incremental learning according to claim 1, characterized in that: In S1, the data set is divided into category sets of multiple stages by category. Specifically, the data set is equally divided into multiple incremental tasks by category.

3. The plant disease identification method based on unsupervised category incremental learning according to claim 1, characterized in that: In S1, the dataset is divided into multiple stage category sets by category. Specifically, half of the total number of categories is used as the initial task, and the remaining categories are equally divided into multiple tasks.

4. The method for plant disease identification based on unsupervised category incremental learning according to claim 2 or 3, characterized in that: The generation process of synthetic samples in S2 is: Where λ is the interpolation coefficient, is a synthetic sample, xj, M ~M represents the sample selected from the buffer M using uniform sampling, and xi,t represents the current task.

5. The method for plant disease identification based on unsupervised category incremental learning according to claim 4, characterized in that: The method for obtaining the two randomly enhanced views of the synthetic sample in S3 is to use two different data enhancement methods, including randomly using rotation, noise addition, scaling or flipping to obtain two data enhanced images.

6. The method for plant disease identification based on unsupervised category incremental learning according to claim 1, characterized in that: During the model training process in S3, the stop gradient mechanism is introduced, that is, during the training process, the gradient is calculated for only one branch, and the stop gradient mechanism is used for the other branch without parameter update. The loss function constructed is: in, is the cosine similarity, and The features are obtained by mapping the prediction heads for the two branches respectively. and They are the branch vectors obtained by the extraction module for the two branches respectively.

7. The method for plant disease identification based on unsupervised category incremental learning according to claim 5 or 6, characterized in that: In S4, the distance between the image feature of the memory data set and each image feature of the test set is calculated, including calculating the Euclidean distance, Manhattan distance, Chebyshev distance or Mahalanobis distance between the image feature of the memory data set and each image feature of the test set.

8. A plant disease recognition system based on unsupervised category incremental learning, characterized in that: The system has a program module corresponding to the steps of the method described in any one of claims 1 to 7, and executes the steps of the plant disease identification method based on unsupervised category incremental learning when running.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps in the plant disease identification method based on unsupervised category incremental learning according to any one of claims 1 to 7 when called by a processor.

Images