A method for identifying pests and diseases based on image recognition

Abstract

The present invention discloses a pest and disease identification method based on image recognition, which relates to the field of image processing and includes the following steps: S1: constructing a pest and disease identification model, which consists of two parts: dynamic data augmentation and dynamic cross-domain adaptation during testing; the dynamic data augmentation part is a learnable implicit augmenter and a fixed explicit augmenter; the dynamic cross-domain adaptation part during testing includes a training phase and a model reasoning phase, the training phase uses yolo for training, and the model reasoning phase is divided into a main model, an auxiliary model and a source model; S2: the image generated by the implicit augmenter is reasoned by a classifier, and the loss is calculated based on the result and the implicit augmenter and the classifier are updated. The technical problem to be solved by the present invention is to provide a pest and disease identification method based on image recognition, which is convenient for pest and disease identification.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a pest identification method based on image recognition. Background Art

[0002] At present, most pest and disease identification research focuses on static scenes, that is, static images taken under specific conditions. Therefore, existing pest and disease identification methods often rely on the independent and identically distributed assumption, but in reality, the occurrence and spread of pests and diseases are often dynamic, and crop diseases in different regions and environments may be different, resulting in non-independent and identically distributed data. This will lead to a decrease in model performance and poor recognition accuracy when the model is applied to new fields or different environments. For example, the uncertainty of crop detection under different weather conditions increases, and new species invasions are difficult to identify. These incorrect classifications can lead to catastrophic consequences.

[0003] For the unseen model, the insect classification accuracy model based on the deep learning method uses a classifier that can distinguish 142 agricultural-related pest species. The model can avoid making predictions when faced with images that do not belong to any class it is trained to predict to improve the accuracy of unseen images. The results show that the OOD detection algorithm can significantly improve the user's trust in pest classification by classifying under uncertain conditions. In the neural network study, explicit and implicit enhancers are configured to optimize the assistant, where the explicit enhancer samples the distribution x~q(x) from the observation sequence, injects Gaussian noise, and generates K enhancements with different noise levels. The implicit enhancer is optimized by maximizing the inter-domain interval and minimizing the diversity and classification error, generating as diverse enhancements as possible, while maintaining consistency with the original semantic information, and not letting the enhancement deviate too much from the original information to provide a noise counterpart, and improves stronger robustness to noise. You Only Look Once (YOLO) is an advanced real-time object detection system. On Pascal Titan X, it processes images at 30 FPS and has an mAP of 57.9% on COCO test-dev. In recent years, it is closely related to the research on crop diseases. For example, an improved plant pest detection method based on YOLOv5m was developed. The SWin Transformer (SWinTR) and Transformer (C3TR) mechanisms were introduced into the YOLOv5m network. The experimental results show that the improved YOLOv5m achieved 95.7% accuracy, 93.1% recall, 94.38% F1 score and 96.4% mean average precision (mAP). At the same time, the proposed model is significantly better than the original YOLOv3, YOLOv4 and YOLOv5m models. Based on the YOLO5s algorithm, the YOLOv5s algorithm was improved to solve the pine wilt disease (PWD) problem, and a real-time and efficient model named PWD-YOLO was obtained. The experimental results show that on a self-constructed dataset, PWD-YOLO exceeds the existing target detection model in terms of model size (2.7 MB), computational complexity (3.5 GFLOPs), parameter volume (1.09 MB) and speed (98.0 frames / s). When it comes to crop pests and diseases, controlling the detection accuracy in different regions and environments to adapt to the changing climate is still a very important issue. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a pest and disease identification method based on image recognition to facilitate pest and disease identification.

[0005] The present invention adopts the following technical solutions to achieve the invention objectives:

[0006] A method for identifying pests and diseases based on image recognition, characterized in that it comprises the following steps:

[0007] S1: Construct a pest and disease recognition model, which consists of two parts: dynamic data augmentation and dynamic cross-domain adaptation during testing;

[0008] The dynamic data augmentation part consists of a learnable implicit augmenter and a fixed explicit augmenter;

[0009] The dynamic cross-domain adaptation part during testing includes the training phase and the model inference phase. The training phase uses YOLO for training, and the model inference phase is divided into the main model, auxiliary model and source model;

[0010] S2: The image generated by the implicit augmenter is inferred by the classifier, and the loss is calculated based on the result and the implicit augmenter and classifier are updated. The images generated by the implicit augmenter and the explicit augmenter are combined as the input for model training;

[0011] S3: training phase, using YOLOv9 for training;

[0012] S4: In the model inference phase, the main model calculates the loss and updates it by the difference between the category, confidence, and detection box results between the main model and the auxiliary model, while the auxiliary model is updated by the exponential moving average of the main model.

[0013] S5: In the case of cross-domain, the auxiliary model determines the size of the inter-domain gap by evaluating the confidence of the prediction. Data augmentation is performed when the main model is trained with the enhanced average pseudo-labels generated by the auxiliary model, thus achieving cross-domain recognition.

[0014] S6: The main model outputs the final result.

[0015] As a further limitation of the technical solution, the specific steps of S2 are:

[0016] S21: Feeding an image into a learnable implicit augmenter Under the guidance of the classifier, the original image is diversified into pest and disease images with different characteristics. For a given image sequence m The goal of the implicit augmenter is to generate a diverse augmented sequence without changing its classification and position information. In order to generate a cross-domain image with sufficient inter-domain interval with the original image, it is proposed to maximize the original image and K Augmented images Inter-domain spacing loss :

[0017] (1)

[0018] represents the Euclidean norm;

[0019] For vector x , The norm is defined as:

[0020] ;

[0021] Represents the operation of finding the maximum value of all elements in set I;

[0022] To stimulate the diversity of implicit augmenters, diversity is used to promote prior losses , whose normalization factor is :

[0023] (2)

[0024] In order to enable the implicit augmenter to learn the correct parameters, a classifier is introduced, using the cross entropy loss Perform training to ensure that the category labels of the augmented sequence are consistent with those of the original sequence;

[0025] (3)

[0026] in: N To store a given sequence m The correct label cl The size of the label space;

[0027] For the k The result of the second augmentation;

[0028] The goal of the classifier is to minimize the categorical cross entropy loss , accurately distinguish the erroneous data and correct data generated by the implicit augmenter, and learn and update when the implicit augmenter continuously generates new augmented data;

[0029] (4)

[0030] in: is the learnable parameter in the classifier;

[0031] is the learning rate of the classifier;

[0032] Represents the loss function The gradient with respect to the parameters θ; this gradient is used to update the classifier parameters to minimize the loss function;

[0033] Implicit Augmenter The goal is to maximize the inter-domain margin loss , minimize diversity loss and categorical cross entropy loss In the case of , more diverse augmented sequences are generated while maintaining the consistency of the augmented sequences with the original sequence labels. The learnable parameters of the implicit augmenter are updated by the following formula:

[0034] (5)

[0035] Among them: The trade-off parameter is selected as ; ; ;

[0036] Learning Rate ;

[0037] is the learnable parameter of the implicit augmenter;

[0038] Represents the total loss function relative to the parameter The gradient of ; this gradient is used to update the implicit augmenter parameters to minimize the loss function;

[0039] S22: Introducing an explicit augmenter , by injecting Gaussian noise at fixed intervals, multiple augmented versions with different noise levels are generated. Specifically, given the original sequence m~q(m) The samples in the image are injected with a series of fixed-interval Gaussian and Poisson composite noise to generate samples with different noise levels. C Augmentation:

[0040] (6)

[0041] (7)

[0042] in: represents a random variable;

[0043] Poisson represents Poisson distribution;

[0044] λ is the parameter of Poisson distribution;

[0045] It means that when m is given The conditional probability distribution of ;

[0046] Indicates an estimated value;

[0047] m Indicates expected value;

[0048] represents normal distribution;

[0049] represents the mean of the normal distribution;

[0050] represents the variance of the normal distribution;

[0051] represents the identity matrix, which means Each component of is independent and identically distributed.

[0052] As a further limitation of the present technical solution, in order to enhance the generalization ability of the source model and make it more flexible and adaptable when facing the target domain, a dynamic data augmentation part is introduced in the training stage. The training data is intelligently enhanced by using a learnable augmenter, thereby expanding the perception range and learning ability of the model.

[0053] As a further limitation of the technical solution, the specific process of S4 is:

[0054] S41: Initialize auxiliary model: In the initial stage of training, the auxiliary model Initialized to be the same as the source pre-trained network, ensuring that the model has a reliable starting point;

[0055] S42: Generate pseudo labels: At each time step t , main model and auxiliary models First, predict the target data and generate the main model prediction and pseudo labels ;

[0056] S43: Main Model Update: Main Model Updated by minimizing the loss between the auxiliary model prediction, the loss function The difference between the primary model prediction and the auxiliary model pseudo-label prediction class is calculated:

[0057] (8)

[0058] in: is the prediction of the target category by the auxiliary model pseudo label;

[0059] is the prediction of the main model category;

[0060] Loss Function The difference between the confidence of the primary model and the auxiliary model for the predicted target is calculated:

[0061] (9)

[0062] in: N is the number of prediction boxes;

[0063] and The auxiliary model and the main model are i The confidence level of the prediction;

[0064] Using loss function Calculate the difference between the detection boxes of the primary model and the auxiliary model:

[0065] (10)

[0066] Smooth L 1 loss is often used to train the bounding box regressor of the target because it can effectively handle the difference between the target box and the predicted box. When the absolute value of the difference between the predicted value and the true value is less than 1, Smooth L 1 The loss function is expressed as L2 loss, which helps to reduce the sensitivity of the model to noise. When the absolute value of the difference is greater than or equal to 1, it turns into L1 loss, which helps to reduce the penalty for outliers and thus improve the robustness of the model.

[0067] According to the above three losses, the time step t Total loss at :

[0068] (11)

[0069] in, , , They are the trade-off parameters of the three losses;

[0070] Update the main model using the total loss:

[0071] (12)

[0072] in: is the learning rate;

[0073] is the main model parameter at time t+1;

[0074] Indicates loss The gradient of

[0075] S44: Auxiliary model weight update: After the main model is updated, the weight of the auxiliary model The smoothing factor is updated by combining the weight of the main model with the exponential moving average. The mixing ratio of new and old weights is controlled to ensure a smooth transition of model weights:

[0076] (13).

[0077] As a further limitation of the present invention, the final prediction is based on the main model When entering data predictions on .

[0078] As a further limitation of the present technical solution, the specific steps of S5 are:

[0079] The size of the interval between domains is determined by evaluating the confidence of the prediction. Data augmentation is applied only when , in order to reduce the error accumulation caused by distribution changes:

[0080] (14)

[0081] (15)

[0082] in: To assist the model b Prediction of inputs;

[0083] is the enhanced average prediction of the auxiliary model;

[0084] is the unenhanced prediction of the auxiliary model;

[0085] The final predicted pseudo-label adopted by the auxiliary model;

[0086] represents the prediction of the pre-trained model;

[0087] is the confidence level;

[0088] is the confidence threshold set;

[0089] Using pre-trained models To evaluate the current input sample The prediction confidence of is used to measure the inter-domain gap between the source domain and the target domain;

[0090] Set a confidence threshold: When the prediction confidence exceeds this threshold, directly adopt the model output As a pseudo label, on the contrary, if the confidence does not reach the threshold, the sample is imposed N A random data augmentation technique is used to improve the accuracy of pseudo labels.

[0091] As a further limitation of this technical solution, random recovery, in order to prevent catastrophic forgetting, sets a 10% recovery probability so that the main model recovers a part of neurons from the source model to avoid the loss of source domain knowledge;

[0092] Consider the master model at time step t The convolutional layer after gradient update based on formula (12):

[0093] (16)

[0094] Where: * represents the convolution operation;

[0095] and Represent the input and output of this layer respectively;

[0096] Represents the trainable convolutional layer weights;

[0097] The random recovery method additionally updates the weights by W :

[0098] (17)

[0099] (18)

[0100] in: represents element-wise multiplication;

[0101] p is a small probability of recovery;

[0102] express M Follows a Bernoulli distribution;

[0103] is the initial weight;

[0104] M is a A mask tensor of the same shape;

[0105] The mask tensor determines Which elements in are restored back to the source weights.

[0106] Compared with the prior art, the advantages and positive effects of the present invention are: through dynamic data augmentation technology, the model of the present invention can learn more diverse feature representations, thereby achieving more accurate identification of pests and diseases in unseen fields. The dynamic cross-domain adaptive part during testing allows the model to make real-time adjustments in the changing target domain, maintaining the ability to respond quickly to new situations. These methods not only enhance the generalization ability of the model, but also improve its cross-domain detection performance, significantly improving the model's ability to detect crop pests and diseases in unseen fields, which is of great practical significance for improving crop yield and quality.

[0107] The present invention has made significant progress in pest and disease detection technology through dynamic data augmentation and dynamic cross-domain adaptation during testing. For agricultural practitioners, accurate detection of pests and diseases is crucial, which is related to whether prevention and control measures can be taken in time to avoid losses. The present invention can significantly improve the adaptability of the model to different climates, environments and other conditions, improve the detection ability of pests and diseases, and provide a new method for the cross-domain pest and disease identification field, so that the yield and quality of crops are guaranteed, which is conducive to the realization of precision agriculture and sustainable development, and helps agricultural practitioners to timely discover and prevent pests and diseases, and better deal with the threats posed by pests and diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] Figure 1 This is a schematic diagram of the pest identification model of the present invention.

[0109] Figure 2 This is a schematic diagram of the dynamic data augmentation part of the present invention.

[0110] Figure 3 This is a schematic diagram of the dynamic cross-domain adaptation part during testing of the present invention. DETAILED DESCRIPTION

[0111] A specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation.

[0112] The present invention comprises the following steps:

[0113] S1: Construct a pest and disease recognition model, which consists of two parts: dynamic data augmentation and dynamic cross-domain adaptation during testing.

[0114] like Figure 1 As shown, the augment part is the dynamic data augmentation part, see Figure 2 ,The dynamic data augmentation part consists of a learnable implicit augmenter and a fixed explicit augmenter;

[0115] The implicit augmenter is learnable. After the enhanced image is classified by the classifier, the inter-domain interval loss, diversity promotion prior loss and category loss are calculated. The implicit augmenter is trained by maximizing the inter-domain interval loss and minimizing the diversity promotion prior loss and category loss, while the classifier is trained by minimizing the category loss. The purpose is to generate a cross-domain augmented image with the same label as the original image and maximize the inter-domain interval.

[0116] The explicit augmenter generates images based on fixed Gaussian and Poisson composite noise injection, providing diverse knowledge for the implicit augmenter.

[0117] like Figure 1 As shown in the figure, the Train part and the Predict part are the dynamic cross-domain adaptive parts during testing. The Train part is the training stage, and the Predict part is the model reasoning stage. The dynamic cross-domain adaptive part during testing includes the training stage and the model reasoning stage. The training stage uses YOLO for training. The model reasoning stage is divided into the main model, the auxiliary model and the source model. The main model is updated by the loss calculated by the main model and the auxiliary model, and the auxiliary model is updated by the exponential moving average. In addition, the main model recovers some neurons from the source model with a recovery probability of 10%, and the final result is output by the main model.

[0118] The main goal is to use the given source data Build a pre-trained model Based on the original data set of the new domain, it is further optimized in the target domain so that it can adapt to the ever-changing environment of the new domain. This process does not rely on access to the original data set of the new domain, but achieves performance improvement through continuous adjustment and optimization in the model reasoning stage. Faced with the ever-changing environment, the goal of this invention is to develop a model that can adapt to these changes and continuously perform target detection. Taking the agricultural field as an example, the environment around crops is constantly evolving due to changes in geographical location, climatic conditions and time. In this context, the model needs to make accurate perception decisions in real time and be able to adapt to these changes quickly. The input of the model, without seeing the data of the target domain are presented one by one in chronological order, which means that at any given point in time t , the model can only access the target domain data at that moment. Based on the time point t Target domain data ,Model Real-time predictions are required , while learning and adjusting itself to better cope with new data that may appear later Since data distribution changes dynamically over time, the model needs to have the ability to learn and evaluate online and needs to be able to gradually adapt to changes in data distribution to improve its performance and robustness in unknown target domains.

[0119] An innovative adaptive strategy is proposed to achieve real-time adaptability of the model in a continuously changing target domain, such as Figure 3 As shown in the figure, ⊙ represents element-by-element multiplication and ⊕ represents addition. This method is based on a pre-trained model that has been trained, and through a dynamic adjustment mechanism, it can quickly respond to the constant changes in the target domain. In order to effectively deal with the error accumulation problem that may occur during self-training, weighted average and enhanced average pseudo-label techniques are adopted. These techniques help reduce the accumulation of errors by intelligently adjusting different pseudo-labels, thereby improving the prediction accuracy of the model. In addition, in order to prevent catastrophic forgetting of the model when adapting to the new target domain, we introduced a random recovery mechanism for the source pre-trained model. This strategy helps the model maintain its grasp of the original knowledge by introducing part of the source model knowledge in a timely manner during the training process, avoiding the loss of key information during the adaptation process to the new domain.

[0120] S2: The image generated by the implicit augmenter is inferred by the classifier, and the loss is calculated based on the result and the implicit augmenter and classifier are updated. The images generated by the implicit augmenter and the explicit augmenter are combined as the input for model training.

[0121] S21: Feeding an image into a learnable implicit augmenter The implicit augmenter is designed to achieve feature generalization. Its core is a learnable parameter that is optimized using adversarial learning. Based on the original sequence, it generates diversified cross-domain augmented images under different weather, different environments, and different visibility conditions. Under the guidance of the classifier, the original image is diversified into pest and disease images with different characteristics. This enables the model to extract representations with domain generalization capabilities from the augmented data, thereby improving the personalized processing capabilities of unseen domains. The implicit augmenter is similar to the generator in GAN (generative adversarial network). For a given image sequence, m The goal of the implicit augmenter is to generate a diverse augmented sequence without changing its classification and position information. In order to generate a cross-domain image with sufficient inter-domain interval with the original image, it is proposed to maximize the original image and K Augmented images Inter-domain spacing loss :

[0122] (1)

[0123] represents the Euclidean norm (also known as norm), for vector x , The norm is defined as:

[0124] ;

[0125] Represents the operation of finding the maximum value of all elements in set I;

[0126] To stimulate the diversity of implicit augmenters, diversity is used to promote prior losses , whose normalization factor is :

[0127] (2)

[0128] In order to enable the implicit augmenter to learn the correct parameters, a classifier is introduced, using the cross entropy loss Perform training to ensure that the category labels of the augmented sequence are consistent with those of the original sequence;

[0129] (3)

[0130] in: N To store a given sequence m The correct label cl The size of the label space;

[0131] For the k The result of the second augmentation;

[0132] The goal of the classifier is to minimize the categorical cross entropy loss , accurately distinguish the erroneous data and correct data generated by the implicit augmenter, and learn and update when the implicit augmenter continuously generates new augmented data;

[0133] (4)

[0134] in: is the learnable parameter in the classifier;

[0135] is the learning rate of the classifier;

[0136] Represents the loss function The gradient with respect to the parameters θ; this gradient is used to update the classifier parameters to minimize the loss function;

[0137] Implicit Augmenter The goal is to maximize the inter-domain margin loss , minimize diversity loss and categorical cross entropy loss In the case of , more diverse augmented sequences are generated while maintaining the consistency of the augmented sequences with the original sequence labels. The learnable parameters of the implicit augmenter are updated by the following formula:

[0138] (5)

[0139] Among them: The trade-off parameter is selected as ; ; ;

[0140] Learning Rate ;

[0141] is the learnable parameter of the implicit augmenter;

[0142] Represents the total loss function relative to the parameter The gradient of ; this gradient is used to update the implicit augmenter parameters to minimize the loss function;

[0143] S22: In order to further incorporate additional information into the implicit augmenter, an explicit augmenter is introduced , it is not trained, but it can provide richer data for instance learning and help learn a wider range of invariant features. For natural images, common augmentation strategies include random rotation, flipping and noise injection. By injecting Gaussian noise at fixed intervals, multiple augmented versions with different noise levels are generated. Specifically, given the original sequence m~q(m) The samples in the image are injected with a series of fixed-interval Gaussian and Poisson composite noise to generate samples with different noise levels. C Augmentation:

[0144] (6)

[0145] (7)

[0146] in: represents a random variable;

[0147] Poisson represents Poisson distribution;

[0148] λ is the parameter of Poisson distribution;

[0149] It means that when m is given The conditional probability distribution of ;

[0150] Indicates an estimated value;

[0151] m Indicates expected value;

[0152] represents normal distribution;

[0153] represents the mean of the normal distribution;

[0154] represents the variance of the normal distribution;

[0155] represents the identity matrix, which means Each component of is independent and identically distributed.

[0156] These augmented versions, together with the new domain augmented images generated by the implicit augmenter, provide a rich data source for model training. In this way, the model can not only learn generalized knowledge in the augmented domain, but also perform personalized processing according to the characteristics of different input samples, thereby showing higher recognition accuracy and robustness in practical applications.

[0157] S3: Training phase, using YOLOv9 for training.

[0158] In order to improve the generalization ability of the source model and make it more flexible and adaptable when facing the target domain, a dynamic data augmentation part is introduced in the training stage. The training data is intelligently enhanced by using a learnable augmenter, thereby expanding the model's perception range and learning ability. In this way, the model can not only learn the characteristics of the original data, but also learn more changes and patterns through the augmented data, which helps the model capture a wider range of data distribution. This enhanced data diversity enables the model to respond more quickly and accurately when facing new situations in the target domain. In addition, the learnable augmenter can self-adjust according to the training feedback of the classifier to generate more effective data augmentation and further optimize the model's learning process. This adaptive data augmentation strategy not only improves the model's adaptability to new environments, but also lays a solid foundation for the long-term stable performance of the model in the target domain.

[0159] S4: In the model inference phase, the main model calculates the loss and updates it by the difference in category, confidence, and detection box results between the main model and the auxiliary model, while the auxiliary model is updated by the exponential moving average of the main model.

[0160] The specific process of S4 is as follows:

[0161] During the self-training process, a core goal is to ensure consistency between the model’s predictions and the generated pseudo-labels, usually given the target data. and Model In the case of The cross entropy between the model predictions and the pseudo labels is used to achieve this. However, while directly using the model’s own predictions as pseudo labels works well in static target domains, this approach may be challenging in dynamically changing target domains. This is because the data distribution in the target domain may shift significantly over time, reducing the accuracy of the pseudo labels. To address this issue, we take an innovative approach to generate high-quality pseudo labels. Specifically, we introduce a weighted average auxiliary model , in order to improve the adaptability and generalization ability of the model in the continuously changing target domain. The following are the implementation steps of this method:

[0162] S41: Initialize auxiliary model: In the initial stage of training (time step t =0), auxiliary model Initialized to be the same as the source pre-trained network, ensuring that the model has a reliable starting point;

[0163] S42: Generate pseudo labels: At each time step t , main model and auxiliary models First, predict the target data and generate the main model prediction and pseudo labels ;

[0164] S43: Main Model Update: Main Model Updated by minimizing the loss between the auxiliary model prediction, the loss function The difference between the primary model prediction and the auxiliary model pseudo-label prediction class is calculated:

[0165] (8)

[0166] in: is the prediction of the target category by the auxiliary model pseudo label;

[0167] is the prediction of the main model category;

[0168] Loss Function The difference between the confidence of the primary model and the auxiliary model for the predicted target is calculated:

[0169] (9)

[0170] in: N is the number of prediction boxes;

[0171] and The auxiliary model and the main model are i The confidence level of the prediction;

[0172] Using loss function Calculate the difference between the detection boxes of the primary model and the auxiliary model:

[0173] (10)

[0174] Smooth L 1 loss is often used to train the bounding box regressor of the target because it can effectively handle the difference between the target box and the predicted box. When the absolute value of the difference between the predicted value and the true value is less than 1 (i.e. |x−y|<1|x−y|<1), Smooth L 1 The loss function is expressed as L2 loss, which helps to reduce the model's sensitivity to noise. When the absolute value of the difference is greater than or equal to 1 (i.e. |x−y|≥1|x−y|≥1), it turns into L1 loss, which helps to reduce the penalty for outliers and thus improve the robustness of the model.

[0175] According to the above three losses, the time step t Total loss at :

[0176] (11)

[0177] in, , , They are the trade-off parameters of the three losses;

[0178] Update the main model using the total loss:

[0179] (12)

[0180] in: is the learning rate;

[0181] is the main model parameter at time t+1;

[0182] Indicates loss The gradient of

[0183] S44: Auxiliary model weight update: After the main model is updated, the weight of the auxiliary model The smoothing factor is updated by combining the weights of the main model with the exponential moving average (EMA). The mixing ratio of new and old weights is controlled to ensure a smooth transition of model weights:

[0184] (13).

[0185] In this way, the auxiliary model The prediction can integrate the model information from past iterations, reducing the performance degradation caused by catastrophic forgetting in long-term continuous adaptation. At the same time, the weighted average consistency target makes the model more robust in the continuous adaptation process, effectively reducing the error accumulation caused by data distribution shift, and improving the model's generalization ability to new unseen domains. This strategy not only improves the performance of the model in the new target domain, but also provides a guarantee for the long-term stable performance of the model in a changing environment.

[0186] S5: In the case of cross-domain, the auxiliary model determines the size of the inter-domain gap by evaluating the confidence of the prediction. Data augmentation is performed when the main model is trained through the enhanced average pseudo-labels generated by the auxiliary model, thereby achieving cross-domain recognition.

[0187] The specific steps of S5 are:

[0188] Data augmentation has been widely used in the existing model training process and usually achieves good results. However, the augmentation methods vary for different data sets, and these methods are usually manually selected or automatically searched to adapt to the needs of the data set. Although adding data augmentation during model testing can improve robustness, these strategies are usually determined for specific environments and remain unchanged once determined, without considering distribution changes that may occur during reasoning.

[0189] In a dynamically changing environment, there may be significant distribution differences between the data distribution in the test phase and the source domain, which may cause the original data enhancement strategy to fail to show the expected effect. In order to meet this challenge, this paper proposes an enhancement strategy based on test time domain offset. Specifically, the size of the inter-domain interval is judged by evaluating the confidence of the prediction. Only when a value less than the confidence threshold is detected, the interval between domains is increased. Data augmentation is applied only when , in order to reduce the error accumulation caused by distribution changes:

[0190] (14)

[0191] (15)

[0192] in: To assist the model b Prediction of inputs;

[0193] is the enhanced average prediction of the auxiliary model;

[0194] is the unenhanced prediction of the auxiliary model;

[0195] The final predicted pseudo-label adopted by the auxiliary model;

[0196] represents the prediction of the pre-trained model;

[0197] is the confidence level;

[0198] is the confidence threshold set;

[0199] Using pre-trained models To evaluate the current input sample The prediction confidence of is used to measure the inter-domain gap between the source domain and the target domain;

[0200] A lower confidence reflects a significant gap between the two domains, while a higher confidence implies that the two domains are similar. Based on this assumption, a confidence threshold is set: when the prediction confidence exceeds this threshold, the output of the model is directly adopted. As a pseudo label, on the contrary, if the confidence does not reach the threshold, the sample is imposed N A random data augmentation technique is used to improve the accuracy of pseudo labels.

[0201] This strategy is based on the following considerations: when the confidence level is higher than the confidence threshold In the case of , excessive data augmentation may have a negative impact on model performance. Therefore, the update of the main model depends on these optimized pseudo labels. The advantages of this approach are:

[0202] 1. Dynamic adaptability: Dynamically adjust whether to apply data enhancement based on the current inter-domain interval, rather than keeping it fixed.

[0203] 2. Reduced error: By applying enhancement only when necessary, it avoids applying enhancements when the confidence level is above the confidence threshold. Unnecessary errors may be introduced.

[0204] 3. Improve efficiency: When the confidence level is below the confidence threshold Applying augmentation, you can more effectively utilize augmentation strategies to improve the adaptability and performance of your model.

[0205] With this strategy, we can ensure that the model can apply data augmentation more flexibly and effectively in the face of continuously changing environments.

[0206] Random recovery: To prevent catastrophic forgetting, a 10% recovery probability is set so that the main model can recover some neurons from the source model to avoid the loss of source domain knowledge.

[0207] When adapting to new data, errors are inevitably introduced. This is particularly serious when facing significant domain shifts in the data sequence, because strong distribution changes may lead to inaccurate and misaligned predictions. Self-training may be counterproductive in this case because it will continuously reinforce wrong predictions with wrong pseudo-labels. In addition, after cross-domain adaptation, the model may forget the original knowledge, so that although it has a good effect in the target domain, the detection effect of the adapted model in the source domain may be lower than that of the unadapted model.

[0208] In order to effectively deal with the problem of forgetting, a novel random recovery mechanism is introduced, which aims to reintroduce knowledge from the source pre-trained model. Specifically, considering the main model at time step t The convolutional layer after gradient update based on formula (12):

[0209] (16)

[0210] Where: * represents the convolution operation;

[0211] and Represent the input and output of this layer respectively;

[0212] Represents the trainable convolutional layer weights;

[0213] The random recovery method additionally updates the weights by W :

[0214] (17)

[0215] (18)

[0216] in: represents element-wise multiplication;

[0217] p is a small probability of recovery;

[0218] express M Follows a Bernoulli distribution;

[0219] is the initial weight;

[0220] M is a A mask tensor of the same shape;

[0221] The mask tensor determines In this way, we can consciously recover the lost knowledge from the source model during self-training, so as to enhance the model's ability to adapt to new data while maintaining the understanding of the original data and avoiding catastrophic forgetting caused by over-adaptation.

[0222] S6: The main model outputs the final result. The final prediction is based on the prediction of the main model on the input data.

[0223] Experimental process of the present invention:

[0224] Experimental setup

[0225] The experiment is based on PyTorch2.2.2+cu121 and trained using NVIDIA GeForce RTX 4090. The number of augmentations for implicit augmenters and explicit augmenters K = C =6, , the average number of enhancements during testing N =32, enhanced average threshold , random recovery probability p =0.1, main model updates learning rate Auxiliary model update smoothing factor The initial learning rate is selected as the hyperparameter during training. lr0 =0.04, final learning rate lr0 * lrf =0.04*0.01, batch size batch =64, training rounds epochs =150.

[0226] Dataset

[0227] The dataset uses a self-collected tomato disease dataset, including 4059 training sets, 451 validation sets, and 450 test sets. The categories are five types of images: tomato_healthy, tomato_leaf blight, tomato_leaf curl, tomato_septoria leaf spot, and tomato_verticulium wilt. The test set is expanded on the basis of the original images to include 4 cross-domain images including Brightness, Fog, Frost, and Snow to test the recognition ability of the model in cross-domain situations.

[0228] Baseline model selection and evaluation metrics

[0229] We selected the YOLO series of v8, v8n, v8s, v8m, v8l, v8x, v9c, and v9e9 models as baseline models. We compared the source models and the two improvements in sequence using the Box Precision, Recall, mAP50, and mAP50-95 standards:

[0230] (19)

[0231] in: TP is the number of true positives (i.e., the number of correctly detected bounding boxes);

[0232] FP is the number of false positives (i.e., the number of backgrounds incorrectly identified as targets);

[0233] (20)

[0234] in: FN is the number of false negatives (i.e., the number of objects that the model failed to detect);

[0235] (twenty one)

[0236] in: It is The area under the precision-recall curve for each category;

[0237] is the total number of categories;

[0238] Table 1 Comparison of the results of the baseline model and dynamic data augmentation

[0239] No Augmented Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.678 0.678 0.615 0.692 0.629 0.68 0.632 0.663 Recall↑ 0.689 0.689 0.749 0.681 0.692 0.71 0.705 0.67 mAP50↑ 0.724 0.724 0.734 0.724 0.662 0.743 0.696 0.673 mAP50-95↑ 0.568 0.568 0.575 0.572 0.523 0.582 0.55 0.526 TraditionalAugmented Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.705 0.705 0.669 0.73 0.777 0.743 0.742 0.729 Recall↑ 0.725 0.725 0.749 0.739 0.721 0.748 0.759 0.743 mAP50↑ 0.791 0.791 0.759 0.796 0.799 0.790 0.811 0.766 mAP50-95↑ 0.631 0.631 0.595 0.623 0.633 0.632 0.643 0.612 IE Augmented Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.766 0.766 0.715 0.758 0.793 0.742 0.771 0.733 Recall↑ 0.761 0.761 0.761 0.788 0.759 0.767 0.754 0.766 mAP50↑ 0.824 0.824 0.804 0.835 0.833 0.817 0.828 0.818 mAP50-95↑ 0.654 0.654 0.637 0.658 0.653 0.646 0.659 0.653

[0240] In order to prove the effectiveness of the dynamic data augmenter, the data was trained after traditional data augmentation and compared with the model after adding the data augmenter. It can be observed from Table 1 that after adding the implicit-explicit data augmenter, the Box Precision, Recall, mAP50 and mAP50-95 of the model without augmentation were improved by 0.0971, 0.0665, 0.113, and 0.0938 on average, and 0.164, 0.107, 0.171, and 0.13 on average, respectively. Compared with the augmentation method of simply adding noise, changing contrast and saturation, the Box Precision, Recall, mAP50 and mAP50-95 of the augmented method in the cross-domain case were improved by 0.03, 0.026, 0.035, and 0.027 on average, and 0.061, 0.049, 0.052, and 0.042 on average, respectively. This proves that our augmentation method is effective and has better augmentation effect than traditional augmentation methods. It can enable the model to learn more cross-domain knowledge to improve the model's cross-domain capability and significantly enhance the model's cross-domain recognition capability.

[0241] Table 2 Comparison of the baseline model and cross-domain adaptation when adding tests

[0242] T=0 Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.678 0.678 0.615 0.692 0.629 0.68 0.632 0.663 Recall↑ 0.689 0.689 0.749 0.681 0.692 0.71 0.705 0.67 mAP50↑ 0.724 0.724 0.734 0.724 0.662 0.743 0.696 0.673 mAP50-95↑ 0.568 0.568 0.575 0.572 0.523 0.582 0.55 0.526 T=10 Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.753 0.753 0.683 0.789 0.708 0.769 0.708 0.737 Recall↑ 0.762 0.762 0.818 0.755 0.778 0.786 0.786 0.754 mAP50↑ 0.812 0.812 0.818 0.792 0.739 0.823 0.777 0.764 mAP50-95↑ 0.66 0.658 0.662 0.65 0.601 0.667 0.64 0.608 T=20 Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.793 0.799 0.724 0.843 0.754 0.809 0.755 0.781 Recall↑ 0.801 0.81 0.862 0.804 0.82 0.829 0.831 0.795 mAP50↑ 0.858 0.865 0.867 0.835 0.787 0.864 0.824 0.811 mAP50-95↑ 0.708 0.701 0.704 0.701 0.644 0.717 0.686 0.657 T=30 Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.81 0.817 0.739 0.857 0.771 0.826 0.772 0.794 Recall↑ 0.817 0.83 0.879 0.823 0.838 0.848 0.848 0.814 mAP50↑ 0.871 0.879 0.881 0.848 0.803 0.88 0.838 0.824 mAP50-95↑ 0.725 0.713 0.716 0.714 0.659 0.731 0.701 0.674

[0243] It can be observed from Table 2 that after adding the cross-domain adaptive method during testing, t =0, the detection accuracy of the two methods is the same, and the detection accuracy gradually improves over time, which proves that the model can learn the knowledge of the target domain when tested in unseen fields and improve the detection ability of the model in unseen fields.

[0244] Table 3 Comparison results of pest and disease identification models

[0245] T=0 Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.766 0.766 0.715 0.758 0.793 0.742 0.771 0.733 Recall↑ 0.761 0.761 0.761 0.788 0.759 0.767 0.754 0.766 mAP50↑ 0.824 0.824 0.804 0.835 0.833 0.817 0.828 0.818 mAP50-95↑ 0.654 0.654 0.637 0.658 0.653 0.646 0.659 0.653 T=10 Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.845 0.851 0.798 0.843 0.859 0.816 0.853 0.818 Recall↑ 0.834 0.834 0.832 0.875 0.852 0.837 0.835 0.841 mAP50↑ 0.888 0.888 0.859 0.892 0.896 0.877 0.881 0.877 mAP50-95↑ 0.714 0.709 0.69 0.718 0.717 0.71 0.715 0.71 T=20 Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.89 0.901 0.85 0.891 0.895 0.877 0.902 0.87 Recall↑ 0.887 0.883 0.878 0.925 0.897 0.882 0.883 0.894 mAP50↑ 0.92 0.923 0.893 0.928 0.93 0.914 0.915 0.914 mAP50-95↑ 0.75 0.747 0.725 0.758 0.755 0.744 0.748 0.748 T=30 Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.906 0.916 0.866 0.905 0.911 0.891 0.915 0.885 Recall↑ 0.901 0.898 0.888 0.942 0.91 0.898 0.896 0.909 mAP50↑ 0.931 0.933 0.903 0.936 0.94 0.927 0.927 0.923 mAP50-95↑ 0.761 0.757 0.736 0.767 0.765 0.754 0.759 0.759

[0246] It can be seen from Table 3 that the detection effect of the pest and disease recognition model is better than that of adding only one model improvement mode, and the initial Box Precision, Recall, mAP50 and mAP50-95 of the method of adding data augmentation are at most 0.164, 0.107, 0.171 and 0.13 higher than the source model. As time goes by, the Box Precision, Recall, mAP50 and mAP50-95 of the model with the cross-domain adaptive method added during testing gradually increase, and reach the highest of 0.916, 0.909, 0.936 and 0.767 at t=30. The results show that our data augmenter can enable the model to learn more cross-domain knowledge during training and improve the accuracy of the model's direct detection in unseen fields. The cross-domain adaptive method during testing enables the model to continuously learn relevant knowledge of the target domain when detecting in the target domain to improve the detection ability in the target domain. In addition, we found in the experiment that the source models with different parameter amounts under the same settings performed differently in the cross-domain case. When exceeding a certain threshold, the more parameters there are, the lower the detection accuracy in the cross-domain case, which indicates that the model is overfitting on the data in the source domain. Our test-time cross-domain adaptive method solves this problem well. Even if the model is overfitting on the source domain data, it can be adjusted to the target domain as soon as possible during testing to balance the weights, so that the model can achieve high accuracy in both the source and target domains.

[0247] Table 4 Detection results of the model adapted to the target domain in the source domain

[0248] Yolov8 Yolov8n Yolov8s Yolov8m Yolov8l Yolov8x Yolov9c Yolov9e Precision↑ 0.891 0.896 0.945 0.92 0.93 0.929 0.927 0.931 Recall↑ 0.92 0.919 0.907 0.918 0.932 0.925 0.961 0.933 mAP50↑ 0.952 0.95 0.955 0.953 0.957 0.965 0.968 0.961 mAP50-95↑ 0.787 0.788 0.785 0.801 0.799 0.801 0.807 0.813

[0249] In order to prove that the model does not suffer from catastrophic forgetting while improving the detection capability of the target domain, and does not forget the knowledge learned from the source domain, we tested the model again in the source domain after adapting it to the target domain. The results are shown in Table 4. Compared with the model without cross-domain adaptation, the accuracy of the model only varies by ±2%, which proves that our random recovery effectively prevents catastrophic forgetting.

[0250] The above disclosure is only a specific embodiment of the present invention, but the present invention is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A method for identifying pests and diseases based on image recognition, characterized in that: The following steps are involved: S1: Construct a pest and disease recognition model, which consists of two parts: dynamic data augmentation and dynamic cross-domain adaptation during testing; The dynamic data augmentation part consists of a learnable implicit augmenter and a fixed explicit augmenter; The dynamic cross-domain adaptation part during testing includes the training phase and the model inference phase. The training phase uses YOLO for training, and the model inference phase is divided into the main model, auxiliary model and source model; S2: The image generated by the implicit augmenter is inferred by the classifier, and the loss is calculated based on the result and the implicit augmenter and classifier are updated. The images generated by the implicit augmenter and the explicit augmenter are combined as the input for model training; S3: training phase, using YOLOv9 for training; S4: In the model inference stage, the main model calculates the loss and updates it by the difference between the category, confidence, and detection box results between the main model and the auxiliary model, while the auxiliary model is updated by the exponential moving average of the main model. The main model restores some neurons from the source model with a 10% recovery probability, and the final result is output by the main model. S5: In the case of cross-domain, the auxiliary model determines the size of the inter-domain gap by evaluating the confidence of the prediction. Data augmentation is performed when the main model is trained with the enhanced average pseudo-labels generated by the auxiliary model, thus achieving cross-domain recognition. S6: The main model outputs the final result.

2. The method for identifying pests and diseases based on image recognition according to claim 1, characterized in that: The specific steps of S2 are: S21: Feeding an image into a learnable implicit augmenter Under the guidance of the classifier, the original image is diversified into pest and disease images with different characteristics. For a given image sequence m The goal of the implicit augmenter is to generate a diverse augmented sequence without changing its classification and position information. In order to generate a cross-domain image with sufficient inter-domain interval with the original image, it is proposed to maximize the original image and K Augmented images Inter-domain spacing loss : （1） represents the Euclidean norm; For vector x , The norm is defined as: ； Represents the operation of finding the maximum value of all elements in set I; To stimulate the diversity of implicit augmenters, diversity is used to promote prior losses , whose normalization factor is : （2） In order to enable the implicit augmenter to learn the correct parameters, a classifier is introduced, using the cross entropy loss Perform training to ensure that the category labels of the augmented sequence are consistent with those of the original sequence; （3） in: N To store a given sequence m The correct label cl The size of the label space; For the k The result of the second augmentation; The goal of the classifier is to minimize the categorical cross entropy loss , accurately distinguish the erroneous data and correct data generated by the implicit augmenter, and learn and update when the implicit augmenter continuously generates new augmented data; （4） in: is the learnable parameter in the classifier; is the learning rate of the classifier; Represents the loss function The gradient with respect to the parameter θ; Implicit Augmenter The goal is to maximize the inter-domain margin loss , minimize diversity loss and categorical cross entropy loss In the case of , more diverse augmented sequences are generated while maintaining the consistency of the augmented sequences with the original sequence labels. The learnable parameters of the implicit augmenter are updated by the following formula: （5） Among them: The trade-off parameter is selected as ; ; ; Learning Rate ; is the learnable parameter of the implicit augmenter; Represents the total loss function relative to the parameter The gradient of S22: Introducing an explicit augmenter , by injecting Gaussian noise at fixed intervals, multiple augmented versions with different noise levels are generated. Specifically, given the original sequence m~q(m) The samples in the image are injected with a series of fixed-interval Gaussian and Poisson composite noise to generate samples with different noise levels. C Augmentation: （6） （7） in: represents a random variable; represents Poisson distribution; is the parameter of Poisson distribution; It means that when m is given The conditional probability distribution of ; Indicates an estimated value; m Indicates expected value; represents normal distribution; represents the mean of the normal distribution; represents the variance of the normal distribution; represents the identity matrix, which means Each component of is independent and identically distributed.

3. The method for identifying pests and diseases based on image recognition according to claim 2, characterized in that: In order to improve the generalization ability of the source model and make it more flexible and adaptable when facing the target domain, a dynamic data augmentation part is introduced in the training stage. The training data is intelligently enhanced by using a learnable augmenter, thereby expanding the model's perception range and learning ability.

4. The method for identifying pests and diseases based on image recognition according to claim 3, characterized in that: The specific process of S4 is as follows: S41: Initialize auxiliary model: In the initial stage of training, the auxiliary model Initialized to be the same as the source pre-trained network, ensuring that the model has a reliable starting point; S42: Generate pseudo labels: At each time step t , main model and auxiliary models First, predict the target data and generate the main model prediction and pseudo labels ; S43: Main Model Update: Main Model Updated by minimizing the loss between the auxiliary model prediction, the loss function The difference between the primary model prediction and the auxiliary model pseudo-label prediction class is calculated: （8） in: is the prediction of the target category by the auxiliary model pseudo label; is the prediction of the main model category; Loss Function The difference between the confidence of the primary model and the auxiliary model for the predicted target is calculated: （9） in: N is the number of prediction boxes; and The auxiliary model and the main model are i The confidence level of the prediction; Using loss function Calculate the difference between the detection boxes of the primary model and the auxiliary model: （10） Smooth L 1 loss is often used to train the bounding box regressor of the target because it can effectively handle the difference between the target box and the predicted box. When the absolute value of the difference between the predicted value and the true value is less than 1, Smooth L 1 The loss function is expressed as loss, which helps reduce the model's sensitivity to noise. When the absolute value of the difference is greater than or equal to 1, it turns into loss, which helps to reduce the penalty for outliers and thus improve the robustness of the model. According to the above three losses, the time step t Total loss at : （11） in, , , They are the trade-off parameters of the three losses; Update the main model using the total loss: （12） in: is the learning rate; is the main model parameter at time t+1; Indicates loss The gradient of S44: Auxiliary model weight update: After the main model is updated, the weight of the auxiliary model The smoothing factor is updated by combining the weight of the main model with the exponential moving average. The mixing ratio of new and old weights is controlled to ensure a smooth transition of model weights: （13）。 5. The method for identifying pests and diseases based on image recognition according to claim 4, characterized in that: The final prediction is based on the main model When entering data predictions on .

6. The method for identifying pests and diseases based on image recognition according to claim 4, characterized in that: The specific steps of S5 are: The size of the interval between domains is determined by evaluating the confidence of the prediction. Data augmentation is applied only when , in order to reduce the error accumulation caused by distribution changes: （14） （15） in: To assist the model b Prediction of inputs; is the enhanced average prediction of the auxiliary model; is the unenhanced prediction of the auxiliary model; The final predicted pseudo-label adopted by the auxiliary model; represents the prediction of the pre-trained model; is the confidence level; is the confidence threshold set; Using pre-trained models To evaluate the current input sample The prediction confidence of is used to measure the inter-domain gap between the source domain and the target domain; Set a confidence threshold: When the prediction confidence exceeds this threshold, directly adopt the model output As a pseudo label, on the contrary, if the confidence does not reach the threshold, the sample is imposed N A random data augmentation technique is used to improve the accuracy of pseudo labels.

7. The method for identifying pests and diseases based on image recognition according to claim 4, characterized in that: The main model update in step S43 also includes a random recovery operation. In order to prevent catastrophic forgetting, a 10% recovery probability is set so that the main model recovers a portion of neurons from the source model to avoid the loss of source domain knowledge. Consider the master model at time step t The convolutional layer after gradient update based on formula (12): （16） Where: * represents the convolution operation; and Represent the input and output of this layer respectively; Represents the trainable convolutional layer weights; The random recovery method additionally updates the weights by W : （17） （18） in: represents element-wise multiplication; p is a small probability of recovery; express M Follows a Bernoulli distribution; is the initial weight; M is a A mask tensor of the same shape; The mask tensor determines Which elements in are restored back to the source weights.

Images