A method for image recognition of crop pests based on causal intervention

Abstract

This invention relates to a causal intervention-based image recognition method for crop pests, which overcomes the shortcomings of existing technologies, such as models focusing on features unrelated to pests, poor robustness, and low recognition accuracy. The invention includes the following steps: acquisition and preprocessing of crop pest images; construction of a causal intervention pest recognition framework; training of the causal intervention pest recognition framework; acquisition of images of crop pests to be identified; and obtaining the crop pest image recognition results. This invention employs a causal intervention strategy, which can effectively influence the distribution of the training dataset, thereby improving the recognition accuracy of the model in different training domains and enhancing the generalization ability and recognition accuracy of deep learning models.

Description

Technical Field

[0001] This invention relates to the field of agricultural image recognition technology, specifically a method for recognizing crop pest images based on causal intervention. Background Technology

[0002] Currently, the main methods for identifying crop pests can be divided into three categories: methods based on traditional human experts, methods based on machine learning, and methods based on deep learning.

[0003] Traditional pest identification methods based on human experts typically suffer from poor real-time performance, low efficiency, and significant subjectivity. Machine learning-based methods usually require specific manual intervention for feature design and generally only produce better results under limited and constrained conditions, which is not conducive to practical agricultural environments. With the development of deep learning technology, the use of deep learning models for pest identification, such as Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), has been extensively studied in the field of pest identification.

[0004] However, existing methods often overlook the inherent problem of the distribution of pest training datasets. When a pest species typically inhabits rice paddies, most images of that species are captured within the same background scene. Therefore, the collected training dataset will inevitably contain a large number of samples with leaves as the background. In this case, the model may focus not only on the pests during training but also on irrelevant environmental attributes such as leaf color and texture. This makes it easy for deep learning models to learn background features unrelated to the pests, resulting in poor robustness and low accuracy in crop pest identification methods.

[0005] Therefore, improving the objectivity, robustness, and accuracy of crop pest identification methods has become an urgent technical problem to be solved. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies, such as models being prone to focusing on features unrelated to pests, poor robustness, and low recognition accuracy, by providing a crop pest image recognition method based on causal intervention to solve the above problems.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] A method for image recognition of crop pests based on causal intervention includes the following steps:

[0009] Acquisition and preprocessing of crop pest images: Acquire several crop pest images, scale the images to 224*224 size, and divide them into training and validation sets according to a 7:3 ratio;

[0010] Construction of a causal intervention pest identification framework: A causal intervention pest identification framework is constructed through a model pre-training layer, a dual training domain, a category core feature learning module, and a fully connected sub-network.

[0011] Training of the causal intervention pest identification framework: The training set is input into the causal intervention pest identification framework for training, and the validation set is used for validation to obtain the optimal result;

[0012] Acquisition of images of crop pests to be identified: Acquire images of crop pests to be identified and perform preprocessing;

[0013] Obtaining crop pest image recognition results: Input the crop pest images to be identified into the trained causal intervention pest recognition framework to obtain crop pest image recognition results.

[0014] The construction of the causal intervention pest identification framework includes the following steps:

[0015] A causal intervention pest identification framework based on a deep learning model architecture is established, which consists of four layers:

[0016] The first layer of the causal intervention pest identification framework is the model pre-training layer, which is used to initialize the parameters of the deep learning model and perform ten generations of pre-training using optimization algorithms and cross-entropy loss.

[0017] The second layer of the causal intervention pest identification framework is to construct a dual training domain. The dual training domain is based on the principle of causal intervention and is constructed by implementing different training sample weight allocation strategies to construct two training domains, namely the original distribution training domain and the inverse distribution training domain.

[0018] The third layer of the causal intervention pest identification framework is the category core feature learning module. The category core feature learning module uses the center ternary loss to enhance and optimize the deep learning model's ability to learn category core features in the dual training domain.

[0019] The fourth layer of the causal intervention pest identification framework is a fully connected subnetwork of deep learning model parameters, and the output of the fully connected subnetwork is connected to the Softmax classifier.

[0020] Set up a dual training domain:

[0021] Construct the original distribution training domain:

[0022] In the original distribution training domain, the distribution of the original training data remains unchanged, thereby encouraging the model to learn stably. The sampling weight of each sample is allocated based on the classification confidence score S obtained in the previous round of training. The probability of selecting each sample is proportional to its weight. The original distribution training domain consists of training samples drawn from each class according to the sampling weight.

[0023] Constructing the inverse distribution training domain:

[0024] In the inverse distribution training domain, a difference is formed from the original distribution training domain. The inverse distribution training domain contains images that were misclassified in the previous round, so that the model can learn from samples that are significantly different from the original distribution.

[0025] The sampling weight of correctly classified samples in the inverse distribution training domain is set to 0 and they will not be added to the inverse distribution training domain. The remaining misclassified samples are weighted according to the classification confidence score 1-S of the previous round. This means that the greater the probability of a sample being misclassified, the greater the probability of it being sampled into the inverse distribution training domain. Based on this process, the construction of the inverse distribution training domain is realized.

[0026] The category core feature learning module is designed to enhance and optimize the deep learning model’s ability to learn category core features in the dual training domain by employing a center ternary loss.

[0027] Loading dual training domain samples:

[0028] The original distributed training domain image samples and the inverse distributed training domain samples are loaded together and merged into a new training set. The new training set contains samples with different distribution characteristics and is optimized using the center ternary loss function to ensure that the model learns the core features of the category under different distribution conditions.

[0029] Define the central ternary loss function:

[0030] The goal of setting the center ternary loss function is to bring the embeddings of a sample closer to its positive sample center, while pushing the embeddings of a sample further away from its negative sample center. A positive sample center refers to the average representation of features from the same class as the sample, while a negative sample center refers to the average representation of features from the class with the largest confidence difference from the sample. Samples of the same class cluster together in the feature space, while samples of different classes are separated.

[0031] Central ternary loss function L tri The formula is as follows:

[0032]

[0033] Where i represents the current input image sample, φ(i) represents the image features extracted by the backbone model, N represents the number of training images, and y +E(y) represents the class corresponding to sample i. + ) represents the class center feature corresponding to sample i, y - E(y) represents the category of the worst-case negative label. - ) represents the class center of the worst predicted negative label, and m is a positive margin constant used to control the distance between samples of the same class and their corresponding class centers.

[0034] The training of the causal intervention pest identification framework includes the following steps:

[0035] The training set is input into the pre-training layer of the model of the causal intervention pest identification framework. At this stage, all training set images are loaded and the SGD optimization algorithm and classification cross-entropy loss are used to perform ten generations of pre-training. Each generation of training is used to validate the deep learning model of the causal intervention pest identification framework with the validation set and the model weights with the best accuracy are saved.

[0036] The classification confidence of image samples in the training set is calculated based on the optimal model weights output by the model pre-training layer. A dual training domain is constructed based on the calculated classification confidence. The dual training domains are then merged and input into the core feature learning module for feature learning. Finally, the model is optimized based on the loss function.

[0037] Define the total loss function L of the causal intervention pest identification framework. total ,

[0038] L total =L cls +λL tri ,

[0039] Among them, L cls L represents the classification cross-entropy loss. tri The centered ternary loss is denoted by λ, which represents the hyperparameter used to weigh the loss.

[0040] Construction of dual training domains: The classification confidence of the training set images is calculated using the optimal model weights output by the model pre-training layer. The model classification confidence is used as the sampling weight of the images. Based on the sampling weights, images are sampled and output as original distribution training domain image samples and inverse distribution training domain samples.

[0041] Training of the category core feature learning module: The output samples from the dual training domains are merged and input into the category core feature learning module;

[0042] Training of a fully connected network: The sample features output by the category core feature learning module are input into the fully connected layer. The output of the fully connected layer is the score for each category. These scores are obtained by multiplying the input features and the weight matrix, plus a bias term.

[0043] The Softmax classifier transforms the output of the fully connected layer into a probability distribution for each class and selects the class with the highest probability as the predicted classification result.

[0044] Based on the classification cross-entropy loss function and the center ternary loss function of the category core feature learning module of the fully connected network, the loss functions of the predicted results and the true results are calculated. The model performance is monitored using the validation set, and the model weight file of the best recognition result on the validation set is saved. The network weights are optimized based on the loss function of the training set and the gradient descent algorithm until the set total number of iterations is reached or until the performance on the validation set no longer improves.

[0045] The construction of the dual training domain includes the following steps:

[0046] Calculate the confidence score: For each pest image in the training dataset, use the best model weights output by the model's pre-trained layers to predict its category and calculate the confidence score S of the prediction.

[0047] Sample weight allocation: The calculated confidence score S is used as the sampling weight for each training sample;

[0048] Construction of the original distribution training domain: The training set is sampled with replacement according to the sample weights. The sampling with replacement step refers to a fixed probability that a sample will be sampled, and that it can be resampled after being sampled. The criterion for resampling is that samples with higher weights have a higher probability of being selected.

[0049] The benchmark for resampling in the original distribution training domain is set as a high probability of selecting samples with high weights. Based on this benchmark, a training domain that maintains the original data distribution characteristics is constructed, namely the original distribution training domain.

[0050] Sample sampling weight reallocation: For correctly classified samples, their weights are set to 0 to exclude them from the inverse distribution training domain; for misclassified samples, the inverse confidence score 1-S is used as their new weights.

[0051] Construction of the inverse distribution training domain: For samples that are misclassified, the weights 1-S are reallocated, and sampling with replacement is performed again. The benchmark for resampling in the inverse distribution training domain is that samples with higher weights have a higher probability of being selected. This biases the selection of samples that are seriously misclassified by the model, thereby constructing the inverse distribution training domain.

[0052] The training of the category core feature learning module includes the following steps:

[0053] Loading dual training domain samples: Loading and merging the original distributed training domain image samples and the inverse distributed training domain samples into a new training set;

[0054] Calculate the class center feature E(y) corresponding to the current training sample. + The process involves selecting all samples of the same class as the given sample from the dual training domain, extracting their features, and then calculating the mean of the features.

[0055] Calculate the class center feature E(y) of the worst predicted negative label for the current training sample. - The sample class with the largest difference in prediction confidence from the sample in the dual training domain is selected to extract features, and then the mean of the features is calculated.

[0056] The center ternary loss is used to minimize the distance between samples from the same class and their class centers, while maximizing the distance between the feature centers of negative sample classes and the samples, in order to maintain tight clustering of intra-class feature spaces and separation of inter-class feature spaces in pest images.

[0057] Beneficial effects

[0058] The present invention provides a method for image recognition of crop pests based on causal intervention. Compared with the prior art, this method can effectively intervene in the distribution of the training dataset by adopting a causal intervention strategy, thereby improving the recognition accuracy of the model in different training domains and enhancing the generalization ability and recognition accuracy of the deep learning model.

[0059] This invention reconstructs the original training dataset into two distinct training domains through domain construction: the original distribution training domain and the inverse distribution training domain. In the original distribution training domain, the model is trained to learn the core features of the categories, while in the inverse distribution domain, the model is trained to focus more on misclassified samples. This effectively reduces the model's dependence on environmental features and improves recognition accuracy.

[0060] Furthermore, this invention employs a center ternary loss to optimize the model's feature learning process. By minimizing the distance between each sample and its corresponding class center, while maintaining tight clustering of intra-class feature spaces and effective separation of inter-class feature spaces, the model can focus more on learning the core features of the class, rather than subtle differences in the background environment.

[0061] This method enables accurate identification of crop pests and improves the robustness and generalization ability of the model. Attached Figure Description

[0062] Figure 1 This is a sequence diagram of the method of the present invention;

[0063] Figure 2 A schematic diagram of a deep convolutional neural network training framework based on causal intervention provided in an embodiment of the present invention;

[0064] Figure 3Visualization results of feature learning under different methods provided in the embodiments of the present invention. Detailed Implementation

[0065] To provide a better understanding of the structural features and effects achieved by the present invention, a detailed description is provided below, accompanied by preferred embodiments and accompanying drawings:

[0066] like Figure 1 As shown, the present invention provides a method for image recognition of crop pests based on causal intervention, comprising the following steps:

[0067] The first step is the acquisition and preprocessing of crop pest images: Acquire several crop pest images, scale the images to 224*224 pixels, and divide them into training and validation sets in a 7:3 ratio.

[0068] The second step is to construct a causal intervention framework for pest identification: such as... Figure 2 As shown, a causal intervention pest identification framework is constructed through a model pre-training layer, a dual training domain, a category core feature learning module, and a fully connected sub-network.

[0069] With the rapid development of causal reasoning in computer vision, various causal intervention methods are being widely used in image classification tasks. Due to limitations in obtaining pest datasets, the collected training datasets will inevitably be in the same environment. In this case, the model may focus not only on pests during training but also on irrelevant environmental attributes, such as the color and texture of leaves on plants damaged by pests. To prevent deep learning models from being affected by the dataset distribution when identifying pests, a neural network training framework based on causal intervention is constructed to achieve more accurate pest image classification.

[0070] The specific steps for constructing the causal intervention pest identification framework are as follows:

[0071] (1) A causal intervention pest identification framework is set up based on a deep learning model architecture. The causal intervention pest identification framework consists of four layers:

[0072] The first layer of the causal intervention pest identification framework is the model pre-training layer, which is used to initialize the parameters of the deep learning model and perform ten generations of pre-training using optimization algorithms and cross-entropy loss.

[0073] The second layer of the causal intervention pest identification framework is to construct a dual training domain. The dual training domain is based on the principle of causal intervention and is constructed by implementing different training sample weight allocation strategies to construct two training domains, namely the original distribution training domain and the inverse distribution training domain.

[0074] The third layer of the causal intervention pest identification framework is the category core feature learning module. The category core feature learning module uses the center ternary loss to enhance and optimize the deep learning model's ability to learn category core features in the dual training domain.

[0075] The fourth layer of the causal intervention pest identification framework is a fully connected subnetwork of deep learning model parameters, and the output of the fully connected subnetwork is connected to the Softmax classifier.

[0076] (2) Setting up dual training domains: Original distribution training domain (Domain 1): In this domain, the distribution of the original training data remains unchanged. The weight of each sample is directly assigned based on the classification confidence score S obtained in the previous round of training. The probability of selecting each sample is proportional to its weight. Domain 1 consists of a combination of samples selected from each class. Inverse distribution training domain (Domain 2): The purpose of this domain is different from Domain 1, containing more misclassified images. Therefore, the weight of correctly classified samples is set to 0 and is not added to Domain 2. The remaining misclassified samples are weighted according to the inverse classification confidence score 1-S, thus constructing an inverse distribution domain mainly dominated by misclassified samples.

[0077] The principle of causal intervention is used to address pest identification problems caused by bias in the training dataset distribution. Causal intervention is a statistical technique that consciously adjusts specific variables to observe their impact on the outcome, aiming to identify genuine causal relationships. In the context of pest identification, the goal of causal intervention is to reduce misclassification due to dataset bias. To mitigate the impact of inherent dataset bias, a dual training domain is constructed by altering the probability of sample selection: an original distribution training domain and an inverse distribution training domain. By minimizing the central ternary loss, the model is encouraged to capture key features that consistently appear across all samples and represent the class, rather than relying on secondary features or noise specific to a particular sample.

[0078] A1) Construct the original distribution training domain:

[0079] In the original distribution training domain, the distribution of the original training data remains unchanged, thereby encouraging the model to learn stably. The sampling weight of each sample is allocated based on the classification confidence score S obtained in the previous round of training. The probability of selecting each sample is proportional to its weight. The original distribution training domain consists of training samples drawn from each class according to the sampling weight.

[0080] A2) Construct the inverse distribution training domain:

[0081] In the inverse distribution training domain, a difference is formed from the original distribution training domain. The inverse distribution training domain contains images that were misclassified in the previous round, so that the model can learn from samples that are significantly different from the original distribution.

[0082] The sampling weight of correctly classified samples in the inverse distribution training domain is set to 0 and they are not added to the inverse distribution training domain. The remaining misclassified samples are weighted according to the classification confidence score 1-S of the previous round. This means that the greater the probability of a sample being misclassified, the greater the probability of it being sampled into the inverse distribution training domain. Based on this process, the inverse distribution training domain is constructed.

[0083] (3) Setting up a category core feature learning module: The category core feature learning module uses center ternary loss to enhance and optimize the deep learning model’s ability to learn category core features in the dual training domain;

[0084] B1) Load dual training domain samples:

[0085] The original distributed training domain image samples and the inverse distributed training domain samples are loaded together and merged into a new training set. The new training set contains samples with different distribution characteristics and is optimized using the center ternary loss function to ensure that the model learns the core features of the category under different distribution conditions.

[0086] B2) Define the central ternary loss function:

[0087] The goal of setting the center ternary loss function is to bring the embeddings of a sample closer to its positive sample center, while pushing the embeddings of a sample further away from its negative sample center. A positive sample center refers to the average representation of features from the same class as the sample, while a negative sample center refers to the average representation of features from the class with the largest confidence difference from the sample. Samples of the same class cluster together in the feature space, while samples of different classes are separated.

[0088] Central ternary loss function L tri The formula is as follows:

[0089]

[0090] Where i represents the current input image sample, φ(i) represents the image features extracted by the backbone model, N represents the number of training images, and y + Represents the class corresponding to sample i, e(y) + ) represents the class center feature corresponding to sample i, y - E(y) represents the category of the worst-case negative label. - ) represents the class center of the worst predicted negative label, and m is a positive margin constant used to control the distance between samples of the same class and their corresponding class centers.

[0091] The third step is to train the causal intervention pest identification framework: input the training set into the causal intervention pest identification framework for training, and use the validation set for validation to obtain the optimal result.

[0092] (1) The training set is input into the pre-training layer of the model of the causal intervention pest identification framework. In this stage, all training set images are loaded and the SGD optimization algorithm and classification cross-entropy loss are used for ten generations of pre-training. Each generation of training is used to validate the deep learning model of the causal intervention pest identification framework with the validation set and save the model weights with the best accuracy. First, ten generations of pre-training are performed. Pre-training only uses classification cross-entropy loss optimization because a suboptimal classifier is needed to obtain a better confidence score. Then, different training domains are sampled. After pre-training, the construction of dual training domains and optimization of center ternary loss are performed.

[0093] The classification confidence of image samples in the training set is calculated based on the optimal model weights output by the model pre-training layer. A dual training domain is constructed based on the calculated classification confidence. The dual training domains are then merged and input into the core feature learning module for feature learning. Finally, the model is optimized based on the loss function.

[0094] (2) Define the total loss function L of the causal intervention pest identification framework. total ,

[0095] L total =L cls +λL tri ,

[0096] Among them, L cls L represents the classification cross-entropy loss. tri The centered ternary loss is denoted by λ, which represents the hyperparameter used to weigh the loss.

[0097] (3) Construction of dual training domains: The classification confidence of the training set images is calculated using the optimal model weights output by the model pre-training layer. The model classification confidence is used as the sampling weight of the images. Based on the sampling weights, the images are sampled and output as original distribution training domain image samples and inverse distribution training domain samples.

[0098] C1) Calculate the confidence score: For each pest image in the training dataset, use the best model weights output by the model's pre-trained layers to predict its category and calculate the confidence score S of the prediction, which reflects the model's confidence in the prediction result.

[0099] C2) Sample weight allocation: The calculated confidence score S is used as the sampling weight for each training sample. The sampling weight will be used in the subsequent sample selection process to construct the training domain.

[0100] C3) Construction of the original distribution training domain: The training set is sampled with replacement according to the sample weights. The sampling with replacement step refers to a fixed probability that a sample will be sampled, and that it can be resampled after being sampled. The criterion for resampling is that samples with higher weights have a higher probability of being selected.

[0101] The benchmark for resampling in the original distribution training domain is set as a high probability of selecting samples with high weights. Based on this benchmark, a training domain that maintains the original data distribution characteristics is constructed, namely the original distribution training domain.

[0102] C4) Sample sampling weight reallocation: For correctly classified samples, set their weights to 0 to exclude them from the inverse distribution training domain; for misclassified samples, use the inverse confidence score 1-S as their new weights.

[0103] C5) Construction of the inverse distribution training domain: For samples that are misclassified, the weights 1-S are reallocated, and sampling with replacement is performed again. The benchmark for resampling in the inverse distribution training domain is that samples with higher weights have a higher probability of being selected. This biases the selection of samples that are seriously misclassified by the model, thereby constructing the inverse distribution training domain.

[0104] At the algorithm level, the indexes of correctly predicted samples are first determined: The stored index information and stored model prediction probabilities are used to obtain the indexes of correctly predicted samples. These correctly predicted sample indices are stored in `correct_index`. Sample sampling weight initialization: Two tensors, `domain1_score` and `domain2_score`, with a size equal to the total number of training samples, are created and initialized to zero. The length of these tensors is equal to the total number of images. Construction of the original distribution training domain: The sampling weights corresponding to the samples are set to the confidence scores obtained from the previous model iteration, and these scores are assigned to `domain1_score`. Construction of the inverse distribution training domain: For correctly classified samples, their weights are set to 0 so they are not included in the inverse distribution training domain. For misclassified samples, `domain2_score` is set to 1.0 - `domain1_score` as their new weights. The updated sampling weight information is applied to the data loader, and samples are sampled with replacement according to the sampling weights to construct a dual-domain training set.

[0105] (4) Training of the category core feature learning module: The output samples of the dual training domains are merged and input into the category core feature learning module.

[0106] D1) Loading dual training domain samples: Load and merge the original distributed training domain image samples and the inverse distributed training domain samples into a new training set; the new training set contains samples with different distribution characteristics, enabling the model to be trained on more diverse data, thereby improving the model's generalization ability to identify pests under different environments and conditions.

[0107] D2) Calculate the class center feature E(y) corresponding to the current training sample. + The process involves selecting all samples of the same class as the given sample from the dual training domain, extracting their features, and then calculating the mean of the features.

[0108] D3) Calculate the class center feature E(y) of the worst predicted negative label for the current training sample. - The sample class with the largest difference in prediction confidence from the sample in the dual training domain is selected to extract features, and then the mean of the features is calculated.

[0109] D4) The center ternary loss is used to minimize the distance between samples from the same class and their class centers, while maximizing the distance between the feature centers of negative sample classes and the samples themselves, thus maintaining tight clustering of intra-class feature spaces and separation of inter-class feature spaces in the pest image. The calculation of class feature centers is achieved by taking the arithmetic mean of the feature vectors of all samples within that class.

[0110] (5) Training of the fully connected network: The sample features output by the category core feature learning module are input into the fully connected layer. The output of the fully connected layer is the score of each category. These scores are obtained by multiplying the input features and weight matrix and adding the bias term.

[0111] The Softmax classifier transforms the output of the fully connected layer into a probability distribution for each class and selects the class with the highest probability as the predicted classification result.

[0112] Based on the classification cross-entropy loss function and the center ternary loss function of the category core feature learning module of the fully connected network, the loss functions of the predicted results and the true results are calculated. The model performance is monitored using the validation set, and the model weight file of the best recognition result on the validation set is saved. The network weights are optimized based on the loss function of the training set and the gradient descent algorithm until the set total number of iterations is reached or until the performance on the validation set no longer improves.

[0113] The fourth step is to acquire images of the crop pests to be identified: acquire images of the crop pests to be identified and perform preprocessing.

[0114] The fifth step is to obtain the image recognition results of crop pests: input the images of crop pests to be identified into the trained causal intervention pest recognition framework to obtain the image recognition results of crop pests.

[0115] In the task of image recognition of crop pests, a series of metrics are used to evaluate the model's performance, including accuracy (ACC), precision, F1 score (F1), and geometric mean (GM). Tables 1, 2, and 3 show the evaluation metrics of the model. The evaluation metrics used are described below:

[0116] A. Accuracy (ACC): Accuracy is one of the simplest and most intuitive evaluation metrics. It represents the ratio of the number of samples correctly predicted by the model to the total number of samples.

[0117] B. Precision: Precision measures the proportion of true positives among samples predicted as positive by the model.

[0118] C. F1 score (F1): The F1 score combines precision and recall, and is a comprehensive evaluation index that is suitable for imbalanced class distributions.

[0119]

[0120] Here, precision represents accuracy, and recall represents recall.

[0121] D. Geometric Mean (GM): The geometric mean is the result of taking the square root of the product of multiple values. It is usually used to calculate the average performance of a classification model across different categories.

[0122]

[0123] Among them, Precision i is the precision for the i-th class, and n is the number of classes. Table 1-3 shows the comparison results on LiDataset, DFSPD, and IP102. The best results are highlighted in bold.

[0124] Table 1: Quantitative results based on the Li dataset

[0125]

[0126] Table 2: Quantitative Results Based on the DFSPD Dataset

[0127]

[0128] Table 3: Quantitative Results Based on the IP102 Dataset

[0129]

[0130]

[0131] Tables 1 and 2 show that when deep learning models are combined with the causal pest identification framework, the recognition performance of all baseline models is significantly improved, highlighting the crucial role of the causal pest identification framework. The ResNet-50 model demonstrates superior performance on both datasets, achieving accuracies of 94.41% and 90.19%, respectively, which are further improved to 95.33% and 92.59% after incorporating the causal pest identification framework. This highlights that the framework further enhances the classification capabilities of efficient networks, demonstrating its effectiveness in improving baseline network performance and mitigating the impact of training data distribution bias.

[0132] Table 3 shows that on the large-scale pest dataset IP102, the basic ResNet-50 model is competitive with some other methods, but still lags behind. However, the ResNet-50 model incorporating the causal intervention pest identification framework achieves the best performance in GM, F1, and ACC metrics, at 57.34%, 64.93%, and 70.60%, respectively, further demonstrating the effectiveness of the causal intervention pest identification framework.

[0133] like Figure 3 As shown, this is a visualization of the features the model focuses on under different methods. The areas biased towards red represent the parts the model considers more important. In the comparison on the left, it can be observed that when the images in the test set match the distribution of the training set, the baseline model focuses on environmental features in addition to the core class features. This behavior is attributed to the biased distribution of the training data. In contrast, the method of this invention ensures that the model maintains its focus on the core class features. In the comparison on the right, when a distribution shift occurs in the test set (the images in the test set are significantly different from those in the training set), the baseline model mainly focuses on mixed background features, but the method of this invention is unaffected in this case and continues to demonstrate stable superiority.

[0134] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for image recognition of crop pests based on causal intervention, characterized in that, Includes the following steps: 11) Acquisition and preprocessing of crop pest images: Acquire several crop pest images, scale the images to 224*224 size, and divide them into training and validation sets according to a 7:3 ratio; 12) Construction of the causal intervention pest identification framework: The causal intervention pest identification framework is constructed through model pre-training layer, dual training domain, category core feature learning module, and fully connected sub-network. The construction of the causal intervention pest identification framework includes the following steps: 121) Based on a deep learning model architecture, a causal intervention pest identification framework is established, comprising four layers: The first layer of the causal intervention pest identification framework is the model pre-training layer, which is used to initialize the parameters of the deep learning model and perform ten generations of pre-training using optimization algorithms and cross-entropy loss. The second layer of the causal intervention pest identification framework is to construct a dual training domain. The dual training domain is based on the principle of causal intervention and is constructed by implementing different training sample weight allocation strategies to construct two training domains, namely the original distribution training domain and the inverse distribution training domain. The third layer of the causal intervention pest identification framework is the category core feature learning module. The category core feature learning module uses the center ternary loss to enhance and optimize the deep learning model's ability to learn category core features in the dual training domain. The fourth layer of the causal intervention pest identification framework is a fully connected subnetwork of deep learning model parameters, and the output of the fully connected subnetwork is connected to the Softmax classifier. 122) Set up a dual training domain: 1221) Construct the original distribution training domain: In the original distribution training domain, the distribution of the original training data remains unchanged, thereby encouraging the model to learn stably. The sampling weight of each sample is allocated based on the classification confidence score S obtained in the previous round of training. The probability of selecting each sample is proportional to its weight. The original distribution training domain consists of training samples drawn from each class according to the sampling weight. 1222) Construct the inverse distribution training domain: In the inverse distribution training domain, a difference is formed from the original distribution training domain. The inverse distribution training domain contains images that were misclassified in the previous round, so that the model can learn from samples that are significantly different from the original distribution. The sampling weight of correctly classified samples in the inverse distribution training domain is set to 0 and they are not added to the inverse distribution training domain. The remaining misclassified samples are weighted according to the classification confidence score 1 - S of the previous round. This means that the greater the probability of a sample being misclassified, the greater the probability of it being sampled into the inverse distribution training domain. Based on this process, the inverse distribution training domain is constructed. 123) Setting up a category core feature learning module: The category core feature learning module uses center ternary loss to enhance and optimize the deep learning model's ability to learn category core features in the dual training domain; 1231) Load dual training domain samples: The original distributed training domain image samples and the inverse distributed training domain samples are loaded together and merged into a new training set. The new training set contains samples with different distribution characteristics and is optimized using the center ternary loss function to ensure that the model learns the core features of the category under different distribution conditions. 232) Define the central ternary loss function: The goal of setting the center ternary loss function is to bring the embeddings of a sample closer to its positive sample center, while pushing the embeddings of a sample further away from its negative sample center. A positive sample center refers to the average representation of features from the same class as the sample, while a negative sample center refers to the average representation of features from the class with the largest confidence difference from the sample. Samples of the same class cluster together in the feature space, while samples of different classes are separated. Central ternary loss function The formula is as follows: ， Where i represents the current input image sample, The image features extracted by the backbone model are represented by N, where N represents the number of training images. Indicates the class corresponding to sample i. This represents the class center feature corresponding to sample i. The category representing the negative label of the worst prediction. The class center represents the negative label of the worst prediction, and m is a positive margin constant used to control the distance between samples of the same class and their corresponding class centers; 13) Training of the causal intervention pest identification framework: The training set is input into the causal intervention pest identification framework for training, and the validation set is used for validation to obtain the optimal result; The training of the causal intervention pest identification framework includes the following steps: 131) Input the training set into the model pre-training layer of the causal intervention pest identification framework. In this stage, load all training set images and perform ten generations of pre-training using the SGD optimization algorithm and classification cross-entropy loss. For each generation of training, use the validation set to validate the deep learning model of the causal intervention pest identification framework and save the model weights with the best accuracy. The classification confidence of image samples in the training set is calculated based on the optimal model weights output by the model pre-training layer. A dual training domain is constructed based on the calculated classification confidence. The dual training domains are then merged and input into the core feature learning module for feature learning. Finally, the model is optimized based on the loss function. 132) Define the total loss function of the causal intervention pest identification framework. , ， in, Represents the classification cross-entropy loss. The centered ternary loss is denoted by λ, which represents the hyperparameter used to weigh the loss. 133) Construction of dual training domains: Calculate the classification confidence of training set images using the best model weights output by the model pre-training layer, use the model classification confidence as the sampling weight of the images, and output the original distribution training domain image samples and the inverse distribution training domain samples according to the sampling weights. 134) Training of the category core feature learning module: The output samples of the dual training domains are merged and input into the category core feature learning module; The training of the category core feature learning module includes the following steps: 1341) Loading dual training domain samples: Loading and merging the original distributed training domain image samples and the inverse distributed training domain samples together into a new training set; 1342) Calculate the class center features corresponding to the current training sample. The features of all samples of the same class as the sample in the dual training domain are selected and extracted, and then the mean of the features is calculated. 1343) Calculate the class center features of the worst predicted negative label for the current training sample. The sample class with the largest difference in prediction confidence from the sample in the dual training domain is selected to extract features, and then the feature mean is calculated. 1344) The center ternary loss is used to minimize the distance between samples from the same class and their class centers, while maximizing the distance between the feature centers of negative sample classes and the samples, in order to maintain the tight clustering of intra-class feature spaces and the separation of inter-class feature spaces in pest images. 135) Training of fully connected networks: The sample features output by the category core feature learning module are input into the fully connected layer. The output of the fully connected layer is the score of each category. These scores are obtained by multiplying the input features and weight matrix and adding a bias term. The Softmax classifier transforms the output of the fully connected layer into a probability distribution for each class and selects the class with the highest probability as the predicted classification result. Based on the classification cross-entropy loss function and the center ternary loss function of the category core feature learning module of the fully connected network, the loss functions of the predicted results and the true results are calculated. The model performance is monitored using the validation set and the model weight file of the best recognition results on the validation set is saved. The network weights are optimized based on the loss function of the training set and the gradient descent algorithm until the set total number of iterations is reached or until the performance on the validation set no longer improves. 14) Acquisition of images of crop pests to be identified: Acquire images of crop pests to be identified and perform preprocessing; 15) Obtaining the image recognition results of crop pests: Input the images of crop pests to be identified into the trained causal intervention pest recognition framework to obtain the image recognition results of crop pests.

2. The method for image recognition of crop pests based on causal intervention according to claim 1, characterized in that, The construction of the dual training domain includes the following steps: 21) Calculate the confidence score: For each pest image in the training dataset, use the best model weights output by the model pre-trained layer to predict its category and calculate the confidence score S of the prediction. 22) Sample weight allocation: The calculated confidence score S is used as the sampling weight for each training sample; 23) Construction of the original distribution training domain: Perform a sampling step with replacement on the training set according to the sample weights. The sampling step with replacement refers to a fixed probability for each sample to be sampled, and setting that it can be resampled after being sampled. The basis for resampling is that samples with higher weights have a higher probability of being selected. The benchmark for resampling in the original distribution training domain is set as a high probability of selecting samples with high weights. Based on this benchmark, a training domain that maintains the original data distribution characteristics is constructed, namely the original distribution training domain. 24) Sample sampling weight reallocation: For correctly classified samples, set their weights to 0 to exclude them from the inverse distribution training domain; for misclassified samples, use the inverse confidence score 1−S as their new weights. 25) Construction of the inverse distribution training domain: For samples that are misclassified, the weights 1-S are reallocated and the distribution sampling with replacement is performed again. The benchmark for resampling in the inverse distribution training domain is that samples with high weights have a higher probability of being selected. This biases the selection of samples that are seriously misclassified by the model, thereby constructing the inverse distribution training domain.

Images