Grain heap surface pest detection model training method, detection method and device

Abstract

The application provides a grain pile surface pest detection model training method, a detection method and equipment, relates to the technical field of image recognition, and the training method comprises the following steps: inputting different types of data samples respectively generated by pre-basing each grain pile surface image data labeled with a pest bounding box label in multiple data domains into two model branches, so that the two model branches sequentially perform backbone network feature extraction and backdoor adjustment on the data samples to obtain target feature maps and pest detection result data; and determining the causal feature alignment total loss corresponding to the current iteration round according to the target feature maps obtained by the two model branches respectively to optimize the two model branches. The application can effectively improve the domain generalization ability of the grain pile surface pest detection model and the adaptability to complex and variable actual detection scenes, can improve the application convenience and effectiveness of the grain pile surface pest detection model, and can further improve the accuracy and reliability of the grain pile surface pest detection result.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a training method, detection method and equipment for a pest detection model on the surface of grain piles. Background Technology

[0002] The reproduction and invasion of stored grain pests are a key factor threatening the safe storage of grain. The resulting infestations not only damage the quantity and quality of grain but may also lead to food safety issues. Therefore, real-time monitoring of pests is of paramount importance. Given the ease of observation of pests on the surface of grain piles, detecting these pests has become an extremely important regulatory task.

[0003] Currently, existing methods for detecting pests on grain pile surfaces automate the detection of pests using target detection models. However, these methods do not consider the domain generalization problem of the models, making them effective only in specific scenarios. When faced with new detection scenarios, it is often necessary to collect new training data and then fine-tune the original model. Therefore, multiple dedicated models need to be maintained for different environments, resulting in significant deployment costs. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method for training a pest detection model on the surface of grain piles, a detection method, and an apparatus to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of this application provides a method for training a pest detection model on the surface of a grain pile, comprising:

[0006] In the current iteration, different types of data samples, pre-generated from grain pile surface image data with bounding boxes labeled with pests in various data domains, are respectively input into the first model branch and the second model branch. This allows the first and second model branches to perform backbone network feature extraction and backdoor adjustment on the different types of data samples to obtain target feature maps and corresponding pest detection results. Furthermore, the total causal feature alignment loss corresponding to the current iteration is determined based on the target feature maps obtained by the first and second model branches respectively, and the first and second model branches are optimized based on this total causal feature alignment loss.

[0007] If the current iteration is the last iteration, then a grain pile surface pest detection model is generated based on the first model branch to output corresponding pest detection results data based on the grain pile surface image data.

[0008] In some embodiments of this application, both the first model branch and the second model branch include: a convolutional neural network, a causal intervention representation module, a contrastive causal feature alignment module, and a detection head connected in sequence; and the convolutional neural networks in the first model branch and the second model branch share weights, and the detection heads in the first model branch and the second model branch also share weights.

[0009] The convolutional neural network is used to extract features from the input data samples to output the corresponding original feature map;

[0010] The causal intervention characterization module is used to determine the confounding factor estimate for the current iteration based on the original feature map corresponding to the data sample, the pest bounding box label corresponding to the data sample, and the confounding factor estimate obtained in the previous iteration; and based on the cross-attention mechanism, to determine the target feature map corresponding to the original feature map after causal intervention characterization based on the confounding factor estimate for the current iteration and the original feature map.

[0011] The contrastive causal feature alignment module is used to transmit the target feature map after causal intervention to the detection head, and is also used to obtain positive samples and difficult negative samples from the target feature map after causal intervention, so as to calculate the contrastive loss corresponding to each positive sample based on the positive samples and difficult negative samples obtained by the two contrastive causal feature alignment modules respectively, and determine the total causal feature alignment loss corresponding to the current iteration based on the contrastive loss corresponding to each positive sample.

[0012] The detection head is used to output the pest detection result data of the data sample according to the target feature map after causal intervention.

[0013] In some embodiments of this application, the causal intervention characterization module includes: a confounding factor estimation unit and a causal intervention cross-attention unit;

[0014] The confounding factor estimation unit is used to obtain a feature vector group consisting of each feature vector falling within the pest bounding box label from the original feature map corresponding to the data sample, based on the pest bounding box label corresponding to the data sample; to use each feature vector as a positive sample corresponding to the data sample to obtain the pest target prediction box corresponding to the data sample based on each positive sample; to calculate the intersection-union ratio (IUR) between the pest target prediction box corresponding to the data sample and the pest bounding box label, and to use the IUR as a weight to determine the weighted average of each positive sample in the feature vector group; and to update the weighted average of the positive samples using an exponential moving average method based on the confounding factor estimation value obtained in the previous iteration to obtain the confounding factor estimation value for the current iteration.

[0015] The causal intervention cross-attention unit is used to sequentially adjust the number of channels and transform the dimensions of the original feature map to obtain the corresponding adjusted feature vector; the adjusted feature vector and the confounding factor estimate of the current iteration are respectively subjected to backdoor adjustment based on cross-attention to obtain the backdoor-adjusted feature vector; the dimension of the backdoor-adjusted feature vector is restored to be the same as the dimension of the original feature map to obtain the corresponding dimension-restored feature map; the dimension-restored feature map is concatenated with the original feature map to obtain the corresponding concatenated feature map; the dimension of the concatenated feature map is reduced to be the same as the dimension of the original feature map to obtain the corresponding target feature map after causal intervention representation.

[0016] In some embodiments of this application, the comparative causal feature alignment module includes a sample mining unit; and the comparative causal feature alignment modules corresponding to the first model branch and the second model branch respectively share the same causal feature alignment unit;

[0017] The sample mining unit is used to take the other feature vectors in the target feature map after causal intervention as negative samples; sort each negative sample in order of confidence from high to low; and select multiple negative samples in order from front to back, with the same number as the number of positive samples, as difficult negative samples.

[0018] The causal feature alignment unit is used to calculate the contrast loss corresponding to each of the positive samples and the difficult negative samples obtained by the sample mining unit in the first model branch and the sample mining unit in the second model branch, respectively, with the goal of reducing the feature distance between the positive samples and increasing the feature distance between the positive samples and the difficult negative samples; and to calculate the weighted average of the contrast losses corresponding to each of the positive samples to obtain the total causal feature alignment loss corresponding to the current iteration round.

[0019] In some embodiments of this application, the grain pile surface pest detection model training method further includes, before generating different types of data samples from grain pile surface image data pre-generated based on various marked pest bounding box labels in multiple data domains:

[0020] Obtain the image data group corresponding to each preset data domain. Each image data group contains multiple grain pile surface image data with bounding box labels marked with pests.

[0021] Randomization operations are performed on the frequency domain and spatial domain of each grain pile surface image data to obtain the enhanced image data corresponding to each grain pile surface image data.

[0022] Different types of data samples are generated based on the surface image data of each grain pile and the corresponding enhanced image data of each surface image data of the grain pile.

[0023] In some embodiments of this application, generating different types of data samples based on the surface image data of each grain pile and the enhanced image data corresponding to each surface image data of each grain pile includes:

[0024] The various image data groups are merged to obtain a first training dataset, and each of the grain pile surface image data in the first training dataset is used as a type of data sample for training the first model branch.

[0025] Furthermore, the enhanced image data corresponding to each of the grain pile surface image data is used as another type of data sample for training the second model branch.

[0026] In some embodiments of this application, generating different types of data samples based on the surface image data of each grain pile and the enhanced image data corresponding to each surface image data of each grain pile includes:

[0027] Each of the image data groups is divided into a second training dataset and a third training dataset, so that the second training dataset and the third training dataset each contain image data of the grain pile surface from different data domains;

[0028] The enhanced image data corresponding to each of the grain pile surface image data in the second training dataset is also added to the second training dataset, and each of the grain pile surface image data and each of the enhanced image data in the second training dataset are used as a type of data sample for training the first model branch.

[0029] In addition, the enhanced image data corresponding to each of the grain pile surface image data in the third training dataset is also added to the third training dataset, and each of the grain pile surface image data and each of the enhanced image data in the third training dataset are used as another type of data sample for training the second model branch.

[0030] In some embodiments of this application, the step of generating a grain pile surface pest detection model based on the first model branch for outputting corresponding pest detection result data based on grain pile surface image data includes:

[0031] The model formed by the convolutional neural network and the detection head in the first model branch is used as a grain pile surface pest detection model for outputting corresponding pest detection results data based on grain pile surface image data, wherein the convolutional neural network in the grain pile surface pest detection model is connected to the detection head.

[0032] The second aspect of this application provides a method for detecting pests on the surface of a grain pile, comprising:

[0033] Collect image data of the surface of the target grain pile within the grain storage space;

[0034] The target grain pile surface image data is input into the grain pile surface pest detection model, so that the grain pile surface pest detection model outputs the pest detection result data corresponding to the target grain pile surface image data; wherein, the grain pile surface pest detection model is trained in advance using the grain pile surface pest detection model training method described in the first aspect.

[0035] A third aspect of this application provides a training device for a pest detection model on the surface of a grain pile, comprising:

[0036] The iterative training module is used to input different types of data samples, which are pre-generated based on grain pile surface image data labeled with pest bounding boxes in various data domains, into the first model branch and the second model branch respectively in the current iteration round. This allows the first model branch and the second model branch to perform backbone network feature extraction and backdoor adjustment on the different types of data samples to obtain target feature maps and corresponding pest detection result data. Furthermore, the module determines the causal feature alignment total loss corresponding to the current iteration round based on the target feature maps obtained by the first model branch and the second model branch respectively, and optimizes the first model branch and the second model branch based on the causal feature alignment total loss.

[0037] The application model output module is used to generate a grain pile surface pest detection model based on the first model branch if the current iteration round is the last iteration round. This model model is used to output the corresponding pest detection results data based on the grain pile surface image data.

[0038] The fourth aspect of this application provides a pest detection device for grain pile surfaces, comprising:

[0039] The image data acquisition module is used to acquire image data of the surface of the target grain pile within the grain storage space;

[0040] The model detection module is used to input the target grain pile surface image data into the grain pile surface pest detection model, so that the grain pile surface pest detection model outputs the pest detection result data corresponding to the target grain pile surface image data; wherein, the grain pile surface pest detection model is trained in advance using the grain pile surface pest detection model training method described in the first aspect.

[0041] A fifth aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the grain pile surface pest detection model training method, and / or to implement the grain pile surface pest detection method.

[0042] The sixth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the grain pile surface pest detection model training method, and / or implements the grain pile surface pest detection method.

[0043] The seventh aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, implements the grain pile surface pest detection model training method, and / or implements the grain pile surface pest detection method.

[0044] The grain pile surface pest detection model training method provided in this application, in the current iteration, inputs different types of data samples, pre-generated based on grain pile surface image data labeled with pest bounding boxes in multiple data domains, into a first model branch and a second model branch respectively. This allows the first and second model branches to perform backbone network feature extraction and backdoor adjustment on the different types of data samples to obtain target feature maps and corresponding pest detection result data. Furthermore, based on the target feature maps obtained by the first and second model branches respectively, the causal feature alignment total loss corresponding to the current iteration is determined, and the first and second model branches are optimized based on this causal feature alignment total loss. If the current iteration is the last iteration, then the first model branch generates a model for detecting pests based on the grain pile surface. This model for detecting pests on grain pile surfaces outputs corresponding pest detection results from grain pile surface image data. It uses grain pile surface image data labeled with pest bounding boxes from various data domains as training samples. During model training, a two-branch domain generalization framework is employed to predict different types of data samples and to perform backdoor adjustments on the features of the data samples. This effectively improves the domain generalization ability of the trained grain pile surface pest detection model and enhances its adaptability to complex and varied real-world detection scenarios. It eliminates the need to maintain multiple dedicated models for different data domains, improving the ease of application and effectiveness of the grain pile surface pest detection model. Consequently, it enhances the accuracy and reliability of grain pile surface pest detection results, providing rapid and accurate automatic detection data for grain pest control.

[0045] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0046] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0047] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings:

[0048] Figure 1 This is a schematic diagram of the first process of a grain pile surface pest detection model training method in one embodiment of this application.

[0049] Figure 2 This is a schematic diagram of a first architecture of the first model branch and the second model branch in one embodiment of this application.

[0050] Figure 3 This is a schematic diagram of the training structure of a grain pile surface pest detection model based on contrastive causal intervention characterization in one embodiment of this application.

[0051] Figure 4 This is a schematic diagram of a second architecture for the first model branch and the second model branch in one embodiment of this application.

[0052] Figure 5 This is a schematic flowchart illustrating the execution of the confounding factor estimation unit in one embodiment of this application.

[0053] Figure 6(a) is a schematic diagram of the data execution flow of the causal intervention characterization module in one embodiment of this application.

[0054] Figure 6(b) is a schematic diagram of the data execution flow of the causal intervention cross-attention unit in one embodiment of this application.

[0055] Figure 7 This is a schematic diagram of a third architecture for the first model branch and the second model branch in one embodiment of this application.

[0056] Figure 8 This is a schematic diagram of the data execution flow framework of the comparison causal feature alignment module in one embodiment of this application.

[0057] Figure 9 This is a schematic diagram illustrating the construction process of a multi-domain grain pile surface pest image dataset in one embodiment of this application.

[0058] Figure 10 This is a schematic diagram of a pest image data acquisition scenario in one embodiment of this application.

[0059] Figure 11 This is a schematic diagram illustrating the working principle of the random domain generation module in one embodiment of this application.

[0060] Figure 12 This is a schematic diagram of the second process of the grain pile surface pest detection model training method in one embodiment of this application.

[0061] Figure 13(a) is a partial schematic diagram of model training in a single-domain mode in one embodiment of this application.

[0062] Figure 13(b) is a partial schematic diagram of model training in a multi-domain mode in one embodiment of this application.

[0063] Figure 14 This is a schematic diagram of the third process of the grain pile surface pest detection model training method in one embodiment of this application.

[0064] Figure 15 This is a schematic diagram of the structure of a grain pile surface pest detection model training device in one embodiment of this application.

[0065] Figure 16 This is a schematic diagram of the structure of a grain pile surface pest detection device according to one embodiment of this application. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.

[0067] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the scheme according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0068] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0069] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0070] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0071] Currently, many grain warehouses are equipped with video surveillance equipment such as network dome cameras, mainly used for detecting pests on the surface of local grain piles. However, traditional methods for monitoring pests on the surface of grain piles mainly rely on manual observation of the pests captured by the dome camera. This method has drawbacks such as being time-consuming, labor-intensive, slow, and prone to errors.

[0072] In recent years, numerous deep learning-based pest detection methods have been proposed, representing a trend towards automated pest detection. For example, a grain storage pest detection system based on convolutional neural networks employs a lightweight VGG16 (Visual Geometry Group 16) model. VGG16 is a convolutional neural network that incorporates weights for classification and regression tasks into the loss function to balance positive and negative samples, thereby accurately identifying areas where pests exist and effectively shortening training and detection time. Its prediction box mechanism further enhances detection capabilities through multi-scale feature extraction. However, this method is primarily suitable for pest detection in simple scenarios such as laboratory glass petri dishes or white sticky insect boards. When used to detect pests on the surface of grain piles, it is prone to misclassifying impurities and gaps between grains as pests.

[0073] Another example is a pest detection method based on the YOLOv7 model in object detection, which integrates the ACmix adaptive convolutional separation module and the Convolutional Block Attention Module (CBAM). ACmix is ​​a novel hybrid model that combines a convolutional neural network (CNN) with a self-attention mechanism, thereby improving sensitivity to small-scale targets, reducing the impact of noise, and improving detection accuracy under conditions such as local occlusion, motion blur, and clutter. However, the training data used in this method is all from a single scene, without considering the possibility that the actual detection data of the target model may not be collected in the same scene as the training data. This can lead to data distribution differences (known as domain shift in computer science), which may result in a decrease in detection performance.

[0074] Another example is a deep learning-based method and system for detecting and identifying rice pests. It uses different models to analyze and identify pest image data from two different background sources: under pest monitoring lights and in the field. Although it distinguishes between the two backgrounds, its dataset is still limited to a specific scenario and fails to cover more environmental variations. Each model is trained only on a single background, lacking cross-domain data support or design for model generalization capabilities.

[0075] This application also proposes a method for detecting stored grain pests based on the YOLOv5s algorithm in object detection models. This method expands the dataset using data augmentation techniques such as rotation, flipping, mirroring, scaling, translation, and mosaic enhancement. Furthermore, it integrates a visual Transformer model (Swin Transformer) structure into the detection head to improve feature extraction capabilities. However, this method essentially only modifies existing general object detection frameworks and does not consider the domain generalization problem that urgently needs to be addressed in practical pest detection.

[0076] In other words, most current deep learning-based methods for monitoring pests on grain pile surfaces do not fully consider the data domain differences between the actual pest detection scenario and the scenario in which the training data was collected. When faced with new detection scenarios, it is often necessary to collect new training data and then fine-tune the original model. Therefore, multiple dedicated models need to be maintained for different environments, resulting in high deployment costs. Furthermore, these methods are usually limited to improvements on existing benchmark deep learning frameworks (such as YOLO and SSD) or only employ simple data augmentation strategies, failing to optimize model performance through domain generalization methods. This leads to limited adaptability to diverse detection scenarios. In short, existing methods for detecting pests on grain pile surfaces do not consider the domain generalization problem of the model, making the model effective only in specific scenarios. When faced with new detection scenarios, it is often necessary to collect new training data and then fine-tune the original model. Therefore, multiple dedicated models need to be maintained for different environments, resulting in high deployment costs.

[0077] Therefore, in order to address the problem that existing methods for detecting pests on the surface of grain piles do not consider the limited domain generalization ability of the model, this application provides a method for training a pest detection model on the surface of grain piles, a method for detecting pests on the surface of grain piles, a device for training a pest detection model on the surface of grain piles to execute the training method, a device for detecting pests on the surface of grain piles to execute the method, an electronic device, a computer-readable storage medium, and a computer program product, which can effectively improve the domain generalization ability of the pest detection model on the surface of grain piles and its adaptability to complex and ever-changing actual detection scenarios.

[0078] The following examples will provide a detailed description.

[0079] Based on this, embodiments of this application provide a method for training a grain pile surface pest detection model that can be implemented by a grain pile surface pest detection model training device, see [link to relevant documentation]. Figure 1 The training method for the pest detection model on the grain pile surface specifically includes the following:

[0080] Step 100: In the current iteration, different types of data samples, pre-generated based on grain pile surface image data labeled with pest bounding boxes in various data domains, are input into the first model branch and the second model branch respectively. This allows the first model branch and the second model branch to perform backbone network feature extraction and backdoor adjustment on the different types of data samples to obtain target feature maps and corresponding pest detection result data. Furthermore, the total causal feature alignment loss corresponding to the current iteration is determined based on the target feature maps obtained by the first model branch and the second model branch respectively. The first model branch and the second model branch are then optimized based on this total causal feature alignment loss.

[0081] In one or more embodiments of this application, the data domain, which can be simply referred to as a domain, refers to a set of pest image data with different feature differences. These differences arise from variations in data acquisition conditions (such as different lighting conditions or different shooting devices). Sets of image data acquired under different conditions correspond to different data domains. Multiple grain pile surface image data used in training the model are labeled with pest bounding box tags, which can be simply referred to as tags; and these tags are used to identify the pest bounding boxes in the grain pile surface image data, i.e., the manually labeled true bounding boxes. Each grain pile surface image data labeled with pest bounding box tags in multiple data domains means that multiple grain pile surface image data correspond to a first data domain, and other multiple grain pile surface image data correspond to a second data domain, and so on, so that the domain generalization ability of the model is considered during the training data selection stage.

[0082] In step 100, the target feature map may include a first target feature map and a second target feature map; the pest detection result data may include first pest detection result data and second pest detection result data. Correspondingly, the first model branch is used to sequentially extract backbone network features and adjust backdoors for one type of data sample to obtain a first target feature map, and output the first pest detection result data corresponding to the first target feature map; the second model branch is used to sequentially extract backbone network features and adjust backdoors for another type of data sample to obtain a second target feature map, and output the first pest detection result data corresponding to the first target feature map; then, based on the first target feature map and the second target feature map, the total causal feature alignment loss corresponding to the current iteration is determined, and the first model branch and the second model branch are optimized based on the total causal feature alignment loss.

[0083] Understandably, the backbone network corresponding to the backbone network feature extraction can be a convolutional neural network (CNN). The backdoor adjustment is an important method for handling confounders in causal inference, used to estimate the true causal effect between variables in observational data. When confounders (variables that affect both causal and outcome variables) exist, directly calculating the association between the causal variable and the outcome variable will lead to erroneous conclusions due to confounding bias (such as Simpson's paradox). Backdoor adjustment separates the pure causal effect of the causal variable on the outcome variable by blocking the "backdoor path" from the causal variable to the outcome variable (i.e., the non-causal path constituted by confounders), thereby eliminating the impact of confounders on the pest detection performance of the model and improving the generalization ability of the model.

[0084] It should also be noted that Causal Feature Alignment Total Loss is a composite loss function designed in multimodal or causal learning frameworks. It aims to improve the interpretability and generalization ability of the model by jointly optimizing the alignment between the model's decision-making mechanism and the real causal path, as well as the semantic consistency of features.

[0085] In other words, during the training of the pest detection model for grain pile surfaces, at least one iteration is required, with step 100 executed in each iteration. Then, it is determined whether the model composed of the first model branch (which can be simply referred to as the first branch) and the second model branch (which can be simply referred to as the second branch) has converged, or whether the current iteration is the preset last iteration. If yes, step 200 is executed; otherwise, the optimized first and second model branches are used to return to step 100 for the next iteration of model training.

[0086] Step 200: If the current iteration is the last iteration, then generate a grain pile surface pest detection model based on the first model branch, which is used to output the corresponding pest detection result data based on the grain pile surface image data.

[0087] As described above, the grain pile surface pest detection model training method provided in this application uses grain pile surface image data labeled with pest bounding boxes from various data domains as data samples for training the model. During model training, a two-branch domain generalization framework is used to predict different types of data samples and backdoor adjustments are made to the features of the data samples. This effectively improves the domain generalization ability of the trained grain pile surface pest detection model and enhances its adaptability to complex and ever-changing actual detection scenarios. It eliminates the need to maintain multiple dedicated models for detection image data from different data domains, thereby improving the ease of application and effectiveness of the grain pile surface pest detection model. Consequently, it improves the accuracy and reliability of grain pile surface pest detection results, providing rapid and accurate automatic detection data for grain pest control.

[0088] To further improve the domain generalization ability of the grain pile surface pest detection model and enhance the accuracy and reliability of grain pile surface pest detection, a grain pile surface pest detection model training method is provided in this application embodiment, see [link to relevant documentation]. Figure 2 The first model branch and the second model branch in the grain pile surface pest detection model training method specifically include: a convolutional neural network 1, a causal intervention representation module 2, a contrastive causal feature alignment module 3, and a detection head 4 connected in sequence; and the convolutional neural network 1 in the first model branch and the second model branch share weights, and the detection head 4 in the first model branch and the second model branch also share weights.

[0089] The convolutional neural network 1 is used to extract features from the input data samples to output the corresponding original feature map M.

[0090] The causal intervention characterization module 2 is used to determine the confounding factor estimate z obtained from the previous iteration based on the original feature map M corresponding to the data sample, the pest bounding box label corresponding to the data sample, and the confounding factor estimate z obtained from the previous iteration. t-1 Determine the confounding factor estimate z for the current iteration round. t Based on the cross-attention mechanism, the confounding factor z is estimated according to the current iteration round. t And the original feature map M determines the target feature map M corresponding to the original feature map after causal intervention representation. b Wherein, the target feature map after causal intervention representation output by the causal intervention representation module 2 in the first model branch is the first target feature map; the target feature map after causal intervention representation output by the causal intervention representation module 2 in the second model branch is the second target feature map.

[0091] Understandably, the principle behind the causal intervention representation module lies in utilizing the "backdoor adjustment" operation of causal inference theory to eliminate spurious correlations between input and output caused by confounding factors during feature modeling in convolutional neural networks (CNNs). By blocking non-causal spurious connections, it prevents the model from relying on non-causal co-occurrence relationships in image data (such as background color or non-causal contextual features) for prediction. This enhances the model's causal prediction capability and strengthens its robustness to domain shift phenomena. Specific implementation includes estimating confounding factors and constructing a backdoor adjustment based on causal intervention cross-attention.

[0092] The comparative causal feature alignment module 3 is used to align the target feature map M after causal intervention characterization. b The data is transmitted to the detection head 4 and is also used to analyze the target feature map M after causal intervention characterization. b Positive samples and difficult negative samples are obtained from the two contrastive causal feature alignment modules 3, and the contrastive loss L corresponding to each positive sample is calculated based on the positive samples and difficult negative samples obtained by each of the two contrastive causal feature alignment modules 3. xi And based on the contrast loss L corresponding to each of the positive samples. xi Determine the total loss for causal feature alignment corresponding to the current iteration round.

[0093] Specifically, the contrastive causal feature alignment module 3 utilizes a contrastive learning mechanism to achieve cross-domain causal feature alignment, aiming to further enhance the model's ability to extract domain-invariant causal features. Simultaneously, by distinguishing between targets on the grain pile surface that resemble pest morphology and background noise, it reduces false detections caused by such interference factors, thereby enhancing the model's detection accuracy and reliability. Its specific technical solutions include hard negative sample mining and causal feature alignment.

[0094] The detection head 4 is used to determine the target feature map M after causal intervention characterization. b The corresponding pest detection results data of the data sample are output.

[0095] Specifically, the grain pile surface stored grain pest detection domain generalization framework, consisting of the first and second model branches, primarily utilizes domain generalization techniques based on contrastive causal feature representations to train the pest detection model, mitigating its performance degradation in real-world detection scenarios. The model's training architecture employs a dual-branch mode, as detailed below. Figure 3As shown, during training, the diversity of the training data distribution is first enhanced through a random domain generation module. Then, the original image data and the enhanced image data are input into two branches respectively, and basic representation and feature extraction operations are performed through the model's backbone network (such as a convolutional neural network). Next, the extracted original feature map M is processed by a causal intervention representation module to block spurious correlations introduced by confounding factors. Finally, cross-domain feature alignment is achieved using a contrastive causal feature alignment module combined with a hard negative sample mining strategy. This process further strengthens the model's ability to extract domain-invariant causal features, while effectively reducing the interference of background noise on pest detection accuracy, significantly improving the model's performance and reliability in complex detection scenarios.

[0096] To further improve the accuracy and reliability of causal intervention characterization in the training process of grain pile surface pest detection models, a grain pile surface pest detection model training method is provided in this application embodiment, see [link to relevant documentation]. Figure 4 The causal intervention characterization module 2 in the grain pile surface pest detection model training method specifically includes: a confounding factor estimation unit 21 and a causal intervention cross-attention unit 22 that are interconnected.

[0097] The confounding factor estimation unit 21 is used to obtain a feature vector group consisting of each feature vector falling within the pest bounding box label from the original feature map M corresponding to the data sample, based on the pest bounding box label corresponding to the data sample; to use each feature vector as a positive sample corresponding to the data sample to obtain the pest target prediction box corresponding to the data sample based on each positive sample; to calculate the intersection-union ratio (IUR) between the pest target prediction box corresponding to the data sample and the pest bounding box label, and to use the IUR as a weight to determine the weighted average of each positive sample in the feature vector group; and to update the weighted average of the positive samples using an exponential moving average method based on the confounding factor estimation value obtained in the previous iteration to obtain the confounding factor estimation value for the current iteration.

[0098] Specifically, see Figure 5 The process executed by the confounding factor estimation unit includes:

[0099] During model training, each batch of input data samples X has dimensions B×3×H×W. After processing by the backbone network, it outputs an original feature map M with dimensions B×C×H×W. Based on the pest bounding box labels (i.e., ground truth boxes) in the dataset, feature vectors f1, f2, ..., f1 that fall within the pest bounding box labels are selected from the B×H×W feature vectors in the original feature map M. N To form a feature vector set F = [f1, f2, ..., f NThe feature map containing the positive samples is used to predict the final pest target. Specifically, the entire feature map is transmitted to the detection head via a causal intervention representation module and a contrastive causal feature alignment module. The detection head outputs the predicted bounding box coordinates and confidence scores. The predicted bounding box coordinates are input to the confounding factor estimation unit to calculate the intersection-union ratio (IU) between the predicted bounding box and the true labeled bounding box. Here, B (Batch Size) represents the batch size, i.e., the number of data samples processed at one time; C (Channels) represents the number of channels, i.e., the number of color or feature channels in the data sample; H (Height) represents the height, i.e., the number of pixels in the vertical direction of the data sample; and W (Width) represents the width, i.e., the number of pixels in the horizontal direction of the data sample.

[0100] Then, using the Intersection over Union (IoU) value between the predicted insect target bounding boxes of the positive samples and the ground truth bounding boxes labeled in the dataset as weights, the weighted average of each positive sample in the feature vector group is calculated, and this is used as the estimate of the confounding factor z. The specific calculation method is as follows:

[0101]

[0102] Among them, f i The feature vector representing a positive sample, where N represents the total number of positive samples in the feature vector group, and IoU i It is the intersection-over-union (IoU) value between the predicted insect target bounding box (i.e., the predicted box) and the insect bounding box label (i.e., the ground truth box) of the i-th positive sample.

[0103] Alternatively, static feature mean estimation (direct averaging of positive sample features) can be used to replace the weighted averaging operation in this application, which can also achieve a certain causal intervention effect.

[0104] During model training, to ensure the stability and continuity of confounding factor estimation, the exponential moving average (EMA) method is used to update the estimated value z of the confounding factors in each batch (i.e., the weighted average of the positive samples in the feature vector group), as shown in Figure 6(a). The update formula is:

[0105]

[0106] Among them, z t-1 It is the confounding factor estimate obtained in the previous iteration t-1, z t f represents the confounding factor estimate for the current iteration round t; α represents the preset estimate weight; i t This represents the feature vector of the positive samples in the current iteration t; This represents the intersection-over-union (IoU) value between the predicted insect target bounding box (i.e., the predicted box) and the insect bounding box label (i.e., the ground truth box) of the i-th positive sample in the current iteration round t.

[0107] The causal intervention cross-attention unit 22 is used to sequentially adjust the number of channels and transform the dimensions of the original feature map M to obtain the corresponding adjusted feature vector; to perform backdoor adjustment based on cross-attention on the adjusted feature vector and the confounding factor estimate of the current iteration to obtain the backdoor adjusted feature vector; to restore the dimension of the backdoor adjusted feature vector to the same dimension as the original feature map M to obtain the corresponding dimension-restored feature map; to concatenate the dimension-restored feature map with the original feature map M to obtain the corresponding concatenated feature map; and to reduce the dimension of the concatenated feature map to the same dimension as the original feature map M to obtain the corresponding target feature map after causal intervention representation.

[0108] Specifically, referring to Figure 6(b), the process executed by the causal intervention cross-attention unit includes:

[0109] The original feature map M output by the backbone network has dimensions B×C×H×W. It is first adjusted for the number of channels by a 1×1 convolution operation. Then, the feature map is transformed into a feature vector with dimensions B×HW×C (i.e., the adjusted feature vector) through a reshape operation. The reshape operation is a commonly used array or tensor shape transformation method in deep learning and scientific computing. Its core is to adjust the dimensional structure of the data without changing the data content.

[0110] Adjusted eigenvectors and estimated confounding factors z t A backdoor adjustment operation based on cross-attention is performed to achieve causal intervention on features. The cross-attention formula is as follows:

[0111]

[0112] Among them, f y (x,z t ) represents the backdoor adjustment operation, x represents the feature vector contained in the original feature map M, and z represents the feature vector. t This represents the estimated confounding factors for the current iteration round t.

[0113] By introducing a fully connected layer W q W k and W v Linear transformations are performed on the query, key, and value respectively to obtain a backdoor-adjusted feature vector to enhance the model's ability to express features.

[0114] The feature vector adjusted by the backdoor will be restored to the dimensions B×C×H×W through a reshape operation to obtain the dimension-restored feature map M'.

[0115] Subsequently, the dimension-restored feature map M' is concatenated with the original feature map M to obtain a concatenated feature map with dimensions B×2C×H×W.

[0116] Finally, the spliced ​​feature map is reduced in dimensionality by 1×1 convolution to obtain the target feature map after causal intervention representation with dimensions of B×C×H×W.

[0117] Alternatively, the effect of this application may be achieved by calculating attention maps of the output features of the two branches and reducing the difference between the attention maps instead of the cross-attention mechanism used in this application.

[0118] To further improve the domain generalization ability of the grain pile surface pest detection model and enhance the accuracy and reliability of grain pile surface pest detection, a grain pile surface pest detection model training method is provided in this application embodiment, see [link to relevant documentation]. Figure 7 The comparative causal feature alignment module 3 in the grain pile surface pest detection model training method specifically includes a sample mining unit 31; and the comparative causal feature alignment modules 3 corresponding to the first model branch and the second model branch respectively share the same causal feature alignment unit 32.

[0119] The sample mining unit 31 is used to take the other feature vectors in the target feature map after causal intervention as negative samples; sort each negative sample in order of confidence from high to low; and select multiple negative samples in order from front to back, with the same number as the number of positive samples, as difficult negative samples.

[0120] In the sample mining unit 31, the positive samples in the target feature map are the same as the positive samples in the confounding factor estimation unit 21, i.e., based on the feature vectors falling within the pest bounding box labels. In the sample mining unit 31, the positive samples already determined in the confounding factor estimation unit 21 can be used directly, or the same method as the confounding factor estimation unit 21 can be used to re-determine the feature vectors falling within the pest bounding box labels as the positive samples.

[0121] Specifically, the data execution flow framework for the comparison causal feature alignment module is as follows: Figure 8As shown. Difficult negative sample mining aims to improve the model's ability to distinguish easily confused features. The specific steps are as follows: First, in the original feature map M output by the backbone network, the feature vector responsible for ultimately predicting the pest target is defined as a positive sample, and the remaining feature vectors are negative samples. Next, the mean confidence score p of all positive sample prediction results (including confidence score and bounding box coordinates) in the current batch is calculated. Assuming the number of positive samples in the current batch is k, the top k negative samples with confidence scores higher than p are selected from all negative samples and designated as difficult negative samples. If the number of negative samples with confidence scores higher than p is less than k, the top k negative samples with the highest confidence scores are selected as difficult negative samples. These difficult negative samples usually correspond to targets or background noise in the image data that are similar in morphology to pests. Because they have high similarity to pest targets and high confidence scores, they are easily misdetected as pests by the model.

[0122] Furthermore, by comparing the feature distance between the negative sample feature vector and the positive sample feature (pest feature) vector, negative samples with a feature distance closer to the positive sample feature are selected as difficult negative samples, replacing the confidence-based method of this application for selecting negative samples.

[0123] Specifically, feature alignment is performed on positive samples and difficult negative samples. The method involves, within a contrastive learning framework, reducing the feature distance between positive samples while increasing the feature distance between positive samples and difficult negative samples through an optimization algorithm. The contrastive loss L for a single positive sample is... xi The calculation formula is as follows:

[0124]

[0125] Where, x i and x j These represent the i-th and j-th positive sample pairs, respectively. Represents a difficult negative sample, where τ is the temperature parameter, and N pos and N neg L represents the number of positive samples and the number of difficult negative samples, respectively. xi For the contrastive loss of a single positive sample, exp() is the natural exponential function.

[0126] The final calculated total loss for causal feature alignment The contrast loss L for all positive samples xi The weighted average is calculated as follows:

[0127]

[0128] Among them, IoU xiThis represents the Intersection over Union (IoU) value between the predicted bounding box and the ground truth bounding box derived from the positive sample. This design is based on the significant differences in the representation quality of pest targets among different positive samples. Specifically, positive samples with higher IoU values ​​with the ground truth bounding boxes can more accurately represent the features of pest targets. Therefore, using IoU as a weighting factor can enhance the model's focus on high-quality positive samples.

[0129] To further improve the effectiveness and reliability of the training data for the grain pile surface pest detection model, and to further enhance the domain generalization ability and accuracy and reliability of the model, the grain pile surface pest detection model training method provided in this application includes the following steps before step 100:

[0130] Step 010: Obtain the image data group corresponding to each preset data domain. Each image data group contains multiple grain pile surface image data with bounding box labels marked with pests.

[0131] Step 020: Randomize the frequency domain and spatial domain of each grain pile surface image data to obtain the enhanced image data corresponding to each grain pile surface image data.

[0132] Step 030: Generate different types of data samples based on the surface image data of each grain pile and the corresponding enhanced image data of each surface image data of the grain pile.

[0133] Specifically, in one specific implementation example of step 010, see [link to example]. Figure 9 The construction process of a multi-domain grain pile surface pest image dataset, which includes image data groups corresponding to each preset data domain, is as follows:

[0134] In a real grain warehouse environment, various image data acquisition conditions are configured, and stored grain pests are artificially released. Based on these conditions, multi-view image data of the artificially released pests is acquired. The acquired pest image data is then labeled and processed, and multi-domain partitioning is performed based on the image data acquisition conditions. Furthermore, to further expand the dataset, this application can also simulate the actual grain warehouse environment under laboratory conditions, acquiring pest image data in the same manner as under actual grain warehouse conditions, thereby enhancing the flexibility of image data acquisition and the richness of the dataset.

[0135] (1) Scenarios for collecting pest image data, such as Figure 10 As shown, the specific implementation method is as follows:

[0136] A network dome camera (PTZ camera), commonly used in grain silos, was employed as the image data acquisition device. The PSN camera was mounted on the grain silo wall using a fixed bracket, with its lens center positioned 1.2 to 1.8 meters vertically above the grain pile surface to achieve optimal shooting angle and imaging resolution. The grain silo could be a simulated indoor area mimicking a real grain silo or a real grain silo. To avoid grain waste, grain past its safe consumption period and no longer suitable for human consumption was placed in the silo. And / or, after acquiring image data of the grain pile surface for training (hereinafter referred to as image data), the grain silo was quickly treated for pest control without affecting grain quality or food safety.

[0137] Next, determine the image data acquisition area. Using the surface of the grain pile directly below the center of the PTZ camera lens as the center point, delineate a semi-circular image data acquisition area with a radius of 3 to 5 meters along the surface of the grain pile. Within the semi-circular image data acquisition area, select 100-150 independent image data acquisition points distributed at equal intervals, with each acquisition point having an area of ​​500 to 600 square centimeters.

[0138] Then, at the collection point, containers such as trays and grain films are placed, and the same type of grain as the surface of the grain pile is evenly spread inside the containers to simulate the surface of a real grain storage pile and to achieve pest isolation and control.

[0139] Subsequently, live adult grain storage pests were evenly released onto the surface of the simulated grain pile to ensure that the distribution and behavior of the pests conformed to the characteristics of natural invasion.

[0140] Finally, the PTZ camera was remotely controlled via the camera control software to capture images of all data acquisition points in a predetermined order, thereby obtaining pest image data. The image data acquisition process in the laboratory environment is consistent with the operation in an actual grain warehouse, covering steps such as determining the image data acquisition area, selecting image data acquisition points, laying the grain, and releasing the pests.

[0141] (2) Image data capture conditions include the following settings:

[0142] ① Use PTZ cameras of different models and years of use to reflect the impact of factors such as PTZ camera resolution, lens aging, and sensor performance degradation on image data quality.

[0143] ② At the image data capture point, grains with different storage times were laid out, including new grains with shorter storage times and old grains. This was to reflect the impact of changes in grain color, blackening, mold, etc., during grain storage on the background information and surrounding contextual features of the image data and the pest targets.

[0144] ③ Considering the significant impact of lighting conditions on the visual characteristics of image data, diverse lighting conditions were used for shooting. By simulating different light intensities, white balance settings, and other conditions, the differences in the visual characteristics of pest image data, such as brightness, contrast, and color saturation, under varying lighting conditions were simulated.

[0145] (3) Data labeling and partitioning: The pest image data was labeled using image data labeling tools such as LabelImg. LabelImg is an open-source image data labeling tool that identifies the species and positions of each pest target in the image data. Subsequently, the collected image dataset was partitioned into data domains according to the set shooting conditions.

[0146] Specifically, in one specific implementation example of step 020, see [link to example]. Figure 11 Step 020 can be generated using a random domain generation module. The principle behind this module is to randomize the frequency and spatial domains of the input image data to expand the coverage of the training data distribution. Its core idea is to perturb domain-varying features in the image data, such as background color and noise—features unrelated to pest characteristics—while simultaneously maintaining domain-invariant features, i.e., the morphology of the pest target remains unchanged, thereby enhancing the domain diversity of the training data. The method's technical advantages lie in: while maintaining the semantic integrity of the input image data, it performs controllable perturbations on domain-varying features, generating new samples with domain differences, thereby improving the sample diversity for model training.

[0147] The specific implementation of step 200 performed by the random domain generation module is as follows:

[0148] (1) Frequency Domain Randomization Steps: First, the image data is transformed from the spatial domain to the frequency domain using Discrete Cosine Transform (DCT) to obtain the corresponding frequency domain spectrum. Next, the frequency domain spectrum is decomposed into low-frequency, mid-frequency, and high-frequency components using a pre-defined bandpass filter bank. The low-frequency and high-frequency components correspond to domain-varying characteristics, while the mid-frequency component corresponds to domain-invariant characteristics. Then, random Gaussian noise is introduced into the low-frequency and high-frequency components. Finally, the frequency domain spectrum after noise injection is reconstructed using Inverse Discrete Cosine Transform (IDCT) to generate enhanced image data. The process is specifically described as follows:

[0149]

[0150] Where X represents the input data sample, and DCT(·) and DCT′(·) represent the Discrete Cosine Transform and Inverse Discrete Cosine Transform, respectively. M(r) is a bandpass filter with a passband of r(R). L ,R H ), where R L and RH It represents the percentage range of the low and high cutoff frequencies in the passband. Function R G () is used to introduce Gaussian noise, described as follows: This represents a standard Gaussian distribution with a mean of 0 and a variance of 1.

[0151] In addition, the discrete cosine transform in the frequency domain randomization operation can be replaced with methods such as discrete Fourier transform or discrete wavelet transform.

[0152] (2) Spatial Domain Randomization Step: Spatial domain randomization employs a color dithering strategy, which randomly rearranges the RGB color channels of the image data according to a preset probability. For example, the RGB color channels can be randomly arranged as GBR, BRG, etc. This channel order reconstruction achieves non-destructive perturbation of the background color of the image data. The method is characterized by enhancing the domain diversity of the training data through controllable color channel substitution while maintaining the semantic integrity of the pest target. Furthermore, the random swapping of the RGB color channels can be modified to other color dithering methods, such as randomly adjusting the brightness and contrast of the image data.

[0153] In order to further improve the ease of application of the grain pile surface pest detection model and reduce the equipment resource occupancy rate, the grain pile surface pest detection model training method provided in this application embodiment further includes the following specific content in step 200:

[0154] Step 210: If the current iteration round is the last iteration round, then the model composed of the convolutional neural network and the detection head in the first model branch is used as the grain pile surface pest detection model for outputting the corresponding pest detection result data based on the grain pile surface image data, wherein the convolutional neural network in the grain pile surface pest detection model is connected to the detection head.

[0155] To further enrich the distribution and combination of training data and enhance the model's adaptability to different detection scenarios, in the grain pile surface pest detection model training method provided in this application embodiment, see... Figure 12 Step 030 in the training method for the detection model of pests on the surface of the grain pile specifically includes the following:

[0156] Step 031: Merge the various image data groups to obtain a first training dataset, and use each grain pile surface image data in the first training set as a type of data sample for training the first model branch.

[0157] And, step 032: use the enhanced image data corresponding to each of the grain pile surface image data as another type of data sample for training the second model branch.

[0158] Referring to Figure 13(a), step 100 can then be performed in single-domain mode: merging all training data and treating them as a single independent and identically distributed data domain. During training, the two model branches are respectively inputted with the original image data and the image data enhanced by the Random Domain Generation (RDG) module, thereby improving the diversity of the data.

[0159] To further enrich the distribution and combination of training data and enhance the model's adaptability to different detection scenarios, in the grain pile surface pest detection model training method provided in this application embodiment, see... Figure 14 Step 030 in the method for training a pest detection model on the surface of a grain pile may further include the following:

[0160] Step 033: Divide each of the image data groups into the second training dataset and the third training dataset respectively, so that the second training dataset and the third training dataset each contain image data of the grain pile surface from different data domains.

[0161] Step 034: Add the enhanced image data corresponding to each of the grain pile surface image data in the second training dataset to the second training dataset, and use each of the grain pile surface image data and each of the enhanced image data in the second training dataset as a type of data sample for training the first model branch.

[0162] And, step 035: add the enhanced image data corresponding to each of the grain pile surface image data in the third training dataset to the third training dataset, and use each of the grain pile surface image data and each of the enhanced image data in the third training dataset as another type of data sample for training the second model branch.

[0163] Referring to Figure 13(b), step 100 can then be executed in a multi-domain mode: both branches randomly receive data from different domains and are enhanced by the random domain generation module to expand the domain coverage of the data and improve the generalization ability of the model. This mode helps the model adapt to multiple data domains and enhances its applicability in different environments, where H1, H2, G1, and G2 are used to represent different data domains, respectively.

[0164] In summary, the grain pile surface pest detection model training method provided in this application addresses the following technical problems existing in the prior art:

[0165] 1) Most existing deep learning-based pest detection methods do not fully consider the differences between the actual pest detection scenarios and the scenarios collected for training data, resulting in weak generalization ability. After the pest detection model is trained and deployed, its detection performance drops significantly compared to the training data.

[0166] 2) Existing deep learning-based pest detection methods primarily use training data from limited pest detection scenarios, and these data are not properly partitioned according to the image data acquisition scenarios, resulting in independently and identically distributed training data. When the pest detection model is deployed to a new detection environment, its detection accuracy drops significantly because it has not encountered a similar data distribution before. This limits the effectiveness of the model to specific scenarios and restricts the practical application of deep learning methods.

[0167] 3) Existing pest detection methods often require collecting new data for training when faced with new detection scenarios, and then fine-tuning the original model. Therefore, multiple dedicated models need to be maintained for different environments, resulting in high deployment costs.

[0168] 4) Existing pest detection methods are usually limited to improvements on existing benchmark deep learning frameworks (such as YOLO, SSD, etc.) or only use simple data augmentation strategies. They fail to optimize model performance through domain generalization methods, resulting in limited adaptability to diverse detection scenarios.

[0169] 5) Traditional deep learning-based pest detection methods are susceptible to confounding factors. For example, certain types of pests often appear in images with a yellow background (background information) or in images where the surface of a grain pile is corn (contextual feature). The model may associate these confounding factors with pest features, creating spurious correlations. Once the background information or contextual features of the image data change, the model's detection performance will significantly decrease.

[0170] 6) Existing methods do not adequately consider how to suppress the impact of background noise on pest detection accuracy. For example, impurities on the surface of grain piles and gaps between grains are similar in shape to pest targets, making it easy for the model to make false detections.

[0171] This application belongs to the field of grain storage pest monitoring technology, specifically involving a domain generalization detection algorithm framework based on contrastive causal intervention representation, which is particularly suitable for visual detection of pests on the surface of grain piles during raw grain storage. This application mainly addresses the problem of decreased pest detection performance in actual storage environments when deep learning detection models suffer from domain shifts between the training data collection scenario and the real application scenario. By constructing an environment-robust domain generalization model, it solves the industry technical bottleneck of insufficient model generalization ability in existing visual detection technologies during grain storage detection caused by domain shift factors such as environmental differences and changes in equipment parameters, thus affecting the accuracy of detection.

[0172] In other words, the grain pile surface pest detection model training method provided in this application aims to solve the problem of decreased detection performance caused by the inconsistency between the actual detection data and the training data distribution, and improve the model's adaptability to complex detection scenarios. Its main contents include: ① Constructing a multi-domain grain pile surface pest image dataset. ② Designing a grain pile surface stored grain pest detection domain generalization framework, which includes: a random domain generation module, a causal intervention representation module, and a contrastive causal feature alignment module. The model framework specifically includes the following:

[0173] 1) Construct a multi-domain pest image dataset: collect pest image data from multiple grain storage scenarios, simulate the domain shift phenomenon that may occur in actual pest detection scenarios, and provide a data foundation for model training and validation.

[0174] 2) Design a random domain generation strategy: enrich the diversity of training data during model training so that the model can learn the features of different domains and improve its generalization ability.

[0175] 3) Design a causal intervention representation module: block the interference of confounding factors (such as image data background and contextual features) at the feature level, ensure that the model relies only on the causal features of pests (such as morphology and texture) for detection, and improve the robustness of the model.

[0176] 4) Design a causal feature alignment module that combines difficult negative sample mining: while improving the model's ability to extract domain-invariant causal features, reduce the impact of background noise on pest detection accuracy, and further improve the model's detection performance and reliability.

[0177] Based on this, the grain pile surface pest detection model training method provided in this application, through the following methods: constructing a multi-domain grain storage pest image dataset; a dual-branch design and module combination method in the grain storage pest detection domain generalization framework; estimation method for confounding factors in the causal intervention representation module; implementation method of backdoor adjustment based on causal intervention cross-attention in the causal intervention representation module; mining mechanism for difficult negative samples in the comparative causal feature alignment module; IoU weighting strategy and its comparative learning process in the comparative causal feature alignment module; image data enhancement method combining frequency domain perturbation and spatial perturbation in the random domain generation module; and model training strategies, including single-domain and multi-domain training methods, achieves the following beneficial effects:

[0178] 1. Application of domain generalization techniques:

[0179] Existing pest detection methods exhibit significant performance degradation when faced with data distributions unseen during training. This application designs three domain generalization modules: a causal intervention representation module, a contrastive causal feature alignment module, and a random domain generation module, integrating them into the pest detection framework. This innovative technical solution effectively alleviates the performance degradation problem caused by data distribution differences in practical applications of pest detection models, thereby significantly improving the model's adaptability to complex and ever-changing real-world detection scenarios and expanding its application scope and practicality.

[0180] 2. Methods for establishing multi-domain pest image datasets:

[0181] This application proposes a method for constructing a multi-domain pest image dataset. Unlike traditional pest detection methods, this application employs a strategy of capturing pest images under actual grain storage and laboratory conditions, fully utilizing the diversity and complexity of real-world storage environments. Simultaneously, this application designs a method for finely dividing the acquired image data into domains by controlling shooting conditions (including but not limited to light intensity, background type, and grain type). This method can generate high-quality training data covering various real-world grain storage scenarios, effectively simulating the diverse and complex situations encountered in real-world pest detection.

[0182] 3. Design of a two-branch generalization framework:

[0183] Existing pest detection methods are mostly limited to simple improvements on existing benchmark deep learning frameworks, without considering the domain generalization problem in actual detection. This application focuses on the model's adaptability to complex detection scenarios. By designing a two-branch domain generalization framework, causal intervention representation, contrastive causal feature alignment, and random domain generation modules are organically integrated. Based on this two-branch structure, training strategies for both single-domain and multi-domain modes are proposed.

[0184] 4. The random domain generation module performs data augmentation to improve domain richness:

[0185] The random domain generation module designed in this application effectively enhances training data by performing randomization operations in the frequency and spatial domains. Unlike traditional data augmentation methods such as translation, flipping, and cropping, which often only change the appearance of image data within a limited range and cannot effectively improve the domain richness of the data, and may even destroy the original semantic information of the image data, the random domain generation module of this application can perturb the domain-related features of image data without destroying the semantic information of the image data, thereby generating more diverse training data. This enhancement method not only effectively expands the domain coverage of the training data, but also enables the model to be exposed to a wider range of feature changes during training, thus significantly improving the model's adaptability and robustness to different detection scenarios.

[0186] 5. Introduce causal intervention representation based on backdoor adjustment:

[0187] This application introduces a causal intervention representation mechanism based on "backdoor adjustment" to eliminate the impact of confounding factors on pest detection performance, thereby improving the model's generalization ability. Existing pest detection methods fail to fully consider the impact of confounding factors (such as background color and contextual information in image data) on model performance during training, leading to a tendency for the model to rely on these spurious correlations for prediction. This results in a significant performance drop when facing new and unseen detection scenarios. To address this issue, this application designs a causal intervention representation module that utilizes a causal intervention cross-attention mechanism to block the interference of confounding factors on model prediction. This module allows the model to focus more on the causal characteristics of the pest itself, such as pest morphology, thus maintaining stable detection performance in new detection scenarios and improving the model's generalization ability.

[0188] 6. Introduce a causal feature alignment strategy that incorporates hard negative sample mining:

[0189] Existing pest detection methods are prone to false detections when dealing with non-pest targets or background noise (such as impurities or gaps between grains) on the surface of grain piles. To address this issue, this application employs a hard negative sample mining technique during model training to automatically identify targets prone to false detection as negative samples. Within a contrastive learning framework, the model not only narrows the feature distance between positive samples but also widens the feature distance between positive and negative samples. This strategy not only enhances the model's ability to extract domain-invariant features but also effectively mitigates false detections, enabling the model to more accurately identify pest targets in complex detection scenarios.

[0190] 7. Training strategies for single-domain and multi-domain modes:

[0191] Existing pest detection methods typically mix all training data together, assuming the data are independent and identically distributed. This approach limits the combinations of data distributions and may lead to insufficient adaptation of the model to changes in data distribution in real-world applications. In contrast, the training strategy proposed in this application can flexibly select single-domain or multi-domain training modes based on whether the training data has domain-specific characteristics. This flexibility not only enriches the combinations of training data distributions but also enhances the model's adaptability to different detection scenarios, enabling the model to better handle diverse and complex data distributions in real-world applications.

[0192] Based on the embodiments of the grain pile surface pest detection model training method provided in the foregoing embodiments, this application also provides an embodiment of a grain pile surface pest detection method, which specifically includes the following:

[0193] Step 300: Collect image data of the surface of the target grain pile within the grain storage space;

[0194] Step 400: Input the target grain pile surface image data into the grain pile surface pest detection model, so that the grain pile surface pest detection model outputs the pest detection result data corresponding to the target grain pile surface image data; wherein, the grain pile surface pest detection model is pre-trained using the grain pile surface pest detection model training method.

[0195] In particular, the grain pile surface pest detection model training method in the embodiments of the grain pile surface pest detection method provided in this application can adopt the processing flow of the above embodiments of the grain pile surface pest detection model training method. Its function will not be repeated here, but can be referred to the detailed description of the above embodiments of the grain pile surface pest detection model training method.

[0196] From a software perspective, this application also provides a training device for the grain pile surface pest detection model, which is used to execute all or part of the aforementioned grain pile surface pest detection model training method. See [link to relevant documentation]. Figure 15 The grain pile surface pest detection model training device specifically includes the following components:

[0197] The iterative training module 10 is used to input different types of data samples, which are pre-generated based on grain pile surface image data with bounding boxes of pests in various data domains, into the first model branch and the second model branch respectively in the current iteration round. This allows the first model branch and the second model branch to perform backbone network feature extraction and backdoor adjustment on the different types of data samples to obtain target feature maps and corresponding pest detection result data. Furthermore, the module determines the causal feature alignment total loss corresponding to the current iteration round based on the target feature maps obtained by the first model branch and the second model branch respectively, and optimizes the first model branch and the second model branch based on the causal feature alignment total loss.

[0198] The application model output module 20 is used to generate a grain pile surface pest detection model based on the first model branch if the current iteration round is the last iteration round. This model model is used to output the corresponding pest detection result data based on the grain pile surface image data.

[0199] The embodiments of the grain pile surface pest detection model training device provided in this application can be used to execute the processing flow of the embodiment of the grain pile surface pest detection model training method in the above embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the above embodiment of the grain pile surface pest detection model training method.

[0200] The training portion of the grain pile surface pest detection model in the aforementioned grain pile surface pest detection model training device can be completed on a server or a client device. The choice can be made based on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations in this regard. If all operations are completed on the client device, the client device may further include a processor for the specific processing of the grain pile surface pest detection model training.

[0201] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0202] The server and the client device can communicate using any suitable network protocol, including those not yet developed as of the date of this application. Such network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Furthermore, such network protocols may also include RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer Protocol) used on top of the aforementioned protocols.

[0203] As described above, the grain pile surface pest detection model training device provided in this application uses grain pile surface image data labeled with pest bounding boxes from various data domains as data samples for training the model. During model training, a two-branch domain generalization framework is used to predict different types of data samples and to adjust the features of the data samples using a backdoor. This effectively improves the domain generalization ability of the trained grain pile surface pest detection model and enhances its adaptability to complex and ever-changing actual detection scenarios. It eliminates the need to maintain multiple dedicated models for detection image data from different data domains, thereby improving the ease of application and effectiveness of the grain pile surface pest detection model. Consequently, it improves the accuracy and reliability of grain pile surface pest detection results, providing rapid and accurate automatic detection results data for grain pest control.

[0204] From a software perspective, this application also provides a grain pile surface pest detection device for performing all or part of the aforementioned grain pile surface pest detection method, see [link to relevant documentation]. Figure 16 The grain pile surface pest detection device specifically includes the following components:

[0205] Image acquisition module 30 is used to acquire image data of the surface of the target grain pile within the grain storage space;

[0206] The model detection module 40 is used to input the target grain pile surface image data into the grain pile surface pest detection model, so that the grain pile surface pest detection model outputs the pest detection result data corresponding to the target grain pile surface image data; wherein, the grain pile surface pest detection model is trained in advance using the grain pile surface pest detection model training method described in the first aspect.

[0207] The embodiments of the grain pile surface pest detection device provided in this application can be used to execute the processing flow of the embodiment of the grain pile surface pest detection method in the above embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the embodiment of the grain pile surface pest detection method in the above embodiments.

[0208] In particular, the grain pile surface pest detection model training method in the embodiments of the grain pile surface pest detection device provided in this application can adopt the processing flow of the above-described embodiments of the grain pile surface pest detection model training method. Its function will not be repeated here, but can be referred to the detailed description of the above-described embodiments of the grain pile surface pest detection model training method.

[0209] The part of the grain pile surface pest detection model training device that performs grain pile surface pest detection can be completed in a server or a client device. The specific choice depends on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations in this regard. If all operations are completed in the client device, the client device may further include a processor for the specific processing of grain pile surface pest detection.

[0210] This application also provides an electronic device, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the grain pile surface pest detection model training method and / or the grain pile surface pest detection method mentioned in the above embodiments. The processor and the memory can be connected via a bus or other means, taking a bus connection as an example. The receiver can be connected to the processor and the memory via wired or wireless means.

[0211] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0212] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the grain pile surface pest detection model training method and / or the program instructions / modules corresponding to the grain pile surface pest detection method in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the grain pile surface pest detection model training method and / or the grain pile surface pest detection method in the above method embodiments.

[0213] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0214] The one or more modules are stored in the memory, and when executed by the processor, they execute the grain pile surface pest detection model training method and / or the grain pile surface pest detection method in the embodiment.

[0215] In some embodiments of this application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected via a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0216] As one implementation method, the functions of the receiver and transmitter in this application can be implemented by transceiver circuits or dedicated transceiver chips, and the processor can be implemented by dedicated processing chips, processing circuits or general-purpose chips.

[0217] As another implementation approach, the server provided in this application embodiment can be implemented using a general-purpose computer. That is, the program code implementing the processor, receiver, and transmitter functions is stored in memory, and the general-purpose processor implements the processor, receiver, and transmitter functions by executing the code in memory.

[0218] This application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of the aforementioned grain pile surface pest detection model training method and / or the grain pile surface pest detection method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0219] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned grain pile surface pest detection model training method and / or the grain pile surface pest detection method.

[0220] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0221] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0222] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0223] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to the embodiments of this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for training a pest detection model on the surface of a grain pile, characterized in that, include: In the current iteration, different types of data samples, which are generated in advance based on grain pile surface image data with various marked pest bounding box labels in multiple data domains, are respectively input into the first model branch and the second model branch, so that the first model branch and the second model branch respectively perform backbone network feature extraction and backdoor adjustment on the different types of data samples to obtain target feature maps and corresponding pest detection result data. Furthermore, the total causal feature alignment loss corresponding to the current iteration round is determined based on the target feature maps obtained by the first model branch and the second model branch respectively, so as to optimize the first model branch and the second model branch based on the total causal feature alignment loss; If the current iteration round is the last iteration round, then a grain pile surface pest detection model is generated according to the first model branch to output the corresponding pest detection result data based on the grain pile surface image data. Both the first model branch and the second model branch include: a convolutional neural network, a causal intervention representation module, a contrastive causal feature alignment module, and a detection head connected in sequence; and the convolutional neural networks in the first model branch and the second model branch share weights, and the detection heads in the first model branch and the second model branch also share weights. The convolutional neural network is used to extract features from the input data samples to output the corresponding original feature map; The causal intervention characterization module is used to determine the confounding factor estimate for the current iteration based on the original feature map corresponding to the data sample, the pest bounding box label corresponding to the data sample, and the confounding factor estimate obtained in the previous iteration; and based on the cross-attention mechanism, to determine the target feature map corresponding to the original feature map after causal intervention characterization based on the confounding factor estimate for the current iteration and the original feature map. The contrastive causal feature alignment module is used to transmit the target feature map after causal intervention to the detection head, and is also used to obtain positive samples and difficult negative samples from the target feature map after causal intervention, so as to calculate the contrastive loss corresponding to each positive sample based on the positive samples and difficult negative samples obtained by the two contrastive causal feature alignment modules respectively, and determine the total causal feature alignment loss corresponding to the current iteration based on the contrastive loss corresponding to each positive sample. The detection head is used to output the pest detection result data of the data sample according to the target feature map after causal intervention.

2. The method for training a pest detection model on the surface of a grain pile according to claim 1, characterized in that, The causal intervention characterization module includes: a confounding factor estimation unit and a causal intervention cross-attention unit; The confounding factor estimation unit is used to obtain a feature vector group consisting of each feature vector falling within the pest bounding box label from the original feature map corresponding to the data sample, based on the pest bounding box label corresponding to the data sample; to use each feature vector as a positive sample corresponding to the data sample to obtain the pest target prediction box corresponding to the data sample based on each positive sample; to calculate the intersection-union ratio (IUR) between the pest target prediction box corresponding to the data sample and the pest bounding box label, and to use the IUR as a weight to determine the weighted average of each positive sample in the feature vector group; and to update the weighted average of the positive samples using an exponential moving average method based on the confounding factor estimation value obtained in the previous iteration to obtain the confounding factor estimation value for the current iteration. The causal intervention cross-attention unit is used to sequentially adjust the number of channels and transform the dimensions of the original feature map to obtain the corresponding adjusted feature vector; the adjusted feature vector and the confounding factor estimate of the current iteration are respectively subjected to backdoor adjustment based on cross-attention to obtain the backdoor-adjusted feature vector; the dimension of the backdoor-adjusted feature vector is restored to be the same as the dimension of the original feature map to obtain the corresponding dimension-restored feature map; the dimension-restored feature map is concatenated with the original feature map to obtain the corresponding concatenated feature map; the dimension of the concatenated feature map is reduced to be the same as the dimension of the original feature map to obtain the corresponding target feature map after causal intervention representation.

3. The method for training a pest detection model on the surface of a grain pile according to claim 2, characterized in that, The comparative causal feature alignment module includes a sample mining unit; and the comparative causal feature alignment modules corresponding to the first model branch and the second model branch respectively share the same causal feature alignment unit. The sample mining unit is used to treat the other feature vectors in the target feature map after causal intervention as negative samples, except for the positive samples. The negative samples are sorted in descending order of confidence, and multiple negative samples, the same number as the number of positive samples, are selected from each of the sorted negative samples in descending order to be designated as difficult negative samples. The causal feature alignment unit is used to calculate the contrast loss corresponding to each of the positive samples and the difficult negative samples obtained by the sample mining unit in the first model branch and the sample mining unit in the second model branch, respectively, with the goal of reducing the feature distance between the positive samples and increasing the feature distance between the positive samples and the difficult negative samples; and to calculate the weighted average of the contrast losses corresponding to each of the positive samples to obtain the total causal feature alignment loss corresponding to the current iteration round.

4. The method for training a pest detection model on the surface of a grain pile according to claim 1, characterized in that, Before generating different types of data samples from grain pile surface image data pre-generated based on various marked pest bounding box labels in multiple data domains, the method further includes: Obtain the image data group corresponding to each preset data domain. Each image data group contains multiple grain pile surface image data with bounding box labels marked with pests. Randomization operations are performed on the frequency domain and spatial domain of each grain pile surface image data to obtain the enhanced image data corresponding to each grain pile surface image data. Different types of data samples are generated based on the surface image data of each grain pile and the corresponding enhanced image data of each surface image data of the grain pile.

5. The method for training a pest detection model on the surface of a grain pile according to claim 4, characterized in that, The step of generating different types of data samples based on the surface image data of each grain pile and the corresponding enhanced image data of each grain pile surface image data includes: The various image data groups are merged to obtain a first training dataset, and each of the grain pile surface image data in the first training dataset is used as a type of data sample for training the first model branch. Furthermore, the enhanced image data corresponding to each of the grain pile surface image data is used as another type of data sample for training the second model branch.

6. The method for training a pest detection model on the surface of a grain pile according to claim 4, characterized in that, The step of generating different types of data samples based on the surface image data of each grain pile and the corresponding enhanced image data of each grain pile surface image data includes: Each of the image data groups is divided into a second training dataset and a third training dataset, so that the second training dataset and the third training dataset each contain image data of the grain pile surface from different data domains; The enhanced image data corresponding to each of the grain pile surface image data in the second training dataset is also added to the second training dataset, and each of the grain pile surface image data and each of the enhanced image data in the second training dataset are used as a type of data sample for training the first model branch. In addition, the enhanced image data corresponding to each of the grain pile surface image data in the third training dataset is also added to the third training dataset, and each of the grain pile surface image data and each of the enhanced image data in the third training dataset are used as another type of data sample for training the second model branch.

7. The method for training a pest detection model on the surface of a grain pile according to claim 1, characterized in that, The step of generating a grain pile surface pest detection model based on the first model branch for outputting corresponding pest detection results data based on grain pile surface image data includes: The model formed by the convolutional neural network and the detection head in the first model branch is used as a grain pile surface pest detection model for outputting corresponding pest detection results data based on grain pile surface image data, wherein the convolutional neural network in the grain pile surface pest detection model is connected to the detection head.

8. A method for detecting pests on the surface of a grain pile, characterized in that, include: Collect image data of the surface of the target grain pile within the grain storage space; The target grain pile surface image data is input into the grain pile surface pest detection model, so that the grain pile surface pest detection model outputs the pest detection result data corresponding to the target grain pile surface image data; wherein, the grain pile surface pest detection model is trained in advance using the grain pile surface pest detection model training method according to any one of claims 1 to 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the grain pile surface pest detection model training method as described in any one of claims 1 to 7, and / or implements the grain pile surface pest detection method as described in claim 8.

Images