Pest semi-supervised object detection method combining active learning and multi-scale view learning

Abstract

The present application relates to a kind of pest semi-supervised target detection method of combining active learning and multiscale view learning, comprising: constructing semi-supervised target detection data set;Data augmentation is carried out to semi-supervised target detection data set;Semi-supervised model is constructed and pseudo label is generated;Multi-scale view learning is carried out by student model, and multi-scale view prediction result is obtained;Semi-supervised training is carried out;Active learning is carried out, and semi-supervised target detection data set is updated;The field corn image to be detected is input into the teacher model after final training, and the final pest detection result is obtained.The present application combines active learning to select some data beneficial to model, which can effectively improve performance to label, maximize the use of labeling cost;Add multiscale view learning structure, capture effective information;Finally, more than 80% of the detection accuracy of labeled data can be realized under the condition of limited 20% of labeled data, which greatly saves the labeling cost.

Description

Technical Field

[0001] The present invention relates to the technical field of human and insect pest target detection, and in particular to a semi-supervised insect pest target detection method combining active learning and multi-scale view learning. Background Art

[0002] Ensuring stable food production is crucial to global food security. Crop yields are affected by a variety of factors, with insect pests being particularly significant. For example, destructive pests like the corn borer typically feed on the mesophyll of corn during its growing season, damaging the leaves and severely impacting corn growth, leading to significant yield declines. Furthermore, the fragile corn stems and leaves are susceptible to collapse. Therefore, timely detection and monitoring of pests is a crucial prerequisite for pest control. Early and thorough prevention and control can effectively mitigate losses caused by pests and is crucial to ensuring stable food production.

[0003] Currently, there are three main mainstream methods for pest target detection: manual inspection, machine vision-based processing, and deep learning-based target detection. These methods have the following limitations: The effectiveness of the first method's inspection depends entirely on human subjectivity, is labor-intensive, and is inefficient. The second method's small target areas are easily affected by factors such as complex backgrounds and lighting, resulting in poor detection results and relying on manually designed feature extraction. Studies have shown that the third method can achieve good results for specific tasks in agricultural scenarios, but building a high-performance model requires a large amount of accurately labeled data. The size of the pest area accounts for a very small proportion of the image, and the complex environmental changes in natural fields create complex backgrounds in the collected data, which greatly complicates manual labeling. Therefore, semi-supervised target detection models that can be trained using a small amount of labeled data are key to solving this problem.

[0004] Existing semi-supervised object detection methods randomly select labeled data without considering which data annotations can bring actual and effective performance improvements. This makes it impossible to avoid redundant and invalid data annotations. In addition, most semi-supervised object detection methods are oriented towards Microsoft's coco (Common Objects in Context) dataset, and few are designed for detecting small target areas of pests and diseases. Therefore, how to design a semi-supervised object detector to accurately detect small pest targets and how to select more meaningful data for annotation under given annotation costs are currently top priorities. Summary of the Invention

[0005] In order to solve the problems of poor detection effect of existing semi-supervised target detectors for small target areas of small corn pests in complex environments, difficulty in feature learning and inability to label effective samples, the purpose of the present invention is to provide a semi-supervised target detection method for pests that combines active learning and multi-scale view learning, which can effectively improve data labeling performance, maximize the use of labeling costs, capture effective information, improve detection accuracy, and greatly save labeling costs.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for semi-supervised target detection of insect pests that combines active learning and multi-scale view learning, the method comprising the following steps in sequence:

[0007] (1) Constructing a semi-supervised target detection dataset: Collecting field corn images and preprocessing them to obtain preprocessed images. Randomly select 5% of the preprocessed images for annotation to obtain annotated images. The annotated images and unannotated images together constitute the semi-supervised target detection dataset.

[0008] (2) Performing data enhancement on the semi-supervised target detection dataset to obtain a data-enhanced semi-supervised target detection dataset: performing standard supervised data enhancement on the labeled data in the semi-supervised target detection dataset to obtain enhanced labeled data; performing semi-supervised strong data enhancement and semi-supervised weak data enhancement on the unlabeled data in the semi-supervised target detection dataset to obtain strongly enhanced unlabeled data and weakly enhanced unlabeled data respectively; the enhanced labeled data, strongly enhanced unlabeled data, and weakly enhanced unlabeled data constitute the data-enhanced semi-supervised target detection dataset;

[0009] (3) Constructing a semi-supervised model and generating pseudo labels: The semi-supervised model includes a teacher model and a student model. The weakly enhanced unlabeled data is input into the teacher model for prediction. The teacher model outputs the prediction results, and an adaptive pseudo label filter is used to generate reliable pseudo labels for unsupervised loss.

[0010] (4) Multi-scale view learning through the student model: downsample the strongly enhanced unlabeled data to obtain the downsampled data, combine the downsampled data and the strongly enhanced unlabeled data to form two-scale view data, input the two-scale view data into the student model, and the student model outputs two view feature maps, namely the downsampled feature map and the un-downsampled feature map. After alignment and fusion, the fused feature map is obtained. The fused feature map and the downsampled feature map are input into the student model, and the student model performs forward propagation to obtain the multi-scale view prediction result;

[0011] (5) Perform semi-supervised training: input the enhanced labeled data into the student model to calculate the supervised loss and obtain the supervised loss; at the same time, input the reliable pseudo-label and multi-scale view prediction results into the teacher model to calculate the unsupervised loss and obtain the unsupervised loss; combine the supervised loss and the unsupervised loss, and train the student model through backpropagation. After training, the teacher model is updated through the EMA exponential moving average algorithm to obtain the updated teacher model. The trained student model and the updated teacher model constitute the semi-supervised model after semi-supervised training;

[0012] (6) Active learning: input the unlabeled data into the semi-supervised model after semi-supervised training, calculate the metric value, and filter out the unlabeled data with larger values ​​according to the metric value for labeling. The labeled data is added to the labeled data part of the semi-supervised target detection dataset, and the semi-supervised target detection dataset is updated;

[0013] (7) Repeat steps (2) to (6) until the amount of labeled data in the semi-supervised target detection dataset reaches 20%, completing the training and obtaining the final trained teacher model. The field corn image to be detected is input into the final trained teacher model to obtain the final pest detection result.

[0014] In step (2), the standard supervised data enhancement includes random cropping and scaling, random flipping and normalization; the weak enhancement includes horizontal flipping, vertical flipping, cropping, resizing, rotation and mosaic; and the strong enhancement includes mixing and random erasing.

[0015] The step (3) specifically includes the following steps:

[0016] (3a) The teacher model performs forward prediction on the weakly enhanced unlabeled data to obtain a prediction result, which includes a confidence score Pred tea And the corresponding target prediction box Bbox tea ; The confidence score Pred tea Sort in descending order, cumulative confidence score Pred tea Get the filter bottom index According to the filter bottom index Get the threshold τ l , filter out the confidence scores less than τ l The confidence score and the corresponding target prediction box are used to obtain the filtered confidence score Pred′ tea And the corresponding target prediction box Bbox′ tea :

[0017]

[0018] Pred′ tea =Predtea [Pred tea >τ l ]

[0019] Among them, sort represents the order of Pred tea Sort in descending order, is the confidence score after descending sorting; sum represents the confidence score of Accumulation;

[0020] The confidence score Pred′ after screening tea Store it in a teacher prediction cache Scores of size 100;

[0021] (3b) The confidence score Pred′ after screening tea Operate separately by category: Pred′ for each category tea Contains bimodal data of positive and negative modes, and performs Gaussian mixture modeling on cached Scores[c]:

[0022]

[0023] in, represents the Gaussian distribution of the negative mode, represents the Gaussian distribution of the positive mode, are the weight, mean and variance of the negative mode respectively, are the weight, mean and variance of the positive mode, c is the total number of categories contained in the semi-supervised target detection dataset; s c Represents the value in the confidence score cache of the category, P(s c ) represents the probability distribution of the confidence buffer containing positive and negative modes;

[0024] Infer the posterior using the EM expectation-maximization algorithm Represents the probability of setting the corresponding target prediction box as a pseudo label according to the confidence score, and determines the adaptive score threshold for each class

[0025]

[0026] use Pred tea Filter to get category pseudo labels The positioning pseudo label is obtained by filtering the soft teacher's jitter screening method Category pseudo-labels and localization pseudo-labels Composing reliable pseudo labels

[0027] In the step (4), the strongly enhanced unlabeled data is downsampled to obtain downsampled data, the downsampled data and the strongly enhanced unlabeled data are combined to form view data of two scales, the view data of the two scales are input into the student model, and the student model outputs two view feature maps, namely the downsampled feature map and the un-downsampled feature map. After alignment and fusion, the fused feature map is obtained. Specifically, the following steps are included:

[0028] (4a) For the strongly enhanced unlabeled data img unlabeled Perform 0.5 times downsampling to obtain downsampled data

[0029]

[0030] Where, downsample is downsampling;

[0031] The downsampled data And the unlabeled data img after strong enhancement unlabeled Composed of two scales of view data, the two scales of view data are input to the feature extraction structure of the student model After that, we get two view feature maps, namely the unsampled feature map X and the sampled feature map X d :

[0032]

[0033] Downsample feature map X d Both the unsampled feature map X and the unsampled feature map X contain 5 layers of feature maps from deep to shallow;

[0034] (4b) Align the two view feature maps to obtain the aligned feature map X′ d :

[0035] X′ d =align(X d ,X)

[0036] In the formula, align means alignment;

[0037] The aligned feature map X′ d Fuse it with the unsampled feature map X to obtain the fused feature map X * :

[0038] X * [i] = ECA Fusion (X ' d [i],X[i])

[0039] Where, ECAFusion represents fusion.

[0040] In step (5), the enhanced labeled data is input into the student model to calculate the supervised loss to obtain the supervised loss; at the same time, the reliable pseudo labels and multi-scale view prediction results are input into the teacher model to calculate the unsupervised loss to obtain the unsupervised loss; the combination of supervised loss and unsupervised loss specifically includes the following steps:

[0041] (5a) Input the enhanced labeled data into the student model to calculate the supervision loss and obtain the supervision loss Loss sup :

[0042]

[0043] Among them, Loss cls is the cross entropy loss, Loss reg is the L1 loss, T() is the standard supervised data augmentation, N is the number of samples in the supervised data, and f stu () is the student model; is the labeled data, y n is the label of the annotated data;

[0044] (5b) Input the reliable pseudo labels and multi-scale view prediction results into the teacher model for unsupervised loss calculation to obtain the unsupervised loss Loss un sup :

[0045]

[0046] Among them, T′() is semi-supervised strong data enhancement, Loss cls is the loss of the soft teacher, M is the number of samples of unlabeled data, is a reliable pseudo label, is the mth unlabeled data;

[0047] (5c) Calculate the total loss Loss:

[0048] Loss=Loss sup +λ unsup Loss un sup

[0049] Where λ unsup is the unsupervised loss weight, λ unsup The value of is 2.0, and the loss is back-propagated to the student model to obtain the updated student model after the tth round, and the teacher model is updated by the EMA exponential moving average algorithm to obtain the updated teacher model:

[0050]

[0051] in, is the updated parameter of the teacher model after the tth round, are the parameters of the teacher model in round t-1, are the updated parameters of the student model after the tth round.

[0052] The step (6) specifically includes the following steps:

[0053] (6a) Using the semi-supervised model after semi-supervised training to calculate metrics on unlabeled data;

[0054] (6b) Calculate the information metric of each image, including teacher-student disagreement, meta distance, and uncertainty, and then perform L2 regularization on these three metrics to obtain the information value of each image;

[0055] (6c) Then, the information value of each image is sorted in descending order, and the first α unlabeled data are selected and added to the labeled dataset, where α refers to 2.5% of all collected data, thus forming a new semi-supervised object detection dataset for a new round of training.

[0056] The step (4b) specifically comprises the following steps:

[0057] (4b1) For the downsampled feature map X d , discard its fifth feature map X d [5], then the first feature map X d [1] After upsampling by 2 times, we get X′ d [1], then X′ d [1] Added to the feature group head position to form the downsampled view feature group X′ aligned with the normal view d [i]:

[0058] X′ d [1]=upsample(X d [1])

[0059] X′ d [i] = X d [i-1],1 <i≤5

[0060] (4b2) The aligned X′ d And X performs ECA efficient channel attention fusion. The specific formula is as follows:

[0061] X1[i]=GAP(X[i])

[0062] X2[i]=GAP(X′ d [i])

[0063] X 12 [i]=Cat(X1[i],X2[i])

[0064] X 12 [i] = Conv1×1(X 12 [i])

[0065] Probs 12 [i]=Softmax(X 12 [i])

[0066] X * [i]=Probs 12 [i][1]*X[i]+Probs 12 [i][2]*X′ d [i]

[0067] Among them, GAP is global average pooling; Cat is tensor splicing; Conv1×1 is a one-dimensional convolution with a convolution kernel size of 3; X1[i] is the result of global average pooling of X[i], X[i] is the feature map of unlabeled data without downsampling, and X2[i] is X ′ d [i] The result after GAP, X 12 [i] is the result of tensor concatenation of X1[i], X2[i], Probs 12 [i] is the fusion weight, X * [i] is X′ d The fused feature map result obtained after ECA efficient channel attention fusion with X.

[0068] The step (6b) specifically comprises the following steps:

[0069] (6b1) Calculate the teacher-student divergence: First, the unlabeled data in the semi-supervised target detection dataset is input into the teacher model without any enhancement to obtain the detection output bboxs tea ,labels tea :

[0070] bboxes tea ,labels tea =f tea (img unlabeled )

[0071] Where bboxs tea is the teacher model f tea () for unlabeled data img unlabeled Target prediction box obtained by forward prediction, labels tea It is bboxs tea The corresponding predicted category label;

[0072] Then input the unlabeled data into the student model and get the corresponding student model prediction:

[0073] bboxes stu ,labels stu =f stu (img unlabeled )

[0074] Where bboxs stu is the student model f stu () for unlabeled data img unlabeled Target prediction box obtained by forward prediction, labels stu It is bboxs stu The corresponding predicted category label;

[0075] The teacher model is then regarded as the true label to calculate the average precision mAP of the student and the teacher. The teacher-student divergence is calculated based on the average precision mAP:

[0076] Divergence = 1-mAP(bboxs tea ,labels tea ,bboxs stu ,labels stu )

[0077] (6b2) Calculate Meta Distance: Use the labeled data to calculate the region of interest feature ROIFeat for each class during the supervision loss calculation, and update the meta feature Meta Feat for each class c :

[0078]

[0079] ROI Feats c =mean(ROI Feats,c)

[0080]

[0081] Among them, GAP is global average pooling, GT is the target prediction box of the labeled data, roi_extractor is a method of extracting features from the feature map of the region of interest ROI, and mean() is the average; Represents the backbone and feature enhancement structure in the student model; img labeled It is the labeled data, ROI Feats c represents the ROI feature of the cth class, is the c-th meta-feature after the t-th round of training update;

[0082] Calculate Meta Distance:

[0083]

[0084]

[0085] Among them, consine() is cosine similarity; ROI Feat unlabeled It is the ROI feature of all categories extracted from unlabeled data; Meta Feat c Meta distance is the meta feature of the current c class; c is the meta-deviation of class c; It is the ROI feature of the cth category extracted from unlabeled data; Represents the backbone and feature enhancement structure in the teacher model;

[0086] The Meta Distance of the final image = mean(Meta distance c );

[0087] (6b3) Calculate the uncertainty:

[0088] Get the unprocessed predictions cls_pred, bbox_pred:

[0089]

[0090] in, It is the structure of the teacher model from its backbone network to the ROI detection head. cls_pred is the class distribution probability of each prediction result, and bbox_pred is the bounding box of each prediction result.

[0091] Calculate the class distribution probability entropy of each prediction result, and average the entropy of each prediction result to obtain img unlabeled Uncertainty:

[0092]

[0093] Where B is img unlabeled The number of target prediction boxes predicted in cls_pred(c i ; b j ) is the j-th target prediction box b j Belongs to category c i probability;

[0094] (6b4) Perform L2 regularization on the three metrics calculated above:

[0095]

[0096] Where Info is the information value of the entire graph.

[0097] It can be seen from the above technical solution that the beneficial effects of the present invention are: First, in order to address the defect that the selection of labeled data for general semi-supervised target detection is random, the present invention combines active learning to select some data that are beneficial to the model and can effectively improve performance for labeling, thereby maximizing the use of labeling costs; Second, the general semi-supervised detection framework is improved by adding a multi-scale view learning structure to strengthen the consistency learning of shallow features and deep semantics of pest and disease targets, and capture effective information; Third, in the pseudo-label screening part, a Gaussian mixture model adaptive filter is introduced to avoid the shortcomings of insufficient pseudo-labels of pest and disease targets in the early stage and the introduction of deviations in the later stage. Ultimately, with a limited 20% labeled data, the detection accuracy of more than 80% labeled data can be achieved, greatly saving the labeling cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 is a flow chart of the method of the present invention;

[0099] Figure 2 Schematic diagram of the multi-scale view learning structure of the present invention;

[0100] Figure 3 Schematic diagram of the active learning structure of the present invention;

[0101] Figure 4 Schematic diagram of the detection results obtained by the present invention. DETAILED DESCRIPTION

[0102] like Figure 1 As shown, a semi-supervised pest detection method combining active learning and multi-scale view learning includes the following steps in sequence:

[0103] (1) Constructing a semi-supervised target detection dataset: Collecting field corn images and preprocessing them to obtain preprocessed images. Randomly select 5% of the preprocessed images for annotation to obtain annotated images. The annotated images and unannotated images together constitute the semi-supervised target detection dataset.

[0104] (2) Performing data enhancement on the semi-supervised target detection dataset to obtain a data-enhanced semi-supervised target detection dataset to achieve consistency regularization and pseudo-label learning during the training process: performing standard supervised data enhancement on the labeled data in the semi-supervised target detection dataset to obtain enhanced labeled data; performing semi-supervised strong data enhancement and semi-supervised weak data enhancement on the unlabeled data in the semi-supervised target detection dataset to obtain strongly enhanced unlabeled data and weakly enhanced unlabeled data respectively; the enhanced labeled data, strongly enhanced unlabeled data, and weakly enhanced unlabeled data constitute the data-enhanced semi-supervised target detection dataset;

[0105] (3) Constructing a semi-supervised model and generating pseudo labels, thereby dynamically screening the current prediction results to obtain reliable pseudo labels: the semi-supervised model includes a teacher model and a student model, the weakly enhanced unlabeled data is input into the teacher model for prediction, the teacher model outputs the prediction results, and an adaptive pseudo label filter is used to generate reliable pseudo labels for unsupervised loss on the prediction results;

[0106] (4) Multi-scale view learning through the student model: downsample the strongly enhanced unlabeled data to obtain the downsampled data, combine the downsampled data and the strongly enhanced unlabeled data to form two-scale view data, input the two-scale view data into the student model, and the student model outputs two view feature maps, namely the downsampled feature map and the un-downsampled feature map. After alignment and fusion, the fused feature map is obtained. The fused feature map and the downsampled feature map are input into the student model, and the student model performs forward propagation to obtain the multi-scale view prediction result;

[0107] (5) Perform semi-supervised training: input the enhanced labeled data into the student model to calculate the supervised loss and obtain the supervised loss; at the same time, input the reliable pseudo-label and multi-scale view prediction results into the teacher model to calculate the unsupervised loss and obtain the unsupervised loss; combine the supervised loss and the unsupervised loss, and train the student model through backpropagation. After training, the teacher model is updated through the EMA exponential moving average algorithm to obtain the updated teacher model. The trained student model and the updated teacher model constitute the semi-supervised model after semi-supervised training;

[0108] (6) Active learning: input the unlabeled data into the semi-supervised model after semi-supervised training, calculate the metric value, and filter out the unlabeled data with larger values ​​according to the metric value for labeling. The labeled data is added to the labeled data part of the semi-supervised target detection dataset, and the semi-supervised target detection dataset is updated;

[0109] (7) Repeat steps (2) to (6) until the amount of labeled data in the semi-supervised target detection dataset reaches 20%, so that more accurate corn pest detection results can be obtained under limited labeling costs. After training is completed, the final trained teacher model is obtained, and the field corn image to be detected is input into the final trained teacher model to obtain the final pest detection result, such as Figure 4 shown.

[0110] In step (2), the standard supervised data enhancement includes random cropping and scaling, random flipping and normalization; the weak enhancement includes horizontal flipping, vertical flipping, cropping, resizing, rotation and mosaic; and the strong enhancement includes mixing and random erasing.

[0111] The step (3) specifically includes the following steps:

[0112] (3a) The teacher model performs forward prediction on the weakly enhanced unlabeled data to obtain a prediction result, which includes a confidence score Pred tea And the corresponding target prediction box Bbox tea ; The confidence score Pred tea Sort in descending order, cumulative confidence score Pred tea Get the filter bottom index According to the filter bottom index Get the threshold τ l , filter out the confidence scores less than τ l The confidence score and the corresponding target prediction box are used to obtain the filtered confidence score Pred′ tea And the corresponding target prediction box Bbox′ tea :

[0113]

[0114] Pred′ tea =Pred tea [Pred tea >τ l ]

[0115] Among them, sort represents the order of Pred tea Sort in descending order, is the confidence score after descending sorting; sum represents the confidence score of Accumulation; The purpose of 0.5 is to prevent the cumulative sum from being too small, which would result in all predictions being filtered out.

[0116] The confidence score Pred′ after screening tea Store it in a teacher prediction cache Scores of size 100;

[0117] (3b) The confidence score Pred′ after screening tea Operate separately by category: Pred′ for each category tea Contains bimodal data of positive and negative modes, and performs Gaussian mixture modeling on cached Scores[c]:

[0118]

[0119] in, represents the Gaussian distribution of the negative mode, represents the Gaussian distribution of the positive mode, are the weight, mean and variance of the negative mode respectively, are the weight, mean and variance of the positive mode, c is the total number of categories contained in the semi-supervised target detection dataset; s c Represents the value in the confidence score cache of the category, P(s c ) represents the probability distribution of the confidence buffer containing positive and negative modes;

[0120] Infer the posterior using the EM expectation-maximization algorithm Represents the probability of setting the corresponding target prediction box as a pseudo label according to the confidence score, and determines the adaptive score threshold for each class

[0121]

[0122] use Pred tea Filter to get category pseudo labels The positioning pseudo label is obtained by filtering the soft teacher's jitter screening method Category pseudo-labels and localization pseudo-labels Composing reliable pseudo labels

[0123] like Figure 2 As shown, in step (4), the strongly enhanced unlabeled data is downsampled to obtain the downsampled data, and the downsampled data and the strongly enhanced unlabeled data are combined into view data of two scales. The view data of the two scales are input into the student model, and the student model outputs two view feature maps, namely the downsampled feature map and the undownsampled feature map. After alignment and fusion, the fused feature map is obtained. Specifically, the following steps are included:

[0124] (4a) For the strongly enhanced unlabeled data img unlabeled Perform 0.5 times downsampling to obtain downsampled data

[0125]

[0126] Where, downsample is downsampling;

[0127] The downsampled data And the unlabeled data img after strong enhancement unlabeled Composed of two scales of view data, the two scales of view data are input to the feature extraction structure of the student model After that, we get two view feature maps, namely the unsampled feature map X and the sampled feature map X d :

[0128]

[0129] Downsample feature map X d Both the unsampled feature map X and the unsampled feature map X contain 5 layers of feature maps from deep to shallow;

[0130] (4b) Align the two view feature maps to obtain the aligned feature map X′ d :

[0131] X′ d =align(X d ,X)

[0132] In the formula, align means alignment;

[0133] The aligned feature map X′ d Fuse it with the unsampled feature map X to obtain the fused feature map X * :

[0134] X * [i] = ECA Fusion (X ' d [i],X[i])

[0135] Where, ECAFusion represents fusion.

[0136] In step (5), the enhanced labeled data is input into the student model to calculate the supervised loss to obtain the supervised loss; at the same time, the reliable pseudo labels and multi-scale view prediction results are input into the teacher model to calculate the unsupervised loss to obtain the unsupervised loss; the combination of supervised loss and unsupervised loss specifically includes the following steps:

[0137] (5a) Input the enhanced labeled data into the student model to calculate the supervision loss and obtain the supervision loss Loss sup :

[0138]

[0139] Among them, Losscls is the cross entropy loss, Loss reg is the L1 loss, T() is the standard supervised data augmentation, N is the number of samples in the supervised data, and f stu () is the student model; is the labeled data, y n is the label of the annotated data;

[0140] (5b) Input the reliable pseudo labels and multi-scale view prediction results into the teacher model for unsupervised loss calculation to obtain the unsupervised loss Loss un sup :

[0141]

[0142] Among them, T′() is semi-supervised strong data enhancement, Loss cls is the loss of the soft teacher, M is the number of samples of unlabeled data, is a reliable pseudo label, is the mth unlabeled data;

[0143] (5c) Calculate the total loss Loss:

[0144] Loss=Loss sup +λ unsup Loss unsup

[0145] Where λ unsup is the unsupervised loss weight, λ unsup The value of is 2.0, and the loss is back-propagated to the student model to obtain the updated student model after the tth round, and the teacher model is updated by the EMA exponential moving average algorithm to obtain the updated teacher model:

[0146]

[0147] in, is the updated parameter of the teacher model after the tth round, are the parameters of the teacher model in round t-1, are the updated parameters of the student model after the tth round.

[0148] like Figure 3 As shown, the step (6) specifically includes the following steps:

[0149] (6a) Using the semi-supervised model after semi-supervised training to calculate metrics on unlabeled data;

[0150] (6b) Calculate the information metric of each image, including teacher-student disagreement, meta distance, and uncertainty, and then perform L2 regularization on these three metrics to obtain the information value of each image;

[0151] (6c) Then, the information value of each image is sorted in descending order, and the first α unlabeled data are selected and added to the labeled dataset, where α refers to 2.5% of all collected data, thus forming a new semi-supervised object detection dataset for a new round of training.

[0152] The step (4b) specifically comprises the following steps:

[0153] (4b1) For the downsampled feature map X d , discard its fifth feature map X d [5], then the first feature map X d [1] After upsampling by 2 times, we get X′ d [1], then X′ d [1] Added to the feature group head position to form the downsampled view feature group X′ aligned with the normal view d [i]:

[0154] X′ d [1]=upsample(X d [1])

[0155] X′ d [i] = X d [i-1],1 <i≤5

[0156] (4b2) The aligned X′ d And X performs ECA efficient channel attention fusion. The specific formula is as follows:

[0157] X1[i]=GAP(X[i])

[0158] X2[i]=GAP(X′ d [i])

[0159] X 12 [i]=Cat(X1[i],X2[i])

[0160] X 12 [i] = Conv1×1(X 12 [i])

[0161] Probs 12 [i]=Softmax(X 12 [i])

[0162] X *[i]=Probs 12 [i][1]*X[i]+Probs 12 [i][2]*X′ d [i]

[0163] Among them, GAP is global average pooling; Cat is tensor concatenation; Conv1×1 is a one-dimensional convolution with a convolution kernel size of 3; X1[i] is the result of global average pooling of X[i], X[i] is the feature map of unlabeled data without downsampling, and X2[i] is X′ d [i] The result after GAP, X 12 [i] is the result of tensor concatenation of X1[i], X2[i], Probs 12 [i] is the fusion weight, X * [i] is X′ d The fused feature map result obtained after ECA efficient channel attention fusion with X.

[0164] The step (6b) specifically comprises the following steps:

[0165] (6b1) Calculate the teacher-student divergence: First, the unlabeled data in the semi-supervised target detection dataset is input into the teacher model without any enhancement to obtain the detection output bboxs tea ,labels tea :

[0166] bboxes tea ,labels tea =f tea (img unlabeled )

[0167] Where bboxs tea is the teacher model f tea () for unlabeled data img unlabeled Target prediction box obtained by forward prediction, labels tea It is bboxs tea The corresponding predicted category label;

[0168] Then input the unlabeled data into the student model and get the corresponding student model prediction:

[0169] bboxes stu ,labels stu =f stu (img unlabeled )

[0170] Where bboxs stu is the student model f stu() for unlabeled data img unlabeled Target prediction box obtained by forward prediction, labels stu It is bboxs stu The corresponding predicted category label;

[0171] The teacher model is then regarded as the true label to calculate the average precision mAP of the student and the teacher. The teacher-student divergence is calculated based on the average precision mAP:

[0172] Divergence = 1-mAP(bboxs tea ,labels tea ,bboxs stu ,labels stu )

[0173] (6b2) Calculate Meta Distance: Use the labeled data to calculate the region of interest feature ROIFeat for each class during the supervision loss calculation, and update the meta feature Meta Feat for each class c :

[0174]

[0175] ROI Feats c =mean(ROI Feats,c)

[0176]

[0177] Among them, GAP is global average pooling, CT is the target prediction box of the labeled data, roi_extractor is a method for extracting features from the feature map of the region of interest ROI, and mean() is the average; Represents the backbone and feature enhancement structure in the student model; img labeled It is the labeled data, ROI Feats c represents the ROI feature of the cth class, is the c-th meta-feature after the t-th round of training update;

[0178] Calculate Meta Distance:

[0179]

[0180]

[0181] Among them, consine() is cosine similarity; ROI Feat unlabeled It is the ROI feature of all categories extracted from unlabeled data; Meta Featc Meta distance is the meta feature of the current c class; c is the meta-deviation of class c; It is the ROI feature of the cth category extracted from unlabeled data; Represents the backbone and feature enhancement structure in the teacher model;

[0182] The Meta Distance of the final image = mean(Meta distance c );

[0183] (6b3) Calculate the uncertainty:

[0184] Get the unprocessed predictions cls_pred, bbox_pred:

[0185]

[0186] in, It is the structure of the teacher model from its backbone network to the ROI detection head. cls_pred is the class distribution probability of each prediction result, and bbox_pred is the bounding box of each prediction result.

[0187] Calculate the class distribution probability entropy of each prediction result, and average the entropy of each prediction result to obtain img unlabeled Uncertainty:

[0188]

[0189] Where B is img unlabeled The number of target prediction boxes predicted in cls_pred(c i ; b j ) is the j-th target prediction box b j Belongs to category c i probability;

[0190] (6b4) Perform L2 regularization on the three metrics calculated above:

[0191]

[0192] Where Info is the information value of the entire graph.

[0193] like Figure 4 As shown in the figure, the blue bounding box represents the pest area target detected by the model, corn refers to the target category, that is, the corn pest area, and the number is the model's confidence score for the detected target. The larger the score, the more certain the model is that this is a pest area target.

[0194] In summary, the present invention combines active learning to select some data that are beneficial to the model and can effectively improve performance for annotation, thereby maximizing the use of annotation costs; improves the general semi-supervised detection framework, adds a multi-scale view learning structure, strengthens the consistency learning of shallow features and deep semantics of pest and disease targets, and captures effective information; in the pseudo-label screening part, introduces a Gaussian mixture model adaptive filter, which can avoid the shortcomings of insufficient pseudo-labels of pest and disease targets in the early stage and the introduction of deviations in the later stage. Ultimately, it can achieve a detection accuracy of more than 80% of the labeled data with a limited 20% of the labeled data, greatly saving the labeling cost.

Claims

1. A semi-supervised pest detection method combining active learning and multi-scale view learning, characterized by: The method comprises the following steps in sequence: (1) Constructing a semi-supervised target detection dataset: Collecting field corn images and preprocessing them to obtain preprocessed images. Randomly select 5% of the preprocessed images for annotation to obtain annotated images. The annotated images and unannotated images together constitute the semi-supervised target detection dataset. (2) Performing data enhancement on the semi-supervised target detection dataset to obtain a data-enhanced semi-supervised target detection dataset: Performing standard supervised data enhancement on the labeled data in the semi-supervised target detection dataset to obtain enhanced labeled data; The unlabeled data in the semi-supervised object detection dataset are subjected to semi-supervised strong data enhancement and semi-supervised weak data enhancement respectively, obtaining the strongly enhanced unlabeled data and the weakly enhanced unlabeled data respectively; The enhanced labeled data, the strongly enhanced unlabeled data, and the weakly enhanced unlabeled data constitute the data-enhanced semi-supervised object detection dataset; (3) Constructing a semi-supervised model and generating pseudo labels: The semi-supervised model includes a teacher model and a student model. The weakly enhanced unlabeled data is input into the teacher model for prediction. The teacher model outputs the prediction results, and an adaptive pseudo label filter is used to generate reliable pseudo labels for unsupervised loss. (4) Multi-scale view learning through the student model: downsample the strongly enhanced unlabeled data to obtain the downsampled data, combine the downsampled data and the strongly enhanced unlabeled data to form two-scale view data, input the two-scale view data into the student model, and the student model outputs two view feature maps, namely the downsampled feature map and the un-downsampled feature map. After alignment and fusion, the fused feature map is obtained. The fused feature map and the downsampled feature map are input into the student model, and the student model performs forward propagation to obtain the multi-scale view prediction result; (5) Perform semi-supervised training: input the enhanced labeled data into the student model to calculate the supervised loss and obtain the supervised loss; at the same time, input the reliable pseudo-label and multi-scale view prediction results into the teacher model to calculate the unsupervised loss and obtain the unsupervised loss; combine the supervised loss and the unsupervised loss, and train the student model through backpropagation. After training, the teacher model is updated through the EMA exponential moving average algorithm to obtain the updated teacher model. The trained student model and the updated teacher model constitute the semi-supervised model after semi-supervised training; (6) Active learning: input the unlabeled data into the semi-supervised model after semi-supervised training, calculate the metric value, and filter out the unlabeled data with larger values ​​according to the metric value for labeling. The labeled data is added to the labeled data part of the semi-supervised target detection dataset, and the semi-supervised target detection dataset is updated; (7) Repeat steps (2) to (6) until the proportion of labeled data in the semi-supervised target detection dataset reaches 20%, the training is completed, and the final trained teacher model is obtained. The field corn image to be detected is input into the final trained teacher model to obtain the final pest detection result; The step (6) specifically includes the following steps: (6a) Using the semi-supervised model after semi-supervised training to calculate metrics on unlabeled data; (6b) Calculate the information metric of each image, including teacher-student disagreement, meta distance, and uncertainty, and then perform L2 regularization on these three metrics to obtain the information value of each image; (6c) Then, the information value of each image is sorted in descending order, and the first α unlabeled data are selected and added to the labeled dataset, where α refers to 2.5% of all collected data, thus forming a new semi-supervised object detection dataset for a new round of training.

2. The method for semi-supervised pest detection combining active learning and multi-scale view learning according to claim 1, characterized in that: In step (2), the standard supervised data enhancement includes random cropping and scaling, random flipping and normalization; the weak enhancement includes horizontal flipping, vertical flipping, cropping, resizing, rotation and mosaic; The strong enhancement includes mixing and random erasing.

3. The method for semi-supervised pest detection combining active learning and multi-scale view learning according to claim 1, characterized in that: The step (3) specifically includes the following steps: (3a) The teacher model performs forward prediction on the weakly enhanced unlabeled data to obtain a prediction result, which includes a confidence score Pred tea And the corresponding target prediction box Bbox tea ; The confidence score Pred tea Sort in descending order, cumulative confidence score Pred tea Get the filter bottom index According to the filter bottom index Get the threshold τ l , filter out the confidence scores and corresponding target prediction boxes with confidence scores less than τl, and obtain the filtered confidence score Pred' tea And the corresponding target prediction box Bbox' tea : Before tea =Prev tea [Previous tea >τ l ] Among them, sort represents the order of Pred tea Sort in descending order, is the confidence score after descending sorting; sum represents the confidence score of Accumulation; The confidence score after screening is Pred' tea Store it in a teacher prediction cache Scores of size 100; (3b) The confidence score after screening is Pred' tea Operate separately by category: Pred' for each category tea Contains bimodal data of positive and negative modes, and performs Gaussian mixture modeling on cached Scores[c]: in, represents the Gaussian distribution of the negative mode, represents the Gaussian distribution of the positive mode, are the weight, mean and variance of the negative mode respectively, are the weight, mean and variance of the positive mode, c is the total number of categories contained in the semi-supervised target detection dataset; s c Represents the value in the confidence score cache of the category, P(s c ) represents the probability distribution of the confidence buffer containing positive and negative modes; Infer the posterior using the EM expectation-maximization algorithm Represents the probability of setting the corresponding target prediction box as a pseudo label according to the confidence score, and determines the adaptive score threshold for each class use Pred tea Filter to get category pseudo labels The positioning pseudo label is obtained by filtering the soft teacher's jitter screening method Category pseudo-labels and localization pseudo-labels Composing reliable pseudo labels 4. The method for semi-supervised pest detection combining active learning and multi-scale view learning according to claim 1, characterized in that: In the step (4), the strongly enhanced unlabeled data is downsampled to obtain downsampled data, the downsampled data and the strongly enhanced unlabeled data are combined to form view data of two scales, the view data of the two scales are input into the student model, and the student model outputs two view feature maps, namely the downsampled feature map and the un-downsampled feature map. After alignment and fusion, the fused feature map is obtained. Specifically, the following steps are included: (4a) For the strongly enhanced unlabeled data img unlabeled Perform 0.5 times downsampling to obtain downsampled data Where, downsample is downsampling; The downsampled data And the unlabeled data img after strong enhancement unlabeled Composed of two scales of view data, the two scales of view data are input to the feature extraction structure of the student model After that, we get two view feature maps, namely the unsampled feature map X and the sampled feature map X d : Downsample feature map X d Both the unsampled feature map X and the unsampled feature map X contain 5 layers of feature maps from deep to shallow; (4b) Align the two view feature maps to obtain the aligned feature map X' d : X′ d =align(X d ,X) In the formula, align means alignment; The aligned feature map X' d Fuse it with the unsampled feature map X to obtain the fused feature map X * : X * [i]=ECAFusion(X’ d [i],X[i]) Where, ECAFusion represents fusion.

5. The method for semi-supervised pest detection combining active learning and multi-scale view learning according to claim 1, characterized in that: In step (5), the enhanced labeled data is input into the student model to calculate the supervised loss to obtain the supervised loss; at the same time, the reliable pseudo labels and multi-scale view prediction results are input into the teacher model to calculate the unsupervised loss to obtain the unsupervised loss; the combination of supervised loss and unsupervised loss specifically includes the following steps: (5a) Input the enhanced labeled data into the student model to calculate the supervision loss and obtain the supervision loss Loss sup : Among them, Loss cls is the cross entropy loss, Loss reg is the L1 loss, T() is the standard supervised data augmentation, N is the number of samples in the supervised data, and f stu () is the student model; is the labeled data, y n is the label of the annotated data; (5b) Input the reliable pseudo labels and multi-scale view prediction results into the teacher model for unsupervised loss calculation to obtain the unsupervised loss Loss unsup : Among them, T'() is semi-supervised strong data enhancement, Loss cls is the loss of the soft teacher, M is the number of samples of unlabeled data, is a reliable pseudo label, is the mth unlabeled data; (5c) Calculate the total loss Loss: Loss=Loss sup +λ unsup Loss unsup Where λ unsup is the unsupervised loss weight, λ unsup The value of is 2.0, and the loss is back-propagated to the student model to obtain the updated student model after the tth round, and the teacher model is updated by the EMA exponential moving average algorithm to obtain the updated teacher model: in, is the updated parameter of the teacher model after the tth round, are the parameters of the teacher model in round t-1, are the updated parameters of the student model after the tth round.

6. The method for semi-supervised pest detection combining active learning and multi-scale view learning according to claim 4, characterized in that: The step (4b) specifically comprises the following steps: (4b1) For the downsampled feature map X d , discard its fifth feature map X d [5], then the first feature map X d [1] After upsampling by 2 times, we get X' d [1], then X' d [1] Added to the feature group head position to form the downsampled view feature group X' aligned with the normal view d [i]: X′ d [1]=upsample(X d [1]) X′ d [i]=X d [i-1],1<i≤5 (4b2) Align X' d And X performs ECA efficient channel attention fusion. The specific formula is as follows: X1[i]=GAP(X[i]) X2[i]=GAP(X′ d [i]) X 12 [i]=Cat(X1[i],X2[i]) X 12 [i]=Conv1×1(X 12 [i]) Probs 12 [i]=Softmax(X 12 [i]) X * [i]=Probs 12 [i][1]*X[i]+Probs 12 [i][2]*X′ d [i] Among them, GAP is global average pooling; Cat is tensor splicing; Conv1×1 is a one-dimensional convolution with a convolution kernel size of 3; X1[i] is the result of global average pooling of X[i], X[i] is the feature map of unlabeled data without downsampling, and X2[i] is X' d [i] The result after GAP, X 12 [i] is the result of tensor concatenation of X1[i], X2[i], Probs 12 [i] is the fusion weight, X * [i] is X' d The fused feature map result obtained after ECA efficient channel attention fusion with X.

7. The method for semi-supervised pest detection combining active learning and multi-scale view learning according to claim 1, characterized in that: The step (6b) specifically comprises the following steps: (6b1) Calculate the teacher-student divergence: First, the unlabeled data in the semi-supervised target detection dataset is input into the teacher model without any enhancement to obtain the detection output bboxs tea ,labels tea : bboxs tea ,labels tea =f tea (img unlabeled ) Where bboxs tea It is the teacher model ftea() for the unlabeled data img unlabeled Target prediction box obtained by forward prediction, labels tea It is bboxs tea The corresponding predicted category label; Then input the unlabeled data into the student model and get the corresponding student model prediction: bboxs stu ,labels stu =f stu (img unlabeled ) Where bboxs stu is the student model f stu () for unlabeled data img unlabeled Target prediction box obtained by forward prediction, labels stu It is bboxs stu The corresponding predicted category label; The teacher model is then regarded as the true label to calculate the average precision mAP of the student and the teacher. The teacher-student divergence is calculated based on the average precision mAP: Divergence=1-mAP(bboxs tea ,labels tea ,bboxs stu ,labels stu ) (6b2) Calculate Meta Distance: Use the labeled data to calculate the region of interest feature ROIFeat for each class during the supervision loss calculation, and update the meta feature Meta Feat for each class c : ROI Feats c =mean(ROI Feats,c) Among them, GAP is global average pooling, GT is the target prediction box of the labeled data, roi_extractor is a method of extracting features from the feature map of the region of interest ROI, and mean() is the average; Represents the backbone and feature enhancement structure in the student model; img labeled It is the labeled data, ROI Feats c represents the ROI feature of the cth class, is the c-th meta-feature after the t-th round of training update; Calculate Meta Distance: Among them, consine() is cosine similarity; ROI Feat unlabeled It is the ROI feature of all categories extracted from unlabeled data; Meta Feat c Meta distance is the meta feature of the current c class; c is the meta-bias of class c; ROIFeat c unlabeled It is the ROI feature of the cth category extracted from unlabeled data; Represents the backbone and feature enhancement structure in the teacher model; The Meta Distance of the final image = mean(Meta distance c ); (6b3) Calculate the uncertainty: Get the unprocessed predictions cls_pred, bbox_pred: in, It is the structure of the teacher model from its backbone network to the ROI detection head. cls_pred is the class distribution probability of each prediction result, and bbox_pred is the bounding box of each prediction result. Calculate the class distribution probability entropy of each prediction result, and average the entropy of each prediction result to obtain img unlabeled Uncertainty: Where B is img unlabeled The number of target prediction boxes predicted in cls_pred(c i ; b j ) is the j-th target prediction box b j Belongs to category c i probability; (6b4) Perform L2 regularization on the three metrics calculated above: Where Info is the information value of the entire graph.

Images