Small sample target detection method based on data association enhancement and feature interaction

Abstract

The invention discloses a small sample target detection method based on data association enhancement and feature interaction. The method comprises the following steps: acquiring a basic class data set for small sample target detection training and a new class data set for fine tuning; taking a target detection network based on two stages of training and fine tuning as a main body framework, and constructing a small sample target detection network containing double branches and backbone network weight sharing; constructing a data association enhancement module, and distinguishing a secondary positive sample and a false positive sample; constructing a two-way mutual support-query feature enhancement module, introducing dense convolution to support a feature guide mechanism, and injecting discriminative semantics which supports centralized learning into a query branch; and carrying out training and fine tuning on network parameters based on the obtained basic type data set and the new type data set to obtain a final small sample target detection model. According to the invention, the overall detection performance of small sample targets is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, more particularly, to a small sample target detection method based on data association enhancement and feature interaction. BACKGROUND

[0002] The research in the field of small sample target detection not only shows key value in real challenges, but also provides strong impetus for the perfection of deep learning theoretical system and the innovation of methodology. As a key research topic in the field of deep learning, small sample target detection provides new strategies for scene applications with data scarcity, explores new learning mechanisms, and also provides new ideas, methods and research directions for other fields of deep learning, promoting the progress of deep learning theory and methodology.

[0003] At present, the mainstream technology for processing small sample target detection generally adopts a two-stage training strategy. First, a pre-training step is performed on the model relying on a sample-rich base class dataset, so that the model learns a general feature representation. Then, through a fine-tuning stage, a small amount of new class samples are adapted, so as to realize the rapid generalization and migration of the model on the new class. Due to the scarcity of new classes, when the model is expanded to new classes, the effect of the model on the new class is far from that of the base class. There are mainly two reasons: (1) the base class and the new class samples are quite different, and the scarcity of new class data samples makes the model unable to fully obtain the performance on the base class. There is a high similarity between the new class and the base class in terms of appearance and feature distribution, which leads to the fact that when the model is expanded to new classes, some base class samples will be misdetected as new classes, or new classes will be detected as base classes, and the model cannot effectively distinguish between base classes and new classes. (2) Most existing small sample target detection algorithms have proposed some methods to enhance the query function by using the support function, but these methods mostly focus on one-way feature enhancement between the query set and the support set, ignoring the two-way promotion between the support and the query features, lacking the "mutual promotion" process, and failing to fully utilize the useful messages that may exist between the support and the query. SUMMARY

[0004] Therefore, the present application provides a small sample target detection method based on data association enhancement and feature interaction to solve the above technical problems.

[0005] To achieve the above purpose, the present application adopts the following technical solutions.

[0006] The small sample target detection method based on data correlation enhancement and feature interaction provided by the application mainly includes two parts: (1) data enhancement: the CLIP model is used to extract the semantic embedding of image features and class label text respectively, and the cosine similarity of the two is calculated. The samples are preliminarily judged by the similarity. This strategy improves the discrimination ability of the model to a certain extent, which helps to further enhance the discriminability of the feature boundary between classes, thereby improving the overall detection performance. (2) Feature interaction: the query feature is used to perform a collective attention operation to generate enhanced support features, and the enhanced support features are used to generate dense kernels to extract more relevant query features, realizing the bidirectional promotion of support and query features, thereby improving the overall detection performance of the model. Specifically, the following steps are included:

[0007] Step S1, obtain the basic class data set for small sample target detection training and the new class data set for fine tuning, and perform data set division according to the N-way K-shot setting.

[0008] Step S2, taking a two-stage training-fine tuning small sample target detection network as the main framework, the network is divided into query branch and support branch two branches, and the feature extraction backbone networks of the two branches share weights; the feature extraction backbone network can be a residual network such as ResNet-50, ResNet-101 or its variants, or a convolutional network with a transformer attention mechanism as the core.

[0009] Step S3, a data correlation enhancement module (ADCE) is constructed, data enhancement is introduced to alleviate the problem of sample scarcity and improve the ability to distinguish between secondary positive samples and false positive samples.

[0010] The data correlation enhancement module includes:

[0011] (1) Cosine similarity calculation. This module is integrated into the model fine-tuning stage, and the model uses CLIP to extract the semantic embedding of image features and class label text, and calculates the cosine similarity of the two.

[0012] Further, the CLIP model is used to calculate the cosine similarity, specifically including:

[0013] 1) using the trained base class detector to infer the candidate box and its confidence score s(x) of the sample x;

[0014] 2) set the confidence score threshold τ to screen the false positive samples FP;

[0015] FP={x|s(x)>τ,y pred ≠y gt} (1)

[0016] wherein, y pred represents the predicted category of the sample, y gt represents the true category of the sample.

[0017] 3) Calculate the semantic similarity Sim(x, T) of the candidate sample image and the target new class text label using the CLIP model;

[0018]

[0019] wherein, x represents the candidate sample image, T represents the target new class text label, f I (·) represents the visual features of the image encoding output, f T (·) represents the text features output by the text encoder.

[0020] (2) False positive sample screening. Samples with high similarity are considered as semantically reliable "secondary positive" samples; while samples with low similarity but still detected as positive by the detector tend to belong to the "false positive" category. This module improves the discrimination ability of the model during the training process to some extent, which helps to enhance the distinguishability of the class boundary, especially when the semantics of the new class and the base class are similar. It shows stronger robustness. Precise screening and reuse of false positive samples to improve the recognition ability of the model for new classes and overall detection performance.

[0021] Further, the false positive sample screening specifically includes:

[0022] 1) Set the similarity threshold β to screen out samples with semantic consistency FP';

[0023] FP' = {x e FP | Sim(x, T) > β} (3)

[0024] 2) Re-label the screened samples and participate in fine-tuning training;

[0025] 3) Calculate the new loss L filter introduced by the CLIP filtered samples;

[0026] L filter = -α t (1-pt) r log(pt) (4)

[0027] wherein, α t is a class balance factor, pt is the predicted probability of the model that the sample x belongs to the class t, and r is an index parameter for adjusting the focus of difficult samples.

[0028] 4) Set the weight hyperparameter, and weight the new loss to the network overall loss L total .

[0029] Ltotal = L cls + L reg + λL filter (5)

[0030] where L cls is the classification loss, L reg is the regression loss, and λ is the loss weight hyper-parameter.

[0031] Step S4, a bidirectional mutual support query-feature enhancement module is constructed to further enhance the class perception ability of the query feature, dense convolution is introduced to support the feature guidance mechanism, and the discriminative semantic learned in the support set is injected into the query branch to realize the attention enhancement of the target region from the feature level.

[0032] In one possible implementation, the construction of the support-query feature enhancement module includes:

[0033] (1) The module mainly includes query-assisted support enhancement and support-assisted query enhancement two branches, which respectively realize the feature interaction between the support set and the query set.

[0034] (2) The query-assisted support enhancement branch uses the query feature to filter out the important information in the support feature that best represents the target class, generates enhanced support features, and uses the obtained support information to enhance the query feature. The specific implementation steps include:

[0035] 1) Generate support feature map F AG . Connect multiple support features of the same class:

[0036] F AG = Concat(F s 1 ,F s 2 ,......,F s k ) (6)

[0037] where F s i represents the i-th support feature in the same class.

[0038] 2) Compress the feature map dimension to obtain the simplified support feature map F':

[0039] F' = CR(F AG ) (7) CR() represents the channel compression operation.

[0040] 3) Generate channel weight GS. The feature map obtained by concatenating the support feature map F' and the query feature F q is transformed by an activation function:

[0041] GS = σ (Concat (F', F q )) (8)

[0042] where σ denotes the Sigmoid activation function.

[0043] 4) Generating enhanced support features.

[0044]

[0045] where denotes channel-wise multiplication, f s is the enhanced support features.

[0046] (3) Adding a dense connection strategy, each layer fully utilizes the features of all previous layers, further promoting the interaction between query and support features. Specifically, the input of the l-th layer not only comes from the previous layer X l-1 , but also contains the splicing results of all feature maps from the initial input X0 to the (l-1)-th layer:

[0047] X l = H l ([X0, X1,..., X l-1 ]) (10)

[0048] where: [X0, X1,..., X l-1 ] denotes the splicing of all previous layer feature maps along the channel dimension; H l (·) denotes the nonlinear transformation function of the l-th layer; X l denotes the output feature map of the current layer. The feature flow mechanism under the dense connection mode, that is, each layer fully utilizes the features of all previous layers, avoids the loss of feature information, especially under the condition of few samples. This feature reuse has a significant improvement on network stability and accuracy performance. After the support features pass through the dense convolution module, the convolution kernel k containing the support class semantics is trained:

[0049] k = DenseConvTrain (f s ) (11)

[0050] (4) The support auxiliary query enhancement branch "injects" the key semantics extracted from the support features into the query features, making the query branch more easily focus on the possible target area and reduce false detection. Specifically, the steps to generate enhanced query features are:

[0051] F q ' = Conv k (F q ) (12)

[0052] where F q is the enhanced query feature, Conv k denotes a convolution using the k parameter.

[0053] Step S5, the extracted features are processed through the data association enhancement module and the support-query feature enhancement module to obtain the final detection target class and position information.

[0054] The small sample target detection model based on data association enhancement and feature interaction provided by the application, the core of which is to fuse the data association enhancement module and the support-query feature enhancement module, the two modules cooperate together, and provide a new small sample target detection solution. Compared with the prior art, the technical scheme provided by the present application has the following beneficial effects:

[0055] (1) A data association enhancement strategy is proposed, which uses the CLIP model to extract the semantic embedding of the image feature and the class label text respectively, and calculates the cosine similarity of the two. The similarity is used to preliminarily judge the sample. This strategy improves the discrimination ability of the model in the training process to some extent, which helps to further enhance the discriminability of the class feature boundary, thereby improving the overall detection performance.

[0056] (2) A support-query feature interaction method is proposed, which uses the query feature to perform a collective attention operation to generate enhanced support features, and then uses the enhanced support features to generate a dense kernel to extract more relevant query features, realizing the bidirectional promotion of support and query features, thereby improving the overall detection performance of the model. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 A flowchart of a small sample target detection method based on data association enhancement and feature interaction provided by the embodiment of the application.

[0058] Figure 2 An architecture diagram of a small sample target detection network provided by the embodiment of the application.

[0059] Figure 3 A data association enhancement strategy structure diagram provided by the embodiment of the application.

[0060] Figure 4 A feature interaction module diagram provided by the embodiment of the application.

[0061] Figure 5 A GradCAM visualization diagram provided by the embodiment of the application (the leftmost column is the original image, the middle column is the baseline method, and the rightmost column is the method proposed by the application). DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0063] Referring to Figure 1 A flowchart of a small sample target detection method based on data association enhancement and feature interaction provided by the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the specific implementation steps of the above method include: Figure 1

[0064] Step 101, obtaining a basic class data set for small sample target detection training and a new class data set for fine-tuning. In this case, two benchmark data sets with wide recognition and authority, PASCAL VOC and MS-COCO, are selected.

[0065] On the PASCAL VOC data set, VOC 2007 and VOC 2012 data are selected as the training set to train the model, and the performance of the model is evaluated by using the VOC 2007 test set. The data set is divided into base classes and new classes. In order to ensure fairness when compared with other methods, three different division strategies of base classes and new classes are designed for the 20 classes of the PASCAL VOC data set. In each division scheme, 5 classes are selected as new classes from the 20 classes by random sampling, and the remaining 15 classes are classified as base classes, and it is strictly ensured that the base class and the new class set do not overlap each other. The specific division settings are as follows: in split 1, the new classes are {bird, bus, cow, motorbike, sofa}, and the remaining 15 classes are base classes; in split 2, the new classes are {aeroplane, bottle, cow, horse, sofa}, and the remaining classes are base classes; and in split 3, the new classes are {boat, cat, motorbike, sheep, sofa}, and the other classes are base classes. During the experiment, the model is first fully trained on the complete labeled data of the base class samples, and then fine-tuned under the condition of only a small number of samples in each new class to verify the learning and generalization ability of the model under the condition of small samples. In the training and fine-tuning stage, the N-way K-shot setting is followed, and the value of K is set to 1, 2, 3, 5 and 10.

[0066] ​When the small sample target detection method is evaluated on the MS-COCO dataset, the dataset contains 80 object categories, and the categories are divided according to the intersection relationship with the PASCAL VOC dataset. Specifically, 60 categories that do not overlap with the PASCAL VOC dataset are determined as base classes, and 20 categories that have class intersection with the PASCAL VOC dataset are set as new classes. There is no overlap between the base classes and the new classes, and the N-way K-shot setting in the small sample learning task is uniformly followed, and each category contains K object instances. Since the MS-COCO dataset has a large number of images and a variety of target instances, this embodiment adopts two relatively high sample number settings of 10-shot and 30-shot to provide sufficient learning samples for the model.

[0067] Step 102, taking the target detection network based on the training-fine tuning two-stage as the main framework, a small sample target detection network containing double branches and backbone network weight sharing is constructed. As shown in Figure 2 This embodiment selects Faster R-CNN as the basic target detection framework, uses ResNet-101 as the main feature extractor, and uses RoIAlign as the RoI feature extractor in the region feature extraction stage. After the base class training is completed, the original full connection layer used for classification in the model is replaced by a global average pooling layer. Specifically, the output features after the res4 block of the backbone network RestNet-101 are selected as the image-level representation, and the Softmax classifier is used for classification prediction and parameter update. The RPN+RoIAlign+two-layer full connection detection head of Faster-RCNN is used, the classification loss L cls The cross entropy is used, and the regression loss L reg Smooth-L1 is used. The query branch inputs the query image I q , and outputs the feature map F q ; the support branch inputs the support image I s , and outputs the feature map F s .

[0068] For the PASCAL VOC dataset, two different training iteration numbers are set in the training, which are 20000 and 10000, and the initial learning rate is set to 0.001 and 0.0005, so as to better control the model parameter update rate. For the MS-COCO dataset with larger data size and more complex categories, the training configuration is expanded. The training iteration is set to 40000 and 20000, and the learning rate is still 0.001 and 0.0005, so as to enhance the learning stability of the model under the condition of multiple categories.

[0069] Step 103, build a data association enhancement module in the support branch to distinguish secondary positive samples and false positive samples. As shown in Figure 3 The data association enhancement module mainly discriminates false positive candidates at the semantic level by introducing CLIP, realizes a more refined sample filtering and enhancement mechanism, alleviates the false positive problem in small sample detection, and enhances the recognition generalization ability of the model to new class targets. The specific implementation steps include:

[0070] Step 1031, use the backbone network trained base class detector to infer the input support image I s , output the candidate box set B = {b i} and its confidence score S = {s i}.

[0071] Step 1032, screen false positive samples: set the score threshold τ = 0.3, screen out candidate boxes with high confidence but conflicting predicted class and real class according to formula (1), as potential false positive samples.

[0072] Step 1033, the new class text label f T (·) is obtained by the CLIP text encoder, and the cosine similarity between the above candidate sample image and f T (·) is calculated according to formula (2).

[0073] Step 1034, set the similarity filtering threshold β = 0.85, compare the similarity, and only keep the samples with semantic consistency with the text label, that is: if Sim(x, T) ≥ β, mark the sample as "secondary positive", and set the class label as the class corresponding to T; Otherwise, mark it as "false positive" and remove it from the training batch.

[0074] Step 1035, add the secondary positive samples to the fine-tuning data subset to further improve the model discrimination ability.

[0075] Step 1036, calculate the new loss introduced by the CLIP filtered samples according to formula (4). In this embodiment, α t = 0.25, r = 2.

[0076] Step 1037, calculate the overall loss of the whole process according to formula (5). In this embodiment, λ = 0.5.

[0077] Step 104, a bidirectional mutual support-query feature enhancement module is constructed, dense convolution is introduced to support the feature guidance mechanism, and discriminative semantic learned by the support set is injected into the query branch. The class perception ability of the query feature is further enhanced through the support-query feature enhancement module, mainly including a query-assisted support enhancement method and a support-assisted query enhancement method, which respectively realize the feature interaction between the support set and the query set. The specific implementation steps include:

[0078] Step 1041, according to formula (6), the K support features under the same category are spliced by channel.

[0079] Step 1042, the feature dimension is simplified by using a channel compression layer, and the channel compression CR(·) adopts 1x1 convolution+BN+ReLU.

[0080] Step 1043, the aggregated support feature F' is combined with the query feature F q After broadcasting in the spatial dimension, splicing is performed, and a Sigmoid function is used to generate a channel weight vector.

[0081] Step 1044, then the weight factor is applied to the support feature to enhance the relevant channel, and an enhanced support feature is generated.

[0082] Step 1045, dense connection kernel generation. L=3 layers of dense blocks are adopted, the size of each convolution kernel is 3x3, the step is 1, the padding is 1, H l (·) is the combination of BN+ReLU+Conv, the support feature is processed by dense convolution according to formula (10) (11), and the convolution kernel k containing the support class semantic is obtained.

[0083] Step 1046, the query feature is convolved by using the kernel k generated by the support feature, and the final enhanced query feature is obtained.

[0084] Step 1047, the enhanced query feature is directly sent to the detection head, and the category and position prediction are completed.

[0085] Step 105, based on the obtained basic class data set and new class data set, the network parameters are trained and fine-tuned to obtain a final small sample target detection model, and small sample target detection in a real scene is performed.

[0086] In order to verify the advancement of the method proposed in the application, the case is compared with the current mainstream small sample target detection method on the new class images of the above data set.

[0087] Based on the PASCAL VOC 2007 test set, N-way K-shot conditions are used for comparative experiments, K takes values of 1, 2, 3, 5, and 10 respectively, and according to the experimental settings of TFA

[28] , three division results of the sample base and new classes are sampled, which are split1, split2 and split3, and the new class detection performance of the model in the 1 / 2 / 3 / 5 / 10-shot environment is tested. nAP50 is used as an evaluation index, and three groups of comparative experiments are carried out with mainstream small sample target detection algorithms, and three groups of experimental results are obtained, wherein Table 1 is the comparative experiment result of split1, Table 2 is the comparative experiment result of split2, and Table 3 is the comparative experiment result of split3.

[0088] From the results in Table 1, it can be seen that the algorithm proposed in the present application achieves higher evaluation accuracy (nAP) under different shot settings, and embodies better small sample target detection capability. Specifically, when the shot number is 10, the model detection accuracy reaches the highest value of 66.3%, which is obviously better than the comparative method. When 2-shot and 3-shot, the method of the present application realizes nAP of 54.2% and 56.2% respectively, which shows that the method of the present application can maintain high detection performance and strong rapid adaptation ability in the case of small sample. Further, with the increase of the number of samples, the detection accuracy of the model shows a steady improvement trend, and the model can better generalize.

[0089] Table 1 Experimental results of different methods under PASCAL VOC dataset split1 Unit: %

[0090]

[0091] According to the experimental results in Tables 2 and 3, under the split2 setting, with the increase of the number of samples, the average accuracy (nAP) of the new class of the method of the present application steadily improves from 30.5% under 1-shot to 52.7% under 10-shot. Under the split3 setting, the model performance also continuously improves with the increase of the number of samples, and nAP improves from 27.6% under 1-shot to 56.5% under 10-shot. The above results show that the method can effectively utilize more sample information to continuously improve the detection performance.

[0092] From the split2 in Table 2 and the split3 in Table 4-3 experimental results, it can be observed that the performance of the method of the present application is relatively weak under 1-shot. But with the increase of the number of samples, the model performance gradually improves and surpasses other methods, and the highest nAP value is achieved under 10-shot. Especially in split3, in addition to the effect of the method of the present application being inferior to other methods under 1-shot, under the rest of the shot settings, the method of the present application all shows the best or close to the best effect. It is worth noting that in the case of a large number of samples, the nAP value of the method of the present application is significantly higher than that of other methods. Through the above experimental results, it is further verified that the method proposed has good detection performance and strong robustness in the small sample target detection task.

[0093] Table 2 Experimental results of different methods under split2 of PASCAL VOC dataset Unit: %

[0094]

[0095] Table 3 Experimental results of different methods under split3 of PASCAL VOC dataset Unit: %

[0096]

[0097] In order to further verify the effectiveness of the data association enhancement module and the feature interaction module on the model, the present case carries out an ablation experiment on the PASCAL VOC dataset NovelSet 1, and the experimental results are shown in Table 4. From the table, it can be seen that the method of the present application exhibits significant performance advantages in 1-shot, 2-shot, 3-shot, 5-shot and 10-shot tasks. We found that the optimization effect of the two components running cooperatively on the model performance is higher than that of a single component. Specifically, this indicates that each component has a positive contribution to the improvement of the model performance. Specifically, by comparing the first row and the second row in Table 4, when the data association enhancement module is introduced, the model nAP is improved by an average of 0.6%. When the QSI module is introduced, the model nAP is improved by an average of 1.0%. The experimental results further show that through the dual mechanism of data association enhancement and feature interaction, the performance of the model in the small sample target detection task is significantly improved.

[0098] Table 4 Ablation experimental results on PASCAL VOC Novel Set 1 Unit: %

[0099]

[0100] In order to more intuitively understand the effectiveness of the method, the embodiment uses GradCAM to perform a visualization experiment. GradCAM calculates the gradient of the prediction score of the target class to the feature map channel through back propagation to obtain a gradient weighting weight. The weight is weighted and summed with the feature map to generate a heat map. The resolution of the GradCAM heat map is consistent with the input image, supporting pixel positioning analysis. The pixel value ranges from 0 to 1, and is usually visualized as a 256-level grayscale image, with the high-light area corresponding to a high contribution level. The pixel value reflects the gradient weighted influence of the input feature on the prediction result, and the higher the value, the more critical the region. In order to enhance the visualization effect, the heat map is converted into pseudo-color through a color scale to enhance the visual contrast. The original image is superimposed according to different transparency values to generate a composite visualization result. GradCAM generates an attention heat map through gradient weighted activation mapping, realizes the explainability of model decision-making in combination with visualization technology, and is widely used in image classification, target detection and other tasks. GradCAM can not only process static tasks but also process dynamic video tasks. The embodiment performs visualization analysis with the help of GradCAM technology, as shown in Figure 5 The Grad-CAM shows the positions mainly focused on by the model when performing target detection, and the network detection region of the method is more concentrated, can capture more specific features of the class, and has a higher utilization rate of features, which also reflects the effectiveness of the method in small sample target detection.

[0101] Compared with the prior art, the technical scheme provided by the embodiment of the present application has the following beneficial effects:

[0102] The small sample target detection model based on data association enhancement and feature interaction provided by the present application introduces a data association enhancement (ADCE) module, uses CLIP to perform image-text similarity calculation, realizes accurate screening and reuse of false positive samples, and alleviates false detection caused by similarity between new classes and base classes. The feature interaction (QSI) is introduced to establish an interaction relationship between the support set and the query set, and a dense convolution is used to support the feature guidance mechanism, so that the discriminative semantics learned in the support set is injected into the query branch, the attention to the target region is enhanced from the feature level, and then a dense kernel with enhanced support function is used to generate related query function. Through comparison experiments, ablation experiments and analysis of visualization results, the effectiveness of the model is verified.

[0103] Although the embodiments of the present application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A small sample target detection method based on data association enhancement and feature interaction, characterized in that, Comprise the following steps: S1, obtain the basic class data set for small sample target detection training and the new class data set for fine tuning, follow the N-way K-shot setting for data set division; S2, take the two-stage training-fine tuning small sample target detection network as the main framework, the network is divided into query branch and support branch two branches, the feature extraction backbone networks of the two branches share weights; S3, construct a data association enhancement module, introduce data enhancement to alleviate the sample scarcity problem, and improve the ability to distinguish between secondary positive samples and false positive samples; S4, construct a bidirectional mutual support support-query feature enhancement module, introduce dense convolution to support the feature guidance mechanism, inject the discriminative semantics learned in the support set into the query branch, and realize the attention enhancement of the target region from the feature level; S5, based on the basic class data set, the network is trained, and then based on the new class data set, the network parameters are fine tuned to obtain the final small sample target detection model for small sample target detection in real scene.

2. The method of claim 1, wherein, The data association enhancement module constructed in step S3 comprises cosine similarity calculation and pseudo-positive sample screening, and the specific steps comprise: Use the trained base class detector to infer the candidate box and confidence score of the sample; Set the confidence score threshold to screen the pseudo-positive samples; Calculate the semantic similarity of the candidate sample image and the target new class text label using the CLIP model; Set the similarity threshold to screen out samples with consistent semantics; The screened samples are relabeled and used for fine tuning training; Calculate the new loss introduced by the CLIP filtered samples and weight them into the network total loss.

3. The method of claim 1, wherein, The support-query feature enhancement module constructed in step S4 comprises: The module includes query-assisted support enhancement and support-assisted query enhancement branches, which realize feature interaction between support set and query set; The query-assisted support enhancement branch uses query features to filter out important information in support features that best represent the target class, and uses the obtained support information to enhance the query features; The support-assisted query enhancement branch injects the key semantics extracted from the support features into the query features, so that the query branch can focus on the possible target area and reduce false detection; Add a dense connection strategy, which fully uses the features of all previous layers at each layer, to further promote the interaction between query and support features.

4. The support-query feature enhancement module for building a two-way mutual assistance according to claim 3, characterized in that, The query-assisted support enhancement branch comprises: Connect multiple support features of the same class to obtain aggregated support feature maps; Simplify the feature dimension by using a channel compression layer; Concatenate the aggregated support features and query features, and use a Sigmoid function to generate a channel weight vector.

5. The support-query feature enhancement module for building a two-way mutual assistance according to claim 3, characterized in that, The support-assisted query enhancement branch comprises: Apply the weight factor generated by the query-assisted support enhancement branch to the support features to enhance the relevant channels; Use dense convolution to establish a connection between the class information in the support features and the query features to obtain a convolution kernel containing support class semantics; Convolve the query features with the above convolution kernel to further promote the interaction between query and support features.

Images