Small sample target detection method and device, setting and readable storage medium

Abstract

The invention relates to the field of target detection, in particular to a small sample target detection method and device, setting and a readable storage medium. The small sample target detection method comprises the following steps: detecting a target picture by using a trained small sample target detection model, and outputting a target detection result; wherein a background suppression module, a multi-scale feature extraction module and a comprehensive prototype pyramid distillation module are added behind a backbone network of the small sample target detection model, and the background suppression module is used for suppressing background features in support features; the multi-scale feature extraction module performs multi-scale pooling and fusion on the support features after background suppression; and the comprehensive prototype pyramid distillation module is used for performing hierarchical distillation on the multi-scale pooling and fused features. According to the method, purer class prototypes can be extracted, generalization is high, and small sample targets can be accurately detected.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a small sample target detection method, device, setting and readable storage medium. Background Art

[0002] Traditional target detection methods are highly dependent on data and have limited generalization capabilities. In practical applications, it is difficult to collect enough and diverse target image samples. As a result, these methods often fail to achieve the expected accuracy and can no longer meet current needs. In contrast, small sample target detection methods allow models to learn effectively under limited data conditions, which is particularly important in the development of surface garbage detection.

[0003] Meta-learning has been proven to be a key technology to solve the small sample problem. Usually, meta-learning-based methods require an additional support branch to extract small sample class prototypes, and then the detector performs classification and detection by measuring the distance between the class prototype and the query feature. Throughout the process, the quality of the class prototype has a direct impact on the detection performance of the model. The input of the support branch is the target image under the rectangular box annotation, but the target object does not fill the entire rectangular box regularly, so the target image will contain background information, and the presence of background information will affect the quality of the subsequent extracted class prototype.

[0004] In summary, existing target detection methods have the following main problems: (1) There is background interference in target support instances; (2) The extracted class prototype information is not comprehensive and pure enough; (3) Traditional target detection methods are highly dependent on datasets and have poor generalization.

[0005] Therefore, it is urgent to design a new small sample target detection method to solve the above problems. Summary of the invention

[0006] The technical problem to be solved by the present invention is to overcome the defect of the existing technology that is heavily dependent on data and provide a detection method for small sample targets. The method can achieve better target detection by only tuning and training on a very small number of target samples, and the model has stronger generalization.

[0007] In order to solve the above technical problems, the technical solution of the present invention is: a small sample target detection method, comprising:

[0008] Use the trained small sample target detection model to detect the target image and output the target detection result;

[0009] A background suppression module, a multi-scale feature extraction module and a comprehensive prototype pyramid distillation module are added to the backbone network of the small sample target detection model. The background suppression module is used to suppress background features in supporting features; the multi-scale feature extraction module performs multi-scale pooling and fusion on the supporting features after background suppression; the comprehensive prototype pyramid distillation module is used to perform hierarchical distillation on the features after multi-scale pooling and fusion.

[0010] The inventor of this patent found that the center of the rectangular frame is often the core feature area of ​​the target. Based on this, a lightweight background suppression module is designed to suppress background features by measuring the feature similarity between the surrounding and center of the supporting features, thereby providing high-quality supporting features for the extraction of class prototypes. Therefore, further, the working process of the background suppression module includes:

[0011] First, select the support feature F s The central area features are spatially pooled to obtain the core features F s c ,

[0012]

[0013] Among them, support features To support the image after passing through the backbone network; (H c ,W c ) is the support feature F s The coordinates of the center point, α is the hyperparameter that determines the size of the central area;

[0014] Then the core feature F s c With support feature F s Perform cosine similarity measurement, and determine the features in the obtained similarity graph that are greater than the threshold β as target features and set them to 1;

[0015] Then the similarity graph and the support feature F s Multiply them to get the support feature map F after noise suppression s BS ;

[0016] The formula for this process is:

[0017]

[0018] Furthermore, the working process of the multi-scale feature extraction module is as follows:

[0019] First, the support feature map after noise suppression Perform spatial average pooling with a kernel size of 2 and a stride of 2, spatial average pooling with a kernel size of 7 and a stride of 7, and global average pooling;

[0020] Then the pooled features are spliced ​​in the spatial dimension to obtain the multi-scale features F s I ; The formula is:

[0021]

[0022] in, represents the average pooling function with kernel size k and step size s; SpatialConcatenate(·) represents dimensional concatenation.

[0023] Furthermore, the working process of the comprehensive prototype pyramid distillation module is:

[0024] The created feature query vector Q is used as the learnable class embedding vector, and three-layer attention calculation is performed. The third-layer attention calculation result Attention3 is used as the class prototype;

[0025] The calculation formula for the first layer is:

[0026]

[0027] The second-level calculation formula is:

[0028]

[0029] The calculation formula for the third layer is:

[0030]

[0031] W1, W2 and W3 represent the first, second and third linear layers respectively, and the input is fed into d k Dimension, Q = {Q1, Q2, Q3}, Q1, Q2 and Q3 represent the feature query vectors of the first layer, the second layer and the third layer respectively. The feature query vectors of each layer are independent of each other, and the feature query vectors between each class are also independent of each other.

[0032] Furthermore, the step of obtaining a trained small sample target detection model is to train, tune and test the small sample target detection model using a small sample target detection dataset to obtain a trained small sample target detection model.

[0033] Furthermore, the small sample target detection dataset includes the target meta-training set Target Meta-Tuning Dataset and the target meta-test set In the target meta-training set M train Train on the target meta-tuning dataset M finetune Tune training on the target meta-test set Mtest The model experimental results were obtained on

[0034] T i train The query image Q and the support set S representing category i in the image include support class instances and background. i The data comes from the base class dataset D which contains a large number of samples. base ; T i finetune and T i test The query image Q of the support class instance and background in the image and the support set S representing category i i The data are all from the small sample target dataset D containing small samples. novel , where the base class dataset D base Contains base category C base , small sample target dataset D novel Contains small sample category C novel ,and

[0035] The present invention also relates to a small sample target detection device, which is used to detect targets using a small sample target detection method.

[0036] The present invention also relates to a device comprising:

[0037] Memory for storing computer programs;

[0038] A processor is used to execute the computer program, and the computer program, when executed by the processor, implements the steps of the small sample target detection method.

[0039] The present invention also relates to a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a method for detecting a small sample target.

[0040] After adopting the above technical solution, the present invention has the following beneficial effects:

[0041] 1. The invention designs a background suppression module, a multi-scale feature extraction module and a comprehensive prototype pyramid distillation module in the model structure. The background suppression module is used to suppress the background features in the supporting features to provide high-quality supporting features for the subsequent extraction of class prototypes. The multi-scale feature extraction module performs multi-scale pooling on the supporting features after background suppression to capture multi-scale information. The comprehensive prototype pyramid distillation module performs hierarchical distillation on the multi-scale features to extract a purer class prototype.

[0042] 2. The background suppression module is based on the strategy of measuring the feature similarity between the target and the background, which effectively suppresses the background features in the supporting features and provides better features for the extraction of class prototypes;

[0043] 3. The comprehensive prototype pyramid distillation module adopts the pyramid hierarchical distillation method to perform prototype distillation on multi-scale support features and extract more comprehensive and pure class prototypes to cope with target matching at different scales in query features;

[0044] 4. The model is trained using a small amount of target data, reducing the traditional method’s reliance on a large amount of labeled data. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of a method for detecting a small sample target using water surface garbage as an example of the present invention;

[0046] Figure 2 is a workflow diagram of the background suppression module of the present invention;

[0047] Figure 3 It is a workflow diagram of the multi-scale feature extraction module and the comprehensive prototype pyramid distillation module of the present invention;

[0048] Figure 4 These are two class prototype diagrams generated in the experiment of the present invention;

[0049] Figure 5 Feature heatmaps of query feature maps for FPD and BINet after prototype assignment;

[0050] Figure 6 This is a visualization result diagram of the reasoning of the present invention. DETAILED DESCRIPTION

[0051] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments in conjunction with the accompanying drawings.

[0052] like Figure 1 , Figure 2 and Figure 3 As shown, a small sample target detection method includes:

[0053] Use the trained small sample target detection model to detect the target image and output the target detection result;

[0054] The backbone network of the small sample target detection model is followed by a background suppression module, a multi-scale feature extraction module and a comprehensive prototype pyramid distillation module. The background suppression module is used to suppress background features in supporting features; the multi-scale feature extraction module performs multi-scale pooling and fusion on supporting features after background suppression; the comprehensive prototype pyramid distillation module is used to perform hierarchical distillation on features after multi-scale pooling and fusion.

[0055] In the prior art, the supporting features are spatially pooled and then dot-product aggregated with the query feature map, or the supporting features are directly fused with the query features for attention. For the former, the class prototype after spatial pooling will incorrectly integrate irrelevant features and discard valuable spatial information. For the latter, the pixel-level class prototype retains spatial information, but lacks abstraction, and direct attention fusion calculation will increase the model calculation burden. Therefore, in this embodiment, a comprehensive prototype pyramid distillation module is designed to extract a more comprehensive and pure class prototype. In addition, considering the diversity of target sizes in query features, relying solely on the original supporting features for prototype distillation will result in the extracted class prototype lacking multi-scale information, which is not conducive to matching targets of different scales in the query features. To solve this problem, before prototype distillation, multi-scale pooling fusion is implemented on the supporting features to ensure that the class prototype adapts to targets of different scales in the query features and improves the universality of matching.

[0056] In summary, the background suppression module in this embodiment is used to suppress the background features in the supporting features, providing high-quality supporting features for the subsequent extraction of class prototypes. The multi-scale feature extraction module performs multi-scale pooling on the supporting features after background suppression to capture multi-scale information. The comprehensive prototype pyramid distillation module performs hierarchical distillation on the multi-scale features to extract purer class prototypes.

[0057] It should be noted that the model in this embodiment is improved on the basis of the FPD (Fine-Grained Prototypes Distillation) model, and the targets may be water surface garbage, animals, and the like.

[0058] like Figure 1 As shown, taking water surface garbage as an example, the specific scheme of the detection method of this small sample target is:

[0059] Step 1: Collect image data of water surface garbage and accurately label the images, including the location and category of the garbage.

[0060] Step 2: To address the problem of scarce data, a small sample learning method is introduced; to address the background noise problem in the supporting features, a background suppression module is designed after the backbone network; to address the problem of poor quality of class prototypes, a comprehensive prototype pyramid distillation module is designed; since simply relying on the original supporting features for prototype distillation will result in the extracted class prototype lacking multi-scale information, a multi-scale feature extraction module is designed before prototype distillation to implement multi-scale pooling fusion of supporting features.

[0061] Step 3: Build a small sample water surface garbage detection model. The query branch is used to input the image to be detected, the support branch is used to input the labeled water surface garbage target instance, and the detection head is used to detect the target on the query image.

[0062] Step 4: Input the base class dataset into the model for training, input the collected surface garbage dataset into the model for tuning, test it on the surface garbage test set, and save the model weights with the best effect on the test set.

[0063] Step 5: Use the saved optimal model to detect the water surface garbage images and complete the small sample water surface garbage detection.

[0064] In one embodiment, Figure 2 As shown, the background suppression module is used to suppress the supporting features F s The working process of the background noise in the video includes:

[0065] First, select the support feature F s The central area features are spatially pooled to obtain the core features F s c ,

[0066]

[0067] Among them, support features To support the image after passing through the backbone network; (H c ,W c ) is the support feature F s The coordinates of the center point, α is the hyperparameter that determines the size of the central area;

[0068] Then the core feature F s c With support feature F s Perform cosine similarity measurement, determine the features in the obtained similarity graph that are greater than the threshold β as target features, set them to 1, and suppress background features whose similarity values ​​are lower than a certain threshold;

[0069] Then the similarity graph and the support feature F s Multiply them to get the support feature map F after noise suppression s BS ;

[0070] The formula for this process is:

[0071]

[0072] The module for background suppression of supporting features in this embodiment is different from the annotation method of Mask in semantic segmentation. In target detection, the target is annotated by a rectangular frame, and the shape of the target is irregular. Therefore, in addition to the target, the rectangular frame will also contain part of the background or other target areas. The supporting image comes from the target area under the rectangular frame annotation, which causes the supporting features extracted by the model to also include background features. The presence of background features will directly affect the quality of the generated class prototype. Through our observation, although the target area cannot fill the entire rectangular frame, the central area of ​​the rectangular frame is usually the core area that best represents the target feature. Therefore, we extract the central area features in the supporting features for spatial pooling, and measure the similarity with the supporting features, and then suppress the background features whose similarity values ​​are lower than a certain threshold, so that the background features in the supporting features can be more effectively suppressed, providing higher quality supporting features for the extraction of class prototypes.

[0073] like Figure 2 As shown in the figure, the target in the support image is a dog, but the background area (grass) occupies almost half of the image. We extract the central area features of the support features and perform spatial pooling to measure the similarity with the support features. For the background area (grass), its feature similarity with the dog is low, so we suppress its features. The target features with higher similarity are retained. Through such steps, we can effectively suppress the background features, thereby providing high-quality support features for subsequent prototype distillation. It is worth noting that our background suppression module has no learnable parameters and complex operations.

[0074] In one embodiment, Figure 3 As shown in Figure 2, the working process of the multi-scale feature extraction module is as follows:

[0075] First, the support feature map after noise suppression Perform spatial average pooling with a kernel size of 2 and a stride of 2, spatial average pooling with a kernel size of 7 and a stride of 7, and global average pooling;

[0076] Then the pooled features are spliced ​​in the spatial dimension to obtain the multi-scale features F s I ; The formula is:

[0077]

[0078] in, represents the average pooling function with kernel size k and step size s; SpatialConcatenate(·) represents dimensional concatenation.

[0079] In one embodiment, Figure 3As shown in the figure, the comprehensive prototype pyramid distillation module uses the embedded vector query method to extract class prototypes, builds a three-layer pyramid structure, and gradually refines the prototype in a layer-by-layer manner. The specific working process of the comprehensive prototype pyramid distillation module is as follows:

[0080] The created feature query vector Q is used as the learnable class embedding vector, and three-layer attention calculation is performed. The third-layer attention calculation result Attention3 is used as the class prototype;

[0081] The calculation formula for the first layer is:

[0082]

[0083] The second-level calculation formula is:

[0084]

[0085] The calculation formula for the third layer is:

[0086]

[0087] W1, W2 and W3 represent the first, second and third linear layers respectively, and the input is fed into d k Dimension, Q = {Q1, Q2, Q3}, Q1, Q2 and Q3 represent the feature query vectors of the first layer, the second layer and the third layer respectively. The feature query vectors of each layer are independent of each other, and the feature query vectors between each class are also independent of each other.

[0088] In one embodiment, the step of obtaining a trained small sample target detection model is to train, tune and test the small sample target detection model using a small sample target detection dataset to obtain a trained small sample target detection model.

[0089] For a single small sample target detection task, it can be expressed as T = {(S1, ..., S N ), Q}, where S i represents the support set of category i, si is the supported target instance of category i, and M is the number of supported target instances. Q is the query image including the supported class instances and the background. Given the task T, the small-shot target detection model should detect all instances of N supported classes in Q. Our small-shot target detection task uses the base class dataset D, which includes the large-shot base class dataset D base and a small sample target dataset D containing small samples novel .

[0090] Small-shot object detection datasets include object meta-training sets Target Meta-Tuning Dataset and the target meta-test set in,

[0091] R train ,R finetune ,R test are the number of training iterations, the number of tuning iterations and the number of testing iterations, respectively, i train The query image Q and the support set S representing category i in the image include support class instances and background. i The data comes from the base class dataset D which contains a large number of samples. base ; T i finetune and T i test The query image Q of the support class instance and background in the image and the support set S representing category i i The data are all from the small sample target dataset D novel , where the dataset D base Contains base category C base , dataset D novel Contains small sample category C novel ,and

[0092] In the tuning and testing of the K-shot environment, T i finetune and T i test Each S in i There are only K supported object instances in . In our study, we adopted a two-stage training strategy widely used in the FSOD field, which is divided into two stages: basic training and tuning training. In the basic training stage, we use the meta-training set M train The training and tuning phase is based on the meta-tuning dataset M finetune Tune and train on the meta-test set M test The model experiment results were obtained.

[0093] In the model tuning phase, given the limited training samples, we copy the weight of the feature query vector that best matches the new class from the base class as the feature query in the new class. In the class prototype aggregation of K samples in the test phase, we use the class prototype and the TopN attention means of the supporting features as weights and then perform a weighted sum.

[0094] In one embodiment, a small sample target detection device is used to detect a target using the small sample target detection method in any of the above embodiments.

[0095] In one embodiment, a device includes:

[0096] Memory for storing computer programs;

[0097] The processor is used to execute a computer program. When the computer program is executed by the processor, the steps of the small sample target detection method in any of the above embodiments are implemented.

[0098] In one embodiment, a readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the small sample target detection method in any of the above embodiments are implemented.

[0099] The advantages of the solutions involved in the above embodiments are demonstrated through specific experiments below.

[0100] A. Experimental Benchmarks

[0101] We systematically evaluate our method on the Pascal VOC and COCO datasets, following the small sample partitioning strategy, supporting sample annotations, and evaluation metrics adopted by Meta R-CNN to ensure comparability and fairness of the results.

[0102] Pascal VOC dataset: This dataset contains 20 categories, which we divide into 15 basic categories and 5 Novel categories. In order to conduct a comprehensive evaluation, three different category combinations are implemented, each with a different combination of basic categories and small sample categories. In the evaluation stage, we selected VOC2007-test as the test set and set the K-shot = {1, 2, 3, 5, 10} environment to test the model performance, focusing on the average precision (AP) of the small sample class at IoU = 0.5 as the core evaluation indicator.

[0103] COCO dataset: This dataset has 80 target categories. We select 20 categories that overlap with Pascal VOC as Novel categories, and the remaining 60 categories are defined as basic categories. We conduct experimental evaluation on the test set under the setting of k-shot = {10,30}, and use the average precision (AP) of the Novel category in the range of IoU = 0.5:0.95 as the experimental standard.

[0104] B. Experimental Details

[0105] We work on the mmfewshot[] framework and implement our method on it. ResNet-101 is selected as the backbone network architecture, and its pre-trained weights on ImageNet are used for weight initialization. The training environment is configured with two NVIDIA 3090GPUs, SGD is used as the optimization strategy, the batch size is set to 8, the momentum value is set to 0.9, and the weight decay factor is set to 1e-4. In the basic training stage, we set the initial learning rate to 0.005 and 0.004 for the Pascal VOC and COCO datasets, and the total number of iterations is 20,000 and 110,000 respectively. After 17,000 and 92,000 iterations, respectively, the learning rate decays to 0.001 times the original. In the fine-tuning training stage, the learning rate is adjusted to 0.001, and no learning rate decay is set. The number of iterations for Pascal VOC under K-shot = {1, 2, 3, 5, 10} is set to {800, 1200, 1600, 2000, 3000}; while for the COCO dataset under K-shot = {10, 30}, 10,000 and 18,000 iterations are performed. For the loss function and other hyperparameter settings not specifically stated, we followed the configuration in Meta R-CNN.

[0106] C. Comparative Experiment

[0107] In Table 1, we show the experimental results of the model in this patent (BINet) and other advanced few-shot object detection (FSOD) methods on the VOC dataset. It includes single-run results and average results of multiple runs under different Novel class samples. The experimental results show that in most cases, BINet's performance exceeds the existing advanced methods and achieves the best experimental results. Specifically, in the 10-shot single-run results on Novel set 1, Novel set 2, and Novel set 3 of Pascal VOC, BINet achieved 1.5%, 2.4%, and 1.2% performance improvements over the previous best results, respectively. In the average results of multiple runs, BINet also showed 1.3%, 1%, and 1.2% performance improvements over the previous best results. These data provide strong support for the effectiveness of our method in the FSOD task. We note that there is a certain gap between BINet and the best results when shot=1. We speculate that since the hyperparameter α cannot be selected to the optimal setting, the background suppression module will suppress some poor target features, which may provide a beneficial effect in the case of very few samples. In addition, some of the embedding vectors that need to be learned in the prototype pyramid distillation module are difficult to accurately generalize to specific classes when the data is extremely scarce.

[0108] Table 1: Comparison of experimental results of small sample object detection models on the VOC dataset (Results are inpercentage, * indicates our reproduced results, bold and underline represent the best and suboptimal results respectively)

[0109]

[0110] Table 2 presents the experimental results of BINet on the COCO dataset with various state-of-the-art small sample object detection (FSOD) methods under various technologies and frameworks. The results show that BINet achieves the best performance compared with other methods under the same technology and architecture. Specifically, BINet achieves a 0.8% performance improvement compared with the previous best 10-shot and 30-shot experimental results, respectively. However, BINet still has room for improvement when compared with methods that use transfer learning techniques or methods under advanced architectures. It is worth noting that our method focuses on optimizing class prototypes without using advanced frameworks or some advanced strategies such as Transformer, DETR, or gradient optimization. We firmly believe that with further improvements, BINet can achieve even more outstanding performance on the challenging COCO dataset.

[0111] Table 2: Comparison of experimental results of small sample target detection models on the COCO dataset (Results are inpercentage, * indicates the experimental results we reproduced, bold and underline represent the best and suboptimal results respectively)

[0112]

[0113]

[0114] D. Ablation experiment

[0115] Ablation experiments are conducted on the Novel Set 1 of PASCAL VOC and the COCO dataset, aiming to optimize the selection of hyperparameters within the module and to demonstrate the effectiveness of the modules and the synergy between them.

[0116] There are two hyperparameters in the background suppression module, namely the hyperparameter α that determines the size of the central region and the threshold hyperparameter β of the similarity measurement. Different selections of α and β will affect the experimental results. In order to determine the optimal value of the hyperparameter, we first conducted an ablation experiment on the module on the Novel Set 1 of PASCAL VOC. Since there are two hyperparameters involved, we conducted an ablation experiment using the controlled variable method. First, we selected the hyperparameter α = {0.45, 0.5, 0.55, 0.6} based on experience. Four sets of experiments were conducted. The experimental results show that the performance is significantly improved when α = {0.5, 0.55}. Then we adjusted the step size to 0.25 and selected α = {0.475, 0.525, 0.575}, while keeping Three more experiments were conducted. It was finally found that when α={0.55,0.5} and The experimental effect is the best when , and all experimental results are shown in Table 3.

[0117] Table 3: The first ablation experiment of α and β hyperparameters in the background suppression module on the Novel Set 1 of PASCAL VOC (Results are in percentage, bold and underline represent the best and second-best results respectively)

[0118]

[0119] Next, we keep α = {0.55, 0.5} and adjust We then conducted 6 more experiments, and the experimental results are shown in Table 4. Finally, we determined that the optimal values ​​of α and β are α = 0.55. Under this set of values, the experimental results of shot = {1,2,3,5,10} achieved the best experimental results of {47.9%, 62.3%, 64.5%, 68.2%, 68.3%}. Compared with the baseline model, the accuracy was improved by {0.2%, 1.6%, 1.5%, 1.3%, 1.1%} respectively.

[0120] Table 4: Second ablation experiment of α and β hyperparameters in the background suppression module on Novel Set 1 of PASCAL VOC (Results are in percentage, bold and underline represent the best and second-best results respectively)

[0121]

[0122] Table 5 shows the results of ablation experiments on different modules on Pascal VOC Novel Set 1. The results show that the background suppression module (BS module), multi-scale feature extraction module and comprehensive prototype pyramid distillation module (the multi-scale feature extraction module and the comprehensive prototype pyramid distillation module are combined, abbreviated as CPPD module) proposed by us can significantly improve the performance of the model, and they have good compatibility. When the BS module and the CPPD module are integrated into the baseline model at the same time, the best experimental results are achieved. Specifically, the experimental results when shot = {1, 2, 3, 5, 10} are {47.7%, 63.0%, 64.0%, 68.8%, 68.9%}. In the case of shot = 3, 5 and 10, the performance is improved by 1.0%, 1.9% and 1.7% respectively compared with the baseline model.

[0123] Table 5: Ablation experiments of different modules on Pascal Voc Novel Set 1 (bold and underline represent the best and second-best results respectively)

[0124]

[0125] In addition, we also conducted ablation experiments on each module on the COCO dataset, and the results are detailed in Table 6. The experimental results show that the BS module and the CPPD module are equally effective on the COCO dataset and are compatible with each other. Finally, after integrating these two modules into the baseline model, they achieved the best performance, reaching 17.0% when shot = 10 and 20.0% when shot = 30. Compared with the baseline model, the performance is improved by 1.1% and 0.8% respectively.

[0126] Table 6: Ablation experiments of different modules on COCO dataset

[0127]

[0128] E. Visualization

[0129] (1) In order to verify that the class prototypes of the CPPD module can effectively represent the targets in the support features, we visualized the attention heatmaps of the feature query and support features in the last layer of the module. Figure 4 As shown in the figure, two class prototypes (one class prototype in each of the upper and lower rows) generated by the attention heat map of the PascalVOC Set 1Novel class are mainly shown. It can be found that the attention points of the feature query are all concentrated on the core position of the target.

[0130] (2) We visualize the feature heatmaps of the query feature graphs of FPD and BINet after prototype assignment. Figure 5As shown in the leftmost column of the figure, the query image has been annotated with a rectangular box according to the annotation file. Through the visualization results, it can be found that compared with the FPD model, the target in the query feature of BINet is more obvious and specific after the prototype is assigned. This makes the feature of the target more prominent, which is more conducive to the detector's recognition and detection of the target.

[0131] (3) Figure 6 The visualization results of BINet training on PASCAL VOC Novel Set 1, Novel Set 2, and Novel Set 3 in a 10-shot environment and reasoning on the VOC2007 test set are presented. It can be seen that BINet can accurately identify and detect the target in most cases. But there are also failure cases, such as the lamb in the picture is blocked by a big lamb, resulting in failure to be detected. Although BINet may not be perfect in detecting some special samples, we have made significant progress compared with previous methods.

[0132] F. Conclusion

[0133] This patent proposes a background suppression module, a multi-scale feature extraction module, and a comprehensive prototype pyramid distillation module, aiming to address the challenges in small sample target detection. Support images usually contain a lot of background information rather than the target object itself. To this end, the background suppression module effectively suppresses the background features in the support features based on the strategy of measuring the feature similarity between the target and the background, providing better features for the extraction of class prototypes. The comprehensive prototype pyramid distillation module uses a pyramid hierarchical distillation method to perform prototype distillation on multi-scale support features, extracting more comprehensive and pure class prototypes to cope with target matching of different scales in the query features. A large number of experiments have confirmed the effectiveness of our modules and their compatibility, and also demonstrated the superiority of our method in performance.

[0134] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for detecting a small sample target, characterized in that: include: Use the trained small sample target detection model to detect the target image and output the target detection result; A background suppression module, a multi-scale feature extraction module and a comprehensive prototype pyramid distillation module are added to the backbone network of the small sample target detection model. The background suppression module is used to suppress background features in supporting features; the multi-scale feature extraction module performs multi-scale pooling and fusion on the supporting features after background suppression; the comprehensive prototype pyramid distillation module is used to perform hierarchical distillation on the features after multi-scale pooling and fusion.

2. The method for detecting small sample targets according to claim 1, characterized in that: The working process of the background suppression module includes: First, select the support feature F s Central area features, and spatial pooling to obtain core features Among them, support features To support the image after passing through the backbone network; (H c ,W c ) is the support feature F s The coordinates of the center point, α is the hyperparameter that determines the size of the central area; Then the core features With support feature F s Perform cosine similarity measurement, and determine the features in the obtained similarity graph that are greater than the threshold β as target features and set them to 1; Then the similarity graph and the support feature F s Multiply them to get the support feature map after noise suppression The specific formula is:

3. The method for detecting small sample targets according to claim 1, characterized in that: The working process of the multi-scale feature extraction module is as follows: First, the support feature map after noise suppression Perform spatial average pooling with a kernel size of 2 and a stride of 2, spatial average pooling with a kernel size of 7 and a stride of 7, and global average pooling; Then the pooled features are spliced ​​in the spatial dimension to obtain multi-scale features The formula is: in, represents the average pooling function with kernel size k and step size s; SpatialConcatenate(·) represents dimensional concatenation.

4. The method for detecting small sample targets according to claim 1, characterized in that: The working process of the comprehensive prototype pyramid distillation module is: The created feature query vector Q is used as the learnable class embedding vector, and three-layer attention calculation is performed. The third-layer attention calculation result Attention3 is used as the class prototype; The calculation formula for the first layer is: The second-level calculation formula is: The calculation formula for the third layer is: W1, W2 and W3 represent the first, second and third linear layers respectively, and the input is fed into d k Dimension, Q = {Q1, Q2, Q3}, Q1, Q2 and Q3 represent the feature query vectors of the first layer, the second layer and the third layer respectively. The feature query vectors of each layer are independent of each other, and the feature query vectors between each class are also independent of each other.

5. The method for detecting small sample targets according to claim 1, characterized in that: The steps of obtaining a trained small sample target detection model are to train, tune and test the small sample target detection model using a small sample target detection dataset to obtain a trained small sample target detection model.

6. The method for detecting small sample targets according to claim 5, characterized in that: Small-shot object detection datasets include object meta-training sets Target Meta-Tuning Dataset and the target meta-test set In the target meta-training set M train Train on the target meta-tuning dataset M finetune Tune training on the target meta-test set M test The model experimental results were obtained on T i train The query image Q and the support set S representing category i in the image include support class instances and background. i The data comes from the base class dataset D which contains a large number of samples. base ; and T i test The query image Q and the support set S representing category i in the image include support class instances and background. i The data are all from the small sample target dataset D containing small samples. novel , where the base class dataset D base Contains base category C base , small sample target dataset D novel Contains small sample category C novel ,and 7. A detection device for small sample targets, characterized in that: Used to detect targets using the small sample target detection method described in any one of claims 1-6.

8. A device, characterized in that: include: Memory for storing computer programs; A processor is used to execute the computer program, and when the computer program is executed by the processor, the steps of the small sample target detection method as described in any one of claims 1 to 6 are implemented.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the small sample target detection method as described in any one of claims 1 to 6 are implemented.

Images