Small sample target detection method based on class knowledge constraint-self-adaption

Abstract

The invention discloses a small sample target detection method based on class knowledge constraint-self-adaption, and the method comprises the steps: introducing a class self-adaption RoI-Head module, dynamically aggregating and querying a class prototype with the strongest feature correlation based on a multi-head attention mechanism, and improving the RoI feature expression capability; a cosine normalization classifier is adopted to unify discrimination boundaries among different categories, and classification deviation caused by category imbalance is relieved; meanwhile, an old knowledge constraint loss function is designed, constraint is kept through the angle of basic category embedding, original knowledge representation is maintained, and disastrous forgetting is reduced. Experimental results show that the method shows excellent incremental detection performance on a plurality of public data sets, achieves the detection capability of newly added categories under the condition of limited training samples, and maintains the recognition performance of basic categories.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and deep learning technology, and specifically relates to a small sample target detection method based on class knowledge constraint-adaptation. Background Art

[0002] Object detection is a fundamental task in computer vision. With the recent development of deep learning, object detection has made significant progress using large-scale datasets. However, in practical applications, obtaining fully labeled data is costly, and models often need to recognize new categories with only a small number of labeled examples. Few-Shot Object Detection (FSOD) has emerged as a promising approach, leveraging a small number of examples to learn new categories after training on a base class. However, most existing FSOD methods employ simple feature aggregation and linear classifiers, making it difficult to maintain recognition of old categories and prone to catastrophic forgetting, a particularly serious problem in incremental learning scenarios. Therefore, the challenge of simultaneously ensuring generalization from a small sample size and retaining old knowledge is of great practical significance. Summary of the Invention

[0003] To solve the above technical problems, the present invention provides a small sample target detection method based on class knowledge constraint-adaptation, which integrates the class adaptive feature aggregation mechanism and the old knowledge retention strategy, while ensuring the ability to detect new categories and significantly reducing the forgetting of basic category knowledge.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention proposes a small sample target detection method based on class knowledge constraint-adaptation, comprising:

[0006] Step 1: Dataset division: Divide the dataset into a training set and a validation set, where the training set is divided into base classes and new classes according to categories;

[0007] Step 2: Base class training: Configure the base class training model, using the Faster R-CNN framework. Through the CA RoI-Head module, use the multi-head attention mechanism to dynamically aggregate the category prototypes most relevant to the query features, generate enhanced RoI features, and save the base class embedding weights.

[0008] Step 3: New class fine-tuning: Build a new class fine-tuning model, introduce a cosine normalization classifier to unify the discrimination boundaries between different classes, design a legacy knowledge-constrained loss function based on the base class embedding weights, and use the training set to fine-tune the new class.

[0009] Furthermore, the step 1 further includes sampling the base class and the new class based on the meta-learning N-Way K-Shot paradigm, and dividing them into a support set and a query set respectively.

[0010] Furthermore, the step 2 includes the CA RoI-Head module performing cross-attention matching on the support set features and the query RoI features through a multi-head attention mechanism with three shared parameters, and outputting weighted fused enhanced RoI features.

[0011] Furthermore, the second step specifically includes:

[0012] Will support features and query RoI features Converted to Q / K / V of attention structure input and input into the multi-head attention mechanism, the outputs are expressed as and ;

[0013] Use multi-head attention mechanism to process output support features , and perform global average pooling to obtain enhanced support class prototypes;

[0014] Perform Cross Attention matching on the query RoI feature Q and the support class prototype, and output the weighted support class prototype ;

[0015] The weighted support class prototype Perform Hadamard product with query RoI features to achieve feature fusion and obtain the base class enhanced RoI features for classification and regression;

[0016] Feed the base class enhanced RoI features into the classifier and regressor for classification and regression operations, and save the class embedding weights of the classifier .

[0017] Furthermore, the step three specifically includes:

[0018] Input the support set and query set of the new class in the training set into the new class detector for feature extraction;

[0019] Generate query RoI feature map based on query features and input it together with support features into CA RoI Head for training;

[0020] Use the class embedding weights of the classifier Calculate the old knowledge constraint loss and then calculate the total loss function;

[0021] The new class enhanced RoI features generated by CA RoI Head are fed into the cosine normalized classifier and regressor for classification and regression.

[0022] Furthermore, in step 3, the formula of the cosine normalization classifier is as follows:

[0023] ,

[0024] in, embedding weights for the classifier classes, is the feature of the input classifier, is a learnable parameter used to control the peak value of the cosine normalized classifier, is the classification probability corresponding to the feature of the input classifier.

[0025] Furthermore, in step 3, the old knowledge constraint loss function is:

[0026] ,

[0027] in, and Class embedding weights before and after fine-tuning for new classes in the object detection model, and To input the features of the classifier before and after fine-tuning the object detection model, Represents the loss weight of OKC Loss, which is determined by the number of classes in the training dataset. is the number of categories in the base class training set, is the number of categories in the training set for the new class.

[0028] Furthermore, the total loss function L of the target detection model is:

[0029] ,

[0030] in, is the classification loss, is the old knowledge constraint loss, is the regression loss, is the binary classification loss of the RPN network, is the regression loss of the RPN network, is the cross entropy loss.

[0031] In a second aspect, the present invention provides an electronic device comprising: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned class knowledge constraint-adaptive small sample target detection method.

[0032] In a third aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned class knowledge constraint-adaptive small sample target detection method.

[0033] The beneficial effects of the present invention are:

[0034] The present invention uses a class-adaptive RoI-Head module and a multi-head attention mechanism to dynamically aggregate the category prototypes with the strongest correlation with the query features, significantly enhancing the expressive power of the RoI features and improving the model's discrimination of different category features, thereby improving the accuracy of detection. Secondly, the cosine normalized classifier is adopted to effectively unify the discrimination boundaries between different categories, alleviate the classification bias problem caused by category imbalance, and further optimize the classification effect. In addition, the designed old knowledge constraint loss function maintains the original knowledge representation by constraining the angle of embedding of the basic category, significantly reduces the catastrophic forgetting of the basic category during the incremental learning process, and ensures that the model can still maintain good recognition performance of the old category when learning new categories. Overall, the present invention achieves a balance between the detection performance of the basic class and the newly added class in the field of small sample target detection, especially in the incremental learning scenario, significantly improves the overall performance and generalization ability of the detection system, and provides an efficient and reliable solution for small sample target detection tasks in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of a small sample target detection method based on class knowledge constraint-adaptation of the present invention;

[0036] Figure 2 Schematic diagram of the overall structure of the class knowledge constraint-adaptive small sample target detection network (CKCA-Det) model of the present invention;

[0037] Figure 3 Schematic diagram of the Class-Adaptive RoI-Head structure;

[0038] Figure 4 A visual comparison of target detection results before and after adding old knowledge constraint loss and cosine normalization classifier. DETAILED DESCRIPTION

[0039] The present invention will be further described below with reference to the accompanying drawings and examples.

[0040] like Figure 1 As shown in FIG, the present invention is a small sample target detection method based on class knowledge constraint-adaptation mainly composed of the following steps:

[0041] Step 1: Prepare a small sample target detection dataset. First, divide the dataset into a training set and a validation set according to the requirements, where the training set is divided into base classes according to the categories. With the new class ,in Based on the meta-learning N-Way K-Shot paradigm, according to the total number of categories N in the training set, each category is sampled separately. (Basic training), (New class fine-tuning) samples as support set, where, in the base class training stage, , ; In the new category fine-tuning stage, , Set it to 1, 2, 3, 5, 10, or 30 according to the specific experimental requirements. After the support set is set, the remaining training set is used as the query set.

[0042] Step 2: Configure the base class training model and build CA RoI-Head and Faster R-CNN. Figure 2 As shown in "BaseTraining", the base class training will be trained using the base class training set. This invention adopts Faster R-CNN as the basic framework for target detection, where the support set and query set are respectively extracted through the shared parameter backbone. In particular, the query feature needs to pass through the RPN network and perform RoI Align to generate the query RoI feature. Then, the proposed Class-Adaptive (CA) RoI-Head is used to replace the ordinary RoI-Head for training. The output features of the CA RoI-Head will be used for classification and regression through the linear classifier Linear cls and the regressor Reg. At the same time, the final base class detector class embedding weights are saved. .

[0043] Among them, the specific form of CA RoI-Head is as follows Figure 3 As shown in Figure 1, it uses three multi-head attention mechanisms. The support set is input into the CA RoI-Head module, which uses the multi-head attention mechanism to weight all category prototypes and obtain the category representation that best matches the current query RoI. This module uses the following processing flow:

[0044] (1) Support features and query RoI features Converted to Q / K / V of attention structure input and input into the multi-head attention mechanism, the outputs are expressed as and , where "shared" means and The multi-head attention mechanism of the input has shared parameters;

[0045] (2) Use multi-head attention mechanism to process output support features , and perform global average pooling to obtain enhanced support class prototypes;

[0046] (3) Cross Attention matching is performed on the query RoI feature Q and the support class prototype, and the weighted support class prototype is output ;

[0047] (4) The weighted support class prototype Perform Hadamard product with query RoI features to achieve feature fusion and obtain the base class enhanced RoI features for classification and regression;

[0048] (5) The base class enhanced RoI features are fed into the classifier (Linear Classifier or Cosine Classifier) ​​and regressor (Bbox Regression) for classification and regression operations, where the Linear Classifier is only used when training the base class and the Cosine Classifier is only used when fine-tuning the new class. The final base class detector class embedding weights are saved. .

[0049] Step 3: Build a new class fine-tuning model and use the cosine normalization classifier to calculate the old knowledge constraint loss OKCLoss. After the base class training is completed, Figure 2 As shown in "Few-Shot Fine-tuning", the new class fine-tuning will use a small number of samples of the base class + the new class training set for training:

[0050] 1. Input the training set (support set + query set) into the backbone of the new class detector for feature extraction;

[0051] 2. Input the query features into the RPN network to generate the query RoI feature map, and input it together with the support features into the CARoI Head for training;

[0052] 3. Use the class embedding weights saved in step 2 Calculate OKC Loss:

[0053] The principle of old knowledge constraint loss OKC Loss is as follows:

[0054] (1) Compare the cosine similarity of the embedding vectors of the base category before and after fine-tuning;

[0055] (2) Constructing an angle-constrained loss function to encourage the base class representation to remain consistent in the semantic space;

[0056] (3) This loss is optimized together with the classification loss and regression loss as part of the total loss function.

[0057] The formula for OKC Loss is as follows:

[0058] ,

[0059] in, and class embedding weights before and after fine-tuning for new classes in the model, and To input the features of the classifier before and after fine-tuning the object detection model, Represents the loss weight of OKC Loss, which is determined by the number of classes in the training dataset. is the number of categories in the base class training set, is the number of categories in the new class training set. The total loss function of the target detection model is designed as follows:

[0060] ,

[0061] in, Used to enrich the inferred object attention vector in meta-learning. It is implemented through a cross entropy loss that encourages the object attention vector to fall into the class to which each object belongs. This is the old knowledge constraint loss. Other loss functions are the well-known loss functions of Faster R-CNN. is the classification loss, is the regression loss, is the binary classification loss of the RPN network, is the regression loss of the RPN network.

[0062] 4. Send the new class enhanced RoI features generated by CA RoI Head to the cosine normalization classifier CosineClassifier and regressor Bbox Regression for classification and regression:

[0063] The principle of the cosine normalization classifier is as follows:

[0064] All class embedding weights and RoI features are L2 normalized before calculating the cosine similarity. This structure uses a learnable scaling parameter The output probability is adjusted to unify the decision boundaries between different categories and suppress the classifier's bias towards new categories. The formula for the cosine normalization classifier is as follows:

[0065] ,

[0066] in, embedding weights for the classifier classes, is the feature of the input classifier, is a learnable parameter used to control the peak value of the cosine normalized classifier.

[0067] The proposed method can be implemented on standard object detection datasets such as PASCAL VOC and MS COCO. It demonstrates excellent detection capabilities for both basic and novel classes in different configurations, from 1-shot to 10-shot, significantly outperforming existing incremental object detection methods. The specific experimental results are as follows:

[0068] Table 1

[0069] As shown in Table 1, CKCA-Det demonstrates superior performance across all settings. On Split 1, compared to the classic iFSOD method (CFA), our method achieves AP50 improvements of 5.7%, 6.8%, 8.8%, 6.3%, and 6.5% for the 1 / 2 / 3 / 5 / 10 shot scenarios, respectively, significantly outperforming other baselines. While slightly inferior to FSCE and FCT in the 2 / 3 / 10 shot settings of Split 2, CKCA-Det still consistently outperforms CFA. Notably, on Split 3, CKCA-Det achieves substantial improvements over CFA (17.2%, 11.2%, 11.2%, 7.5%, and 6.4% for 1 / 2 / 3 / 5 / 10 shots, respectively), with the performance improvement decreasing with increasing sample size.

[0070] To further validate the generalization capability of CKCA-Det, we conducted incremental few-shot object detection experiments on the MS COCO dataset, which has 60 base classes, 20 novel classes, and a 10 / 30 shot configuration. As shown in Table 2, CKCA-Det maintains excellent performance even on this large-scale benchmark. In the 10-shot setting, CKCA-Det improves the classic incremental method CFA by 5.2% (nAP) and 4.4% (AP50), and by 14.5% compared to ONCE. For the 30-shot scenario, CKCA-Det outperforms CFA by 2.6% (nAP) and 1.7% (AP50). Notably, CKCA-Det demonstrates superior few-shot generalization compared to state-of-the-art FSOD methods such as CME, maintaining performance advantages of 0.6% and 0.4% in the 10 / 30 shot configurations, respectively. These results strongly demonstrate the effectiveness and scalability of our method for large-scale and complex scenes.

[0071] Table 2

[0072] Table 3

[0073] Finally, Table 3 shows the incremental learning performance and ablation study of CKCA-Det. When the model only includes the Class Adaptive RoI (CA RoI) component, the AP50:95 for novel classes reaches 16.1%, while base class performance is limited to 23.4%, indicating that feature aggregation alone cannot fully preserve base class knowledge. After incorporating the OKC loss, novel class performance slightly decreases by 0.4%, while base class and overall metrics significantly improve by 3.8% and 2.7%, respectively. This demonstrates the effectiveness of the OKC loss in mitigating catastrophic forgetting by regularizing base class embedding updates. Compared to state-of-the-art methods, CKCA-Det achieves comprehensive performance superior to ONCE. Figure 4 The visualization results of CKCA-Det are shown, and it can be clearly seen that the detection accuracy is improved after adding OKC Loss.

[0074] The above embodiments are preferred implementation schemes of the present invention. In addition, the present invention can also be implemented in other ways. Any obvious replacement without departing from the concept of the present technical solution is within the scope of protection of the present invention.

[0075] In a second aspect, the present invention provides an electronic device comprising: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned class knowledge constraint-adaptive small sample target detection method.

[0076] In a third aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned class knowledge constraint-adaptive small sample target detection method.

[0077] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A small sample target detection method based on class knowledge constraint-adaptation, characterized in that: The method comprises: Step 1: Dataset division: Divide the dataset into a training set and a validation set, where the training set is divided into base classes and new classes according to categories; Step 2: Base class training: Configure the base class training model, using the Faster R-CNN framework. Through the CA RoI-Head module, use the multi-head attention mechanism to dynamically aggregate the category prototypes most relevant to the query features, generate enhanced RoI features, and save the base class embedding weights. Step 3: New class fine-tuning: Build a new class fine-tuning model, introduce a cosine normalization classifier to unify the discrimination boundaries between different classes, design a legacy knowledge-constrained loss function based on the base class embedding weights, and use the training set to fine-tune the new class.

2. The small sample target detection method based on class knowledge constraint-adaptation according to claim 1 is characterized in that: The step 1 further includes sampling the base class and the new class based on the meta-learning N-Way K-Shot paradigm, and dividing them into a support set and a query set respectively.

3. The small sample target detection method based on class knowledge constraint-adaptation according to claim 2 is characterized in that: The step 2 includes the CA RoI-Head module performing cross-attention matching on the support set features and the query RoI features through a multi-head attention mechanism with three shared parameters, and outputting weighted fused enhanced RoI features.

4. The small sample target detection method based on class knowledge constraint-adaptation according to claim 3 is characterized in that: The second step specifically includes: Will support features and query RoI features Converted to Q / K / V of attention structure input and input into the multi-head attention mechanism, the outputs are expressed as and ; Use multi-head attention mechanism to process output support features , and perform global average pooling to obtain enhanced support class prototypes; Perform Cross Attention matching on the query RoI feature Q and the support class prototype, and output the weighted support class prototype ; The weighted support class prototype Perform Hadamard product with query RoI features to achieve feature fusion and obtain the base class enhanced RoI features for classification and regression; Feed the base class enhanced RoI features into the classifier and regressor for classification and regression operations, and save the class embedding weights of the classifier .

5. The small sample target detection method based on class knowledge constraint-adaptation according to claim 1 is characterized in that: The step three specifically includes: Input the support set and query set of the new class in the training set into the new class detector for feature extraction; Generate query RoI feature map based on query features and input it together with support features into CA RoI Head for training; Use the class embedding weights of the classifier Calculate the old knowledge constraint loss and then calculate the total loss function; The new class enhanced RoI features generated by CA RoI Head are fed into the cosine normalized classifier and regressor for classification and regression.

6. The small sample target detection method based on class knowledge constraint-adaptation according to claim 5 is characterized in that: In step 3, the formula of the cosine normalization classifier is as follows: ， in, embedding weights for the classifier classes, is the feature of the input classifier, is a learnable parameter used to control the peak value of the cosine normalized classifier, is the classification probability corresponding to the feature of the input classifier.

7. The method for detecting small-sample targets based on class knowledge constraint-adaptation according to claim 5, characterized in that: In step 3, the old knowledge constraint loss function is: ， in, and Class embedding weights before and after fine-tuning for new classes in the object detection model, and To input the features of the classifier before and after fine-tuning the object detection model, Represents the loss weight of OKC Loss, which is determined by the number of classes in the training dataset. is the number of categories in the base class training set, is the number of categories in the training set for the new class.

8. The method for detecting small sample targets based on class knowledge constraint-adaptation according to claim 7, characterized in that: The total loss function L of the target detection model is: ， in, is the classification loss, is the old knowledge constraint loss, is the regression loss, is the binary classification loss of the RPN network, is the regression loss of the RPN network, is the cross entropy loss.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs; Wherein, when one or more programs are executed by the one or more processors, the one or more processors implement the small sample target detection method based on class knowledge constraint-adaptation as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that Executable instructions are stored thereon, which, when executed by a processor, enable the processor to implement a class knowledge constraint-adaptive small sample target detection method as described in any one of claims 1-8.

Images