Small sample target detection method and system based on misleading information elimination
By obtaining the category features of base class data in small sample object detection and external knowledge-guided feature extraction, the problem of insufficient learning of new class features and classifiers being susceptible to background interference is solved, and the recognition accuracy of new class and model specificity is improved. Task adaptability.
Patent Information
- Application Number
- CN202411263586.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2025-01-07
AI Technical Summary
The existing small sample object detection method has insufficient learning of new class features, classifiers are susceptible to background information interference, and high error rate of new class recognition, resulting in poor detection performance of models in new categories.
By obtaining the category characteristics of the base class data, fixing other model component parameters in the object detection model, randomly initializing the classifier parameters, using external knowledge to guide the feature extractor, training on the base class data set, and on the new class data set Fine-tuning is performed to constrain the classifier similarity of new classes and background classes using cosine similarity loss, enhancing the model's ability to identify new categories.
The model's learning efficiency and recognition accuracy for new categories are improved, the confusion between new classes and background categories is reduced, the model's adaptability to specific tasks is enhanced, and the recognition accuracy of new categories is improved.
Smart Images
Figure CN119273886A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of target detection, and proposes a small sample target detection method and system based on misleading information elimination. Background Art
[0002] In the existing field of small sample target detection, there are two mainstream technical routes: small sample target detection based on meta-learning and small sample target detection based on transfer learning.
[0003] The key to the small sample object detection method based on meta-learning is "learning how to learn". It constructs a large number of training tasks by sampling from the base class dataset, so that the model can master the ability to quickly generalize to new categories in the process of constantly adapting to new tasks, and finally achieve the goal of completing object detection with only a small number of samples.
[0004] Small-sample object detection based on transfer learning aims to learn generalizable knowledge from base class data and transfer it to detection tasks of new categories. Generally, this type of method consists of two training stages, and its network structure is similar to that of traditional object detection algorithms. In the first stage, the base training stage, the detector uses a large number of base class samples for learning to obtain a base model. The purpose of this stage is to enable the model to master general object detection capabilities. In the second stage, the fine-tuning stage, the model uses a small amount of training data from new categories to update the classifier of the base model to adapt it to the new detection task. In the fine-tuning stage, the parameters of the feature extractor are usually kept fixed or updated only at a small learning rate to prevent overfitting. Through these two stages of training, small-sample object detection methods based on transfer learning can effectively utilize the knowledge of base class data and quickly adapt to new detection tasks, achieving good performance in small-sample scenarios.
[0005] However, small sample target detection based on meta-learning and small sample target detection based on transfer learning still have the following shortcomings.
[0006] 1. Insufficient learning of new category features: Due to the lack of labeled data for new categories in the base training phase, the model mainly learns to identify the features of the base categories. Instances of new categories are treated as background during training, causing the model to tend to extract background information of new category instances rather than their unique discriminative features.
[0007] 2. Classifiers are easily disturbed by background information: In small sample learning, new categories have only a small amount of labeled data, and the extracted features often contain more background information. This makes the classifier easily misled and difficult to effectively distinguish new categories from background categories.
[0008] 3. High error rate in identifying new categories: The model lacks prior knowledge about new categories and is easily disturbed by non-discriminative features, resulting in a high error rate in classifying new category instances, which ultimately affects the performance of detecting new category targets. Summary of the invention
[0009] In view of the above shortcomings, the present invention provides a small sample target detection method and system based on misleading information elimination, which aims to introduce external knowledge guidance, eliminate misleading information of new class identification existing in the model, and enhance the model's recognition of new classes.
[0010] In order to achieve the above-mentioned object of the invention, the technical solution of the present invention includes the following contents.
[0011] A small sample target detection method based on misleading information elimination, the method comprising:
[0012] Get the category features of each category in the base class dataset;
[0013] The classifier parameters in the target detection model are fixed based on the category features, and the target detection model is trained on the base class dataset to obtain the parameters θ of other model components in the target detection model. other ; Wherein, the other model components include: backbone network, RPN network, RCNN network (including classification network and bounding box regression network);
[0014] Using the parameter θ other After fixing the parameters of other model components in the target detection model and randomly initializing the classifier parameters, the target detection model is trained on the base class dataset to obtain the classifier parameters θ classifier ;
[0015] Based on the parameters θ of the other model components other and the parameters θ of the classifier classifier , generate the base model;
[0016] Fine-tuning the parameters of the base model on a small sample data set consisting of new class data and base class data to obtain a trained object detection model;
[0017] Complete the target detection task based on the trained target detection model.
[0018] Furthermore, the step of obtaining the category features of each category in the base category data set includes:
[0019] Get the category names contained in the base class dataset;
[0020] The category name is input into the large model to obtain the category features of the category.
[0021] Furthermore, the step of obtaining the category features of each category in the base category data set includes:
[0022] Input the base class data into the large-scale text model to obtain the description of the i-th category Among them, 1≤j≤N des , N des The number of descriptions given for large-scale text models;
[0023] Will describe Input to CLIP's text encoder to get this description After the feature vector of The feature vectors of are averaged to obtain the category features of the i-th category.
[0024] Furthermore, the weights in the classifier include weights corresponding to the base class and weights corresponding to the background class; wherein, when training the target detection model on the base class data set, the weights corresponding to the base class are fixed, and the weights corresponding to the background class are learned.
[0025] Furthermore, when the target detection model is trained on the base class data set, the loss of the target detection model is L1=L RPN +L reg +L cls Among them, L RPN is the loss of the RPN network, L reg is the regression loss in the RCNN network, L cls is the classification loss in the RCNN network.
[0026] Furthermore, the target detection model is trained on the base class data set, and the loss of the target detection model is L2=L cls Among them, L cls is the classification loss in the RCNN network.
[0027] Furthermore, fine-tuning the parameters of the base model on a small sample data set consisting of new class data and base class data includes:
[0028] Initializing the weights of the corresponding base classes and the corresponding background classes in the classifier based on the base model, and randomly initializing the weights of the corresponding new classes;
[0029] On a small sample data set consisting of new class data and base class data, the parameters of the base model are fine-tuned based on the loss L3; wherein the loss L3 = L RPN +L reg +L cls +L cosine , L RPN is the loss of the RPN network, Lreg is the regression loss in the RCNN network, L cls is the classification loss in the RCNN network, N new is the number of new class data, N base is the number of base class data, W0 is the background class weight, W i is the weight of the new class, when N base +1≤i≤N base +N new , i is the category number of the new category.
[0030] A small sample target detection system based on misleading information elimination, the system comprising:
[0031] The base training module is used to obtain the category features of each category in the base class dataset; fix the classifier parameters in the target detection model based on the category features, and train the target detection model on the new class dataset to obtain the parameters θ of other model components in the target detection model. other ; Wherein, the other model components include: backbone network, RPN network, RCNN network (including classification network and bounding box regression network); using the parameter θ other After fixing the parameters of other model components in the target detection model and randomly initializing the classifier parameters, the target detection model is trained on the base class dataset to obtain the classifier parameters θ classifier ; Based on the parameters θ of the other model components other and the parameters θ of the classifier classifier , generate the base model;
[0032] A fine-tuning module, used to fine-tune the parameters of the base model on a small sample data set consisting of new class data and base class data to obtain a trained object detection model;
[0033] The detection module is used to complete the target detection task based on the trained target detection model.
[0034] An electronic device, characterized in that the electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements any of the above-mentioned small sample target detection methods based on misleading information elimination.
[0035] A computer-readable storage medium, characterized in that computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a processor, any of the above-mentioned small sample target detection methods based on misleading information elimination is implemented.
[0036] Compared with the prior art, the present invention has at least the following beneficial effects:
[0037] The present invention enhances the model's understanding of the semantic relationship between categories by extracting discriminative information in the categories to be identified from external knowledge, improves the model's learning efficiency for new categories, and helps the model distinguish between discriminative information and misleading information.
[0038] The present invention adopts a feature extractor based on external knowledge guidance to enhance the interpretability of the model, so that the learned feature extractor can retain the feature relationship between different categories, and at the same time make the learned features include discriminative features that are beneficial to identifying the target category as much as possible.
[0039] The present invention improves the adaptability of the model to specific tasks through classifier reconstruction, so that it can better handle specific classification tasks based on specific data sets.
[0040] The present invention reduces confusion between the new class and the background class by constraining the classifier similarity of the new class and the background class, thereby improving the recognition accuracy of the model for the new class. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 The overall structure diagram of the present invention. DETAILED DESCRIPTION
[0042] The present invention is further described in detail below in conjunction with specific embodiments. The given examples are only for illustrating the present invention, but not for limiting the scope of the present invention.
[0043] The small sample target detection method based on misleading information elimination of the present invention can effectively eliminate information that interferes with the recognition of new categories by introducing external knowledge, strengthen the learning of discriminative features of new categories, thereby improving the classification accuracy of the classifier for new category instances, and ultimately enhance the model's recognition ability for new categories.
[0044] Specifically, if Figure 1 As shown, the present invention includes two stages: base training and fine-tuning.
[0045] 1. Basic training stage.
[0046] The base training stage of the present invention includes three steps: category attribute extraction based on a large model, feature learning based on external knowledge guidance, and reconstruction of the base class classifier.
[0047] 1. Category attribute extraction based on large models
[0048] This step extracts the category features of the category of interest through a large-scale text pre-training model or a visual-language pre-training model. Specifically, assuming that the category name of the base class is Among them, N baseis the number of categories of the base class, when 1≤i≤N base When , i is the category number of the base class. There are two category feature extraction methods:
[0049] (1) Coarse-grained category feature extraction, that is, input the category name into the large model, extract the features of the category name, and use them as the category features of the category. For example, in the CLIP model, "a photo of a[class name]" can be input into the text encoder to extract the category features containing semantic information. [class name] is 1≤i≤N base .
[0050] (2) Extract fine-grained category features, that is, through large-scale text models, obtain the fine-grained attributes of the category. For example, you can input "What does a [class name] look like?" into models such as ChatGPT or Claude3 to obtain a description of the i-th category. Among them, N des is the number of descriptions given for the large model. Then, 1≤j≤N des Input into the text encoder of CLIP to obtain the feature vector of each attribute. Finally, take the average of the embedding vectors of all attributes to obtain the final category feature.
[0051] The category features obtained by the two methods are expressed as
[0052] 2. Feature learning based on external knowledge guidance
[0053] This step is based on the category feature W obtained from the large model pre To learn the feature extractor. Specifically, the present invention uses W pre Initialize the classifier parameters in the model, then fix the classifier parameters, and optimize other components in the model based on the data contained in the base class dataset. It should be noted that in addition to the weights corresponding to the base class, the classifier also requires weights corresponding to the background class. The present invention randomly initializes the weights corresponding to the background class during initialization, and only fixes the weights corresponding to the base class during model update. The weights corresponding to the background class are learnable. The loss function of the model update keeps the loss of the two-stage detection model Faster R-CNN unchanged:
[0054] L=L RPN +L reg +L cls
[0055] Among them, L RPN is the loss of the RPN network, Lreg is the regression loss in the RCNN network, L cls is the classification loss in the RCNN network.
[0056] 3. Base class classifier reconstruction
[0057] In addition to completing feature learning based on base class data, in order to make the classifier parameters better adapt to specific detection tasks, the present invention no longer uses W pre Instead of using L as a classifier parameter, the model’s classifier is randomly initialized and the classifier is relearned based on the base class data while fixing the parameters of other model components. In the process of reconstructing the base class classifier, L is no longer needed. RPN and L reg Two losses, but only through L cls Update the model's classifier.
[0058] After the base class classifier is reconstructed, the resulting model is the base model. The base model has the ability to detect base class objects and contains generalized knowledge that helps detect other categories.
[0059] 2. Fine-tuning stage.
[0060] In the fine-tuning phase, the parameters of the backbone network, RPN network, bounding box regression network, and classifier in the base model are fine-tuned based on a small sample data set consisting of new classes and base classes. Specifically, the weights of the classifier corresponding to the base class and background class are initialized by the corresponding parameters in the base model, and the weights corresponding to the new class are randomly initialized. At this time, W represents the classifier weights of all categories. The training loss is:
[0061] L=L RPN +L reg +L cls +L cosine
[0062] Among them, the first three losses are the same as the losses in the base training stage. cosine The main purpose of L is to minimize the similarity between the new class weight and the background class weight, helping the model to distinguish features that may mislead the recognition of new classes. cosine It can be defined as:
[0063]
[0064] Among them, W0 is the background class weight, cos is the cosine similarity, N new is the number of new categories, when N base +1≤i≤N base +N new , i is the category number of the new category.
[0065] In summary, the present invention extracts discriminative information in the category to be identified through a large-scale pre-trained model. The powerful semantic understanding ability of the large model is used to extract semantic information of the category to be identified from text data. This semantic information includes a description of the category attributes, which can effectively capture the key features of the category to be identified.
[0066] The present invention enhances the learning ability of the feature extractor based on external knowledge guidance. External knowledge is used to provide additional information for the feature extractor, helping the model to better understand the differences and connections between categories and guiding the model to learn more effective feature representations.
[0067] The present invention reconstructs the classifier based on the learned instance feature representation to enhance the adaptability of the model to specific tasks. By adjusting and reconstructing the classifier, the model can better adapt to the needs of specific tasks, identify the key features of the adapted tasks, and improve the recognition accuracy of the model for specific tasks.
[0068] The present invention constrains the classifier similarity of the new class and the background class based on cosine similarity. By constraining the similarity, confusion between the new class and the background class can be avoided, effectively preventing the new class from being mistakenly identified as the background class, thereby improving the sensitivity of the model to the features of the new class, thereby achieving the purpose of improving the model's recognition accuracy for the new class.
[0069] Experimental Results
[0070] In order to verify the effectiveness of the method, the present invention conducted experiments on the Pascal VOC dataset, and the experimental results are shown in Table 1. Table 1 compares the performance of the present invention with the current SOTA methods TFA and DeFRCN in the field of small sample target detection. It can be seen from Table 1 that except for the 10-shot setting where the performance of the present invention is slightly lower than that of DeFRCN, in the other three settings, the present invention has achieved a consistent improvement compared with DeFRCN, with an average performance improvement of 1.50%. Compared with TFA, the present invention has achieved performance improvements in all settings. The experimental results illustrate the effectiveness of the present invention.
[0071] Table 1 Small sample target detection performance of the present invention on the Pascal VOC dataset (%)
[0072]
[0073]
[0074] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology in this field to understand the content of the present invention and implement it, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.
Claims
1. A small sample target detection method based on misleading information elimination, characterized in that: The method comprises: Get the category features of each category in the base class dataset; The classifier parameters in the target detection model are fixed based on the category features, and the target detection model is trained on the base class dataset to obtain the parameters θ of other model components in the target detection model. other ; Wherein, the other model components include: backbone network, RPN network, RCNN network; Using the parameter θ other After fixing the parameters of other model components in the target detection model and randomly initializing the classifier parameters, the target detection model is trained on the base class dataset to obtain the classifier parameters θ classifier ; Based on the parameters θ of the other model components other and the parameters θ of the classifier classifier , generate the base model; Fine-tuning the parameters of the base model on a small sample data set consisting of new class data and base class data to obtain a trained object detection model; Complete the target detection task based on the trained target detection model.
2. The method according to claim 1, characterized in that The step of obtaining the category features of each category in the base category data set includes: Get the category names contained in the base class dataset; The category name is input into the large model to obtain the category features of the category.
3. The method according to claim 1, characterized in that The step of obtaining the category features of each category in the base category data set includes: Input the base class data into the large-scale text model to obtain the description of the i-th category Among them, 1≤j≤N des , N des The number of descriptions given for large-scale text models; Will describe Input to CLIP's text encoder to get this description After the feature vector of The feature vectors of are averaged to obtain the category features of the i-th category.
4. The method according to claim 1, characterized in that: The weights in the classifier include weights corresponding to base classes and weights corresponding to background classes; wherein, when training a target detection model on a base class data set, the weights corresponding to the base classes are fixed, and the weights corresponding to the background classes are learned.
5. The method according to claim 1, characterized in that When the target detection model is trained on the base class data set, the loss of the target detection model is L1=L RPN +L reg +L cls Among them, L RPN is the loss of the RPN network, L reg is the regression loss in the RCNN network, L cls is the classification loss in the RCNN network.
6. The method according to claim 1, characterized in that The target detection model is trained on the base class data set, and the loss of the target detection model is L2=L cls Among them, L cls is the classification loss in the RCNN network.
7. The method according to claim 1, characterized in that The fine-tuning of the parameters of the base model on a small sample data set consisting of new class data and base class data includes: Initializing the weights of the corresponding base classes and the corresponding background classes in the classifier based on the base model, and randomly initializing the weights of the corresponding new classes; On a small sample data set consisting of new class data and base class data, the parameters of the base model are fine-tuned based on the loss L3; wherein the loss L3 = L RPN +L reg +L cls +L cosion , L RPN is the loss of the RPN network, L reg is the regression loss in the RCNN network, L cls is the classification loss in the RCNN network, N new is the number of new class data, N base is the number of base class data, W0 is the background class weight, W i is the weight of the new class, when N base +1≤i≤N base +N new , i is the category number of the new category.
8. A small sample target detection system based on misleading information elimination, characterized in that: The system comprises: The base training module is used to obtain the category features of each category in the base class data set; fix the classifier parameters in the target detection model based on the category features, and train the target detection model on the base class data set to obtain the parameters θ of other model components in the target detection model other ; Wherein, the other model components include: backbone network, RPN network, RCNN network; using the parameter θ other After fixing the parameters of other model components in the target detection model and randomly initializing the classifier parameters, the target detection model is trained on the base class dataset to obtain the classifier parameters θ classifier ; Based on the parameters θ of the other model components other and the parameters θ of the classifier classifier , generate the base model; A fine-tuning module, used to fine-tune the parameters of the base model on a small sample data set consisting of new class data and base class data to obtain a trained object detection model; The detection module is used to complete the target detection task based on the trained target detection model.
9. An electronic device, characterized in that: The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the small sample target detection method based on misleading information elimination as described in any one of claims 1-7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the small sample target detection method based on misleading information elimination as described in any one of claims 1-7 is implemented.