Implicit feature contrastive learning-based few-shot object detection method
By employing an implicit feature contrastive learning method, we improved the fine-grained prototype distillation network and feature fusion module, which solved the problem of low feature utilization in small sample target detection and improved the detection performance and stability of the model.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- CHONGQING UNIV OF TECH
- Filing Date
- 2025-05-28
- Publication Date
- 2026-07-30
AI Technical Summary
In existing technologies for small-sample target detection, the limited number of samples leads to low feature utilization, making it difficult to achieve a target detection model with good generalization.
We employ an implicit feature contrastive learning method. By improving the fine-grained prototype distillation network model, we design a feature mixing module and a normalized feature aggregation module to generate enhanced features and perform implicit contrastive learning, thereby improving the model's feature learning ability and stability.
It improves the performance of target detection with small samples, enhances the model's learning of intra-class and inter-class relationships, reduces covariate bias, and improves detection accuracy and speed.
Smart Images
Figure CN2025097634_30072026_PF_FP_ABST
Abstract
Description
A few-shot object detection method based on implicit feature contrastive learning Technical Field
[0001] This invention relates to the field of computer vision technology, specifically to a few-sample object detection method based on implicit feature contrastive learning. Background Technology
[0002] Object detection is a fundamental task in computer vision, and deep learning-based methods have seen significant advancements in recent years. However, traditional object detection requires large amounts of labeled data to improve performance, which presents challenges in many real-world applications, such as limited data availability, high data acquisition costs, and significant difficulties. Therefore, few-shot object detection has emerged as a solution.
[0003] Few-shot object detection aims to classify and locate objects in images using a small number of labeled samples. Due to the scarcity of data, few-shot object detection is a challenging task. It assumes that the base class has a sufficient number of samples, while each new class has only a few. Therefore, the key issue is how to train a well-generalized object detection model using a small number of labeled samples.
[0004] However, existing methods for small sample sizes often suffer from insufficient sample diversity to characterize the complete sample distribution. Current mainstream methods do not make high utilization of features from a small number of samples, resulting in a significant gap in this type of method.
[0005] Therefore, a new solution is needed to address the above problems. Summary of the Invention
[0006] The purpose of this invention is to provide a few-sample target detection method based on implicit feature contrastive learning to solve the technical problems mentioned in the background art.
[0007] To achieve the above objectives, the present invention provides the following technical solution: a few-sample target detection method based on implicit feature contrastive learning, comprising at least the following steps:
[0008] S1: Obtain a small sample object detection dataset, and perform data partitioning and preprocessing on the small sample object detection dataset;
[0009] S2: Improve the Fine-Grained Prototype Distillation (FPD) network model and design an implicit contrastive learning framework for model training;
[0010] S3: Design a feature blending module to generate enhanced features using the Mixup feature blending method;
[0011] S4: Design a normalized feature aggregation module and use batch normalization to process the aggregated features;
[0012] S5: Input the base class images into the constructed network and save the base training model after training is completed;
[0013] S6: Fine-tune the base training model using the new class of images, save the fine-tuned model, and finally load the model for model validation to obtain the detection results.
[0014] Furthermore, the small sample target detection dataset in step 1 includes at least the Pascal VOC 2007 and Pascal VOC 2012 datasets, with the training and validation sets of Pascal VOC 2007 and Pascal VOC 2012 serving as the training set and the test set of Pascal VOC 2007 serving as the test set.
[0015] Furthermore, the partitioning of the small sample target detection dataset in step 1 includes at least the following steps:
[0016] The training set is divided into two parts: 15 classes are selected from the 20 classes in the training set as the base training dataset, and the remaining 5 classes are selected as the new class dataset.
[0017] During the base training phase, the model is trained using a large amount of data with detailed annotation information from 15 classes. During the fine-tuning phase, the model is fine-tuned using a small number of labeled samples from 20 classes, including the base class and the new class.
[0018] Furthermore, the data preprocessing of the small sample target detection dataset in step 1 includes at least the following steps: adjusting the size of the query image to a maximum of 1333x800 pixels, and adjusting the size of the instances cropped from the supporting images to 224x224 pixels.
[0019] Furthermore, S2 includes at least the following steps:
[0020] First, the input image is processed through the first three layers of the ResNet-101 feature extraction backbone network pre-trained on the ImageNet dataset to obtain intermediate features of the query image and supporting images;
[0021] Secondly, feature aggregation is used to aggregate intermediate features of the query image and supporting images to form a feature map;
[0022] Subsequently, the feature map from the above process is fed into the Region Proposal Network (RPN) to obtain the region of interest, and then passed through the fourth layer of the backbone network ResNet-101 to obtain the final region of interest feature map.
[0023] Next, the intermediate features of the supporting image are passed through the fourth layer of the backbone network ResNet-101 to obtain high-level features. Then, features of the same type as the query image are filtered out from the high-level features, and one of them is randomly selected as a positive sample feature.
[0024] Then, features that are different from the query image in the high-level features are filtered out, and two of them are randomly selected and sent to the feature mixing module to generate an enhanced negative sample feature;
[0025] Next, positive and negative sample features are concatenated to obtain feature-enhanced positive and negative sample pairs, which are then used to generate class-level prototype features via a prototype network.
[0026] Finally, feature aggregation is used to aggregate the region of interest features and class-level prototype features. The generated aggregated feature matrix is normalized and then fed into the detection head for implicit comparison.
[0027] Feature enhancement improves the model's ability to learn features, while aggregated feature normalization reduces covariate bias within the model, making network training more stable and accelerating convergence. Implicit contrastive learning helps the model learn better intra-class and inter-class relationships, thereby improving the performance of small sample object detection.
[0028] Furthermore, step 3, which uses the Mixup feature fusion method to generate enhanced features, includes at least the following steps:
[0029] First, two features are obtained from supporting high-level image features and features that distinguish different classes of the query image;
[0030] Next, the Beta distribution is used to generate the hyperparameters. Defined weight coefficients The formula is as follows:
[0031]
[0032] Then, for the two features According to the weighting coefficients Mixing yields the characteristics of the mixture. The formula is as follows:
[0033] .
[0034] Furthermore, step 4, which uses batch normalization to process the aggregated features, includes at least the following steps:
[0035] Obtain the query sample features and supporting sample features, aggregate the query sample features and supporting sample features, and normalize the aggregated features.
[0036] First, for a given feature ,in It refers to the batch size and the average value. and variance The calculation formula is as follows:
[0037]
[0038] in It is the eigenvector of the eigenvector. One element;
[0039] Then, the calculated mean and variance are used to standardize each input to obtain the standardized output. The formula is as follows:
[0040]
[0041] in It is a small constant used to prevent the denominator from being zero.
[0042] Furthermore, step 5 includes at least the following steps:
[0043] The input base training dataset is processed by an improved FPD network model for feature extraction, feature aggregation, implicit contrast, and detection head output.
[0044] After training is complete, save the model with the optimal weights.
[0045] Furthermore, step 6 includes at least the following steps:
[0046] The training weight model is loaded with a base model, the fine-tuned dataset is input, and the improved FPD network model is used for feature extraction, feature aggregation, implicit contrast, detection head output, and after training is completed, the optimal fine-tuned weight model is saved, the weight model is loaded, and the test set is input for accuracy testing.
[0047] Compared with the prior art, the beneficial effects of the present invention are:
[0048] This invention proposes a novel concept, implicit contrastive learning, by analyzing the relationship between contrastive learning and feature aggregation methods. It also introduces hybrid feature sampling to generate enhanced features and further proposes an implicit feature contrastive learning (IFCL) module. By constructing enhanced positive and negative sample features and implicitly comparing them with RoI feature evidence from the query image, the model's ability to learn feature representation is improved, thereby enhancing the performance of small sample target detection. Attached Figure Description
[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0050] Figure 1 is a flowchart illustrating the small-sample target detection method of the present invention;
[0051] Figure 2 is a schematic diagram of the dietary comparative framework of the present invention;
[0052] Figure 3 is a schematic diagram of the Mixup feature mixing module of the present invention; Detailed Implementation
[0053] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0054] Referring to Figures 1 to 3, a few-shot target detection method based on implicit feature contrastive learning is described. Figure 1 shows a flowchart of the few-shot target detection method provided in a specific embodiment of the present invention, and the specific steps are as follows:
[0055] Training phase
[0056] A few-shot object detection method based on implicit feature contrastive learning includes the following steps:
[0057] Step 1: Obtain a small sample object detection dataset, including at least the Pascal VOC 2007 and Pascal VOC 2012 datasets. Use the training and validation sets of Pascal VOC 2007 and Pascal VOC 2012 as the training set, and the test set of Pascal VOC 2007 as the test set.
[0058] Then, the training set is divided into 15 classes from the 20 classes in the training set as the base training dataset, and the remaining 5 classes are used as new class data.
[0059] During the base training phase, the model is trained using a large amount of data with detailed annotation information from 15 classes. During the fine-tuning phase, the model is fine-tuned using a small number of labeled samples from 20 classes, including the base class and the new class.
[0060] The size of the query image is adjusted to a maximum of 1333x800 pixels, and the size of instances cropped from the supporting images is adjusted to 224x224 pixels.
[0061] Step 2: Improve the FPD network model by designing an implicit feature contrastive learning framework for model training. The specific steps are as follows:
[0062] First, the input image is processed through the first three layers of the ResNet-101 feature extraction backbone network pre-trained on the ImageNet dataset to obtain intermediate features of the query image and supporting images.
[0063] Secondly, feature aggregation is used to aggregate intermediate features of the query image and supporting images to form a feature map;
[0064] Subsequently, the feature map from the above process is fed into the Region Proposal Network (RPN) to obtain the region of interest, and then passed through the fourth layer of the backbone network ResNet-101 to obtain the final region of interest feature map.
[0065] Next, the intermediate features of the supporting image are passed through the fourth layer of the backbone network ResNet-101 to obtain high-level features. Then, features of the same type as the query image are filtered out from the high-level features, and one of them is randomly selected as a positive sample feature.
[0066] Then, features that are different from the query image in the high-level features are filtered out, and two of them are randomly selected and sent to the feature mixing module to generate an enhanced negative sample feature;
[0067] Next, positive and negative sample features are concatenated to obtain feature-enhanced positive and negative sample pairs, which are then used to generate class-level prototype features via a prototype network.
[0068] Finally, feature aggregation is used to aggregate the region of interest features and class-level prototype features. The generated aggregated feature matrix is normalized and sent to the detection head for implicit comparison.
[0069] Feature enhancement improves the model's ability to learn features, while aggregated feature normalization reduces covariate bias within the model, making network training more stable and accelerating convergence. Implicit contrastive learning helps the model learn better intra-class and inter-class relationships, thereby improving the performance of small sample object detection.
[0070] Step 3: Use the Mixup feature blending method to generate enhanced features.
[0071] First, two features are obtained from supporting high-level image features and features that distinguish different classes of the query image;
[0072] Next, the Beta distribution is used to generate the hyperparameters. Defined weight coefficients The formula is as follows:
[0073]
[0074] Then, for the two features According to the weighting coefficients Mixing yields the characteristics of the mixture. The formula is as follows:
[0075] .
[0076] Step 4: Process the aggregated features using batch normalization.
[0077] Obtain the query sample features and supporting sample features, aggregate the query sample features and supporting sample features, and normalize the aggregated features.
[0078] First, for a given feature ,in It refers to the batch size and the average value. and variance The calculation formula is as follows:
[0079]
[0080] in It is the eigenvector of the eigenvector. One element;
[0081] Then, the calculated mean and variance are used to standardize each input to obtain the standardized output. The formula is as follows:
[0082]
[0083] in It is a small constant used to prevent the denominator from being zero.
[0084] Step 5: Input the base training dataset and perform feature extraction, feature aggregation, implicit contrast, and head detection output using the improved FPD network model;
[0085] After training is complete, save the model with the optimal weights.
[0086] Step 6: Load the base training weight model, input the fine-tuning dataset, perform feature extraction, feature aggregation, implicit contrast, detect the head output through the improved FPD network model, after training is completed, save the optimal fine-tuned weight model, load the weight model, and input it into the test set for accuracy testing.
[0087] Verification phase
[0088] A few-shot object detection method based on implicit feature contrastive learning is presented. To verify the algorithm's effectiveness, comparative experiments were conducted using the PASCAL VOC dataset. All experiments were performed on a PyTorch framework with an NVIDIA RTX-3090 GPU, with a batch size of 4 images and the SGD optimizer used. During the base training phase, the model was trained on the VOC dataset for 30k iterations. The learning rate was 0.005, decaying by 0.1 times at 17k iterations. During the fine-tuning phase, the learning rate was set to 0.001. The evaluation metric was the AP50 score for new classes in the test set.
[0089] Table 1 Experimental results on PASCAL VOC
[0090] Comparative experiments using FPD as the baseline model revealed significant improvements across multiple metrics. Subsequent comparisons with several mainstream methods also yielded optimal or near-optimal results. This demonstrates that our proposed method can effectively improve detection performance for small targets.
[0091] We designed ablation experiments to verify the contributions of the IFCL module in the base training and fine-tuning phases, and to demonstrate its greater improvement when working together. The experiments were designed with FPD as the baseline, and each ablation experiment was conducted without changing any hyperparameters. The results are shown in Table 3.
[0092] Table 3 Ablation experimental results on the PASCAL VOC dataset
[0093] Compared to the baseline, using the IFCL module in both stages improved accuracy. Applying IFCL only in the fine-tuning stage improved AP50 by 1.5%, 0.7%, 0.5%, and 0.4% respectively in Novel Set 1. Applying IFCL only in the base training stage improved AP50 by 5.4%, 1.9%, 1.5%, and 1.6% respectively. This is because the goal of the base training stage is to learn the basic feature representations of the samples. Introducing the IFCL module at this stage helps the model better capture important features, improves feature discrimination ability using a large amount of training data, reduces the risk of overfitting, promotes model optimization in an effective direction, enhances feature diversity and robustness, and makes the model perform better when learning different class boundaries, thereby improving overall performance. In contrast, in the fine-tuning stage, the limited labeled data makes the feature representations learned by the model less targeted, resulting in poor performance.
[0094] When the IFCL module is used in both phases, in AP 50The proposed method achieved improvements of 5.8%, 2.4%, 1.9%, and 3.2% respectively, demonstrating that it can perform better on small target detection.
[0095] In summary:
[0096] By constructing an implicit contrastive learning framework, the model learns stronger intra-class and inter-class relationships. The Mixup feature fusion method is applied to generate enhanced positive and negative sample features, helping the model learn a more complete sample feature distribution. Furthermore, normalizing the aggregated features reduces covariate bias within the model, making network training more stable. These methods are expected to alleviate the problems in few-shot object detection and improve its performance.
[0097] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
Claims
1. A few-sample target detection method based on implicit feature contrastive learning, characterized in that: At least the following steps are included: S1: Obtain a small sample object detection dataset, and perform data partitioning and preprocessing on the small sample object detection dataset; S2: Improve the FPD network model and design an implicit contrastive learning framework for model training; S3: Design a feature blending module to generate enhanced features using the Mixup feature blending method; S4: Design a normalized feature aggregation module and use batch normalization to process the aggregated features; S5: Input the base class images into the constructed network and save the base training model after training is completed; S6: Fine-tune the base training model using the new class of images, save the fine-tuned model, and finally load the model for model validation to obtain the detection results.
2. The few-sample target detection method based on implicit feature contrastive learning according to claim 1, characterized in that: The small sample object detection dataset in step 1 includes at least the Pascal VOC 2007 and Pascal VOC 2012 datasets, with the training and validation sets of Pascal VOC 2007 and Pascal VOC 2012 used as the training set and the test set of Pascal VOC 2007 used as the test set.
3. The few-sample target detection method based on implicit feature contrastive learning according to claim 2, characterized in that: Step 1, which involves partitioning the small sample object detection dataset, includes at least the following steps: The training set is divided into two parts: 15 classes are selected from the 20 classes in the training set as the base training dataset, and the remaining 5 classes are selected as the new class dataset. During the base training phase, the model is trained using a large amount of data with detailed annotation information from 15 classes. During the fine-tuning phase, the model is fine-tuned using a small number of labeled samples from 20 classes, including the base class and the new class.
4. The few-sample target detection method based on implicit feature contrastive learning according to claim 2, characterized in that: The data preprocessing of the small sample object detection dataset in step 1 includes at least the following steps: adjusting the size of the query image to a maximum of 1333x800 pixels, and adjusting the size of the instances cropped from the supporting images to 224x224 pixels.
5. A few-sample target detection method based on implicit feature contrastive learning according to claim 1, characterized in that: S2 includes at least the following steps: First, the input image is processed through the first three layers of the ResNet-101 feature extraction backbone network pre-trained on the ImageNet dataset to obtain intermediate features of the query image and supporting images. Secondly, feature aggregation is used to aggregate intermediate features of the query image and supporting images to form a feature map; Subsequently, the feature map from the above process is fed into the Region Proposal Network (RPN) to obtain the region of interest, and then passed through the fourth layer of the backbone network ResNet-101 to obtain the final region of interest feature map. Next, the intermediate features of the supporting image are passed through the fourth layer of the backbone network ResNet-101 to obtain high-level features. Then, features of the same type as the query image are filtered out from the high-level features, and one of them is randomly selected as a positive sample feature. Then, features that are different from the query image in the high-level features are filtered out, and two of them are randomly selected and sent to the feature mixing module to generate an enhanced negative sample feature; Next, positive and negative sample features are concatenated to obtain feature-enhanced positive and negative sample pairs, which are then used to generate class-level prototype features via a prototype network. Finally, feature aggregation is used to aggregate the region of interest features and class-level prototype features. The generated aggregated feature matrix is normalized and sent to the detection head for implicit comparison. Feature enhancement improves the model's ability to learn features, while aggregated feature normalization reduces covariate bias within the model, making network training more stable and accelerating convergence. Implicit contrastive learning helps the model learn better intra-class and inter-class relationships, thereby improving the performance of small sample object detection.
6. The few-sample target detection method based on implicit feature contrastive learning according to claim 1, characterized in that: Step 3, which uses the Mixup feature blending method to generate enhanced features, includes at least the following steps: First, two features are obtained from supporting high-level image features and features that distinguish different classes of the query image; Next, the Beta distribution is used to generate the hyperparameters. Defined weight coefficients The formula is as follows: ， Then, for the two features According to the weighting coefficients Mixing yields the characteristics of the mixture. The formula is as follows: 。 7. A few-sample target detection method based on implicit feature contrastive learning according to claim 1, characterized in that: Step 4 includes at least the following steps: Obtain the query sample features and supporting sample features, aggregate the query sample features and supporting sample features, and normalize the aggregated features; First, for a given feature ,in It refers to the batch size and the average value. and variance The calculation formula is as follows: ， in It is the eigenvector of the eigenvector. One element; Then, the calculated mean and variance are used to standardize each input to obtain the standardized output. The formula is as follows: ， in It is a small constant used to prevent the denominator from being zero.
8. A few-sample target detection method based on implicit feature contrastive learning according to claim 1, characterized in that: Step 5 includes at least the following steps: The input base training dataset is processed by an improved FPD network model for feature extraction, feature aggregation, implicit contrast, and detection head output. After training is complete, save the model with the optimal weights.
9. A few-sample target detection method based on implicit feature contrastive learning according to claim 1, characterized in that: Step 6 includes at least the following steps: The training weight model is loaded with a base model, the fine-tuned dataset is input, and the improved FPD network model is used for feature extraction, feature aggregation, implicit contrast, detection head output, and after training is completed, the optimal fine-tuned weight model is saved, the weight model is loaded, and the test set is input for accuracy testing.