Feature token-based feature fusion efficient few-sample target detection method

Abstract

The invention belongs to the field of image processing, and discloses a feature token-based feature fusion efficient few-sample target detection method, which comprises first-stage pre-training and second-stage small sample fine adjustment. In the fine tuning stage, the pre-trained backbone network is frozen to completely retain the strong generalization feature extraction capability of the backbone network, meanwhile, a feature token feature fusion module is inserted between the backbone network and a detection head, and only the feature token feature fusion module and the detection head are subjected to fine tuning. The feature token feature fusion module adopts a novel double-path architecture, interacts with a feature graph through a learnable feature token, and can adaptively enhance feature representation related to a new class, so that adaptation to a new task is accurately completed on the premise of not interfering basic knowledge. Besides, in order to further improve the practicability of the model, a phantom separable convolution module is also introduced to construct a feature token fusion network, so that the parameter quantity and the calculation complexity of the model are remarkably reduced while the performance is ensured.

Description

Technical Field

[0001] This invention belongs to the field of image processing and relates to an efficient few-sample target detection method based on feature token-based feature fusion. Background Technology

[0002] In recent years, driven by deep learning technology, the field of general object detection has made groundbreaking progress, with models such as Faster R-CNN, YOLO, and Transformer demonstrating outstanding performance on various benchmark datasets. However, the core reason these models can achieve such ideal results is that they rely on massive and accurately labeled training datasets. In many real-world applications, such as objects falling from heights, autonomous driving, and high-altitude remote sensing, obtaining massive amounts of labeled samples is often extremely difficult or costly.

[0003] To address the technical bottlenecks posed by data sparsity, few-shot object detection (FSOD) has gradually developed. FSOD aims to enable models to quickly learn and detect previously unseen "new" object classes from only a few (e.g., 1 to 10) labeled samples. Currently, the mainstream approach in FSOD is a two-stage strategy centered on "pre-training-fine-tuning": first, a powerful feature extractor is pre-trained on a "base class" containing a large number of samples; then, the model is fine-tuned using a small number of "new" class samples, transferring the learned knowledge to the new task.

[0004] Despite its widespread adoption, this paradigm faces a thorny dilemma during the fine-tuning phase. On the one hand, while fully fine-tuning the entire pre-trained model allows it to learn new class features effectively, it can easily lead to a "catastrophic forgetting" of base class knowledge, significantly impairing the valuable generalization ability acquired through large-scale data training. On the other hand, fine-tuning only the final detection head to preserve base class knowledge leaves the feature extractors in the backbone network "unaware" of the visual characteristics of new classes, resulting in insufficient feature adaptability and severely limiting detection performance. Summary of the Invention

[0005] This invention provides an efficient few-shot object detection method based on token-based feature fusion. To address the sharp contradiction between "knowledge preservation" and "task adaptation," this invention proposes a token-based fusion training strategy (TFFT) and a corresponding token-based fusion network, TFFNet. See the description below for details:

[0006] The technical solution of the present invention:

[0007] An efficient few-shot object detection method based on token-based feature fusion comprises a one-stage pre-training and a two-stage few-shot fine-tuning. The core idea of ​​this invention is: in the fine-tuning stage, the powerful generalization feature extraction capability of the pre-trained backbone network is fully preserved by freezing it. Simultaneously, a token feature fusion module (TFF) is inserted between the backbone network and the detection head, and only this TFF module and the detection head are fine-tuned. The token feature fusion module adopts a novel dual-path architecture, interacting with the feature map through learnable tokens, adaptively enhancing feature representations related to new classes, thereby accurately adapting to new tasks without interfering with fundamental knowledge. Furthermore, to further improve the model's practicality, this invention introduces a Ghost Separable Convolution (GSConv) module to construct the token fusion network, effectively reducing the number of model parameters and computational complexity while maintaining stable model performance.

[0008] The specific steps are as follows:

[0009] (1) Design the feature token fusion network model architecture;

[0010] The feature token fusion network model is designed, and its overall architecture follows a backbone network-neck network-detection head structure, with the following improvements introduced:

[0011] 1) Integrate phantom separable convolutional modules into the backbone network;

[0012] The phantom separable convolutional module first learns the spatial features of each input channel independently through deep convolution, capturing local spatial information and generating intermediate feature maps. Then, a 1x1 pointwise convolution is used to process the intermediate feature maps. Mapping to key features Further, through a series of linear operations, from key features... Ghostly features derived from the middle Finally, the key features and ghost features are concatenated along the channel dimension to form the final output feature map, the number of which is consistent with the input channels of the module receiving the output feature map.

[0013] 2) Introduce a feature token feature fusion module between the backbone network and the neck network;

[0014] The feature token feature fusion module adopts a dual-path feature enhancement strategy, including a feature token interaction path and a feature weighting path;

[0015] In the feature token interaction path, the input feature map is first processed by a 1*1 convolution. Projecting onto a low-dimensional space to generate projected features Among them, feature map For the output of the backbone network, , For compression ratio, For the number of channels, For batch size, The height of the feature map, The width of the feature map is then used; then, the learnable token set obtained through random initialization is utilized. As a convolution kernel, it is used in conjunction with projected features. Perform convolution operations to generate feature interaction maps. ,in, The number of learnable tokens; finally, the feature interaction graph is processed using the Sigmoid activation function. The numerical mapping is within the interval [0,1], representing the correlation strength of features at each spatial location;

[0016] In the feature weighting path, the input features are first weighted by a 3x3 convolution. Feature extraction and feature map generation are performed. The output of the backbone network is then processed by global average pooling to extract global context information, followed by 1x1 convolution to generate feature weights. These feature weights are then normalized by Softmax and weighted on the feature interaction graph M generated by the feature token interaction path. Finally, a weighted summation operation is used to generate enhanced features. ,in, Let i be the i-th channel of the feature interaction graph M;

[0017] (2) Training strategy for feature token fusion network model;

[0018] The feature token fusion network model employs a two-stage training strategy:

[0019] The PASCAL VOC benchmark dataset was used, and the dataset was divided into 15 base classes. and 5 new class datasets The training and test sets are randomly divided, and each new class is divided into only K labeled samples;

[0020] The first stage involves pre-training on the base class dataset, and then training the feature token fusion network model on the base class dataset, which contains a large number of labeled samples. End-to-end pre-training is performed on top of this;

[0021] The second stage is the few-sample transfer learning stage; targeting new types of datasets. The feature token fusion network model is fine-tuned using a small number of labeled K-shot samples. Specifically, the model parameters of the feature token fusion network model obtained in the first stage of pre-training are reloaded, and the parameters of the backbone network are frozen. At this time, only the parameters of the feature token feature fusion module and the detection head are updated.

[0022] (3) Dynamic hybrid enhancement strategy;

[0023] The dynamic blending enhancement strategy is to dynamically and probabilistically combine two complementary enhancement techniques: pixel-level interpolation (Mixup) and region-level stitching (CutMix).

[0024] Dynamic Hybrid Enhancement Strategy A probability combination of pixel-level interpolation and region-level stitching:

[0025]

[0026] in, For the input image, The original labels, including bounding box coordinates and category, are assigned according to preset probabilities. Apply pixel-level interpolation, It is a uniform distribution Random variables sampled in the middle, and These are enhancement operators for pixel-level interpolation and region-level stitching, respectively.

[0027] Pixel-level interpolation is used to interpolate two image data. and Perform linear interpolation to construct a union of all bounding boxes from both images, generating new training samples, defined as follows:

[0028]

[0029]

[0030] in, It is the pixel data of the image; This is the corresponding tag data; It is a mixing coefficient, which is a weighted average of two image samples and their labels, obtained by sampling from a beta distribution;

[0031] Region-level stitching is the process of stitching images together. A local region stitched into an image Generate new samples at the corresponding positions. :

[0032]

[0033]

[0034] in, It is a binary mask representing the cropped region, determined by random coordinates. To define; It is the area of ​​the cutting region. ; This represents element-wise multiplication;

[0035] To protect labeled regions from occlusion during image stitching, a semantic protection mechanism is introduced; specifically: In the region-level stitching operation, a candidate cropping region b is randomly sampled; if this candidate cropping region b matches any ground truth bounding box in the cropped image... The intersection-union ratio exceeds the threshold τ_iou, i.e. If the condition is not met, the candidate cropping region b is rejected and resampling is performed until a region that meets the conditions is found.

[0036] The beneficial effects of this invention are:

[0037] (1) The feature token network and the adapted two-stage training strategy proposed in this invention solve the contradiction between knowledge retention and task adaptation in traditional fine-tuning methods;

[0038] (2) The proposed feature token fusion module can efficiently enhance the feature representation of small sample categories and improve the detection performance of the model;

[0039] (3) The constructed feature token fusion network architecture achieves an excellent balance between model efficiency and performance. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of a feature token fusion network framework.

[0041] Figure 2 This is a schematic diagram of the feature token fusion training strategy.

[0042] Figure 3 This is a schematic diagram of the phantom separable convolutional module.

[0043] Figure 4 This is a schematic diagram of the feature token feature fusion module.

[0044] Figure 5 The diagram shows a visual comparison of the feature maps of the phantom separable convolutional module and the standard convolutional module. (a) is the feature map of the standard convolution, and (b) is the feature map of the phantom separable convolution. Detailed Implementation

[0045] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0046] The core objective of small sample target detection technology is to achieve accurate detection of new target categories under the constraint of a small number of labeled samples (usually only a few samples). Its core goal is to break the bottleneck of traditional target detection's dependence on large-scale labeled data.

[0047] Feature fusion technology is a core component of object detection. It emphasizes the effective integration of features at different levels (shallow detail features and deep semantic features) and at different scales (small target features and large target features) to compensate for the shortcomings of single feature representation capabilities. Traditional methods, such as feature pyramid networks, fuse features through multiple paths, but in small sample scenarios, the fused features often lack discriminative power due to insufficient learning of new category features, thus affecting the accuracy of target recognition.

[0048] (1) Design the feature token fusion network model architecture;

[0049] See Figure 1 and Figure 2 To achieve an effective balance between real-time performance and detection accuracy in small-sample target detection tasks, this invention proposes a feature token fusion network model. The feature token fusion network model follows the efficient YOLO paradigm, with its overall architecture based on the classic core architecture of "Backbone - Neck - Head" (see Figure 1). While retaining the original module design, the following key improvements are introduced:

[0050] First, to further reduce model complexity and optimize inference speed, the proposed phantom separable convolutional module is integrated into the backbone network. The core objective of this phantom separable convolutional module is to mine rich feature information with lower computational overhead, thereby improving the model's running speed without a significant performance degradation.

[0051] Secondly, to better adapt to feature learning for small sample classes in the second stage, a feature token fusion module is innovatively introduced between the backbone network and the neck network. The feature token fusion module aims to achieve effective fusion and transformation of base class and new class features, providing a more robust feature representation for subsequent fine-tuning.

[0052] The feature token feature fusion network model proposed in this invention adopts a two-stage training strategy, such as... Figure 2 As shown, the aim is to effectively utilize base class data for general feature learning and to quickly adapt to new category targets.

[0053] The first phase targets the base class dataset. Pre-training was performed on a base class dataset containing a large number of labeled samples. The next step involves completing the end-to-end pre-training process. The main goal of this stage is to enable the backbone network and the detection head to learn powerful general visual feature extraction capabilities and object detection knowledge. By fully utilizing base class data, the feature token feature fusion module can establish a generalized representation of common object features, laying the foundation for subsequent adaptation to new categories.

[0054] After the first phase of training is completed, the feature token fusion network model enters the few-shot transfer learning phase. For each new class, we perform fine-tuning training of the feature token fusion network model using a small number of labeled samples (K-shot). Specifically, we reload the parameters of the feature token fusion network model obtained in the first phase of pre-training and freeze the parameters of the backbone network. At this time, only the parameters of the feature token fusion module and the detection head are updated.

[0055] This strategy not only effectively preserves the generalized feature representations with strong generalization capabilities already learned by the backbone network on the base classes, but also enables the feature token fusion network model to quickly and efficiently adapt to the detection tasks of new target classes by adjusting only the local parameters that are more relevant to the task and category. This targeted parameter update greatly improves the detection performance of the feature token fusion network model under conditions of scarce data and small sample sizes.

[0056] 1) Integrate phantom separable convolutional modules into the backbone network;

[0057] join Figure 3 The phantom separable convolutional module first learns the spatial features of each input channel independently through deep convolution, capturing local spatial information and generating intermediate feature maps. Then, a 1x1 pointwise convolution is used to process the intermediate feature maps. Mapping to key features Further drawing inspiration from the phantom convolution module, a series of linear operations are used to extract key features. Ghostly features derived from the middle These ghost features are variants of the key features, eliminating the need for learning from scratch. This allows for the generation of a rich variety of features at extremely low computational cost, compensating for the insufficient number of basic features and significantly reducing the number of parameters. Finally, the key features and ghost features are concatenated along the channel dimension to form the final output feature map, whose number of channels matches the input channels of the module receiving the output feature map.

[0058] See [link to visualization of feature maps generated by the phantom separable convolution module and comparison with feature maps extracted by standard convolution]. Figure 5The phantom separable convolution module successfully reduces the number of model parameters while maintaining good feature map similarity. This study proposes a phantom separable convolution module that innovatively combines the advantages of depthwise separable convolution and phantom convolution, achieving an effective balance between detection accuracy and computational efficiency. Furthermore, the module design is based on rigorous feature similarity analysis to verify the redundancy assumption, providing a solid theoretical basis for the structural design of the phantom separable convolution module and effectively ensuring its practical effectiveness and stable reliability in real-world applications.

[0059] 2) Introduce a feature token feature fusion module between the backbone network and the neck network;

[0060] The core challenge of small-sample object detection lies in the limited number of labeled samples, making it difficult for the model to learn effective feature representations. Traditional fine-tuning methods have significant shortcomings: directly fine-tuning the entire model can easily lead to overfitting; while fine-tuning only the detection head fails to fully utilize the rich feature knowledge learned by the backbone network during the pre-training of basic categories.

[0061] To address this issue, a feature token feature fusion module is proposed, a novel feature enhancement mechanism that allows fine-tuning of the feature token feature fusion layer and detection head while freezing the backbone network parameters to maintain basic feature extraction capabilities. In this way, the model can more fully adapt to new category features and learn effective feature representations for small sample categories, making it particularly suitable for small sample object detection scenarios.

[0062] See Figure 4 The feature token feature fusion module adopts a dual-path feature enhancement strategy, including two core components: the feature token interaction path and the feature weighting path.

[0063] In the feature token interaction path, the input feature map is first processed by a 1*1 convolution. Projecting onto a low-dimensional space to generate projected features Among them, feature map For the output of the backbone network, , For compression ratio, For the number of channels, For batch size, The height of the feature map, The width of the feature map is then used; then, the learnable token set obtained through random initialization is utilized. As a convolution kernel, it is used in conjunction with projected features. Perform convolution operations to generate feature interaction maps. ,in, The number of learnable tokens; finally, using the Sigmoid activation function, the feature interaction graph is... The values ​​are projected onto the [0,1] interval to characterize the correlation strength between features at different spatial locations;

[0064] In the feature weighting path, the first step is to perform a 3x3 convolution on the input feature map. Perform feature extraction on this feature map. This is the output of the backbone network; then global average pooling is used to obtain global context information, and then 1*1 convolution is used to obtain feature weights; these feature weights are normalized by Softmax and then weighted on the feature interaction graph M generated by the feature token interaction path. Finally, a weighted summation operation is used to generate enhanced features. ,in, Let be the i-th channel of the feature interaction graph M; this enhanced feature effectively fuses the original feature information with the semantic information captured by Tokens.

[0065] This dual-path design enables the feature token feature fusion module to significantly improve the feature representation capability of small sample classes while maintaining the stability of the backbone network, through learnable feature tokens and an adaptive feature weighting mechanism.

[0066] (2) Training strategy for feature token fusion network model;

[0067] For the feature token fusion network model, a two-stage training strategy is adopted:

[0068] The PASCAL VOC benchmark dataset was selected as the experimental data. It was divided into 15 base class datasets and 5 new class datasets. The training set and test set were randomly split, and only K labeled samples were set for each new class.

[0069] The first stage involves pre-training on the base class dataset, and then training the feature token fusion network model on the base class dataset, which contains a large number of labeled samples. End-to-end pre-training is performed on top of this;

[0070] The second stage is the few-shot transfer learning stage, which focuses on new types of datasets. The corresponding training process is carried out, and the feature token fusion network model is fine-tuned using a small number of labeled K-shots provided; specifically as follows: the model parameters of the feature token fusion network model obtained in the first stage of pre-training are reloaded, and the parameters of the backbone network are frozen; at this time, only the parameters of the feature token feature fusion module and the detection head are updated;

[0071] (3) Dynamic hybrid enhancement strategy;

[0072] Based on the training strategy and feature token fusion network proposed above, this invention also designs a dynamic hybrid augmentation strategy. In small-sample object detection tasks, traditional augmentation strategies such as rotation and cropping, while increasing the amount of data, have limited sample diversity and are difficult to effectively simulate the data distribution of the real world. On the other hand, hybrid augmentation methods such as pixel-level interpolation, although capable of creating richer samples, may destroy key semantic features such as object edges due to global linear interpolation of pixels. The dynamic hybrid augmentation strategy (DHA) proposed in this invention precisely solves the above problems.

[0073] The core idea of ​​the dynamic hybrid augmentation strategy is to dynamically and probabilistically combine two complementary augmentation techniques: pixel-level interpolation (Mixup) and region-level stitching (CutMix). In this way, the dynamic hybrid augmentation strategy can not only generate semantically ambiguous but continuously distributed samples to promote boundary learning between categories, but also generate stitched samples that retain key semantic features to improve the model's localization ability, thereby achieving more efficient and robust data augmentation.

[0074] A dynamic blending strategy is adopted, defined as follows: A probability combination of pixel-level interpolation and region-level stitching:

[0075] Dynamic Hybrid Enhancement Strategy A probability combination of pixel-level interpolation and region-level stitching:

[0076]

[0077] in, For the input image, As the initial label, it encompasses bounding box coordinates and category information, and will be assigned according to a pre-set probability. Apply pixel-level interpolation, It is a uniform distribution Random variables sampled in the middle, and These are enhancement operators for pixel-level interpolation and region-level stitching, respectively.

[0078] Pixel-level interpolation is used to interpolate two image data. and Linear interpolation is performed to construct the union of all object boxes from the two images, generating new training samples. This method can generate virtual samples that smoothly transition between training samples, enhancing the model's generalization ability in class boundary regions; defined as follows:

[0079]

[0080]

[0081] Region-level stitching is the process of stitching images together. A local region stitched into an image Generate new samples at the corresponding positions. Unlike pixel-level interpolation, region-level stitching can preserve more complete local semantic information, which helps improve the model's localization accuracy.

[0082]

[0083]

[0084] in, It is a binary mask representing the cropped region, determined by random coordinates. To define; It is the area of ​​the cutting region. ; This represents element-wise multiplication;

[0085] We employ a dynamic switching mechanism, which alternates between two enhancement strategies with random probabilities to avoid overfitting caused by a single enhancement mode.

[0086] To protect labeled regions from occlusion during image stitching, a semantic protection mechanism is introduced; specifically: when performing region-level stitching, a candidate cropping region b is obtained through random sampling; if this candidate cropping region b matches any ground truth bounding box in the cropped image... The intersection-union ratio exceeds the threshold τ_iou, i.e. In this case, the candidate cropped region b will be discarded and resampling will continue until a region that meets the criteria is finally found. This mechanism ensures that the stitching operation does not destroy the original important target and preserves the key monitoring signals.

[0087] Since samples generated by hybrid augmentation may contain composite label structures from multiple images (e.g., mixed class weights in pixel-level interpolation), standard loss functions are not directly applicable. Therefore, an adaptation of the loss function is necessary. For samples generated by pixel-level interpolation, the detection loss L_total is correspondingly linearly interpolated:

[0088]

[0089] Here, L-det is the standard detection loss function. For samples generated by CutMix, the loss function is directly calculated on the union of all adjusted bounding boxes. Compared to traditional augmentation methods that only change the geometry of samples, the dynamic hybrid augmentation strategy generates samples with new semantics through cross-sample combination, significantly improving the efficiency and diversity of data augmentation in small-sample scenarios. With its dynamic switching mechanism and semantic protection strategy, the dynamic hybrid augmentation strategy effectively avoids the loss of key features while enriching sample diversity. This strategy not only solves the synchronization problem between bounding boxes and image augmentation, but also significantly improves the detection accuracy and generalization performance of the model in small-sample learning through the expanded sample set.

[0090] The feasibility of the above-mentioned small-sample target detection method is verified below with specific experimental settings:

[0091] All experiments were conducted on the PASCAL VOC benchmark dataset, which is widely used in the field of few-shot object detection. Following standard practice, the data from VOC 2007 trainval and VOC 2012 trainval were merged for the basic training phase of the model.

[0092] Class Division: The PASCAL VOC dataset contains 20 object categories. These 20 categories were divided into 15 base classes and 5 new classes. To avoid the randomness of a single class division, two different base / new class division schemes were used in experiments to comprehensively evaluate the model's generalization ability.

[0093] Evaluation Protocol: During the fine-tuning and testing phases for new classes, a standard K-shot configuration was followed, where K takes values ​​of 1, 2, 3, 5, and 10, representing that only K labeled samples are available for fine-tuning for each new class. To eliminate performance fluctuations caused by random sampling, each K-shot configuration was repeated 10 times independently, with the support set resampled in each round. The final performance metric was the average of these 10 rounds of experiments. The standard mAP@50 was used as the core evaluation metric for model detection performance.

[0094] Expected Results: This invention effectively resolves the contradiction between "knowledge preservation" and "task adaptation" in few-shot target detection by introducing a phantom separable convolution module and a feature token fusion module. Compared to the baseline model, it achieves synergistic optimization of performance and efficiency. Specifically, under 1-shot, 2-shot, 3-shot, 5-shot, and 10-shot settings for few-shot detection tasks, the mean accuracy (mAP) of the final ensemble model is improved by 0.7, 1.1, 0.8, 0.1, and 0.8 percentage points, respectively, significantly enhancing the detection accuracy for new target categories. Simultaneously, thanks to the lightweight design of the phantom separable convolution module and the efficient feature interaction mechanism of the feature token fusion module, the model parameter count is reduced to 2.3M, a 30.3% reduction compared to the baseline model, significantly lowering computational resource consumption and deployment barriers. This invention achieves model lightweighting while ensuring improved detection performance in few-shot scenarios, making it particularly suitable for resource-constrained applications such as edge devices and real-time monitoring, demonstrating significant practical value.

Claims

1. A high-efficiency, few-shot target detection method based on feature token-based feature fusion, characterized in that, The steps are as follows: (1) Design the feature token fusion network model architecture; The feature token fusion network model is designed, and its overall architecture follows a backbone network-neck network-detection head structure, with the following improvements introduced: 1) Integrate phantom separable convolutional modules into the backbone network; The phantom separable convolutional module first learns the spatial features of each input channel independently through deep convolution, capturing local spatial information and generating intermediate feature maps. Then, a 1x1 pointwise convolution is used to process the intermediate feature maps. Mapping to key features Further, through a series of linear operations, from key features... Ghostly features derived from the middle Finally, the key features and ghost features are concatenated along the channel dimension to form the final output feature map, the number of which is consistent with the input channels of the module receiving the output feature map. 2) Introduce a feature token feature fusion module between the backbone network and the neck network; The feature token feature fusion module adopts a dual-path feature enhancement strategy, including a feature token interaction path and a feature weighting path; In the feature token interaction path, the input feature map is first processed by a 1*1 convolution. Projecting onto a low-dimensional space to generate projected features Among them, feature map For the output of the backbone network, , For compression ratio, For the number of channels, For batch size, The height of the feature map, The width of the feature map is then used; then, the learnable token set obtained through random initialization is utilized. As a convolution kernel, it is used in conjunction with projected features. Perform convolution operations to generate feature interaction maps. ,in, The number of learnable tokens; finally, the feature interaction graph is processed using the Sigmoid activation function. The numerical mapping is within the interval [0,1], representing the correlation strength of features at each spatial location; In the feature weighting path, the input features are first weighted by a 3x3 convolution. Feature extraction and feature map generation are performed. The output of the backbone network is then processed by global average pooling to extract global context information, followed by 1x1 convolution to generate feature weights. These feature weights are then normalized by Softmax and weighted on the feature interaction graph M generated by the feature token interaction path. Finally, a weighted summation operation is used to generate enhanced features. ,in, Let i be the i-th channel of the feature interaction graph M; (2) Training strategy for feature token fusion network model; The feature token fusion network model employs a two-stage training strategy: The PASCAL VOC benchmark dataset was used, and the dataset was divided into 15 base classes. and 5 new class datasets The training and test sets are randomly divided, and each new class is divided into only K labeled samples; The first stage involves pre-training on the base class dataset, and then training the feature token fusion network model on the base class dataset, which contains a large number of labeled samples. End-to-end pre-training is performed on top of this; The second stage is the few-sample transfer learning stage; targeting new types of datasets. The feature token fusion network model is fine-tuned using a small number of labeled K-shot samples. Specifically, the model parameters of the feature token fusion network model obtained in the first stage of pre-training are reloaded, and the parameters of the backbone network are frozen. At this time, only the parameters of the feature token feature fusion module and the detection head are updated. (3) Dynamic hybrid enhancement strategy; The dynamic blending enhancement strategy is to dynamically and probabilistically combine two complementary enhancement techniques: pixel-level interpolation (Mixup) and region-level stitching (CutMix). Dynamic Hybrid Enhancement Strategy A probability combination of pixel-level interpolation and region-level stitching: ； in, For the input image, The original labels, including bounding box coordinates and category, are assigned according to preset probabilities. Apply pixel-level interpolation, It is a uniform distribution Random variables sampled in the middle, and These are enhancement operators for pixel-level interpolation and region-level stitching, respectively. Pixel-level interpolation is used to interpolate two image data. and Perform linear interpolation to construct a union of all bounding boxes from both images, generating new training samples, defined as follows: ； ； in, It is the pixel data of the image; This is the corresponding tag data; It is a mixing coefficient, which is a weighted average of two image samples and their labels, obtained by sampling from a beta distribution; Region-level stitching is the process of stitching images together. A local region stitched into an image Generate new samples at the corresponding positions. : ； ； in, It is a binary mask representing the cropped region, determined by random coordinates. To define; It is the area of ​​the cutting region. ; This represents element-wise multiplication; To protect labeled regions from occlusion during image stitching, a semantic protection mechanism is introduced; specifically: In the region-level stitching operation, a candidate cropping region b is randomly sampled; if this candidate cropping region b matches any ground truth bounding box in the cropped image... The intersection-union ratio exceeds the threshold τ_iou, i.e. If the condition is not met, the candidate cropping region b is rejected and resampling is performed until a region that meets the conditions is found.

Images