Small sample target detection method and device based on style augmentation and storage medium

Abstract

The application provides a small sample target detection method based on style augmentation, comprising the following steps: extracting content features of a query image and style features of support images, fusing the content features and the style features to generate style enhanced samples, extracting embedding features of the style enhanced samples, and calculating the angular similarity between the embedding features and class center vectors; applying an angular interval to the angular similarity of a target class, and scaling the angular similarity of all classes using a scaling factor; outputting a class label of the target through a classifier, outputting a position bounding box of the target using a detection head, and obtaining a final detection result. Through the technical framework of data layer style augmentation and feature layer discriminative constraint collaborative optimization, the distribution diversity of the training data is significantly expanded without destroying the foreground semantics, and a strong discriminative measurement is constructed in a high-dimensional feature space, so that the detection accuracy and robustness of the model in a data scarce scene are improved.

Description

Technical Field

[0001] This invention relates to the field of image detection and recognition technology, and in particular to a method, device and storage medium for small sample target detection based on style augmentation. Background Technology

[0002] In recent years, deep learning technology has achieved remarkable success in the field of object detection, but its performance heavily relies on the support of large-scale, high-quality labeled data. However, in practical applications such as remote sensing monitoring, industrial quality inspection, and medical image analysis, collecting and labeling sufficient data for a large number of new object categories is costly and even difficult to achieve. This strong dependence on data greatly limits the widespread application of traditional deep detection models.

[0003] To address these issues, few-shot object detection (FSOD) technology emerged. Its core idea is to pre-train the model on a base class with a large number of samples to acquire general feature representation capabilities, and then quickly adapt it using only a small number of samples (i.e., "support samples") for each new class. While this paradigm alleviates the dependence on data, existing methods still face many serious challenges when adapting to new classes. First, at the data level, the limited number of samples leads to a severe lack of intra-class diversity, making the model prone to overfitting to the few samples seen during training, and struggling to generalize to different shapes, lighting, and backgrounds of the same type of object. Traditional data augmentation methods (such as flipping and rotating) can only bring limited linear transformations, while generative model-based methods struggle to effectively maintain the semantic structure of foreground objects while enhancing background diversity.

[0004] Secondly, at the level of feature discriminability, in high-dimensional feature spaces, the conventional softmax loss function tends to learn separable features, but it is difficult to ensure that the learned features have sufficient discriminability. That is, the distribution of features of the same class is not compact enough, and there is a lack of sufficient spacing between features of different classes. This phenomenon is amplified dramatically under the condition of few samples. Limited samples make it extremely difficult to control intra-class variance and distinguish between classes, ultimately leading to a large amount of inter-class confusion in the model during testing.

[0005] In summary, existing few-shot target detection techniques mainly suffer from the following bottlenecks: data augmentation methods are limited and struggle to balance sample diversity and semantic consistency; the feature embedding space lacks effective discriminative constraints, leading to intra-class dispersion and inter-class confusion. Therefore, there is an urgent need for an innovative method that can collaboratively address both data diversity and feature discriminative enhancement to improve the robustness and generalization ability of models in data-scarce scenarios. Summary of the Invention

[0006] This invention proposes a few-sample object detection method, device, and storage medium based on style augmentation. It solves the problems in the prior art of how to ensure that the foreground semantics are not destroyed while significantly increasing the diversity of background and environment to improve the model's generalization ability when training samples are extremely scarce; and how to establish effective discriminative constraints in high-dimensional embedding space to improve intra-class aggregation and inter-class separability.

[0007] The technical solution of this invention is implemented as follows:

[0008] The first aspect of this invention provides a small-sample target detection method based on style augmentation, comprising the following steps:

[0009] S1, Feature Extraction: Encode the input query image and support images respectively, and extract the content features of the query image and the style features of the support images;

[0010] S2, Style Fusion: Using an adaptive instance normalization method, the content features and style features are fused to generate style-enhanced samples;

[0011] S3, Similarity metric: Style-enhanced samples are fed into the detection backbone network to extract embedded features, and the angular similarity between the embedded features and the predefined center vectors of each category is calculated;

[0012] S4, Discriminative constraint: Apply an angular interval to the angular similarity of the target class and use a scaling factor to scale the angular similarity of all classes;

[0013] S5, Detection Output: Based on the scaled similarity of each category, the classifier outputs the category label of the target, and the detection head outputs the location bounding box of the target to obtain the final detection result.

[0014] Specifically, in step S1, a pre-trained convolutional neural network is used as a shared encoder to process the query image. and supporting images Encode, respectively in L Extracting content features from each network layer With style characteristics ,in, And calculate the channel mean and standard deviation of each layer feature.

[0015] Specifically, in step S2, the adaptive instance normalization operation is performed according to the following formula:

[0016] ;

[0017] in, This indicates the normalization operation of the adaptive implementation; This represents the mean. It represents the standard deviation.

[0018] Furthermore, in step S2, during multi-layer feature fusion, a learnable fusion coefficient is introduced for each layer. Then the first i The fusion characteristics of the layers are:

[0019] ;

[0020] ;

[0021] in, It is the sigmoid activation function. The weight matrix is ​​a learnable matrix. It is used to control the balance between content retention and style injection.

[0022] Specifically, step S3 includes:

[0023] For embedded features x With category center vector Perform L2 normalization separately:

[0024] ;

[0025] in, and Representing the normalized embedding features and the first... j Each category center vector The L2 norm of a vector;

[0026] The cosine of the normalized embedded features and the class center vector is calculated as the angular similarity:

[0027] ;

[0028] in, For vectors and The angle between them.

[0029] Specifically, step S4 includes:

[0030] For target category y Calculate their angular similarity Replace with ,in, m The angular interval is additive.

[0031] Multiply the angular similarity of all categories by a scaling factor. s The scaled similarity scores of each category are then used as the classification logical values.

[0032] Specifically, in step S5, the method for outputting the target's category label through the classifier is as follows:

[0033] Using the Arcface function as a classifier, the classification logical values ​​are converted into class prediction probabilities:

[0034] ;

[0035] in, Representing embedded features x Belongs to the real category y The predicted probability.

[0036] Furthermore, in step S5, the detection head employs a region proposal network and a regression branch;

[0037] The region proposal network is used to generate candidate regions on the feature map;

[0038] The regression branch is used to predict the location coordinates of the candidate region. x , y, w , h ),in, x , y, w , h These represent the center coordinates, length, and width of the candidate region, respectively.

[0039] Based on the predicted probability of the category and the predicted location coordinates, a non-maximum suppression algorithm is applied to filter out the final detection results. box , cls ),in, box The bounding box representing the location of the target. cls The category label representing the target.

[0040] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to implement the steps of the target detection method.

[0041] A third aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the target detection method.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] (1) This invention effectively solves the key problem of poor model generalization ability under few sample conditions by constructing a technical framework for collaborative optimization of "data layer style augmentation" and "feature layer discriminative constraint". This scheme can significantly expand the distribution diversity of training data without destroying the foreground semantics, and construct a highly discriminative metric in the high-dimensional feature space, thereby improving the detection accuracy and robustness of the model in data-scarce scenarios.

[0044] (2) This invention introduces the style fusion mechanism of Adaptive Instance Normalization (AdaIN) to effectively combine the style features of the supporting image with the content features of the query image. This method can generate enhanced samples with different styles such as background, texture, and lighting, but with the semantic structure of the foreground target remaining unchanged. Compared with traditional geometric transformation enhancement, this invention can achieve more realistic and diverse data augmentation from the essential feature level, thereby significantly enriching the distribution of training data and effectively alleviating the model overfitting problem caused by the scarcity of samples.

[0045] (3) By introducing learnable fusion coefficients in multi-layer feature fusion, this invention achieves dynamic and refined control over the degree of injection of content information and style information. This mechanism allows the model to adaptively adjust the fusion strategy according to different network layers (such as low-level texture and high-level semantics), ensuring that while enhancing style diversity, the core content semantic information is preserved to the greatest extent, further consolidating the quality and effectiveness of the generated samples.

[0046] (4) This invention constructs a highly discriminative supervision signal by calculating the angular similarity between the embedded features and the class center vector, and by introducing an additive angular interval and a scaling factor. This mechanism explicitly compresses the intra-class feature distribution in the angular space while expanding the inter-class feature interval, forcing the model to learn feature representations with higher cohesion and separability. This fundamentally solves the problem of loose feature distribution and easy inter-class confusion under conditions of few samples, and greatly improves the accuracy of classification. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating a small-sample target detection method based on style augmentation according to the present invention. Detailed Implementation

[0049] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0050] Reference Figure 1 The first aspect of this invention provides a small-sample target detection method based on style augmentation, comprising the following steps:

[0051] S1, Feature Extraction: Encode the input query image and support images respectively, and extract the content features of the query image and the style features of the support images;

[0052] Query Image With supporting image The input system first preprocesses both components to normalize their scale and distribution:

[0053] Scaling the shorter side of the image to between 600 and 800 pixels;

[0054] Then, pixel values ​​are normalized to make them follow a zero-mean, unit-variance distribution. ,in and These represent the global pixel mean and standard deviation of the training dataset, respectively. During the training phase, light data augmentation (such as random flipping, color jitter, or slight cropping) can be applied to the images with a certain probability to introduce slight randomness and reduce the risk of overfitting. Subsequently, the images are scaled to a uniform resolution. H × W This ensures that subsequent features are aligned.

[0055] To enhance stylistic diversity, samples are taken from a supporting image set or a pre-built external style library. K One style sample (preferred in this embodiment) K =3). Use a pre-trained convolutional neural network with shared weights (such as VGG19, truncated to ReLU4-1 layer) as the encoder. f For query images and each style sample Encode. Separately in... L Content features of the query image are extracted from 4 network layers (e.g., relu1_1, relu2_1, relu3_1, relu4_1, i.e., L=4). Style characteristics of style samples ,in, And calculate the channel mean and standard deviation of each layer feature.

[0056] S2, Style Fusion: Using an adaptive instance normalization method, the content features and style features are fused to generate style-enhanced samples; this step aims to inject the style of the supporting image into the content of the query image to generate enhanced samples that both preserve semantics and are rich in variation.

[0057] Adaptive Instance Normalization (AdaIN): For the first instance... i The layer uses the AdaIN operation for style injection, and the calculation formula is as follows:

[0058] ;

[0059] in, This indicates the normalization operation of the adaptive implementation; This represents the mean. This represents the standard deviation. This operation will query the characteristics. The channel mean and standard deviation are aligned to style features. The statistics are used to achieve style transfer.

[0060] To avoid style injection from disrupting key semantics, a learnable fusion coefficient is introduced for each layer during multi-layer feature fusion. This coefficient is calculated using a small projection network (such as a 1×1 convolution) and a sigmoid activation function: ,in Let be a learnable weight matrix, such that Then the first i The final fusion features of the layers for:

[0061] ;

[0062] ;

[0063] in, The sigmoid activation function has a fusion coefficient. Used to control the balance between content retention and style injection, initially, It can be set to close to 1, so that the model prioritizes retaining content and gradually learns the optimal fusion ratio during training.

[0064] In the AdaIN operation, to avoid minor disruptions to the foreground semantics caused by numerical instability, an alignment threshold (e.g., 10) can be set. -3 When the difference between the calculated style statistics and content statistics is less than the threshold, style injection can be weakened, prioritizing the stability of the foreground semantics.

[0065] In multi-layer feature fusion, a hierarchical fixed weight strategy can be used to replace or initialize the learnable fusion coefficients. .

[0066] For low-level features (such as ReLU1_1 and ReLU2_1), which contain more texture and color information, higher style weights can be set. (Recommended range 1.0 to 1.5) to emphasize the introduction of surface diversity such as background style and lighting.

[0067] For high-level features (such as ReLU3_1 and ReLU4_1), which carry more semantic content, higher content weights can be assigned. (Recommended range 2.0–3.0) to ensure that the semantic and structural consistency of the target object’s category is not compromised.

[0068] Then, merge the features of all layers. Input a lightweight convolutional decoder Reconstructed into the final style-enhanced image. This enables multi-scale style enhancement.

[0069] To improve training efficiency and stability, the generated... K Each augmented sample is screened. An initial detection backbone and detection head are fed in, and their overall detection confidence score is obtained. This embodiment employs a strategy of selecting the variant with the highest score, i.e. , and selected Used for subsequent steps.

[0070] S3, Similarity metric: Enhances selected style samples The high-dimensional embedding features are then fed into the formal training backbone network (such as ResNet) to extract them. x Then the embedded features are calculated. x Angular similarity with predefined category center vectors;

[0071] First, regarding embedded features x With category center vector Perform L2 normalization separately:

[0072] ;

[0073] in, and Representing the normalized embedding features and the first... j Each category center vector The L2 norm of a vector;

[0074] Then, the cosine value between the normalized embedded features and the class center vector is calculated as the angular similarity:

[0075] ;

[0076] in, For vectors and The angle between them.

[0077] S4, Discriminative Constraint: To enhance the discriminativeness of features, an additive angular interval is applied to the angular similarity of the target class, and a scaling factor is used to scale the angular similarity of all classes;

[0078] For the target category (the true category of the sample) y Calculate their angular similarity Replace with ,in, m The additive angular interval (selectable range is 0.4-0.5, preferred in this embodiment) is preferred. m = 0.45);

[0079] Angular similarity of all categories (for each category) y for For other categories j for Multiply by scaling factor s (The selectable range is 30-64, and this embodiment is preferred.) s = 32), and obtain the scaled similarity of each category as the classification logits.

[0080] S5, Detection Output: Based on the scaled similarity of each category, the classifier outputs the category label of the target, and the detection head outputs the location bounding box of the target to obtain the final detection result.

[0081] Using the Arcface function as a classifier, the classification logical values ​​are converted into class prediction probabilities:

[0082] ;

[0083] in, Representing embedded features x Belongs to the real category y The predicted probability is calculated, and the category with the highest probability is the predicted label.

[0084] The feature maps of the style-enhanced samples are input into the detection head, which employs a Region Proposal Network (RPN) and a regression branch.

[0085] The region proposal network is used to generate candidate regions on the feature map;

[0086] The regression branch is used to predict the location coordinates of the candidate region. x , y, w , h ),in, x , y, w , h These represent the center coordinates, length, and width of the candidate region, respectively.

[0087] Based on the predicted probability of the category and the predicted location coordinates, the non-maximum suppression (NMS) algorithm is applied to filter out the final detection results. box , cls ),in, box The bounding box representing the location of the target. cls The category label representing the target.

[0088] During training, the total loss function It consists of a weighted sum of three parts:

[0089] Style generation constraint loss The difference between the calculated content and style reconstruction;

[0090] Angle-based loss determination ;

[0091] Detecting regression loss It is used for bounding box coordinate regression.

[0092] The total loss is minimized by an optimizer (such as AdamW), and the parameters of the style augmentation module and the detection model are jointly optimized.

[0093] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to implement the steps of the target detection method.

[0094] A third aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the target detection method.

[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A small-sample target detection method based on style augmentation, characterized in that, Includes the following steps: S1, Feature Extraction: Encode the input query image and support images respectively, and extract the content features of the query image and the style features of the support images; S2, Style Fusion: Using an adaptive instance normalization method, the content features and style features are fused to generate style-enhanced samples; S3, Similarity metric: Style-enhanced samples are fed into the detection backbone network to extract embedded features, and the angular similarity between the embedded features and the predefined center vectors of each category is calculated; S4, Discriminative constraint: Apply an angular interval to the angular similarity of the target class and use a scaling factor to scale the angular similarity of all classes; S5, Detection Output: Based on the scaled similarity of each category, the classifier outputs the category label of the target, and the detection head outputs the location bounding box of the target to obtain the final detection result.

2. The small-sample target detection method based on style augmentation as described in claim 1, characterized in that, In step S1, a pre-trained convolutional neural network is used as a shared encoder to process the query image. and supporting images Encode, respectively in L Extracting content features from each network layer With style characteristics ,in, And calculate the channel mean and standard deviation of each layer feature.

3. The small-sample target detection method based on style augmentation as described in claim 2, characterized in that, In step S2, the adaptive instance normalization operation is performed according to the following formula: ； in, This indicates the normalization operation of the adaptive implementation; This represents the mean. It represents the standard deviation.

4. The small-sample target detection method based on style augmentation as described in claim 3, characterized in that, In step S2, during multi-layer feature fusion, a learnable fusion coefficient is introduced for each layer. Then the first i The fusion characteristics of the layers are: ； ； in, It is the sigmoid activation function. The weight matrix is ​​a learnable matrix. It is used to control the balance between content retention and style injection.

5. The small-sample target detection method based on style augmentation as described in claim 1, characterized in that, Step S3 specifically includes: Embedded features x With category center vector Perform L2 normalization separately: ； in, and Representing the normalized embedding features and the first... j Each category center vector The L2 norm of a vector; The cosine of the normalized embedded features and the class center vector is calculated as the angular similarity: ； in, For vectors and The angle between them.

6. The small-sample target detection method based on style augmentation as described in claim 1, characterized in that, Step S4 specifically includes: For target category y Calculate their angular similarity Replace with ,in, m The angular interval is additive. Multiply the angular similarity of all categories by a scaling factor. s The scaled similarity scores of each category are then used as the classification logical values.

7. The small-sample target detection method based on style augmentation as described in claim 6, characterized in that, In step S5, the method for outputting the target's category label through the classifier is as follows: Using the Arcface function as a classifier, the classification logical values ​​are converted into class prediction probabilities: ； in, Representing embedded features x Belongs to the real category y The predicted probability.

8. The small-sample target detection method based on style augmentation as described in claim 7, characterized in that, In step S5, the detection head uses a region proposal network and a regression branch; The region proposal network is used to generate candidate regions on the feature map; The regression branch is used to predict the location coordinates of the candidate region. x , y,w , h ),in, x , y,w , h These represent the center coordinates, length, and width of the candidate region, respectively. Based on the predicted probability of the category and the predicted location coordinates, a non-maximum suppression algorithm is applied to filter out the final detection results. box , cls ),in, box The bounding box representing the location of the target. cls The category label representing the target.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the target detection method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the target detection method as described in any one of claims 1 to 8.

Images