A small sample target detection method based on support feature diffusion generation

Abstract

This invention relates to the fields of image processing and artificial intelligence, and particularly to a few-shot object detection method based on support feature diffusion generation. The method includes steps such as constructing a few-shot object detection base class meta-learning training set and grouping image data within the training set. This invention introduces a support feature diffusion generation module, applying a diffusion model to the generation and optimization of support features. During the training phase, the inverse process of adding noise to features is learned, enabling the model to recover robust support feature representations from noise. During the testing phase, through progressive reverse diffusion, a more stable and diverse set of support features can be generated from a small number of support samples, effectively mitigating the feature instability and overfitting problems caused by few samples.

Description

Technical Field

[0001] This invention relates to the fields of image processing and artificial intelligence, and in particular to a small sample target detection method based on support feature diffusion generation. Background Technology

[0002] Object detection is an essential tool for scene perception in fields such as autonomous driving and security monitoring. It can quickly extract the location and attribute information of key targets in a scene, providing strong support for task planning and decision-making based on scene perception.

[0003] Traditional object detection methods rely on analyzing human target characteristics to design target features and using classifiers for recognition. While effective for simple scenes and tasks, their adaptability decreases significantly in complex scenarios. In recent years, object detection methods, represented by deep learning, have made great strides in the field of scene perception. Deep learning-based methods automatically learn target features through multi-layer convolutional neural networks, enabling them to better adapt to scene changes and have become the mainstream method for scene perception.

[0004] Deep learning-based object detection methods typically have a large number of parameters, which are adjusted through backpropagation, usually requiring multiple iterations. This process is driven by a large amount of data. Deep learning-based object detection methods generally show good performance for categories that are relatively consistent with the training distribution, which requires a relatively large number of samples in that category, typically thousands. However, for some specific scenarios, the number of samples collected is relatively limited, and the cost of collecting large numbers of samples is too high, prompting the development of few-sample object detection methods. Few-sample object detection methods can usually complete training and learning for a small number of categories, meeting the application requirements.

[0005] Deep learning-based few-shot object detection methods typically employ meta-learning to transfer the model's learning capabilities from large-sample categories to few-shot categories. However, these models often exhibit low classification performance. To address this issue, variational feature aggregation is introduced to reconstruct and sample the support class distribution, using this sampled distribution for auxiliary classification of few samples. However, the support features obtained through variational feature aggregation suffer from insufficient stability and feature diversity. Furthermore, the use of linear classification also limits the model's discriminative ability, making it difficult to achieve good few-shot object detection performance. Summary of the Invention

[0006] This invention addresses the shortcomings of existing meta-learning-based few-shot target detection methods by providing a few-shot target detection method that supports feature diffusion generation, thereby improving the performance of few-shot target detection.

[0007] The objective of this invention is achieved through the following technical solution: A few-sample target detection method based on support for feature diffusion generation includes the following steps: S1. Construct a base class meta-learning training set for few-shot object detection, which includes a base class meta-learning query training set and a base class meta-learning support training set; wherein, the base class meta-learning support training set is obtained by selecting N images from each class in the base class image training set, where N is a positive integer not greater than 30. S2. Divide the image data in the small-sample object detection base class meta-learning training set into groups to form base class training sample groups. Each base class training sample group includes batch_size images from the base class meta-learning query training set. Zhangji meta-learning supports training set images, where batch_size is the number of samples in each iteration. The number of categories in the base class; S3. The images from the training set of base class elements for small sample target detection in S2 are fed into the small sample target detection network generated based on support feature diffusion for forward propagation processing. The small sample target detection network based on feature diffusion generation includes a backbone network, a candidate region generation module, a support image feature extraction module, a query image feature extraction module, a support feature diffusion generation module, a feature aggregation module, a cosine class regression module, and a bounding box localization module. The backbone network, candidate region generation module, support image feature extraction module, query image feature extraction module, feature aggregation module, and bounding box localization module adopt modules from the mature technology MetaFR-CNN (Meta Faster Region-based Convolutional Neural Network).

[0008] The system comprises the following modules: a backbone network for extracting high-dimensional features from the image; a candidate region generation module for coarse localization of target boxes and determination of background; a support image feature extraction module for extracting support features corresponding to the coarsely localized target boxes from the high-dimensional features of the support image and averaging them according to category; a query image feature extraction module for extracting query features corresponding to the coarsely localized target boxes from the high-dimensional features of the query image; a support feature diffusion generation module for generating more stable and generalized support features; a feature ensemble module for multiplying the support features with the query features to obtain the final features used for category and boundary regression; and a cosine category regression module and a bounding box localization module for outputting the target recognition results.

[0009] Step S3 specifically includes the following sub-steps: S31. The images in the small sample target detection base class meta-learning training set are grouped according to the sample grouping in S2, and then fed into the backbone network and candidate region generation module of the small sample target detection network based on support feature diffusion to obtain the basic features and candidate region extraction results of the base class meta-learning query training set and the base class meta-learning support training set. and This is used to provide support for the subsequent generation of support feature sets; S32, will and The inputs are fed into the query image feature extraction module and the support image feature extraction module to obtain the interest pooling features of the query image and the interest pooling features of the support image, respectively. ; S33. Apply the interest pooling features of the support image obtained in S32. Noise is gradually added within a time step T to obtain the features at each time point t; during forward propagation, noise is randomly selected at time point t; the noise follows a Gaussian distribution with a mean of 0 and a covariance matrix of identity matrix I, i.e. Add noise using the following formula:

[0010]

[0011] In the formula, , , and It is standard Gaussian noise of the same dimension. For noise variance scheduling; Features after adding noise The data is fed into the support feature diffusion generation module to obtain the support feature set. .

[0012] In step S33, the feature diffusion generation module is supported for using pooling features of interest. After adding noise, the feature is encoded using a fully connected network encoder for high-dimensional mapping, and then decoded using a fully connected network decoder to achieve forward diffusion of the noisy pooled features. The formula for the mapping process is as follows:

[0013] in, This is a fully connected network encoder, consisting of fully connected layers, batch normalization layers, and a LeakyReLU activation function. The LeakyReLU activation function is:

[0014] in, Here, 'a' represents the hidden layer feature value of the fully connected network encoder, where 'a' is a very small positive number, typically 0.001. This is a fully connected network decoder, consisting of fully connected layers, batch normalization layers, and a sigmoid activation function. The sigmoid activation function is:

[0015] in, These are the hidden layer feature values ​​of the fully connected network encoder.

[0016] S34, will and The input is fed into the feature aggregation module to obtain new query image features, specifically interest pooling features. The feature aggregation module processes the following steps:

[0017] in Let i be the support vector corresponding to the i-th category. This is a tensor splicing operation; S35. Utilize the query image features obtained in S34 to perform interest pooling features. The values ​​are input into the cosine classification regression module and the bounding box localization module, respectively, to obtain the coordinates and category of the bounding box.

[0018] In step S35, the cosine category regression module consists of a fully connected layer, and the bounding box localization module consists of a fully connected layer.

[0019] The specific functions implemented by the newly added cosine category regression module in this invention are as follows: The interest pooling feature Fb_q_new of the query image is input into the cosine classification regression module to obtain better bounding box classification results. The cosine classification regression module consists of fully connected layers, and the formula for cosine classification regression is:

[0020] in, The weights of the fully connected layer, This is a scaling factor used to adjust the numerical range of cosine similarity, enhancing the classifier's ability to distinguish feature differences. For the first Group prediction, for The transpose of .

[0021] In step S3, the pooling features of interest in the support set image are gradually noise-added and diffused, which helps to generate a more stable support set.

[0022] In step S3, a feature diffusion generation module is supported. This module reconstructs the feature distribution of the new class and the old class through a diffusion mechanism. During training, the module gradually adds noise to enable the network to predict noise. During testing, it gradually removes noise to recover stable support features.

[0023] In step S3, the cosine class regression module calculates the cosine similarity between the sample vector to be classified and the known classes, and determines the class of the sample based on the similarity.

[0024] S4. Calculate the binary regression loss and target localization loss of the candidate region generation module in the small sample target detection network based on support feature diffusion; The binary regression loss in step S4 is:

[0025] in, For tags, To predict the probability of it being the foreground, Valid sample size; The target location loss is:

[0026] in This is the predicted border offset. Actual offset; The number of positive samples To smooth the L1 norm loss, The formula is as follows:

[0027] in, .

[0028] S5. Calculate the category regression loss and bounding box regression loss in a few-sample object detection network based on support feature diffusion generation; The formula for the category regression loss in step S5 is:

[0029] Where N is the total number of candidate boxes involved in the calculation. For the first The probability that each candidate box is predicted as the true class. For the first The true category of each candidate box; The formula for bounding box regression loss is:

[0030] in This is the predicted border offset. The true offset; N is the number of positive samples.

[0031] S6. Sum the losses calculated in S4 and S5, calculate the average, and perform backpropagation to update the weights of the small sample target detection network generated based on the support feature diffusion. In step S6, the total loss of the few-sample object detection network based on support feature diffusion generation is: .

[0032] S7. Iteratively execute steps S3 to S6 until the few-shot object detection network converges, and save the weights of the few-shot object detection network generated based on support feature diffusion as the training weights for base class meta-learning. S8. Construct a few-shot meta-learning query image dataset. The number of samples in each class of the few-shot meta-learning query image dataset is equal to the number of samples in each class of the base class meta-learning support training set. The few-shot meta-learning query image dataset and the base class meta-learning support image dataset form a new class of few-shot object detection meta-learning training set. S9. Divide the image data in the training set of the new class small sample object detection base class meta-learning into groups to form new class training sample groups; each new class training sample group includes batch_size images from the new class meta-learning query training set. Zhangji meta-learning supports training set images, where batch_size is the number of samples in each iteration. This is the sum of the number of categories in the new class and the base class. S10. Change the number of categories in the category regression module of the few-shot object detection network generated based on support feature diffusion to the sum of the base class categories and the new class categories, and use the base class meta-learning training weights as the initial weights of the modified few-shot object detection network generated based on support feature diffusion. S11. The training set images of the new class few-shot target detection base class meta-learning in S9 are sent to the modified few-shot target detection network based on support feature diffusion in step S10 for forward propagation. Step S11 includes the following sub-steps; S111. The training set for the new class few-shot target detection base class meta-learning is grouped according to the sample grouping in S9, and then fed into the backbone network and candidate region generation module of the few-shot target detection network based on support feature diffusion to obtain the basic features and candidate region extraction results of the new class meta-learning query training set and the new class meta-learning support training set. and This is used to provide support for the subsequent generation of support feature sets; S112, will and The inputs are fed into the query image feature extraction module and the support image feature extraction module to obtain the interest pooling features of the query image and the interest pooling features of the support image, respectively. ; S113. Apply the interest pooling features of the support image obtained in S112. Noise is gradually added within a time step of 1000 to obtain the features at each time t; during forward propagation, noise is randomly added at time t; the noise follows a Gaussian distribution with a mean of 0 and a covariance matrix of identity matrix I, i.e. Add noise using the following formula: ; ;

[0033] in , , and It is standard Gaussian noise of the same dimension. For noise variance scheduling, the noisy features are fed into the support feature diffusion generation module to obtain the support feature set. ; S114, will and The input is fed into the feature aggregation module to obtain new query image features, specifically interest pooling features. The feature aggregation module processes the following steps:

[0034] in For the first Support vectors corresponding to each category This is a tensor splicing operation; S115. Utilize the query image features obtained in S114 to perform interest pooling features. The values ​​are input into the cosine classification regression module and the bounding box localization module, respectively, to obtain the coordinates and category of the bounding box. The specific functions implemented by the cosine category regression module in this sub-step are as follows: The interest pooling feature Fn_q_new from the query image is input into the cosine classification regression module to obtain better bounding box classification results. The cosine classification regression module consists of fully connected layers, and the formula for cosine classification regression is:

[0035] in, The weights of the fully connected layer, This is a scaling factor used to adjust the numerical range of cosine similarity, enhancing the classifier's ability to distinguish feature differences. For the i-th prediction, the dimension is... .

[0036] S12. Calculate the binary classification regression loss and target localization loss of the candidate region generation module of the modified small sample target detection network based on the support feature diffusion generation. In step S12, the binary classification regression loss of the candidate region generation module of the modified network is changed. The calculation formula is:

[0037] in, For tags, To predict the probability of it being the foreground, Valid sample size; In step S12, the target localization loss of the candidate region generation module of the modified network is... The calculation formula is:

[0038] in This is the predicted border offset. Actual offset; The number of positive samples.

[0039] S13. Calculate the category regression loss and target regression loss of the small sample target detection network generated based on the support feature diffusion after the modification; In step S13, the category regression loss The calculation formula is:

[0040] Where N is the total number of candidate boxes involved in the calculation. For the first The probability that each candidate box is predicted as the true class. For the first The true category of each candidate box; In step S13, the target regression loss The calculation formula is:

[0041] in This is the predicted border offset. The true offset; N is the number of positive samples.

[0042] S14. The losses calculated in S12 and S13 are summed and averaged, and backpropagation is performed to update the weights of the small sample target detection network generated based on the support feature diffusion. In step S14, the total loss of the modified few-sample object detection network based on support feature diffusion is: .

[0043] S15. Iteratively execute steps S11 to S14 until the modified few-sample object detection network based on support feature diffusion converges. Save the weights of the few-sample object detection network based on support feature diffusion at this time as training weights for new class element learning. S16. Input the small sample test image dataset into the small sample target detection network generated by support feature diffusion obtained in S15 for detection, and output the identified target boxes, the category information, confidence score and target location of each target box.

[0044] Step S16 includes the following sub-steps: S161. Construct a test support image dataset using a few-sample meta-learning query image dataset and a base class meta-learning support image dataset; S162. Input the small sample test images and test support image datasets into the small sample object detection network obtained in S15; obtain the basic features and candidate region extraction results of the small sample test images and test support image datasets through the backbone network and candidate region generation module, respectively. and This is used to provide support for the subsequent generation of support feature sets; S163, will and The inputs are fed into the query image feature extraction module and the support image feature extraction module to obtain the interest pooling features of the query image and the interest pooling features of the support image, respectively. ; S164. Set the step size for diffusion inverse denoising to... The supporting feature set features are recovered from time t to time 0; the pooling features of interest in the test supporting image dataset are then retrieved. The input is fed into a fully connected network encoder that supports feature diffusion generation to obtain encoded features, which are then fed into a fully connected network decoder that also supports feature diffusion generation to obtain decoded features. These decoded features are then used as input and processed sequentially through both the fully connected network encoder and decoder, repeating this process. The support feature set after diffusion denoising is obtained; the formula for the diffusion denoising process is:

[0045] in To obtain the interest pooling features of the test supporting image dataset for S163, for Features after denoising Let be the encoded features output by the fully connected network decoder supporting the feature diffusion generation module at time t. ; The formula is as follows:

[0046]

[0047] ; S165. Input the support feature set after diffusion denoising and the interest pooling features of the small sample test images into the feature aggregation module to obtain the interest pooling features of the aggregated small sample test images. Will and The input is fed into the feature aggregation module to obtain new query image features, specifically interest pooling features. The feature aggregation module processes the following steps:

[0048] in Let i be the support vector corresponding to the i-th category. This is a tensor splicing operation; S166. Pool the query image features obtained in S165 with interest-based features. The values ​​are input into the cosine classification regression module and the bounding box localization module, respectively, to obtain the coordinates and class confidence of the bounding box. The cosine classification regression module consists of a fully connected layer, which predicts the class information of the bounding box and outputs the confidence. The bounding box localization module consists of a fully connected layer, which predicts the offset of the bounding box. The specific functions implemented by the cosine category regression module in this sub-step are as follows: The interest pooling feature Ft_n_q_new of the query image is input into the cosine classification regression module to obtain better bounding box classification results. The cosine classification regression module consists of fully connected layers, and the formula for cosine classification regression is:

[0049] in, The weights of the fully connected layer, This is a scaling factor used to adjust the numerical range of cosine similarity, enhancing the classifier's ability to distinguish feature differences. For the first Group prediction.

[0050] The predicted bounding box is obtained by adding the offset of the bounding box output by the bounding box localization module to Proposal_n_q. The category corresponding to the maximum category confidence score output by the cosine classification regression module is taken as the category of the bounding box, and this maximum confidence score is taken as the category confidence score of the bounding box. Finally, all the bounding boxes, confidence scores, and category information are combined, and duplicate boxes are removed by the NMS (Non-Maximum Suppression) operation to output the final identified target boxes. Each target box corresponds to a category information, confidence score, and target location.

[0051] In step 16, the input condition for the support feature diffusion generation module is the interest pooling features of the test support image dataset. The support feature set is gradually recovered from the interest pooling features of the test support image dataset through stepwise denoising during the testing phase.

[0052] The beneficial effects of this invention are: 1) This invention introduces a support feature diffusion generation module, applying the diffusion model to the generation and optimization of support features. During the training phase, the model learns the inverse process of adding noise to features, enabling it to recover robust support feature representations from noise. During the testing phase, through progressive reverse diffusion, a more stable and diverse set of support features can be generated from a small number of support samples, effectively alleviating the feature instability and overfitting problems caused by small samples.

[0053] 2) This invention uses a cosine category regression module to replace traditional inner product operations with cosine similarity for classification. By calculating the directional similarity between query features and the weight vectors of each category, and introducing a learnable scaling factor, the model's ability to distinguish between class differences in the feature space is enhanced, making it particularly suitable for scenarios with dense feature distributions and small sample sizes.

[0054] 3) This invention employs a two-stage training and dynamic class expansion mechanism (base class training → new class fine-tuning), achieving effective knowledge transfer and rapid adaptation. This method eliminates the need to retrain the entire network; it only requires fine-tuning the classifier head and using the diffusion module to generate new class support features, enabling efficient expansion to new classes and demonstrating good practicality and scalability. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating a specific implementation of the small-sample target detection method based on feature diffusion generation according to the present invention. Figure 2 This is a flowchart illustrating a specific implementation of the detection model described in this invention; Detailed Implementation

[0056] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] See Figures 1-2 The present invention provides a technical solution: Example

[0058] like Figure 1 and Figure 2 As shown, a few-sample target detection method based on support feature diffusion generation includes the following steps: S1. Construct a small sample object detection base class meta-learning training set with a large number of samples using the PASCAL VOC dataset. The small sample object detection base class meta-learning training set has 15 classes, including target types such as aeroplane, bicycle, bird, boat, bottle, bus, car, cat, chair, cow, dining table, dog, horse, motorbike, and person. Select 10 images from each class in the base class image training set as the base class meta-learning support training set. The base class meta-learning query training set and the base class meta-learning support training set are combined to form the small sample object detection base class meta-learning training set. S2. Group the image data in the small sample target detection base class meta-learning training set into base class training sample groups. Each base class training sample group includes 1 base class meta-learning query training set image and 10×15 base class meta-learning support training set images.

[0059] S3. The images from the training set of base class elements for small sample target detection in S2 are fed into the small sample target detection network generated based on support feature diffusion for forward propagation processing. The small sample target detection network based on feature diffusion generation includes a backbone network, a candidate region generation module, a support image feature extraction module, a query image feature extraction module, a support feature diffusion generation module, a feature aggregation module, a cosine class regression module, and a bounding box localization module. The backbone network, candidate region generation module, support image feature extraction module, query image feature extraction module, feature aggregation module, and bounding box localization module adopt modules from the mature technology MetaFR-CNN (Meta Faster Region-based Convolutional Neural Network).

[0060] The system comprises the following modules: a backbone network for extracting high-dimensional features from the image; a candidate region generation module for coarse localization of target boxes and determination of background; a support image feature extraction module for extracting support features corresponding to the coarsely localized target boxes from the high-dimensional features of the support image and averaging them according to category; a query image feature extraction module for extracting query features corresponding to the coarsely localized target boxes from the high-dimensional features of the query image; a support feature diffusion generation module for generating more stable and generalized support features; a feature ensemble module for multiplying the support features with the query features to obtain the final features used for category and boundary regression; and a cosine category regression module and a bounding box localization module for outputting the target recognition results.

[0061] Step S3 specifically includes the following sub-steps: S31. The images in the small sample target detection base class meta-learning training set are grouped according to the sample grouping in S2, and then fed into the backbone network and candidate region generation module of the small sample target detection network based on support feature diffusion to obtain the basic features and candidate region extraction results of the base class meta-learning query training set and the base class meta-learning support training set. and , , The vector dimensions are 1×1024×58×42 and 150×1024×58×42, respectively. , The quantities are 512 and 150×512 respectively, which are used to support the subsequent generation of support feature sets; S32, will and The inputs are fed into the query image feature extraction module and the support image feature extraction module to obtain the interest pooling features of the query image and the interest pooling features of the support image, respectively. ; , The dimensions are 1×512×1024×1×1 and 15×1024×1×1, respectively.

[0062] S33. Apply the interest pooling features of the support image obtained in S32. Noise is gradually added within a time step T to obtain the features at each time point t; during forward propagation, noise is randomly selected at time point t; the noise follows a Gaussian distribution with a mean of 0 and a covariance matrix of identity matrix I, i.e. Add noise using the following formula:

[0063]

[0064] In the formula, , , and It is standard Gaussian noise of the same dimension. For noise variance scheduling.

[0065] Features after adding noise The data is fed into the support feature diffusion generation module to obtain the support feature set. . Its size is 15×1024×1×1.

[0066] In step S33, the feature diffusion generation module is supported for using pooling features of interest. After adding noise, the feature is encoded using a fully connected network encoder for high-dimensional mapping, and then decoded using a fully connected network decoder to achieve forward diffusion of the noisy pooled features. The formula for the mapping process is as follows:

[0067] in, This is a fully connected network encoder, consisting of fully connected layers, batch normalization layers, and a LeakyReLU activation function. The LeakyReLU activation function is:

[0068] in, Here, 'a' represents the hidden layer feature value of the fully connected network encoder, where 'a' is a very small positive number, typically 0.001. This is a fully connected network decoder, consisting of fully connected layers, batch normalization layers, and a sigmoid activation function. The sigmoid activation function is:

[0069] in, These are the hidden layer feature values ​​of the fully connected network encoder.

[0070] S34, will and The input is fed into the feature aggregation module to obtain new query image features, specifically interest pooling features. The feature aggregation module processes the following steps:

[0071] in This is the support vector corresponding to the i-th category, with a dimension of 1×1×1024×1×1. This is a tensor splicing operation; S35. Utilize the query image features obtained in S34 to perform interest pooling features. The data are input into the cosine classification regression module and the bounding box localization module, respectively, to obtain the coordinates and categories of the bounding boxes. The cosine classification regression module consists of a fully connected layer with an input dimension of 1024 and an output dimension equal to the sum of the number of background elements and the number of categories, which equals 11. The bounding box localization module consists of a fully connected layer with an input dimension of 1024 and an output dimension of 4, predicting the offset of the bounding boxes.

[0072] In step S35, the cosine category regression module consists of a fully connected layer, and the bounding box localization module consists of a fully connected layer.

[0073] The specific functions implemented by the newly added cosine category regression module in this invention are as follows: The interest pooling feature Fb_q_new of the query image is input into the cosine classification regression module to obtain better bounding box classification results. The cosine classification regression module consists of fully connected layers, and the formula for cosine classification regression is:

[0074] in, The weights of the fully connected layer, This is a scaling factor used to adjust the numerical range of cosine similarity, enhancing the classifier's ability to distinguish feature differences. For the first Group prediction, with a dimension of 1×15. for The transpose of .

[0075] In step 3, the pooling features of the support set image are gradually noise-added and diffused, which helps to generate a more stable support set.

[0076] Step 3 supports a feature diffusion generation module, which reconstructs the feature distribution of the new and old classes through a diffusion mechanism. During training, this module gradually adds noise to enable the network to predict noise, and during testing, it gradually removes noise to recover stable support features.

[0077] In step 3, the cosine class regression module calculates the cosine similarity between the sample vector to be classified and the known classes, and determines the class of the sample based on the similarity.

[0078] S4. Calculate the binary regression loss and target localization loss of the candidate region generation module in the small sample target detection network based on support feature diffusion; The binary regression loss in step S4 is:

[0079] in, For tags, To predict the probability of it being the foreground, Valid sample size; The target location loss is:

[0080] in This is the predicted border offset. Actual offset; The number of positive samples To smooth the L1 norm loss, The formula is as follows:

[0081] in, .

[0082] S5. Calculate the category regression loss and bounding box regression loss in a few-sample object detection network based on support feature diffusion generation; The formula for the category regression loss in step S5 is:

[0083] Where N is the total number of candidate boxes involved in the calculation. For the first The probability that each candidate box is predicted as the true class. For the first The true category of each candidate box; The formula for bounding box regression loss is:

[0084] in This is the predicted border offset. The true offset; N is the number of positive samples.

[0085] S6. Sum the losses calculated in S4 and S5, calculate the average, and perform backpropagation to update the weights of the small sample target detection network generated based on the support feature diffusion. In step S6, the total loss of the few-sample object detection network based on support feature diffusion generation is: .

[0086] S7. Iteratively execute steps S3 to S6 until the few-shot object detection network converges, and save the weights of the few-shot object detection network generated based on support feature diffusion as the training weights for base class meta-learning. S8. Construct a few-shot meta-learning query image dataset using the PASCAL VOC training dataset. The categories include potted plant, sheep, sofa, train, and tvmonitor, with 10 samples per category. The base class meta-learning query training set and the base class meta-learning support training set are combined to form a new class of few-shot object detection base class meta-learning training set.

[0087] S9. Group the image data in the training set of base class meta-learning for new class small sample object detection into new class training sample groups; each new class training sample group includes 1 new class meta-learning query training set image and 10×20 base class meta-learning support training set images.

[0088] S10. Change the number of categories in the category regression module of the few-shot object detection network generated based on support feature diffusion to a sum of 20 for the base class categories and the new class categories, and use the base class meta-learning training weights as the initial weights of the modified few-shot object detection network generated based on support feature diffusion. S11. The training set images of the new class few-shot target detection base class meta-learning in S9 are sent to the modified few-shot target detection network based on support feature diffusion in step S10 for forward propagation. Step S11 includes the following sub-steps; S111. The training set for the new class few-shot target detection base class meta-learning is grouped according to the sample grouping in S9, and then fed into the backbone network and candidate region generation module of the few-shot target detection network based on support feature diffusion to obtain the basic features and candidate region extraction results of the new class meta-learning query training set and the new class meta-learning support training set. and ; , The vector dimensions are 1×1024×58×42 and 200×1024×58×42, respectively. , The quantities are 512 and 200×512 respectively, used to support the subsequent generation of support feature sets; S112, will and The inputs are fed into the query image feature extraction module and the support image feature extraction module to obtain the interest pooling features of the query image and the interest pooling features of the support image, respectively. ; , The dimensions are 1×512×1024×1×1 and 20×1024×1×1, respectively.

[0089] S113. Apply the interest pooling features of the support image obtained in S112. Noise is gradually added within a time step of 1000 to obtain the features at each time t; during forward propagation, noise is randomly added at time t; the noise follows a Gaussian distribution with a mean of 0 and a covariance matrix of identity matrix I, i.e. Add noise using the following formula:

[0090]

[0091] in , , and It is standard Gaussian noise of the same dimension. For noise variance scheduling; the features after adding noise The data is fed into the support feature diffusion generation module to obtain the support feature set. ; S114, will and The input is fed into the feature aggregation module to obtain new query image features, specifically interest pooling features. The feature aggregation module processes the following steps:

[0092] in For the first The support vectors for each category have a dimension of 1×1×1024×1×1. This is a tensor splicing operation; S115. Utilize the query image features obtained in S114 to perform interest pooling features. The inputs are fed into the cosine classification regression module and the bounding box localization module, respectively, to obtain the coordinates and categories of the bounding boxes. The cosine classification regression module consists of a fully connected layer with an input dimension of 1024 and an output dimension equal to 21 (the sum of the number of background elements and the number of categories). The bounding box localization module consists of a fully connected layer with an input dimension of 1024 and an output dimension of 4, predicting the offset of the bounding boxes.

[0093] The specific functions implemented by the cosine category regression module in this sub-step are as follows: The interest pooling feature Fn_q_new from the query image is input into the cosine classification regression module to obtain better bounding box classification results. The cosine classification regression module consists of fully connected layers, and the formula for cosine classification regression is:

[0094] in, The weights of the fully connected layer, This is a scaling factor used to adjust the numerical range of cosine similarity, enhancing the classifier's ability to distinguish feature differences. For the i-th prediction, the dimension is... .

[0095] S12. Calculate the binary classification regression loss and target localization loss of the candidate region generation module of the modified small sample target detection network based on the support feature diffusion generation. In step S12, the binary classification regression loss of the candidate region generation module of the modified network is changed. The calculation formula is:

[0096] in, For tags, To predict the probability of it being the foreground, Valid sample size; In step S12, the target localization loss of the candidate region generation module of the modified network is... The calculation formula is:

[0097] in This is the predicted border offset. Actual offset; The number of positive samples.

[0098] S13. Calculate the category regression loss and target regression loss of the small sample target detection network generated based on the support feature diffusion after the modification; In step S13, the category regression loss The calculation formula is:

[0099] Where N is the total number of candidate boxes involved in the calculation. For the first The probability that each candidate box is predicted as the true class. For the first The true category of each candidate box; In step S13, the target regression loss The calculation formula is:

[0100] in This is the predicted border offset. The true offset; N is the number of positive samples.

[0101] S14. The losses calculated in S12 and S13 are summed and averaged, and backpropagation is performed to update the weights of the small sample target detection network generated based on the support feature diffusion. In step S14, the total loss of the modified few-sample object detection network based on support feature diffusion is: .

[0102] S15. Iteratively execute steps S11 to S14 until the modified few-sample object detection network based on support feature diffusion converges. Save the weights of the few-sample object detection network based on support feature diffusion at this time as training weights for new class element learning. S16. Input the small sample test image dataset into the small sample target detection network generated by support feature diffusion obtained in S15 for detection, and output the identified target boxes, the category information, confidence score and target location of each target box.

[0103] Step S16 includes the following sub-steps: S161. Use untrained data from the PASCAL VOC dataset as few-shot test images and the training set for meta-learning of base classes for new few-shot object detection as the test support image dataset. Categories include: aeroplane, bicycle, bird, boat, bottle, bus, car, cat, chair, cow, dining table, dog, horse, motorbike, person, potted plant, sheep, sofa, train, and TV monitor. S162. Input the small sample test images and test support image datasets into the small sample object detection network obtained in S15; obtain the basic features and candidate region extraction results of the small sample test images and test support image datasets through the backbone network and candidate region generation module, respectively. and , , The vector dimensions are 1×1024×58×42 and 200×1024×58×42, respectively. , The quantities are 512 and 200×512 respectively, which are used to support the subsequent generation of support feature sets; S163, will and The inputs are fed into the query image feature extraction module and the support image feature extraction module to obtain the interest pooling features of the query image and the interest pooling features of the support image, respectively. ; , The dimensions are 1×512×1024×1×1 and 20×1024×1×1, respectively.

[0104] S164. Set the step size for diffusion inverse denoising to... The supporting feature set features are recovered from time t to time 0; the pooling features of interest in the test supporting image dataset are then retrieved. The input is fed into a fully connected network encoder that supports feature diffusion generation to obtain encoded features, which are then fed into a fully connected network decoder that also supports feature diffusion generation to obtain decoded features. These decoded features are then used as input and processed sequentially through both the fully connected network encoder and decoder, repeating this process. The diffusion-denoised support feature set is obtained in this step; the final denoised support feature dimension is 20×1024. The formula for the diffusion denoising process is:

[0105] in To obtain the interest pooling features of the test supporting image dataset for S163, for Features after denoising Let be the encoded features output by the fully connected network decoder supporting the feature diffusion generation module at time t. ; The formula is as follows:

[0106]

[0107] ; S165. Input the support feature set after diffusion denoising and the interest pooling features of the small sample test images into the feature aggregation module to obtain the interest pooling features of the aggregated small sample test images. Will and The input is fed into the feature aggregation module to obtain new query image features, specifically interest pooling features. The feature aggregation module processes the following steps:

[0108] in Let be the support vector corresponding to the i-th category, with a dimension of 1×1×1024×1×1. This is a tensor splicing operation; S166. Pool the query image features obtained in S165 with interest-based features. The inputs are fed into the cosine classification regression module and the bounding box localization module, respectively, to obtain the coordinates and class confidence of the bounding boxes. The cosine classification regression module consists of a fully connected layer with an input dimension of 512×1024 and an output dimension of 512×21. The bounding box localization module consists of a fully connected layer with an input dimension of 512×1024 and an output dimension of 512×4, predicting the offset of the bounding boxes.

[0109] The specific functions implemented by the cosine category regression module in this sub-step are as follows: The interest pooling feature Ft_n_q_new of the query image is input into the cosine classification regression module to obtain better bounding box classification results. The cosine classification regression module consists of fully connected layers, and the formula for cosine classification regression is:

[0110] in, The weights of the fully connected layer, This is a scaling factor used to adjust the numerical range of cosine similarity, enhancing the classifier's ability to distinguish feature differences. For the prediction group, the dimension is 1×21.

[0111] The predicted bounding box is obtained by adding the offset of the bounding box output by the bounding box localization module to Proposal_n_q. The category corresponding to the maximum category confidence score output by the cosine classification regression module is taken as the category of the bounding box, and this maximum confidence score is taken as the category confidence score of the bounding box. Finally, all the bounding boxes, confidence scores, and category information are combined, and duplicate boxes are removed by the NMS (Non-Maximum Suppression) operation to output the final identified target boxes. Each target box corresponds to a category information, confidence score, and target location.

[0112] In step 16, the input condition for the support feature diffusion generation module is the interest pooling features of the test support image dataset. The support feature set is gradually recovered from the interest pooling features of the test support image dataset through stepwise denoising during the testing phase.

[0113] The method was tested on the PASCAL VOC test dataset containing only 5 new categories. The test results are shown in Table 1 below:

[0114] Table 1 In Table 1, Method 1 represents the benchmark method Meta-R-CNN, and Method 2 represents the case using Support Feature Diffusion Generation. It can be seen that the map is significantly improved, which demonstrates the effectiveness of this method.

[0115] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A small-sample target detection method based on support feature diffusion generation, characterized in that: Includes the following steps: S1. Construct a base class meta-learning training set for few-shot object detection, which includes a base class meta-learning query training set and a base class meta-learning support training set; wherein, the base class meta-learning support training set is obtained by selecting N images from each class in the base class image training set, where N is a positive integer not greater than 30. S2. Divide the image data in the small-sample object detection base class meta-learning training set into groups to form base class training sample groups. Each base class training sample group includes batch_size images from the base class meta-learning query training set. Zhangji meta-learning supports training set images, where batch_size is the number of samples in each iteration. The number of categories in the base class; S3. The images from the training set of base class elements for small sample target detection in S2 are fed into the small sample target detection network generated based on support feature diffusion for forward propagation processing. The small sample target detection network based on feature diffusion generation includes a backbone network, a candidate region generation module, a support image feature extraction module, a query image feature extraction module, a support feature diffusion generation module, a feature aggregation module, a cosine class regression module, and a bounding box localization module. S4. Calculate the binary regression loss and target localization loss of the candidate region generation module in the small sample target detection network based on support feature diffusion; S5. Calculate the category regression loss and bounding box regression loss in a few-sample object detection network based on support feature diffusion generation; S6. Sum the losses calculated in S4 and S5, calculate the average, and perform backpropagation to update the weights of the small sample target detection network generated based on the support feature diffusion. S7. Iteratively execute steps S3 to S6 until the few-shot object detection network converges, and save the weights of the few-shot object detection network generated based on support feature diffusion as the training weights for base class meta-learning. S8. Construct a few-shot meta-learning query image dataset. The number of samples in each class of the few-shot meta-learning query image dataset is equal to the number of samples in each class of the base class meta-learning support training set. The few-shot meta-learning query image dataset and the base class meta-learning support image dataset form a new class of few-shot object detection meta-learning training set. S9. Divide the image data in the training set of the new class small sample object detection base class meta-learning into groups to form new class training sample groups; each new class training sample group includes batch_size images from the new class meta-learning query training set. Zhangji meta-learning supports training set images, where batch_size is the number of samples in each iteration. The sum of the number of categories in the new class and the base class; S10. Change the number of categories in the category regression module of the few-shot object detection network generated based on support feature diffusion to the sum of the base class categories and the new class categories, and use the base class meta-learning training weights as the initial weights of the modified few-shot object detection network generated based on support feature diffusion. S11. The training set images of the new small sample target detection base class in S9 are sent to the modified small sample target detection network based on support feature diffusion in step S10 for forward propagation. S12. Calculate the binary classification regression loss and target localization loss of the candidate region generation module of the modified small sample target detection network based on the support feature diffusion generation. S13. Calculate the category regression loss and target regression loss of the small sample target detection network generated based on the support feature diffusion after the modification; S14. The losses calculated in S12 and S13 are summed and averaged, and backpropagation is performed to update the weights of the small sample target detection network generated based on the support feature diffusion. S15. Iteratively execute steps S11 to S14 until the modified few-sample object detection network based on support feature diffusion converges. Save the weights of the few-sample object detection network based on support feature diffusion at this time as training weights for new class element learning. S16. Input the small sample test image dataset into the small sample target detection network generated by support feature diffusion obtained in S15 for detection, and output the identified target boxes, the category information, confidence score and target location of each target box.

2. The small sample target detection method based on support feature diffusion generation according to claim 1, characterized in that: Step S3 specifically includes the following sub-steps: S31. The images in the small sample target detection base class meta-learning training set are grouped according to the sample grouping in S2, and then fed into the backbone network and candidate region generation module of the small sample target detection network based on support feature diffusion to obtain the basic features and candidate region extraction results of the base class meta-learning query training set and the base class meta-learning support training set. and This is used to provide support for the subsequent generation of support feature sets; S32, will and The inputs are fed into the query image feature extraction module and the support image feature extraction module to obtain the interest pooling features of the query image and the interest pooling features of the support image, respectively. ; S33. Apply the interest pooling features of the support image obtained in S32. Noise is gradually added within a time step T to obtain the features at each time point t; during forward propagation, noise is randomly selected at time point t; the noise follows a Gaussian distribution with a mean of 0 and a covariance matrix of identity matrix I, i.e. Add noise using the following formula: ； ； In the formula, , , and It is standard Gaussian noise of the same dimension. For noise variance scheduling; Features after adding noise The data is fed into the support feature diffusion generation module to obtain the support feature set. ; S34, will and The input is fed into the feature aggregation module to obtain new query image features, specifically interest pooling features. The feature aggregation module processes the following steps: ； in Let i be the support vector corresponding to the i-th category. This is a tensor splicing operation; S35. Utilize the query image features obtained in S34 to perform interest pooling features. The values ​​are input into the cosine classification regression module and the bounding box localization module, respectively, to obtain the coordinates and category of the bounding box.

3. The small sample target detection method based on support feature diffusion generation according to claim 2, characterized in that: In step S33, the feature diffusion generation module is supported for using pooling features of interest. After adding noise, the feature is encoded using a fully connected network encoder for high-dimensional mapping, and then decoded using a fully connected network decoder to achieve forward diffusion of the noisy pooled features. The formula for the mapping process is as follows: ； in, This is a fully connected network encoder, consisting of fully connected layers, batch normalization layers, and a LeakyReLU activation function. The LeakyReLU activation function is: ； in, Here, 'a' represents the hidden layer feature value of the fully connected network encoder, where 'a' is a positive number, typically 0.

001. This is a fully connected network decoder, consisting of fully connected layers, batch normalization layers, and a sigmoid activation function. The sigmoid activation function is: ； in, These are the hidden layer feature values ​​of the fully connected network encoder.

4. The small sample target detection method based on support feature diffusion generation according to claim 2, characterized in that: In step S35, the cosine class regression module consists of fully connected layers, and the formula for cosine class regression is: ； in, The weights of the fully connected layer, This is a scaling factor used to adjust the numerical range of cosine similarity, enhancing the classifier's ability to distinguish feature differences. For the first Group prediction, for The transpose of .

5. The small sample target detection method based on support feature diffusion generation according to claim 4, characterized in that: The binary regression loss in step S4 is: ； in, For tags, To predict the probability of it being the foreground, Valid sample size; The target location loss is: ； in This is the predicted border offset. Actual offset; The number of positive samples To smooth the L1 norm loss, The formula is as follows: ； in, .

6. The small sample target detection method based on support feature diffusion generation according to claim 5, characterized in that: The formula for the category regression loss in step S5 is: ； Where N is the total number of candidate boxes involved in the calculation. For the first The probability that each candidate box is predicted as the true class. For the first The true category of each candidate box; The formula for bounding box regression loss is: ； in This is the predicted border offset. True offset; N is the number of positive samples; In step S6, the total loss of the few-sample object detection network based on support feature diffusion generation is: 。 7. The small sample target detection method based on support feature diffusion generation according to claim 1, characterized in that: Step S11 includes the following sub-steps; S111. The training set for the new class few-shot target detection base class meta-learning is grouped according to the sample grouping in S9, and then fed into the backbone network and candidate region generation module of the few-shot target detection network based on support feature diffusion to obtain the basic features and candidate region extraction results of the new class meta-learning query training set and the new class meta-learning support training set. and This is used to provide support for the subsequent generation of support feature sets; S112, will and The inputs are fed into the query image feature extraction module and the support image feature extraction module to obtain the interest pooling features of the query image and the interest pooling features of the support image, respectively. ; S113. Apply the interest pooling features of the support image obtained in S112. Noise is gradually added within a time step of 1000 to obtain the features at each time t; during forward propagation, noise is randomly added at time t; the noise follows a Gaussian distribution with a mean of 0 and a covariance matrix of identity matrix I, i.e. Add noise using the following formula: ； ； in , , and It is standard Gaussian noise of the same dimension. For noise variance scheduling; the features after adding noise The data is fed into the support feature diffusion generation module to obtain the support feature set. ; S114, will and The input is fed into the feature aggregation module to obtain new query image features, specifically interest pooling features. The feature aggregation module processes the following steps: ； in For the first Support vectors corresponding to each category This is a tensor splicing operation; S115. Utilize the query image features obtained in S114 to perform interest pooling features. The values ​​are input into the cosine classification regression module and the bounding box localization module, respectively, to obtain the coordinates and category of the bounding box. The cosine category regression formula in this sub-step is: ； in, The weights of the fully connected layer, This is a scaling factor used to adjust the numerical range of cosine similarity, enhancing the classifier's ability to distinguish feature differences. For the i-th prediction, the dimension is... .

8. The small sample target detection method based on support feature diffusion generation according to claim 1, characterized in that: In step S12, the binary classification regression loss of the candidate region generation module of the modified network is changed. The calculation formula is: ； in, For tags, To predict the probability of it being the foreground, Valid sample size; In step S12, the target localization loss of the candidate region generation module of the modified network is... The calculation formula is: ； in This is the predicted border offset. Actual offset; The number of positive samples; In step S13, the category regression loss The calculation formula is: ； Where N is the total number of candidate boxes involved in the calculation. For the first The probability that each candidate box is predicted as the true class. For the first The true category of each candidate box; In step S13, the target regression loss The calculation formula is: ； in This is the predicted border offset. True offset; N is the number of positive samples; In step S14, the total loss of the modified few-sample object detection network based on support feature diffusion is: 。 9. The small sample target detection method based on support feature diffusion generation according to claim 1, characterized in that: Step S16 includes the following sub-steps: S161. Construct a test support image dataset using a few-sample meta-learning query image dataset and a base class meta-learning support image dataset; S162. Input the small sample test images and test support image datasets into the small sample object detection network obtained in S15; The basic features and candidate region extraction results of the small sample test images and test support image datasets were obtained through the backbone network and candidate region generation module, respectively. and This is used to provide support for the subsequent generation of support feature sets; S163, will and The inputs are fed into the query image feature extraction module and the support image feature extraction module to obtain the interest pooling features of the query image and the interest pooling features of the support image, respectively. ; S164. Set the step size for diffusion inverse denoising to... The features of the supporting feature set are restored from time t to time 0. The test will support interest pooling features in image datasets. The input is fed into a fully connected network encoder that supports a feature diffusion generation module to obtain encoded features, and then fed into a fully connected network decoder that supports a feature diffusion generation module to obtain decoded features. The decoded features are then used as input and processed sequentially by a fully connected network encoder and a fully connected network decoder that support feature diffusion generation modules. This process is repeated. The support feature set after diffusion denoising is obtained; the formula for the diffusion denoising process is: ； in To obtain the interest pooling features of the test supporting image dataset for S163, for Denoising features Let be the encoded features output by the fully connected network decoder supporting the feature diffusion generation module at time t. ; The formula is as follows: ； ； ； S165. Input the support feature set after diffusion denoising and the interest pooling features of the small sample test images into the feature aggregation module to obtain the interest pooling features of the aggregated small sample test images. Will and The input is fed into the feature aggregation module to obtain new query image features, specifically interest pooling features. The feature aggregation module processes the following steps: ； in Let i be the support vector corresponding to the i-th category. This is a tensor splicing operation; S166. Pool the query image features obtained in S165 with interest-based features. The values ​​are input into the cosine classification regression module and the bounding box localization module, respectively, to obtain the coordinates and class confidence of the bounding box. The cosine classification regression module consists of a fully connected layer, which predicts the class information of the bounding box and outputs the confidence. The bounding box localization module consists of a fully connected layer, which predicts the offset of the bounding box. The cosine category regression formula in this sub-step is: ； in, The weights of the fully connected layer, This is a scaling factor used to adjust the numerical range of cosine similarity, enhancing the classifier's ability to distinguish feature differences. For the first Group prediction.

Images