A few-sample target detection method guided by dynamic latent features

Abstract

This invention relates to a few-sample object detection method guided by dynamic latent features. The method involves acquiring a sample dataset, constructing a few-sample object detection model guided by dynamic latent features, training the model with base class data from the sample dataset, freezing the model's local parameters, and then adjusting the model with new class data from the sample dataset. The method then inputs the target data into the adjusted model to obtain the few-sample object detection results. This invention makes feature representation more refined and diverse, improving the overall detection effect and the model's generalization ability. Through multi-scale information extraction and multi-similarity guidance, it significantly enhances the model's effective utilization of supporting features, thereby strengthening the predictive ability of targets.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing, and in particular to a small-sample target detection method based on dynamic latent features. Background Technology

[0002] In recent years, deep learning-based object detectors have achieved remarkable results, mainly relying on large amounts of high-quality training data and accurate bounding box annotations. However, this process is costly, and data annotation is tedious and time-consuming. In contrast, humans can quickly learn new concepts with limited data. To reduce the cost of manual annotation and narrow the gap between detectors and human learning capabilities, few-shot object detection has emerged. Its goal is to achieve efficient and accurate detection even with scarce samples, thereby better addressing various challenges in real-world applications.

[0003] Traditional small sample object detection methods typically follow a two-stage training paradigm. The first stage uses a large number of basic samples to train the model and build a general object detector. The second stage fine-tunes the model on a small number of new samples, hoping to improve detection performance when data is extremely scarce.

[0004] However, due to the limited number of new class samples, the model may be insufficient in representing supporting features, thus limiting detector performance. Furthermore, data imbalance makes the model more prone to learning base class features during training, neglecting key information about new classes and reducing detection accuracy for those classes. To address these issues, some methods attempt to improve representational power by weighting and integrating supporting and query features. However, these methods often rely on fixed feature abstractions, failing to fully utilize information at different scales and struggling to capture the complex relationships between supporting and query features, thus limiting the model's adaptability in small-sample scenarios. Summary of the Invention

[0005] This invention solves the problems existing in the prior art and provides a small sample target detection method based on dynamic latent features.

[0006] The technical solution adopted in this invention is a small sample target detection method based on dynamic latent features. The method acquires a sample dataset, constructs a small sample target detection model based on dynamic latent features, trains the model with base class data in the sample dataset, freezes the local parameters of the model, and adjusts (fine-tunes) the model with new class data in the sample dataset.

[0007] Input the target data into the adjusted model to obtain small sample target detection results.

[0008] Preferably, the small sample target detection model includes support branches and query branches;

[0009] The query branch includes a backbone network, an RPN module, and a regional interest alignment module arranged in sequence. The RPN module is used to generate candidate boxes, and the output of the regional interest alignment is aggregated with the output of the support branch and then output.

[0010] The support branch includes a backbone network and a fusion layer. The outputs of the backbone networks of the query branch and the support branch are aligned and then input into the fusion layer. The fusion layer is followed by a latent feature reconstruction module and a dynamic multi-scale similarity guidance module.

[0011] Preferably, the query branch and the support branch share the backbone network parameters.

[0012] Preferably, the output of the backbone network of the query branch is aligned dimensionally with the output of the backbone network supporting the branch through a global average pooling layer and an expansion module.

[0013] Preferably, the latent feature reconstruction module includes an encoder arranged in sequence, two convolutional blocks arranged in parallel, and the outputs of the two convolutional blocks are reparameterized and input into the decoder.

[0014] Preferably, the dynamic multi-scale similarity guidance module includes a multi-scale information generator for generating features at different scales. The features at different scales are input into the multi-similarity feature guidance module, processed by multiple similarity layers, and then concatenated and output. The processing here includes weighted fusion and guidance.

[0015] Preferably, the first branch reduces information loss and learns a high-resolution representation of features through multi-branch convolution operations; the second branch focuses on a low-resolution representation of features through global max pooling and deconvolution operations; and the third branch combines global average pooling and channel attention mechanisms to aggregate the feature representations of the first and second branches, thereby enhancing information representation capabilities.

[0016] Preferably, similarity scores between features at different scales are calculated based on a hybrid similarity metric, the internal weights of the support features are dynamically adjusted, and the features at three different scales are concatenated to obtain dynamically optimized multi-scale support features.

[0017] Preferably, the model is trained using base class data from the sample dataset, and the loss function is...

[0018] L = L rpn +L LFR +L meta +L cls +L reg

[0019] Among them, L rpn For the loss of RPN, L LFR L is the reconstruction loss of the latent feature reconstruction module. metaL is the meta-classification loss of the model (the loss in the baseline model). cls For classification loss (ordinary image classification loss), L reg For regression loss.

[0020] Preferably, after training the model with base class data in the sample dataset, the local parameters of the model are frozen, including the parameters of the backbone network, RPN module, and region interest alignment module.

[0021] This invention relates to a few-sample target detection method guided by dynamic latent features. The method involves acquiring a sample dataset, constructing a few-sample target detection model guided by dynamic latent features, training the model with base class data in the sample dataset, freezing the local parameters of the model, adjusting the model with new class data in the sample dataset, and inputting the data to be detected into the adjusted model to obtain the few-sample target detection results.

[0022] The beneficial effects of this invention are as follows:

[0023] (1) The Latent Feature Reconstruction (LFR) module is proposed to add latent information contained in the image to query features and supporting features, thereby enriching the representation ability of features and making up for the lack of information due to insufficient samples. This module encodes and reconstructs features to mine the hidden information in the latent space, making the feature representation more delicate and diverse. It not only improves the expressive ability of features and enables the model to effectively learn key information when faced with a small number of samples, but also provides richer and more accurate information support for subsequent detection tasks, thereby improving the overall detection effect and the generalization ability of the model.

[0024] (2) A Dynamic Multi-Scale Similarity Guidance (DMSG) module is designed to dynamically adjust the internal weights of supporting features according to the needs of the query image, thereby highlighting key information that is highly related to the query image and effectively suppressing useless background noise and possible occlusion interference. The DMSG module extracts multi-level information through a multi-scale information generator, enhancing the model's ability to understand complex scenes and ensuring that key features can be captured at different perspectives and scales. The DMSG module uses multiple similarity layers to quantify and model the similarity relationship between features at different levels, which can more accurately capture the connection between supporting features and the query image and highlight key prediction information. Through the extraction of multi-scale information and the guidance of multiple similarities, the effective use of supporting features by the model is significantly improved, thereby enhancing the prediction ability of the target. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0026] Figure 2 This is a model diagram of the present invention.

[0027] Figure 3 This is a structural diagram of the multi-scale generator module of the present invention.

[0028] Figure 4 This is a structural diagram of the multi-similarity feature guidance module of the present invention. Detailed Implementation

[0029] The present invention will be further described in detail below with reference to embodiments, but the scope of protection of the present invention is not limited thereto.

[0030] This invention relates to a small-sample target detection method guided by dynamic latent features, the method mainly comprising the following steps:

[0031] (1) Obtain the sample dataset;

[0032] (2) Construct a small sample target detection model based on dynamic latent features;

[0033] (3) After training the model with the base class data in the sample dataset, freeze the local parameters of the model;

[0034] (4) Adjust the model using new class data from the sample dataset;

[0035] (5) Input the data to be detected into the adjusted model to obtain the small sample target detection results.

[0036] The following steps provide a detailed explanation.

[0037] (1) Obtain the sample dataset;

[0038] Obtain the object detection dataset and divide it into a base class dataset and a new class dataset to create a few-sample detection task;

[0039] The data in the sample dataset is obtained from more than one dataset.

[0040] In the implementation of this invention, the PASCAL VOC and MS COCO datasets were selected as datasets;

[0041] For the PASCAL VOC dataset, the same three random partition settings as the TFA model are used. Each partition contains 20 classes, with 15 as base classes and 5 as new classes. Each new class contains K = 1, 2, 3, 5, 10 annotated instances.

[0042] For the MS COCO dataset, 20 categories that overlap with PASCAL VOC are selected from 80 object categories as new classes, and the remaining 60 categories are used as base classes. K values ​​are set to 10 and 30.

[0043] Few sample detection tasks are created using the partitioned dataset, with each task containing supporting sample data and query sample data.

[0044] (2) Construct a small sample target detection model based on dynamic latent features;

[0045] The small sample target detection model includes support branches and query branches;

[0046] The query branch includes a backbone network, an RPN module, and a regional interest alignment module arranged in sequence. The output of the regional interest alignment is aggregated with the output of the support branch and then output.

[0047] The support branch includes a backbone network and a fusion layer. The outputs of the backbone networks of the query branch and the support branch are aligned and then input into the fusion layer. The fusion layer is followed by a latent feature reconstruction module and a dynamic multi-scale similarity guidance module.

[0048] In this invention, aggregated query features (output of region interest alignment) and multi-scale support features (output of dynamic multi-scale similarity guidance module) are used, and the generated aggregated features are input into the detection head. The detection head is used to classify the target (class head) and regress the bounding box (box head), and finally output the prediction result.

[0049] Here, ResNet-101 is used as the backbone network to extract basic features of the image. The query branch and the support branch share the backbone network parameters. After feature extraction using the backbone network, the query features are refined using a global average pooling layer and expanded to the same dimension as the support features.

[0050] The latent feature reconstruction module includes an encoder arranged in sequence, two convolutional blocks arranged in parallel, and the outputs of the two convolutional blocks are reparameterized and input into the decoder.

[0051] In this invention, query features and supporting features are reconstructed using a latent feature reconstruction module to add latent information contained in the image. Latent Feature Reconstruction (LFR) uses variational inference to encode query features and supporting features, mapping the conditional distribution of the samples to a multivariate Gaussian distribution. Then, a reparameterization technique is used to reconstruct the latent code z from this distribution. z contains additional information representation, and the formula for calculating z is as follows:

[0052]

[0053] During training, LFR follows a reparameterized setting, but during inference, μ is used directly as the sampling result, where ε ~ N(0,1), ⊙ represents element-wise multiplication, and the approximate posterior mean μ and variance σ are obtained through two convolutional blocks, satisfying...

[0054]

[0055] in, It is a standard 3×3 convolution, F enc The encoder in LFR ultimately reconstructs features by resampling the query and support features in z. The LFR reconstruction loss is L... LFR The calculation formula is as follows:

[0056]

[0057] Where μ is the approximate posterior mean, σ is the approximate posterior variance, and X... / s These are query features and supporting features extracted from the backbone network, X C / s It is a feature reconstructed by LFR, but this is not the final feature used. The final feature used is the potential code z.

[0058] The dynamic multi-scale similarity guidance module includes a multi-scale information generator for generating features at different scales. These features are input into the multi-similarity feature guidance module, processed through multiple similarity layers, concatenated, and then output.

[0059] The different scales of features here are used to characterize global features, regional features, and local features.

[0060] In this invention, the Dynamic Multi-Scale Similarity Guidance (DMSG) module is used to reduce the detector's bias towards the base class;

[0061] The first part is to generate multi-scale information of features using a multi-scale information generator, which consists of three branches.

[0062] The first branch, High-level, mainly consists of downsampling modules. It utilizes multi-branch convolutions to reduce information loss and learn high-resolution representations of features. Here, it is the output of concatenated first-level convolutions (1×1conv) and second-level convolutions (3×3conv), the output of concatenated first-level convolutions (1×1conv), second-level convolutions (3×3conv), and third-level convolutions (3×3conv) are fused, and then output after concatenation through fourth-level convolutions (depthseparable convolutions) and fifth-level convolutions (1×1conv).

[0063]

[0064] The second low-level branch learns low-resolution representations of features through global max pooling and deconvolution. Specifically, a first-level convolution (1×1 conv) is input into both the global max pooling branch and the deconvolution branch, and the results are then fused. The global max pooling branch consists of a sequentially connected global max pooling layer (GMP), a deconvolution layer (DeConv), and a convolutional layer (3×3 conv). The deconvolution branch consists of a sequentially connected deconvolution layer (DeConv) and a convolutional layer (3×3 conv). The fused result is then output to the deconvolution layer (DeConv) and the convolutional layer (1×1 conv) before being output.

[0065]

[0066] The third branch, Mid-level, aggregates feature representations from the other two branches. To improve the adaptability of region features to variations in object scale, the output of the first-level convolution is multiplied by a global average pooling branch and a depthwise separable convolutional layer. The global average pooling branch includes a sequentially set global average pooling layer (GAP), a convolutional layer (1×1 conv), and a sigmoid activation function. The result of this multiplication is fused with the output of the fourth-level convolution (depthwise separable convolution) in the first branch and the output of the fused deconvolutional layer (DeConv) in the second branch, and then output through a convolutional layer (1×1 conv).

[0067]

[0068] Where DW() represents depthwise separable convolution, De() represents deconvolution, GAP() represents global average pooling, GMP() represents global max pooling, and S() represents the sigmoid function.

[0069] The second part calculates similarity scores between features at different scales based on a hybrid similarity metric, dynamically adjusts the internal weights of the support features, and concatenates the features at three different scales to obtain dynamically optimized multi-scale support features. Based on these multi-scale features, DMSG uses a multi-similarity feature guidance module to generate global similarity, regional similarity, and local similarity (classified by global features, regional features, and local features at different scales) to provide more refined and comprehensive guidance for the support features.

[0070] Specifically, after obtaining multi-scale feature information, the features at each scale are divided into support features and query features. Taking the high-resolution features output by the High-level branch as an example, the multi-similarity feature guidance module uses different similarity layers to calculate the similarity scores between them. The calculation formulas for the multi-similarity layers are as follows.

[0071]

[0072] Among them, D c It is cosine similarity, D e It is the Euclidean distance, D m is the Manhattan distance, i represents the i-th image, and n is the total number of images.

[0073] Then, the various similarity measures are weighted and fused to obtain the global similarity score, which serves as the similarity evaluation index at this scale.

[0074] Based on this, the internal weights of the support features are dynamically adjusted. Finally, the features at three different scales are concatenated to obtain dynamically optimized multi-scale support features.

[0075] (3) Train the model using the base class data in the sample dataset;

[0076] During training, SGD was used as the optimizer, with the momentum parameter set to 0.9, the weight decay factor set to 1e-4, the batch size set to 10, and the learning rate set to 0.006 during the training phase to ensure the stability and efficiency of the optimization process.

[0077] The model is trained using the base class data in the sample dataset, and the loss function is...

[0078] L = L rpn +L LFR +L meta +L cls +L reg

[0079] Among them, L rpn For the loss of RPN, L LFR L is the reconstruction loss of the latent feature reconstruction module. meta For the meta-classification loss of the model, L cls For classification loss, L reg For regression loss;

[0080] The base class data is input into the model, and the model parameters are continuously optimized through the backpropagation algorithm to finally obtain the base class model.

[0081] (4) Freeze the local parameters of the model and adjust the model with new class data in the sample dataset;

[0082] After training the model with base class data from the sample dataset, the local parameters of the model are frozen to be the parameters of the backbone network, RPN module, and region interest alignment module.

[0083] In this invention, most parameters of the base class model are frozen, including those of the backbone network, RPN module, and region interest alignment module, to preserve the general feature representation capability of the base class model. Subsequently, the model is fine-tuned using new class data. During the fine-tuning stage, only some trainable parameters are adjusted, and the learning rate is set to 0.001.

[0084] It is important to note that each category in the new class data contains only a small number of annotated instances, so the fine-tuning process aims to efficiently adapt to the new class features while maintaining good detection performance on the base class.

[0085] (5) Input the data to be detected into the adjusted model to obtain the small sample target detection results.

[0086] By comprehensively evaluating the trained model on the test set and analyzing its detection accuracy across different data categories, we can thoroughly verify the model's generalization ability and practical application performance. Ultimately, we obtain an optimized model capable of adapting to new datasets, providing reliable support for small-sample object detection tasks.

[0087] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0089] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0090] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0091] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0092] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A small-sample target detection method based on dynamic latent features, characterized in that: The method obtains a sample dataset and constructs a small sample target detection model guided by dynamic latent features. The small sample target detection model includes support branches and query branches; The query branch includes a backbone network, an RPN module, and a regional interest alignment module arranged in sequence. The output of the regional interest alignment is aggregated with the output of the support branch and then output. The support branch includes a backbone network and a fusion layer. The outputs of the backbone networks of the query branch and the support branch are aligned and then input into the fusion layer. After the fusion layer, a latent feature reconstruction module and a dynamic multi-scale similarity guidance module are sequentially arranged. The latent feature reconstruction module includes an encoder and two convolutional blocks arranged in sequence. The outputs of the two convolutional blocks are reparameterized and then input into the decoder. The dynamic multi-scale similarity guidance module includes a multi-scale information generator, which generates features at different scales. It calculates the similarity scores between features at different scales based on a hybrid similarity metric and dynamically adjusts the internal weights of the support features. The features at different scales are input into the multi-similarity feature guidance module, processed by the multi-similarity layer, concatenated, and output. After training the model with base class data from the sample dataset, freeze the local parameters of the model, and then adjust the model with new class data from the sample dataset. Input the target data into the adjusted model to obtain small sample target detection results.

2. The method for small-sample target detection based on dynamic latent features as described in claim 1, characterized in that: The query branch and support branch share the same trunk network parameters.

3. The small sample target detection method based on dynamic latent features as described in claim 1, characterized in that: The output of the query branch's backbone network is aligned with the output dimension of the backbone network supporting the branch through a global average pooling layer and an expansion module.

4. The small sample target detection method based on dynamic latent features as described in claim 1, characterized in that: The first branch reduces information loss and learns a high-resolution representation of features through multi-branch convolution operations; The second branch focuses on low-resolution representations of features through global max pooling and deconvolution operations; the third branch combines global average pooling and channel attention mechanisms to aggregate the feature representations of the first and second branches.

5. A small-sample target detection method based on dynamic latent features as described in claim 4, characterized in that: By concatenating features from three different scales, we obtain dynamically optimized multi-scale support features.

6. The small-sample target detection method based on dynamic latent features as described in claim 1, characterized in that: The model is trained using the base class data in the sample dataset, and the loss function is... ， in, For RPN loss, The reconstruction loss of the latent feature reconstruction module, The meta-classification loss of the model, For classifying losses, For regression loss.

7. The small sample target detection method based on dynamic latent features as described in claim 1, characterized in that: After training the model with base class data from the sample dataset, the local parameters of the model are frozen to be the parameters of the backbone network, RPN module, and region interest alignment module.

Images