Subspace foreground perception based few-shot object detection method and system

Abstract

Embodiments of the present invention provide a few-shot target detection method and system based on subspace foreground perception. The method includes acquiring a support image and a query image; inputting the support image and the query image into a trained few-shot target detection model based on subspace foreground perception to obtain the target detection result output by the few-shot target detection model based on subspace foreground perception. The few-shot target detection model based on subspace foreground perception is trained at least based on a bidirectional contrastive loss function. The target detection result includes at least a detection box, target classification, confidence score, and regression parameters. The few-shot target detection model based on subspace foreground perception is used to perform target detection on the query image based at least on a foreground perception module and multiple subspace-based feature aggregation modules to obtain the target detection result.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a few-shot target detection method and system based on subspace foreground perception. Background Technology

[0002] As one of the most challenging tasks in computer vision, object detection involves classifying and locating multiple objects in an image or video. Deep learning-based object detection techniques require a large number of training samples; under limited sample conditions, the performance of existing successful detection models deteriorates significantly or even fails. In real-world scenarios, due to the rapid iteration and update speed, collecting and labeling large datasets with diverse categories is extremely difficult, requiring not only time and manpower but also technical support. Therefore, focusing on improving the detection accuracy of models for new object classes under limited sample conditions is a worthwhile research direction. Currently, the mainstream methods for solving the limited sample problem are meta-learning and transfer learning. Meta-learning-based limited sample object detection utilizes a large number of meta-tasks to train the model, enabling it to handle different tasks, thus achieving the goal of detecting new object classes with only a small number of new sample samples after fine-tuning the model. The drawback is the high computational cost. Transfer learning-based limited sample object detection trains the model on a source domain containing a large amount of labeled data and then transfers it to the target domain. The drawback is poor model performance when the distribution of source and target domain data differs significantly. Considering both computational cost and model performance, this invention chooses the meta-learning method to address the limited sample object detection problem. Most meta-learning-based few-shot object detection models model the relationship between known samples (support samples) and unknown samples (query samples) to achieve the interaction of category information between support samples and query samples. However, when these few-shot object detection models are faced with images with complex backgrounds, they have difficulty acquiring high-quality features and achieving fine-grained aggregation and target matching, thus making it difficult to obtain satisfactory accuracy.

[0003] Existing meta-learning-based few-shot object detection methods mostly improve upon these methods by focusing on feature enhancement and feature aggregation, fully learning information from a small number of supporting samples to guide the model's detection.

[0004] In terms of feature enhancement: Meta R-CNN and DCNet use channel masking to highlight the foreground information of features, that is, adding an extra channel to the RGB image, assigning a value of 1 to spatial pixels within the ground truth bounding box, and assigning a value of 0 to pixels outside the bounding box. DAnA uses attention to enhance the foreground information of features, enabling the model to adaptively process foreground features that contribute more to the task, and achieving a more accurate representation of foreground information.

[0005] Regarding feature aggregation: Standard few-shot object detection is divided into global feature aggregation and RoI feature aggregation. Global feature aggregation aims to aggregate supporting features and query features, and the aggregated features are used to generate candidate bounding box features. RoI feature aggregation aims to aggregate supporting features and query RoI features, and the aggregated features are used to achieve the final object detection.

[0006] However, the existing technology has many problems, including but not limited to the following:

[0007] 1) Regarding feature enhancement, in channel mask foreground enhancement, all foreground features are treated as equally important, which is not conducive to the expression of foreground semantic features. Attention-based foreground enhancement uses the same processing mode for all channels without fine-tuning for specific semantics, causing the model to be unable to distinguish foreground information with different semantics well, thus affecting the model's performance.

[0008] 2) Regarding feature aggregation, vector-based aggregation is simple and intuitive, but it loses some important spatial feature information, thus failing to fully learn the complex relationships between features and being limited to linear combinations. This method performs poorly in locating small targets in complex backgrounds. While convolution-based feature aggregation can effectively extract features locally and retain a certain degree of spatial information, in practical applications, the loss of supporting feature spatial information may lead to incomplete correspondences between supporting features and query features. This incomplete correspondence affects the model's feature matching, thereby impacting the accuracy and robustness of target detection. Attention-based aggregation uses the same fusion method for all subspace features, leading to mutual interference between different semantics, directly affecting the expressive power of the aggregated features, and thus impacting the model's detection accuracy.

[0009] Specifically, the key task of a few-shot object detection model based on meta-learning is to match support features with query features. However, when the support class is glass bottle, if plastic bottles with similar shapes, sizes, and transparency appear in the query image, the model struggles to find discriminative differences in the extracted features, thus incorrectly classifying the plastic bottles as foreground features and resulting in erroneous detection results. Summary of the Invention

[0010] This invention provides a few-sample target detection method and system based on subspace foreground perception, aiming to solve at least the following technical problems: (1) performance degradation under few-sample conditions; (2) technical obstacles and labor costs of data collection and annotation; (3) large computational load and high resource requirements of meta-learning; (4) poor performance of transfer learning when data distribution is large; (5) limitations of feature enhancement: ① the channel masking method fails to distinguish the importance of foreground features, which is not conducive to the expression of foreground semantic features; ② the attention mechanism fails to refine the processing for specific semantics, resulting in the model being unable to effectively distinguish foreground information with different semantics. (6) Limitations of feature aggregation: ① Vector-based methods lose important spatial feature information, limiting the ability to learn complex relationships between features; ② Although convolution-based methods can extract local features and retain spatial information, the loss of spatial information of supporting features may lead to inaccurate feature matching; ③ The aggregation method of attention mechanism fails to effectively distinguish different semantic features, resulting in a decrease in feature expression ability; (7) When the supporting class and the class in the query image are similar in shape, size and transparency, the model has difficulty distinguishing them, resulting in incorrect detection results; (8) In complex backgrounds, existing technologies perform poorly in the localization of small targets, limiting the accuracy and robustness of the model in practical applications. The embodiments provided by this invention achieve efficient differentiation between foreground targets and background in query images, enhance the perception ability of foreground information, and realize refined feature aggregation of different semantic entities, improving the accuracy of feature expression and enhancing the model's ability to match positive supporting samples. In addition, for images in complex backgrounds, the detection accuracy of the model under such difficult conditions is improved.

[0011] Specifically, the embodiments of the present invention provide the following technical solutions:

[0012] In a first aspect, embodiments of the present invention provide a few-sample target detection method based on subspace foreground perception, comprising:

[0013] Get supporting images and query images;

[0014] The supporting image and the query image are input into a trained few-shot target detection model based on subspace foreground awareness to obtain the target detection result output by the few-shot target detection model based on subspace foreground awareness. The few-shot target detection model based on subspace foreground awareness is trained at least on a bidirectional contrastive loss function. The target detection result includes at least a detection box, target classification, confidence score, and regression parameters.

[0015] The subspace-based foreground perception few-shot target detection model is used to perform target detection on the query image based on at least a foreground perception module and multiple subspace-based feature aggregation modules to obtain the target detection result.

[0016] Furthermore, the method also includes:

[0017] The few-shot target detection model based on subspace foreground perception includes a feature extraction network module, a foreground perception module, a first subspace-based feature aggregation module, a region proposal network module, a RoI alignment module, and a second subspace-based feature aggregation module.

[0018] Furthermore, the method also includes:

[0019] The feature extraction network module is used for:

[0020] Receive the input of the supporting image and the query image;

[0021] Support features are extracted from the support image, and query features are extracted from the query image; and

[0022] The query features are input to the foreground perception module, and the supporting features are input to the pooling module.

[0023] Furthermore, the method also includes:

[0024] The foreground perception module is used for:

[0025] Receive the query features input from the feature extraction network module;

[0026] Enhance the foreground information in the query features; and

[0027] The enhanced query features are input into the first subspace-based feature aggregation module.

[0028] Furthermore, the method also includes:

[0029] The pooling module is used for:

[0030] Receive the supporting features input from the feature extraction network module;

[0031] The supporting features are pooled; and

[0032] The pooled support features are input into the second subspace-based feature aggregation module.

[0033] Furthermore, the method also includes:

[0034] The first subspace-based feature aggregation module is used for:

[0035] Receive the supporting features and the enhanced query features input from the foreground perception module;

[0036] The supporting features and the enhanced query features are aggregated to generate a first aggregated feature; and

[0037] The first aggregated feature is input into the region proposal network module.

[0038] Furthermore, the method also includes:

[0039] The region suggestion network module is used for:

[0040] Receive the first aggregated feature input from the first subspace-based feature aggregation module;

[0041] Based on the first aggregated features, candidate box features are generated; and

[0042] The candidate box features are input into the RoI alignment module.

[0043] Furthermore, the method also includes:

[0044] The RoI alignment module is used for:

[0045] Receive the candidate box features input from the region suggestion network module;

[0046] Based on the candidate box features, query RoI features are generated;

[0047] Align the query RoI features; and

[0048] The aligned query RoI features are input into the second subspace-based feature aggregation module.

[0049] Furthermore, the method also includes:

[0050] The second subspace-based feature aggregation module is used for:

[0051] Receive the queried RoI features input from the RoI alignment module and the pooled support features input from the pooling module;

[0052] The pooled support features and the queried RoI features are aggregated to generate a second aggregated feature; and

[0053] The target detection result is generated based on the second aggregated feature.

[0054] Secondly, embodiments of the present invention also provide a few-sample target detection system based on subspace foreground perception, comprising:

[0055] The image acquisition module is used to acquire supporting images and query images;

[0056] The object detection module is used to input the supporting image and the query image into a trained few-shot object detection model based on subspace foreground awareness, and obtain the object detection result output by the few-shot object detection model based on subspace foreground awareness. The few-shot object detection model based on subspace foreground awareness is trained at least based on a bidirectional contrastive loss function. The object detection result includes at least a detection box, object classification, confidence score, and regression parameters. The few-shot object detection model based on subspace foreground awareness is used to perform object detection on the query image based at least on a foreground awareness module and multiple subspace-based feature aggregation modules to obtain the object detection result.

[0057] Furthermore, in a third aspect, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the steps of the aforementioned few-sample target detection method based on subspace foreground perception.

[0058] Furthermore, in a fourth aspect, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the aforementioned few-sample target detection method based on subspace foreground perception.

[0059] As can be seen from the above technical solutions, the few-sample target detection method and system based on subspace foreground perception provided by the embodiments of the present invention aim to solve the following problems: (1) performance degradation under few-sample conditions; (2) technical obstacles and labor costs in data collection and labeling; (3) large computational load and high resource requirements in meta-learning; (4) poor performance of transfer learning when the data distribution is large; and (5) limitations of feature enhancement: ① the channel masking method fails to distinguish the importance of foreground features, which is not conducive to the expression of foreground semantic features; ② the attention mechanism fails to refine the processing of specific semantics, resulting in the model being unable to effectively distinguish foreground information of different semantics. (6) Limitations of feature aggregation: ① Vector-based methods lose important spatial feature information, limiting the ability to learn complex relationships between features; ② Although convolution-based methods can extract local features and retain spatial information, the loss of spatial information of supporting features may lead to inaccurate feature matching; ③ The aggregation method of attention mechanism fails to effectively distinguish different semantic features, resulting in a decrease in feature expression ability; (7) When the supporting class and the class in the query image are similar in shape, size and transparency, the model has difficulty distinguishing them, resulting in incorrect detection results; (8) In complex backgrounds, existing technologies perform poorly in the localization of small targets, limiting the accuracy and robustness of the model in practical applications. The embodiments provided by this invention achieve efficient differentiation between foreground targets and background in query images, enhance the perception ability of foreground information, and realize refined feature aggregation of different semantic entities, improving the accuracy of feature expression and enhancing the model's ability to match positive supporting samples. In addition, for images in complex backgrounds, the detection accuracy of the model under such difficult conditions is improved. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0061] Figure 1 A flowchart of a few-sample target detection method based on subspace foreground perception provided in an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of the overall framework of a few-shot target detection model based on subspace foreground perception provided in an embodiment of the present invention;

[0063] Figure 3A schematic diagram of the structure of a foreground perception module (FAM) provided in an embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram of the structure of a subspace-based feature aggregation module (SFAM) provided in an embodiment of the present invention;

[0065] Figure 5 A schematic diagram of a few-shot target detection system based on subspace foreground perception provided in an embodiment of the present invention; and

[0066] Figure 6 This is a schematic diagram of an electronic device provided according to an embodiment of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0068] The various terms and phrases used in this invention have their general meanings known to those skilled in the art; however, this invention still intends to provide a more detailed description and explanation of these terms and phrases. If any term or phrase used herein deviates from its known meaning, the meaning as expressed in this invention shall prevail; and if not defined in this application, it shall have the meaning commonly understood by those skilled in the art.

[0069] The existing technologies have many problems, including but not limited to the following: (1) performance degradation under few sample conditions; (2) technical obstacles and labor costs in data collection and labeling; (3) large computational load and high resource requirements for meta-learning; (4) poor performance of transfer learning when there are large differences in data distribution; (5) limitations of feature enhancement: ① the channel masking method fails to distinguish the importance of foreground features, which is not conducive to the expression of foreground semantic features; ② the attention mechanism fails to refine the processing for specific semantics, resulting in the model being unable to effectively distinguish foreground information of different semantics; (6) limitations of feature aggregation: ① the vector-based method loses important spatial feature information, which limits the ability to learn complex relationships between features; ② although the convolution-based method can extract local features and retain spatial information, the loss of spatial information of supporting features may lead to inaccurate feature matching; ③ the aggregation method of the attention mechanism fails to effectively distinguish different semantic features, resulting in a decrease in feature expression ability; (7) when the supporting class and the class in the query image are similar in shape, size and transparency, the model has difficulty distinguishing them, resulting in incorrect detection results; (8) in complex backgrounds, the existing technologies perform poorly in the localization of small targets, which limits the accuracy and robustness of the model in practical applications.

[0070] In response to this, in a first aspect, one embodiment of the present invention provides a method.

[0071] The following is combined with Figure 1 This invention describes a few-sample target detection method based on subspace foreground perception.

[0072] Figure 1 This is a flowchart of a few-sample target detection method based on subspace foreground perception provided in an embodiment of the present invention.

[0073] In this embodiment, it should be noted that the few-shot target detection method based on subspace foreground perception may include at least the following steps:

[0074] S100: Obtain supporting images and query images;

[0075] S200: Input the supporting image and the query image into a trained subspace foreground-aware few-shot target detection model to obtain the target detection result output by the subspace foreground-aware few-shot target detection model. The subspace foreground-aware few-shot target detection model is trained at least on a bidirectional contrastive loss function. The target detection result includes at least a detection box, target classification, confidence score, and regression parameters. The subspace foreground-aware few-shot target detection model is used to perform target detection on the query image based at least on a foreground perception module and multiple subspace-based feature aggregation modules to obtain the target detection result.

[0076] The following is combined with Figure 2The method of the present invention describes a trained few-shot target detection model based on subspace foreground perception.

[0077] Figure 2 This is a schematic diagram of the overall framework of a few-shot target detection model based on subspace foreground perception provided in an embodiment of the present invention.

[0078] like Figure 2 As shown, the few-shot object detection model based on subspace foreground awareness uses a Faster RegionWith Convolutional Neural Network (Faster R-CNN) as its basic detection framework. First, a feature extraction network (or feature extraction network module) with shared parameters extracts features from both the support and query images. Then, a Foreground Awareness Module (FAM) enhances the foreground information in the query features. In the first stage, a Subspace-based Feature Aggregation Module (SFAM) aggregates the support and query features, and the aggregated features (or first aggregated features) are input into a Region Proposal Network (RPN) module to generate candidate bounding box features. In the second stage, another SFAM further aggregates the query RoI features with the support features to obtain detection features (or second aggregated features), and then calculates classification and regression parameters. Furthermore, during the training phase of the few-shot target detection model based on subspace foreground perception, two-way contrastive loss (TCL) is used to constrain the spatial similarity between support features and query features, thereby enhancing the model's ability to match positive support foreground targets.

[0079] In this embodiment, it should be noted that the few-shot target detection method based on subspace foreground perception may further include: the few-shot target detection model based on subspace foreground perception includes a feature extraction network module, a foreground perception module, a first subspace-based feature aggregation module, a region proposal network module, a RoI alignment module, and a second subspace-based feature aggregation module.

[0080] In this embodiment, it should be noted that the few-shot target detection method based on subspace foreground perception may further include: a feature extraction network module for receiving input support images and query images; extracting support features from the support images and query features from the query images; and inputting the query features to the foreground perception module and the support features to the pooling module.

[0081] In this embodiment, it should be noted that the few-sample target detection method based on subspace foreground perception may further include: a pooling module for receiving supporting features input from the feature extraction network module; performing pooling processing on the supporting features; and inputting the pooled supporting features to a second subspace-based feature aggregation module.

[0082] In this embodiment, it should be noted that the few-shot target detection method based on subspace foreground perception may further include: a region proposal network module for receiving a first aggregated feature input from a first subspace-based feature aggregation module; generating candidate box features based on the first aggregated feature; and inputting the candidate box features to a RoI alignment module.

[0083] In this embodiment, it should be noted that the few-shot target detection method based on subspace foreground perception may further include: a RoI alignment module for receiving candidate box features input from a region proposal network module; generating query RoI features based on the candidate box features; aligning the query RoI features; and inputting the aligned query RoI features into a second subspace-based feature aggregation module.

[0084] The following description Figure 3 The foreground perception module (FAM) in the method of this invention.

[0085] Figure 3 This is a schematic diagram of the structure of the foreground perception module (FAM) provided in an embodiment of the present invention.

[0086] In this embodiment, it should be noted that the few-shot target detection method based on subspace foreground perception may further include: a foreground perception module for receiving query features input from a feature extraction network module; enhancing the foreground information in the query features; and inputting the enhanced query features into a first subspace-based feature aggregation module.

[0087] Specifically, in few-shot object detection models based on meta-learning, when there is no significant difference between foreground and background information in the query image, the RPN in Faster R-CNN will confuse the foreground and background information in the query features, leading to missed or false detections of foreground region candidate boxes. To address this, this invention proposes a Foreground Awareness Module (FAM), which aims to enhance the foreground information in the query features using spatial and channel hybrid attention and a threshold function, thereby avoiding background information interfering with the RPN's generation of foreground region candidate boxes.

[0088] More specifically, such as Figure 3As shown, the Foreground Awareness (FAM) module can be divided into a channel attention module and a spatial attention module. The channel attention module adjusts the channel information of the query features, highlighting key features of the foreground target while attenuating background and redundant information. The spatial attention module focuses on feature adjustment in the spatial dimension, analyzing the importance of features at each location to highlight spatial region information relevant to the foreground target. Furthermore, to focus on foreground information more precisely, this module is applied to different subspace features, each representing specific semantic information. Foreground awareness is then performed based on this specific semantic information, avoiding interference between different semantic features.

[0089] Further integration Figure 3 In embodiments of the present invention, query features can be... Classified by channel dimension N There are 3 subspace features, where N is a natural number, H represents the height of the feature map (i.e., the number of vertical pixels in the feature map), W represents the width of the feature map (i.e., the number of horizontal pixels in the feature map), and C represents the number of channels in the map (i.e., the depth or number of features at each location in the feature map). Each subspace feature is represented as... Then, the channel attention module (CAM) and spatial attention module (SAM) are concatenated for processing. The corresponding enhanced subspace features are obtained. :

[0090] ;

[0091] CAM and SAM can be formulated as follows:

[0092] ;

[0093] ;

[0094] Here, GAP represents global average pooling, used to integrate spatial information and extract global channel information. Sum represents summation, summing the pixel values ​​corresponding to the features in each channel, compressing channel information and extracting key local spatial information. Norm represents normalization, standardizing pixel values ​​in space to between 0 and 1, reducing the amplitude deviation of pixel values ​​between different features, making the model process input data more stably, and improving the model's generalization ability. The Sigmoid activation function introduces non-linearity, increasing the robustness of the module. T represents the threshold function, used to assist attention, further enhancing foreground information, attenuating background information, and eliminating redundant information, defined as:

[0095] ;

[0096] ;

[0097] Where τ represents the threshold, which changes dynamically based on the input features and can be controlled by hyperparameters. β Make adjustments. m This represents the total number of pixels for the feature. In CAM, m It equals the number of channels of the feature; while in SAM, m It equals the product of the width and height of the feature. Finally, N The enhanced subspace features are concatenated along the channel dimension to obtain the enhanced query features. .

[0098] The following description Figure 4 The subspace-based feature aggregation module SFAM in the method of this invention.

[0099] Figure 4 This is a schematic diagram of the structure of the subspace-based feature aggregation module SFAM provided in an embodiment of the present invention.

[0100] Different subspace features represent different semantic entities. If the same attention factor is used for feature aggregation on all subspace features, the unique information and differences of each subspace will be ignored, which will introduce interference into the aggregation process, affect the expression of aggregated features, and thus affect the detection performance of the model. Based on this, the embodiments of the present invention propose a subspace-based feature aggregation module SFAM, which assigns different attention factors to different semantic entities to achieve refined feature aggregation.

[0101] In this embodiment, it should be noted that the few-shot target detection method based on subspace foreground perception may further include: a first subspace-based feature aggregation module for receiving supporting features and enhanced query features input from the foreground perception module; performing aggregation processing on the supporting features and enhanced query features to generate a first aggregated feature; and inputting the first aggregated feature to the region proposal network module.

[0102] In this embodiment, it should be noted that the few-sample target detection method based on subspace foreground perception may further include: a second subspace-based feature aggregation module for receiving query RoI features input from the RoI alignment module and pooled support features input from the pooling module; performing aggregation processing on the pooled support features and query RoI features to generate a second aggregated feature; and generating a target detection result based on the second aggregated feature.

[0103] like Figure 4 As shown, first, support features are... Query features enhanced by the foreground awareness module Classified by channel dimension N Each subspace feature, and then, for each pair of subspace features and Perform aggregation.

[0104] In addition, two independent fully connected layers are used for processing. and To facilitate information exchange at the channel level and extract key information. and They are reshaped and multiplied to obtain the spatial similarity of supporting features and query features under specific semantics, and then utilized... softmax Normalization is performed to obtain the corresponding similarity score. :

[0105] ;

[0106] in, and These are the average embedding values ​​of all pixels in the corresponding subspace features. Then, using... Weighted support features Obtain the corresponding subspace n The aggregation characteristics. N The aggregation features are concatenated to obtain .Will and The final aggregated feature is obtained by splicing. .

[0107] Furthermore, it should be noted that the key task of the few-shot object detection model based on meta-learning is to match support features with query features. However, in the existing technology, when the support class is, for example, a glass bottle, and the query image contains plastic bottles with similar shapes, sizes, and transparency, the existing model has difficulty finding discriminative differences in the extracted features, and thus incorrectly classifies the plastic bottle class as the foreground class, resulting in incorrect detection results.

[0108] In the few-shot target detection method based on subspace foreground awareness provided in this embodiment of the invention, it should be noted that the training of the few-shot target detection model based on subspace foreground awareness can be based on a bidirectional contrastive loss function. Using class prototypes as reference features, the similarity within the class of support features and query features is increased, while the inter-class similarity between positive support samples and negative support samples is reduced, thereby enhancing the model's ability to match positive support samples and distinguish negative support samples.

[0109] Specifically, the bidirectional contrastive loss function is defined as:

[0110] ;

[0111] in, Indicates the first i The feature vectors of the positive support class. Indicates the first i Feature vectors of negative support classes and These represent the class prototype features of positive and negative support samples, respectively. K The average of the feature vectors corresponding to each category is obtained:

[0112] ;

[0113] The cosine similarity function is calculated as follows:

[0114] ;

[0115] in, and They are vectors , In the i eigenvalues ​​of dimension (i =1,2,…, N ), N This represents the channel dimension of the feature vector. By continuously optimizing the loss function during training, the similarity between positive and negative support features is reduced, while the similarity between each support feature and its corresponding class prototype is increased. This enhances intra-class aggregation, reduces misclassification of query samples, and improves the reliability of the model.

[0116] This algorithm includes a three-part loss function in both the training and fine-tuning phases:

[0117] ;

[0118] in, This represents the RPN prediction loss, used to measure the accuracy of the generated candidate box categories and regression parameters. This represents the prediction loss of Faster R-CNN, used to measure the model's performance in object detection over a specific region. For bidirectional contrastive loss, the metric supports both inter-class and intra-class spatial distances, with λ set to 0.01. Optimization of the three-part loss function ensures the model's ability to identify foreground targets in complex backgrounds.

[0119] Furthermore, based on comparative analysis of various models and numerous experiments, the performance of the few-shot target detection model based on subspace foreground perception in the subspace foreground perception-based method is verified, as shown in Table 1:

[0120] ;

[0121] Table 1

[0122] It should be noted that, in order to evaluate the detection performance of this invention on general targets (such as animals, vehicles, furniture, etc.), the evaluation results of the few-shot target detection model based on subspace foreground perception in this invention and 11 existing models on the PASCAL VOC dataset were compared and analyzed.

[0123] The evaluation metric is mAP50, with K=1, 3, 5, and 10. Furthermore, 15 categories from PASCAL VOC are selected as base classes, and the remaining 5 categories are used as new classes. The new categories are divided into three sets: Set 1 includes birds, buses, cows, motorcycles, and sofas; Set 2 includes airplanes, bottles, cows, horses, and sofas; and Set 3 includes boats, cats, motorcycles, sheep, and sofas. However, these are merely examples and should not limit the application of this invention to other forms of data.

[0124] Referring to Table 1, which shows the experimental results of this model and 11 advanced models on three new class sets with K=1, 3, 5 and 10 settings, the best results are indicated in bold and the second best results are indicated in underline.

[0125] For set 1, the few-sample target detection model based on subspace foreground perception achieves the best detection accuracy under all four K settings.

[0126] For set 2, when K=1 and 5, the detection accuracy of this model is the best; when K=3, the detection accuracy of this few-sample target detection model based on subspace foreground perception is the third best, which is 1.6% and 1.1% lower than the best model and the second best model, respectively; when K=10, the detection accuracy of this few-sample target detection model based on subspace foreground perception is the second best, which is 0.6% lower than the best model.

[0127] For set 3, the few-sample target detection model based on subspace foreground perception has the highest detection accuracy when K=1, 3, 5, and 10.

[0128] In summary, compared with 11 existing models, the detection accuracy of this few-sample target detection model based on subspace foreground perception is the best in most cases. In particular, for sets 1 and 3, the performance of this invention is optimal under all K settings. Based on this, the effectiveness and advancement of the embodiments of this invention have been successfully verified.

[0129] As can be seen from the above technical solutions, the few-sample target detection method based on subspace foreground perception provided by the embodiments of the present invention first designs a foreground perception module, which uses attention combined with a threshold function to enhance the foreground information of query features. Then, a feature aggregation module based on subspace is designed to achieve refined feature aggregation for specific semantics, accurately guide the detection network to allocate specific support information for each query region, and avoid mutual interference between different semantic information. Finally, the clustering idea is introduced, and the difference of feature information of the same category is constrained by bidirectional contrast loss, thereby improving the model's ability to match positive support samples. Based on this, the following problems are solved: (1) performance degradation under few-sample conditions; (2) technical obstacles and labor costs of data collection and annotation; (3) large computational load and high resource requirements of meta-learning; (4) poor performance of transfer learning when the data distribution is large; (5) limitations of feature enhancement: ① the channel masking method fails to distinguish the importance of foreground features, which is not conducive to the expression of foreground semantic features; ② the attention mechanism fails to perform refined processing for specific semantics, resulting in the model being unable to effectively distinguish foreground information of different semantics. (6) Limitations of feature aggregation: ① Vector-based methods lose important spatial feature information, limiting the ability to learn complex relationships between features; ② Although convolution-based methods can extract local features and retain spatial information, the loss of spatial information of supporting features may lead to inaccurate feature matching; ③ The aggregation method of attention mechanism fails to effectively distinguish different semantic features, resulting in a decrease in feature expression ability; (7) When the supporting class and the class in the query image are similar in shape, size and transparency, the model has difficulty distinguishing them, resulting in incorrect detection results; (8) In complex backgrounds, existing technologies perform poorly in the localization of small targets, limiting the accuracy and robustness of the model in practical applications. The embodiments provided by this invention achieve efficient differentiation between foreground targets and background in query images, enhance the perception ability of foreground information, and realize refined feature aggregation of different semantic entities, improving the accuracy of feature expression and enhancing the model's ability to match positive supporting samples. In addition, for images in complex backgrounds, the detection accuracy of the model under such difficult conditions is improved.

[0130] Based on the same inventive concept, in another aspect, an embodiment of the present invention proposes a few-sample target detection system based on subspace foreground perception.

[0131] The following is combined with Figure 5 The present invention describes a few-shot target detection system based on subspace foreground perception. The few-shot target detection system based on subspace foreground perception described below can be referred to in correspondence with the few-shot target detection method based on subspace foreground perception described above.

[0132] Figure 5 This is a schematic diagram of a few-sample target detection system based on subspace foreground perception provided in an embodiment of the present invention.

[0133] In this embodiment, it should be noted that the few-shot target detection system 100 based on subspace foreground perception includes: an image acquisition module 110 for acquiring support images and query images; and a target detection module 120 for inputting the support images and query images into a trained few-shot target detection model based on subspace foreground perception to obtain the target detection result output by the few-shot target detection model based on subspace foreground perception. The few-shot target detection model based on subspace foreground perception is trained at least based on a bidirectional contrastive loss function. The target detection result includes at least a detection box, target classification, confidence score, and regression parameters. The few-shot target detection model based on subspace foreground perception is used to perform target detection on the query image based at least on the foreground perception module and multiple subspace-based feature aggregation modules to obtain the target detection result.

[0134] As can be seen from the above technical solutions, the present invention provides a few-sample target detection system based on subspace foreground perception, which solves the following problems: (1) performance degradation under few-sample conditions; (2) technical obstacles and labor costs in data collection and labeling; (3) large computational load and high resource requirements in meta-learning; (4) poor performance of transfer learning when data distribution is large; and (5) limitations of feature enhancement: ① the channel masking method fails to distinguish the importance of foreground features, which is not conducive to the expression of foreground semantic features; ② the attention mechanism fails to refine the processing of specific semantics, which makes the model unable to effectively distinguish foreground information with different semantics. (6) Limitations of feature aggregation: ① Vector-based methods lose important spatial feature information, limiting the ability to learn complex relationships between features; ② Although convolution-based methods can extract local features and retain spatial information, the loss of spatial information of supporting features may lead to inaccurate feature matching; ③ The aggregation method of attention mechanism fails to effectively distinguish different semantic features, resulting in a decrease in feature expression ability; (7) When the supporting class and the class in the query image are similar in shape, size and transparency, the model has difficulty distinguishing them, resulting in incorrect detection results; (8) In complex backgrounds, existing technologies perform poorly in the localization of small targets, limiting the accuracy and robustness of the model in practical applications. The embodiments provided by this invention achieve efficient differentiation between foreground targets and background in query images, enhance the perception ability of foreground information, and realize refined feature aggregation of different semantic entities, improving the accuracy of feature expression and enhancing the model's ability to match positive supporting samples. In addition, for images in complex backgrounds, the detection accuracy of the model under such difficult conditions is improved.

[0135] Since the few-shot target detection system based on subspace foreground perception provided in this embodiment can be used to execute the few-shot target detection method based on subspace foreground perception described in the above embodiments, the specific working principle can be found in the description of the above method embodiments.

[0136] In this embodiment, it should be noted that the various modules in the few-sample target detection system based on subspace foreground perception of this invention can be integrated into one unit or deployed separately. These modules can be combined into one module or further divided into multiple sub-units.

[0137] On another front, based on the same inventive concept, yet another embodiment of the present invention provides an electronic device.

[0138] Figure 6 This is a schematic diagram of an electronic device provided according to an embodiment of the present invention.

[0139] In this embodiment, it should be noted that the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logic instructions in the memory 630 to execute a few-shot object detection method based on subspace foreground perception. This method includes: acquiring a support image and a query image; inputting the support image and the query image into a trained few-shot object detection model based on subspace foreground perception to obtain the object detection result output by the model. The few-shot object detection model based on subspace foreground perception is trained at least on a bidirectional contrastive loss function. The object detection result includes at least a bounding box, object classification, confidence score, and regression parameters. The few-shot object detection model based on subspace foreground perception is used to perform object detection on the query image based at least on a foreground perception module and multiple subspace-based feature aggregation modules to obtain the object detection result.

[0140] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform a few-shot target detection method based on subspace foreground perception. The few-shot target detection method based on subspace foreground perception includes: acquiring a support image and a query image; inputting the support image and the query image into a trained few-shot target detection model based on subspace foreground perception to obtain a target detection result output by the few-shot target detection model based on subspace foreground perception. The few-shot target detection model based on subspace foreground perception is trained at least based on a bidirectional contrastive loss function. The target detection result includes at least a detection box, a target classification, a confidence score, and regression parameters. The few-shot target detection model based on subspace foreground perception is used to perform target detection on the query image based at least on a foreground perception module and multiple subspace-based feature aggregation modules to obtain the target detection result.

[0142] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0144] Furthermore, in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0145] Furthermore, in this invention, the terms "embodiment," "this embodiment," "yet another embodiment," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A subspace foreground-aware based few-shot object detection method, characterized in that, The method comprises: obtaining a support image and a query image; inputting the support image and the query image into a trained subspace foreground-aware few-shot object detection model to obtain an object detection result output by the subspace foreground-aware few-shot object detection model, wherein the object detection result at least comprises a detection box, an object classification, a confidence, and regression parameters, wherein the subspace foreground-aware few-shot object detection model is configured to perform object detection on the query image based on at least a foreground perception module, a first subspace-based feature aggregation module, and a second subspace-based feature aggregation module to obtain the object detection result, wherein the at least foreground perception module, the first subspace-based feature aggregation module, and the second subspace-based feature aggregation module perform object detection on the query image to obtain the object detection result, comprising: by the foreground perception module: dividing query features extracted based on the query image into a plurality of subspace features according to a channel dimension; performing enhancement processing on the plurality of subspace features in the spatial and channel dimensions to obtain a plurality of enhanced subspace features corresponding to the plurality of subspace features; concatenating the plurality of enhanced subspace features according to the channel dimension to generate enhanced query features; and inputting the enhanced query features into the first subspace-based feature aggregation module; by the first subspace-based feature aggregation module: dividing support features extracted based on the support image and the received enhanced query features into a first plurality of pairs of subspace features according to a channel dimension; calculating similarity scores of the first plurality of pairs of subspace features; calculating a first plurality of subspace weighted support features using the similarity scores of the first plurality of pairs of subspace features and corresponding support features in the first plurality of pairs of subspace features; concatenating the first plurality of subspace weighted support features to obtain a first weighted support feature; concatenating the first weighted support feature and the enhanced query features to generate a first aggregated feature; and inputting the first aggregated feature into the second subspace-based feature aggregation module; by the second subspace-based feature aggregation module: dividing the support features extracted based on the support image and query RoI features generated based on the first aggregated feature into a second plurality of pairs of subspace features according to a channel dimension; calculating similarity scores of the second plurality of pairs of subspace features; calculating a second plurality of subspace weighted support features using the similarity scores of the second plurality of pairs of subspace features and corresponding support features in the second plurality of pairs of subspace features; concatenating the second plurality of subspace weighted support features to obtain a second weighted support feature; concatenating the second weighted support feature and the query RoI features to generate a second aggregated feature, and calculating classification and regression parameters, generating and outputting the object detection result, The few-shot object detection model based on subspace foreground perception is trained at least based on a bidirectional contrast loss function, the bidirectional contrast loss function is used to increase the similarity between the support features and the query features in the same sample, and reduce the similarity between the positive support sample and the negative support sample, The bidirectional contrast loss function is: ； representing the feature vector of the i positive support class, representing the feature vector of the i negative support class, and representing the class prototype feature of the positive and negative support samples, respectively, are obtained by averaging the feature vectors of the K corresponding class. ； denotes the cosine similarity function, calculated as: ； and are vectors , The eigenvalues i in the first (i =1,2,…, N dimension are N denotes the channel dimension of the eigenvectors.

2. The method of claim 1, wherein, The few-shot object detection model based on subspace foreground perception comprises a feature extraction network module, a foreground perception module, a first subspace-based feature aggregation module, a region proposal network module, an RoI alignment module, and a second subspace-based feature aggregation module.

3. The method of claim 2, wherein the sub-space foreground-aware based few-shot object detection method is characterized by, The feature extraction network module is used to: receive the input support image and query image; extract support features from the support image and query features from the query image; and input the query features into the foreground perception module and the support features into a pooling module. The foreground perception module is used to:

4. The method of claim 3, wherein, receive the query features input from the feature extraction network module; enhance the foreground information in the query features; and input the enhanced query features into the first subspace-based feature aggregation module. The pooling module is used to:

5. The method of claim 4, wherein, receive the support features input from the feature extraction network module; perform pooling processing on the support features; and input the support features after the pooling processing into the second subspace-based feature aggregation module. The first subspace-based feature aggregation module is used to:

6. The method of claim 5, wherein the sub-space foreground-aware based few-shot object detection method is characterized by, receive the support features and the enhanced query features input from the foreground perception module; perform aggregation processing on the support features and the enhanced query features to generate first aggregation features; and input the first aggregation features into the region proposal network module. The region proposal network module is used to:

7. The method of claim 6, wherein, receive the first aggregation features input from the first subspace-based feature aggregation module; generate candidate box features based on the first aggregation features; and input the candidate box features into the RoI alignment module. The RoI alignment module is used to:

8. The method of claim 7, wherein, receive the candidate box features input from the region proposal network module; generate query RoI features based on the candidate box features; align the query RoI features; and input the aligned query RoI features into the second subspace-based feature aggregation module. The second subspace-based feature aggregation module is used to: receive the query RoI features input from the RoI alignment module and the support features after the pooling processing input from the pooling module; 9. The method of claim 8, wherein, perform aggregation processing on the support features after the pooling processing and the query RoI features to generate second aggregation features; and generate the object detection result based on the second aggregation features. comprises: an image acquisition module configured to acquire a support image and a query image; ​ 10. A subspace foreground-aware based few-shot object detection system, comprising: ​ ​ The target detection module is configured to input the support image and the query image into a trained subspace foreground-aware few-shot target detection model to obtain a target detection result output by the subspace foreground-aware few-shot target detection model, the subspace foreground-aware few-shot target detection model being trained based on at least a bidirectional contrast loss function, and the target detection result including at least a detection frame, target classification, confidence, and regression parameter. The target detection module is configured to input the support image and the query image into a trained subspace foreground-aware few-shot target detection model to obtain a target detection result output by the subspace foreground-aware few-shot target detection model, the subspace foreground-aware few-shot target detection model being trained based on at least a bidirectional contrast loss function, and the target detection result including at least a detection frame, target classification, confidence, and regression parameter. The target detection module is configured to input the support image and the query image into a trained subspace foreground-aware few-shot target detection model to obtain a target detection result output by the subspace foreground-aware few-shot target detection model, the subspace foreground-aware few-shot target detection model being trained based on at least a bidirectional contrast loss function, and the target detection result including at least a detection frame, target classification, confidence, and regression parameter. The target detection module is configured to input the support image and the query image into a trained subspace foreground-aware few-shot target detection model to obtain a target detection result output by the subspace foreground-aware few-shot target detection model, the subspace foreground-aware few-shot target detection model being trained based on at least a bidirectional contrast loss function, and the target detection result including at least a detection frame, target classification, confidence, and regression parameter. The target detection module is configured to input the support image and the query image into a trained subspace foreground-aware few-shot target detection model to obtain a target detection result output by the subspace foreground-aware few-shot target detection model, the subspace foreground-aware few-shot target detection model being trained based on at least a bidirectional contrast loss function, and the target detection result including at least a detection frame, target classification, confidence, and regression parameter. The target detection module is configured to input the support image and the query image into a trained subspace foreground-aware few-shot target detection model to obtain a target detection result output by the subspace foreground-aware few-shot target detection model, the subspace foreground-aware few-shot target detection model being trained based on at least a bidirectional contrast loss function, and the target detection result including at least a detection frame, target classification, confidence, and regression parameter. The target detection module is configured to input the support image and the query image into a trained subspace foreground-aware few-shot target detection model to obtain a target detection result output by the subspace foreground-aware few-shot target detection model, the subspace foreground-aware few-shot target detection model being trained based on at least a bidirectional contrast loss function, and the target detection result including at least a detection frame, target classification, confidence, and regression parameter. The target detection module is configured to input the support image and the query image into a trained subspace foreground-aware few-shot target detection model to obtain a target detection result output by the subspace foreground-aware few-shot target detection model, the subspace foreground-aware few-shot target detection model being trained based on at least a bidirectional contrast loss function, and the target detection result including at least a detection frame, target classification, confidence, and regression parameter. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The sub-space foreground perception-based few-shot target detection model is trained based on at least a bidirectional contrast loss function, the bidirectional contrast loss function is used to increase the similarity between the support features and the query features in the same sample, and reduce the similarity between the positive support sample and the negative support sample, The bidirectional contrast loss function is: ； Indicates the first i The feature vectors of the positive support class. Indicates the first i Feature vectors of negative support classes and These represent the class prototype features of positive and negative support samples, respectively. K The average of the feature vectors corresponding to each category is obtained: ； denotes the cosine similarity function, calculated as: ； and are vectors , The eigenvalues i of the first (i =1,2,…, N ) are N denotes the channel dimension of the eigenvectors.

Images