Small-sample target detection method and system based on subspace foreground perception

Abstract

The embodiment of the invention provides a few-sample target detection method and system based on subspace foreground perception. The method comprises the following steps: acquiring a support image and a query image; and inputting the support image and the query image into a trained sub-space foreground perception-based few-sample target detection model to obtain a target detection result output by the sub-space foreground perception-based few-sample target detection model, the small-sample target detection model based on subspace foreground perception is obtained at least based on bidirectional contrast loss function training, and the target detection result at least comprises a detection frame, target classification, confidence and regression parameters, and the sub-space foreground perception-based few-sample target detection model is used for performing target detection on the query image at least based on the foreground perception module and the plurality of sub-space-based feature aggregation modules so as to obtain a target detection result.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a few-sample target detection method and system based on subspace foreground perception. Background Art

[0002] As one of the most challenging tasks in computer vision, object detection involves classifying and localizing multiple objects in an image or video. Deep learning-based object detection technology requires a large number of training samples. Under low-sample conditions, the performance of existing successful detection models is significantly reduced or even fails. In real-world scenarios, due to the slow iteration and update speed, collecting and annotating datasets with diverse data types and large volumes is extremely difficult. This requires not only time and manpower, but also technical support. Therefore, focusing on addressing low-sample conditions and improving the model's detection accuracy for new object classes is a worthy research direction. Currently, the mainstream approaches to addressing the low-sample problem include meta-learning and transfer learning. Meta-learning-based low-sample object detection uses a large number of meta-tasks to train a model, equipping it with the ability to handle different tasks. This allows the model to detect new object classes by fine-tuning the model using only a small number of new class samples. However, this approach suffers from high computational complexity. Transfer learning-based low-sample object detection uses a model trained on a source domain containing a large amount of labeled data and then transfers it to the target domain. However, this approach suffers from poor performance when the data distributions differ significantly between the source and target domains. Taking into account both computational complexity and model performance, the present invention selects meta-learning as a method for low-sample object detection. Most meta-learning-based few-shot object detection models achieve the interaction of support sample and query sample category information by modeling the relationship between known samples (support samples) and unknown samples (query samples). However, when these few-shot object detection models are faced with images with complex backgrounds, they find it difficult to obtain high-quality features and achieve refined aggregation and target matching, making it difficult to achieve satisfactory accuracy.

[0003] Existing meta-learning-based few-shot object detection mostly improves from two aspects: feature enhancement and feature aggregation, fully learning the information of a small number of supporting samples to guide model detection.

[0004] In terms of feature enhancement, Meta R-CNN and DCNet use channel masking to highlight the foreground information of features. This involves adding an additional channel to the RGB image and assigning a value of 1 to spatial pixels within the ground-truth annotated box and 0 to pixels outside of it. DAnA uses attention to enhance the foreground information of features, enabling the model to adaptively process foreground features that contribute more to the task, achieving more accurate representation of foreground information.

[0005] In terms of feature aggregation, standard few-shot object detection is divided into overall feature aggregation and RoI feature aggregation. Overall feature aggregation aims to aggregate support features and query features, and the aggregated features are used to generate candidate box features. RoI feature aggregation aims to aggregate support features and query RoI features, and the aggregated features are used to achieve final object detection.

[0006] However, there are many problems in the prior art, including but not limited to the following:

[0007] 1) In terms of feature enhancement, channel-masked foreground enhancement treats all foreground features as equally important, which is detrimental to the representation of foreground semantic features. Attention-based foreground enhancement uses the same processing mode for all channels, without refined processing for specific semantics. This results in the model being unable to distinguish foreground information with different semantics, thus affecting model performance.

[0008] 2) In terms of feature aggregation, the vector-based aggregation method is simple and intuitive, but it will lose some important spatial feature information, so it cannot fully learn the complex relationship between features and is limited to linear combinations. This method performs poorly in locating small targets in complex backgrounds. Although convolution-based feature aggregation can effectively extract features locally and retain a certain degree of spatial information, in practical applications, the loss of supporting feature space information may lead to incomplete correspondence between supporting features and query features. This incomplete correspondence will affect the feature matching of the model, thereby affecting the accuracy and robustness of target detection. The aggregation method based on the attention mechanism uses the same fusion method for all subspace features, resulting in mutual interference between different semantics, which directly affects the expressiveness of the aggregated features and thus affects the detection accuracy of the model.

[0009] Specifically, the key task of meta-learning-based few-shot object detection models is to match support features with query features. However, when the support class is glass bottles, if the query image contains plastic bottles of similar shape, size, and transparency, the model has difficulty finding discriminative differences in the extracted features, and may mistakenly classify the plastic bottles as foreground, resulting in incorrect detection results. Summary of the Invention

[0010] The present invention provides a few-shot target detection method and system based on subspace foreground perception, aiming to solve at least the following technical problems: (1) performance degradation under few-shot conditions; (2) technical barriers and labor costs for data collection and annotation; (3) large amount of meta-learning computation and high resource requirements; (4) poor effect of transfer learning when data distribution varies greatly; (5) limitations of feature enhancement: ① the channel masking method fails to distinguish the importance of foreground features, which is not conducive to the expression of foreground semantic features; ② the attention mechanism fails to perform fine processing for specific semantics, resulting in the model being unable to effectively distinguish foreground information with different semantics. (6) Limitations of feature aggregation: ① Vector-based methods lose important spatial feature information, limiting the ability to learn complex relationships between features; ② Although convolution-based methods can extract local features and retain spatial information, the loss of spatial information of supporting features may lead to inaccurate feature matching; ③ The aggregation method of the attention mechanism fails to effectively distinguish different semantic features, resulting in a decrease in feature expression ability; (7) When the support class and the class in the query image are similar in shape, size, and transparency, the model is difficult to distinguish, resulting in incorrect detection results; (8) Under complex backgrounds, the existing technology performs poorly in locating small targets, limiting the accuracy and robustness of the model in practical applications. The embodiment provided by the present invention achieves efficient distinction between foreground targets and background in the query image, enhances the perception of foreground information, and realizes refined feature aggregation of different semantic entities, improves the accuracy of feature expression, and enhances the ability of the model to match positive support samples. In addition, for images in complex backgrounds, the detection accuracy of the model under such difficult conditions is improved.

[0011] Specifically, the embodiments of the present invention provide the following technical solutions:

[0012] In a first aspect, an embodiment of the present invention provides a method for detecting a small number of targets based on subspace foreground perception, comprising:

[0013] Get support images and query images;

[0014] Inputting the support image and the query image into a trained subspace foreground-aware few-shot target detection model, obtaining a target detection result output by the subspace foreground-aware few-shot target detection model, wherein the subspace foreground-aware few-shot target detection model is trained based on at least a bidirectional contrast loss function, and the target detection result includes at least a detection box, a target classification, a confidence score, and a regression parameter.

[0015] The subspace foreground perception-based few-shot target detection model is used to perform target detection on the query image based on at least a foreground perception module and a plurality of subspace-based feature aggregation modules to obtain the target detection result.

[0016] Furthermore, the method further comprises:

[0017] The subspace foreground perception-based few-shot target detection model includes a feature extraction network module, a foreground perception module, a first subspace-based feature aggregation module, a region proposal network module, an RoI alignment module, and a second subspace-based feature aggregation module.

[0018] Furthermore, the method further comprises:

[0019] The feature extraction network module is used to:

[0020] receiving the support image and the query image as input;

[0021] extracting support features from the support image and query features from the query image; and

[0022] The query features are input into the foreground perception module, and the supporting features are input into the pooling module.

[0023] Furthermore, the method further comprises:

[0024] The foreground perception module is used to:

[0025] receiving the query feature input from the feature extraction network module;

[0026] enhancing foreground information in the query features; and

[0027] The enhanced query features are input into the first subspace-based feature aggregation module.

[0028] Furthermore, the method further comprises:

[0029] The pooling module is used to:

[0030] receiving the supporting features input from the feature extraction network module;

[0031] performing pooling processing on the supporting features; and

[0032] The support features after the pooling process are input into the second subspace-based feature aggregation module.

[0033] Furthermore, the method further comprises:

[0034] The first subspace-based feature aggregation module is used to:

[0035] receiving the supporting features and the enhanced query features input from the foreground perception module;

[0036] Aggregating the supporting features and the enhanced query features to generate a first aggregated feature; and

[0037] The first aggregated features are input into the region proposal network module.

[0038] Furthermore, the method further comprises:

[0039] The region proposal network module is used to:

[0040] receiving the first aggregated features input by the first subspace-based feature aggregation module;

[0041] Based on the first aggregated features, generating candidate frame features; and

[0042] The candidate box features are input into the RoI alignment module.

[0043] Furthermore, the method further comprises:

[0044] The RoI alignment module is used to:

[0045] Receiving the candidate box features input by the region proposal network module;

[0046] Generate query RoI features based on the candidate box features;

[0047] Aligning the query RoI features; and

[0048] The aligned query RoI features are input into the second subspace-based feature aggregation module.

[0049] Furthermore, the method further comprises:

[0050] The second subspace-based feature aggregation module is used to:

[0051] Receiving the query RoI features input by the RoI alignment module and the pooled support features input by the pooling module;

[0052] Aggregating the pooled supporting features and the query RoI features to generate a second aggregated feature; and

[0053] Based on the second aggregated features, the target detection result is generated.

[0054] In a second aspect, an embodiment of the present invention further provides a few-shot target detection system based on subspace foreground perception, comprising:

[0055] An image acquisition module, used to acquire support images and query images;

[0056] A target detection module is used to input the support image and the query image into a trained few-sample target detection model based on subspace foreground perception to obtain a target detection result output by the few-sample target detection model based on subspace foreground perception, wherein the few-sample target detection model based on subspace foreground perception is trained at least based on a bidirectional contrast loss function, and the target detection result includes at least a detection box, a target classification, a confidence level, and a regression parameter, wherein the few-sample target detection model based on subspace foreground perception is used to perform target detection on the query image based on at least a foreground perception module and multiple subspace-based feature aggregation modules to obtain the target detection result.

[0057] Furthermore, in the third aspect, an embodiment of the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that when the processor executes the program, the steps of the above-mentioned method for detecting a small number of targets based on subspace foreground perception are implemented.

[0058] Furthermore, in the fourth aspect, an embodiment of the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the above-mentioned subspace foreground perception-based few-sample target detection method are implemented.

[0059] It can be seen from the above technical solutions that the embodiments of the present invention provide a few-shot target detection method based on subspace foreground perception and a few-shot target detection system based on subspace foreground perception, which are intended to solve (1) performance degradation under few-shot conditions based on at least a foreground perception module FAM, a subspace-based feature aggregation module SFAM, a two-way contrast loss function, etc.; (2) technical barriers and labor costs for data collection and annotation; (3) large amount of meta-learning computation and high resource requirements; (4) poor effect of transfer learning when data distribution differences are large; (5) limitations of feature enhancement: ① the channel mask method fails to distinguish the importance of foreground features, which is not conducive to the expression of foreground semantic features; ② the attention mechanism fails to perform fine processing for specific semantics, resulting in the model being unable to effectively distinguish foreground information of different semantics. (6) Limitations of feature aggregation: ① Vector-based methods lose important spatial feature information, limiting the ability to learn complex relationships between features; ② Although convolution-based methods can extract local features and retain spatial information, the loss of spatial information of supporting features may lead to inaccurate feature matching; ③ The aggregation method of the attention mechanism fails to effectively distinguish different semantic features, resulting in a decrease in feature expression ability; (7) When the support class and the class in the query image are similar in shape, size, and transparency, the model is difficult to distinguish, resulting in incorrect detection results; (8) Under complex backgrounds, the existing technology performs poorly in locating small targets, limiting the accuracy and robustness of the model in practical applications. The embodiment provided by the present invention achieves efficient distinction between foreground targets and background in the query image, enhances the perception of foreground information, and realizes refined feature aggregation of different semantic entities, improves the accuracy of feature expression, and enhances the ability of the model to match positive support samples. In addition, for images in complex backgrounds, the detection accuracy of the model under such difficult conditions is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 A flowchart of a few-shot target detection method based on subspace foreground perception provided by one embodiment of the present invention;

[0062] Figure 2 A schematic diagram of the overall framework of a subspace foreground-aware few-shot object detection model provided by one embodiment of the present invention;

[0063] Figure 3A schematic diagram of the structure of a foreground awareness module FAM provided in one embodiment of the present invention;

[0064] Figure 4 A schematic diagram of the structure of a subspace-based feature aggregation module SFAM provided in one embodiment of the present invention;

[0065] Figure 5 A schematic structural diagram of a few-shot target detection system based on subspace foreground perception provided by an embodiment of the present invention; and

[0066] Figure 6 A schematic diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0068] Various terms and phrases used herein have their general meanings as understood by persons of ordinary skill in the art. Nevertheless, the present invention intends to provide a more detailed description and explanation of these terms and phrases. If any term or phrase used herein is inconsistent with its generally understood meaning, the meaning as set forth in this invention shall prevail. Furthermore, if not defined herein, it shall have the meaning as generally understood by persons of ordinary skill in the art.

[0069] The existing technologies have many problems, including but not limited to the following: (1) performance degradation under few-sample conditions; (2) technical barriers and labor costs for data collection and annotation; (3) large amount of meta-learning computation and high resource requirements; (4) poor performance of transfer learning when data distribution is highly different; (5) limitations of feature enhancement: ① the channel mask method fails to distinguish the importance of foreground features, which is not conducive to the expression of foreground semantic features; ② the attention mechanism fails to perform fine processing for specific semantics, resulting in the model being unable to effectively distinguish foreground information with different semantics. (6) limitations of feature aggregation: ① the vector-based method loses important spatial feature information, limiting the ability to learn complex relationships between features; ② the convolution-based method can extract local features and retain spatial information, but the loss of spatial information of the support features may lead to inaccurate feature matching; ③ the aggregation method of the attention mechanism fails to effectively distinguish different semantic features, resulting in a decrease in feature expression ability; (7) when the support class and the class in the query image are similar in shape, size, transparency, etc., the model is difficult to distinguish, resulting in incorrect detection results; (8) in complex backgrounds, the existing technologies perform poorly in localizing small targets, limiting the accuracy and robustness of the models in practical applications.

[0070] To this end, in a first aspect, an embodiment of the present invention provides a method.

[0071] The following combination Figure 1 The present invention describes a few-shot target detection method based on subspace foreground perception.

[0072] Figure 1 This is a flowchart of a method for detecting few-shot targets based on subspace foreground perception provided by one embodiment of the present invention.

[0073] In this embodiment, it should be noted that the subspace foreground-aware few-shot target detection method may include at least the following steps:

[0074] S100: Acquire supporting images and query images;

[0075] S200: Input the support image and the query image into the trained few-shot target detection model based on subspace foreground perception, and obtain the target detection result output by the few-shot target detection model based on subspace foreground perception. The few-shot target detection model based on subspace foreground perception is trained at least based on a two-way contrast loss function, and the target detection result includes at least a detection box, a target classification, a confidence level, and a regression parameter. The few-shot target detection model based on subspace foreground perception is used to perform target detection on the query image based on at least a foreground perception module and multiple subspace-based feature aggregation modules to obtain a target detection result.

[0076] The following combination Figure 2The trained subspace foreground-aware few-shot object detection model in the method of the present invention is described.

[0077] Figure 2 A schematic diagram of the overall framework of a subspace foreground-aware few-shot object detection model provided by one embodiment of the present invention.

[0078] like Figure 2 As shown in the figure, the subspace foreground-aware few-shot object detection model uses the Faster RegionWith Convolutional Neural Network (Faster R-CNN) as the basic detection framework. First, a parameter-sharing feature extraction network (or feature extraction network module) is used to extract features from the support image and the query image. Then, the foreground awareness module (FAM) is used to enhance the foreground information in the query features. Then, in the first stage, the support and query features are aggregated using the subspace-based feature aggregation module (SFAM). The aggregated features (or first aggregated features) are input into the Region Proposal Network (RPN) module to generate candidate box features. Then, in the second stage, another SFAM is used to further aggregate the query RoI features with the support features to obtain detection features (or second aggregated features) and calculate classification and regression parameters. In addition, during the training stage of the subspace foreground-aware few-shot object detection model, the two-way contrastive loss (TCL) is used to constrain the spatial similarity between support features and query features, thereby enhancing the model's ability to match foreground targets of the positive support class.

[0079] In this embodiment, it should be noted that the few-sample target detection method based on subspace foreground perception may also include: the few-sample target detection model based on subspace foreground perception includes a feature extraction network module, a foreground perception module, a first subspace-based feature aggregation module, a region proposal network module, an RoI alignment module, and a second subspace-based feature aggregation module.

[0080] In this embodiment, it should be noted that the few-sample target detection method based on subspace foreground perception can also include: a feature extraction network module for receiving input support images and query images; extracting support features from the support images, and extracting query features from the query images; and inputting the query features into the foreground perception module, and inputting the support features into the pooling module.

[0081] In this embodiment, it should be noted that the few-sample target detection method based on subspace foreground perception can also include: a pooling module for receiving supporting features input from the feature extraction network module; and performing pooling processing on the supporting features; and inputting the pooled supporting features into a second subspace-based feature aggregation module.

[0082] In this embodiment, it should be noted that the few-sample target detection method based on subspace foreground perception may also include: a region proposal network module is used to receive a first aggregated feature input by a first subspace-based feature aggregation module; based on the first aggregated feature, a candidate box feature is generated; and the candidate box feature is input into the RoI alignment module.

[0083] In this embodiment, it should be noted that the few-sample target detection method based on subspace foreground perception can also include: an RoI alignment module for receiving candidate box features input by the region proposal network module; generating query RoI features based on the candidate box features; aligning the query RoI features; and inputting the aligned query RoI features into a second subspace-based feature aggregation module.

[0084] The following is a description Figure 3 The foreground awareness module FAM in the method of the present invention.

[0085] Figure 3 A schematic diagram of the structure of a foreground awareness module FAM provided in one embodiment of the present invention.

[0086] In this embodiment, it should be noted that the few-sample target detection method based on subspace foreground perception can also include: a foreground perception module for receiving query features input from a feature extraction network module; enhancing the foreground information in the query features; and inputting the enhanced query features into a first subspace-based feature aggregation module.

[0087] Specifically, in a few-shot object detection model based on meta-learning, when there is no significant difference between the foreground and background information in the query image, the RPN in Faster R-CNN will confuse the foreground and background information in the query features, resulting in missed or false detection of foreground region candidate boxes. To this end, this paper proposes a foreground-aware module (FAM), which aims to enhance the foreground information in the query features by utilizing spatial and channel-wise mixed attention and a threshold function, thereby preventing background information from interfering with the RPN's generation of foreground region candidate boxes.

[0088] More specifically, if Figure 3As shown in the figure, the foreground perception module (FAM) can be divided into a channel attention module and a spatial attention module. The channel attention module can be used to adjust the channel information of the query features, highlighting the key features of the foreground target and attenuating background and redundant information. The spatial attention module focuses on feature adjustment in the spatial dimension, analyzing the importance of features at each location to highlight spatial regions related to the foreground target. Furthermore, to focus more precisely on foreground information, this module is applied to different subspace features. Each subspace feature represents specific semantic information. Foreground perception is performed based on this specific semantic information, avoiding mutual interference between different semantic features.

[0089] Further integration Figure 3 In the embodiment of the present invention, the query feature According to the channel dimension, it is divided into N subspace features, where N is a natural number, H represents the height of the feature map (that is, the number of vertical pixels in the feature map), W represents the width of the feature map (that is, the number of horizontal pixels in the feature map), C represents the number of channels of the map (that is, the depth or number of features at each position in the feature map), and each subspace feature is represented as Then, the channel attention module (Channel·Attention·Module, CAM) and the spatial attention module (Spatial·Attention·Module, SAM) are used in series to process Get the corresponding enhanced subspace features

[0090]

[0091] Among them, CAM and SAM can be formulated as:

[0092] CAM(x)=T(GAP(x))+GAP(x) (2)

[0093] SAM(x)=Sigmoid[T(Sum(Norm(x)))+Sum(Norm(x))] (3)

[0094] Among them, GAP represents global average pooling, which is used to integrate spatial information and extract global channel information. Sum represents the summation operation, which sums the channel pixels corresponding to the feature, compresses the channel information and extracts local key spatial information. Norm represents normalization, which standardizes the pixel values ​​of the space to between 0 and 1, reduces the pixel value amplitude deviation between different features, enables the model to process input data more stably, and improves the generalization ability of the model. The Sigmoid activation function introduces nonlinearity to increase the robustness of the module. T represents the threshold function, which is used to assist attention, further enhance foreground information, attenuate background information, and eliminate redundant information. It is defined as:

[0095]

[0096]

[0097] Where τ represents the threshold, which changes dynamically according to the input features and can be adjusted by the hyperparameter β, and m represents the total number of pixels in the feature. In CAM, m is equal to the number of channels of the feature; while in SAM, m is equal to the product of the width and height of the feature. Finally, the N enhanced subspace features are concatenated according to the channel dimension to obtain the enhanced query feature

[0098] The following is a description Figure 4 The subspace-based feature aggregation module SFAM in the method of the present invention.

[0099] Figure 4 A schematic diagram of the structure of a subspace-based feature aggregation module SFAM provided in one embodiment of the present invention.

[0100] Different subspace features represent different semantic entities. If the same attention factor is used for feature aggregation for all subspace features, the unique information and differences of each subspace will be ignored, resulting in interference introduced into the aggregation process, affecting the expression of the aggregated features, and thus affecting the detection performance of the model. Based on this, an embodiment of the present invention proposes a subspace-based feature aggregation module SFAM, which assigns different attention factors to different semantic entities to achieve refined feature aggregation.

[0101] In this embodiment, it should be noted that the few-sample target detection method based on subspace foreground perception may also include: a first subspace-based feature aggregation module for receiving supporting features and enhanced query features input from the foreground perception module; aggregating the supporting features and enhanced query features to generate a first aggregated feature; and inputting the first aggregated feature into the region proposal network module.

[0102] In this embodiment, it should be noted that the few-sample target detection method based on subspace foreground perception can also include: a second subspace-based feature aggregation module for receiving the query RoI features input by the RoI alignment module and the pooled support features input by the pooling module; aggregating the pooled support features and query RoI features to generate second aggregated features; and generating target detection results based on the second aggregated features.

[0103] like Figure 4 As shown, first support features and query features enhanced by the foreground-aware module Divide into N subspace features according to the channel dimension, and then, for each pair of subspace features and Perform aggregation.

[0104] In addition, two independent fully connected layers are used to process and To interact with channel-level information and extract key information and Reshape and multiply them to obtain the spatial similarity under the specific semantics of the support feature and the query feature, and use softmax to normalize them to obtain the corresponding similarity score

[0105]

[0106] Among them and are the average embedding values ​​of all pixels in the corresponding subspace features. Then, a Weighting support features for weights Get the aggregate features of the corresponding subspace n. Concatenate N aggregate features to get Will and Splicing to get the final aggregate feature F a .

[0107] Furthermore, it should be noted that the key task of the meta-learning-based few-shot object detection model is to match support features with query features. However, in the prior art, when the support class is, for example, a glass bottle, and plastic bottles of similar shape, size, and transparency appear in the query image, the model in the prior art finds it difficult to find discriminative differences in the extracted features, and thus mistakenly classifies the plastic bottle class as a foreground class, resulting in erroneous detection results.

[0108] In the few-sample target detection method based on subspace foreground perception provided by an embodiment of the present invention, regarding the training of the few-sample target detection model based on subspace foreground perception, it should be noted that the training of the few-sample target detection model based on subspace foreground perception can be based on a two-way contrast loss function, with the class prototype as the reference feature, increasing the similarity within the support feature and the query feature class, and reducing the inter-class similarity between the positive support samples and the negative support samples, thereby enhancing the model's ability to match positive support samples and distinguish negative support samples.

[0109] Specifically, the two-way contrast loss function is defined as:

[0110]

[0111] in, represents the eigenvector of the i-th positive support class, represents the feature vector of the i-th negative support class, f p and f n Represent the prototype features of positive and negative support samples respectively, which are obtained by averaging the feature vectors of K corresponding categories:

[0112]

[0113] <·> represents the cosine similarity function, and the calculation process is:

[0114]

[0115] Among them, f 1，i and f 2，i where ∈ R, ∈ F1, ∈ F2 are the eigenvalues ​​of vectors f1 and f2 in the i-th dimension (i = 1, 2, ..., N), where N represents the channel dimension of the feature vector. By continuously optimizing the loss function during the training phase, the similarity between positive and negative supporting features is reduced, while the similarity between each supporting feature and its corresponding class prototype is increased. This enhances intra-class aggregation, reduces misclassification of query samples, and improves model reliability.

[0116] This algorithm contains three parts of the loss function in both training and fine-tuning stages:

[0117] L=Lpn+Ldet+λLTCL

[0118] Lrpn represents the RPN prediction loss, which measures the accuracy of the generated candidate box categories and regression parameters. Laet represents the Faster R-CNN prediction loss, which measures the model's object detection performance in a specific area. LTCL is the two-way contrast loss, which measures the spatial distance between supported sample classes and within classes. The loss is set to 0.01. By optimizing these three loss functions, the model's ability to identify foreground objects in complex backgrounds is guaranteed.

[0119] Furthermore, based on a large number of experiments comparing and analyzing various models, the effectiveness of the subspace foreground-aware few-shot target detection model in the subspace foreground-aware few-shot target detection method is verified as shown in Table 1:

[0120]

[0121] Table 1

[0122] It should be noted that in order to evaluate the detection performance of the present invention for general targets (such as animals, vehicles, furniture, etc.), a comparative analysis was conducted on the evaluation results of the subspace foreground perception-based few-shot target detection model in the present invention and 11 models in the existing technology on new types of data on the PASCAL VOC dataset.

[0123] The evaluation metric mAP50 is used with K = 1, 3, 5, and 10. Furthermore, 15 categories from PASCAL VOC are selected as base classes, and the remaining five categories are selected as new classes. The new classes are divided into three sets: Set 1 includes bird, bus, cow, motorcycle, and sofa; Set 2 includes airplane, bottle, cow, horse, and sofa; and Set 3 includes boat, cat, motorcycle, sheep, and sofa. However, these are examples only and should not limit the application of this invention to other forms of data.

[0124] Refer to Table 1, which shows the experimental results of this model and 11 advanced models on three new class sets under the settings of K = 1, 3, 5 and 10. The best results are bolded and the suboptimal results are underlined.

[0125] For set 1, the detection accuracy of this few-shot target detection model based on subspace foreground perception is optimal under all four K settings.

[0126] For set 2, when K = 1 and 5, the detection accuracy of this model is the best; when K = 3, the detection accuracy of this few-shot target detection model based on subspace foreground perception is third, which is 1.6% and 1.1% lower than the optimal model and the second-best model respectively; when K = 10, the few-shot target detection model based on subspace foreground perception is second-best, and the detection accuracy is slightly lower than the optimal model by 0.6%.

[0127] For set 3, the detection accuracy of this few-shot target detection model based on subspace foreground perception is the highest when K = 1, 3, 5, and 10.

[0128] In summary, compared with 11 existing models, in most cases, the detection accuracy of the few-shot target detection model based on subspace foreground perception can achieve the best, especially for Set 1 and Set 3. The performance of the present invention can reach the best under all K settings. Based on this, the effectiveness and advancement of the embodiments of the present invention have been successfully verified.

[0129] As can be seen from the above technical solutions, the embodiment of the present invention provides a method for detecting few-shot targets based on subspace foreground perception. First, a foreground perception module is designed to enhance the foreground information of query features by using attention combined with a threshold function. Then, a feature aggregation module based on subspace is designed to achieve refined feature aggregation for specific semantics, accurately guide the detection network to allocate specific support information for each query region, and avoid mutual interference of different semantic information. Finally, the clustering idea is introduced to use the two-way contrast loss to constrain the differences in feature information of the same category, thereby improving the model's ability to match positive support samples. Based on this, (1) performance degradation under few-shot conditions; (2) technical barriers and labor costs for data collection and annotation; (3) large amount of meta-learning computation and high resource requirements; (4) poor effect of transfer learning when data distribution is greatly different; (5) limitations of feature enhancement: ① the channel mask method fails to distinguish the importance of foreground features, which is not conducive to the expression of foreground semantic features; ② the attention mechanism fails to perform refined processing for specific semantics, resulting in the model being unable to effectively distinguish foreground information with different semantics. (6) Limitations of feature aggregation: ① Vector-based methods lose important spatial feature information, limiting the ability to learn complex relationships between features; ② Although convolution-based methods can extract local features and retain spatial information, the loss of spatial information of supporting features may lead to inaccurate feature matching; ③ The aggregation method of the attention mechanism fails to effectively distinguish different semantic features, resulting in a decrease in feature expression ability; (7) When the support class and the class in the query image are similar in shape, size, and transparency, the model is difficult to distinguish, resulting in incorrect detection results; (8) Under complex backgrounds, the existing technology performs poorly in locating small targets, limiting the accuracy and robustness of the model in practical applications. The embodiment provided by the present invention achieves efficient distinction between foreground targets and background in the query image, enhances the perception of foreground information, and realizes refined feature aggregation of different semantic entities, improves the accuracy of feature expression, and enhances the ability of the model to match positive support samples. In addition, for images in complex backgrounds, the detection accuracy of the model under such difficult conditions is improved.

[0130] Based on the same inventive concept, on the other hand, an embodiment of the present invention proposes a few-shot target detection system based on subspace foreground perception.

[0131] The following combination Figure 5 The present invention provides a description of the few-sample target detection system based on subspace foreground perception. The few-sample target detection system based on subspace foreground perception described below and the few-sample target detection method based on subspace foreground perception described above can refer to each other.

[0132] Figure 5 A schematic diagram of the structure of a few-shot target detection system based on subspace foreground perception provided by one embodiment of the present invention.

[0133] In this embodiment, it should be noted that the few-sample target detection system 100 based on subspace foreground perception includes: an image acquisition module 110, used to acquire a support image and a query image; a target detection module 120, used to input the support image and the query image into a trained few-sample target detection model based on subspace foreground perception, and obtain the target detection result output by the few-sample target detection model based on subspace foreground perception. The few-sample target detection model based on subspace foreground perception is trained at least based on a two-way contrast loss function, and the target detection result includes at least a detection box, a target classification, a confidence level, and a regression parameter. The few-sample target detection model based on subspace foreground perception is used to perform target detection on the query image based on at least a foreground perception module and multiple subspace-based feature aggregation modules to obtain a target detection result.

[0134] It can be seen from the above technical solutions that the embodiment of the present invention provides a few-shot target detection system based on subspace foreground perception, which solves (1) performance degradation under few-shot conditions; (2) technical barriers and labor costs for data collection and annotation; (3) large amount of meta-learning computation and high resource requirements; (4) poor effect of transfer learning when data distribution varies greatly; (5) limitations of feature enhancement: ① the channel masking method fails to distinguish the importance of foreground features, which is not conducive to the expression of foreground semantic features; ② the attention mechanism fails to perform fine processing for specific semantics, resulting in the model being unable to effectively distinguish foreground information with different semantics. (6) Limitations of feature aggregation: ① Vector-based methods lose important spatial feature information, limiting the ability to learn complex relationships between features; ② Although convolution-based methods can extract local features and retain spatial information, the loss of spatial information of supporting features may lead to inaccurate feature matching; ③ The aggregation method of the attention mechanism fails to effectively distinguish different semantic features, resulting in a decrease in feature expression ability; (7) When the support class and the class in the query image are similar in shape, size, and transparency, the model is difficult to distinguish, resulting in incorrect detection results; (8) Under complex backgrounds, the existing technology performs poorly in locating small targets, limiting the accuracy and robustness of the model in practical applications. The embodiment provided by the present invention achieves efficient distinction between foreground targets and background in the query image, enhances the perception of foreground information, and realizes refined feature aggregation of different semantic entities, improves the accuracy of feature expression, and enhances the ability of the model to match positive support samples. In addition, for images in complex backgrounds, the detection accuracy of the model under such difficult conditions is improved.

[0135] Since the few-sample target detection system based on subspace foreground perception provided by the embodiment of the present invention can be used to execute the few-sample target detection method based on subspace foreground perception described in the above embodiment, the specific content of its working principle can be found in the introduction of the above method embodiment.

[0136] In this embodiment, it should be noted that the various modules in the subspace foreground-aware few-shot object detection system of the embodiment of the present invention can be integrated into one or deployed separately. The above modules can be combined into one module or further split into multiple sub-units.

[0137] On the other hand, based on the same inventive concept, another embodiment of the present invention provides an electronic device.

[0138] Figure 6 A schematic diagram of an electronic device provided according to an embodiment of the present invention.

[0139] In this embodiment, it should be noted that the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call the logic instructions in the memory 630 to execute the subspace foreground perception-based few-sample target detection method, which includes: obtaining a supporting image and a query image; inputting the supporting image and the query image into a trained subspace foreground perception-based few-sample target detection model to obtain a target detection result output by the subspace foreground perception-based few-sample target detection model, the subspace foreground perception-based few-sample target detection model is trained at least based on a bidirectional contrast loss function, and the target detection result includes at least a detection box, a target classification, a confidence level, and a regression parameter, wherein the subspace foreground perception-based few-sample target detection model is used to perform target detection on the query image based on at least a foreground perception module and multiple subspace-based feature aggregation modules to obtain a target detection result.

[0140] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0141] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform a few-sample target detection method based on subspace foreground perception, the few-sample target detection method based on subspace foreground perception comprising: obtaining a supporting image and a query image; inputting the supporting image and the query image into a trained few-sample target detection model based on subspace foreground perception, and obtaining a target detection result output by the few-sample target detection model based on subspace foreground perception, the few-sample target detection model based on subspace foreground perception is trained at least based on a bidirectional contrast loss function, and the target detection result includes at least a detection box, a target classification, a confidence level, and a regression parameter, wherein the few-sample target detection model based on subspace foreground perception is used to perform target detection on the query image based on at least a foreground perception module and a plurality of subspace-based feature aggregation modules to obtain a target detection result.

[0142] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0143] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0144] In addition, in the present invention, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0145] In addition, in the present invention, the descriptions with reference to the terms "embodiment", "present embodiment", "further embodiment", etc. mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in an appropriate manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A few-shot target detection method based on subspace foreground perception, characterized in that: include: Obtain support images and query images; Inputting the support image and the query image into a trained few-shot target detection model based on subspace foreground perception, obtaining a target detection result output by the few-shot target detection model based on subspace foreground perception, wherein the few-shot target detection model based on subspace foreground perception is trained based on at least a bidirectional contrast loss function, and the target detection result at least includes a detection box, a target classification, a confidence level, and a regression parameter, Among them, the subspace foreground perception-based few-sample target detection model is used to perform target detection on the query image based on at least a foreground perception module and multiple subspace-based feature aggregation modules to obtain the target detection result.

2. The method for detecting a small number of targets based on subspace foreground perception according to claim 1, characterized in that: The subspace foreground perception-based few-sample target detection model includes a feature extraction network module, a foreground perception module, a first subspace-based feature aggregation module, a region proposal network module, a RoI alignment module, and a second subspace-based feature aggregation module.

3. The method for detecting a small number of targets based on subspace foreground perception according to claim 2, characterized in that: The feature extraction network module is used for: receiving the support image and the query image as input; extracting support features from the support image and extracting query features from the query image; and The query features are input into the foreground perception module, and the supporting features are input into the pooling module.

4. The method for detecting a small number of targets based on subspace foreground perception according to claim 3, characterized in that: The foreground perception module is used for: Receiving the query feature input from the feature extraction network module; enhancing foreground information in the query features; and The enhanced query features are input into the first subspace-based feature aggregation module.

5. The method for detecting a small number of targets based on subspace foreground perception according to claim 4, characterized in that: The pooling module is used to: receiving the supporting features input from the feature extraction network module; Performing pooling on the supporting features; and The support features after the pooling process are input into the second subspace-based feature aggregation module.

6. The method for detecting a small number of targets based on subspace foreground perception according to claim 5, characterized in that: The first subspace-based feature aggregation module is used for: receiving the supporting features and the enhanced query features input from the foreground awareness module; Aggregating the supporting features and the enhanced query features to generate a first aggregated feature; and The first aggregated features are input into the region proposal network module.

7. The method for detecting a small number of targets based on subspace foreground perception according to claim 6, characterized in that: The region proposal network module is used to: Receiving the first aggregated feature input by the first subspace-based feature aggregation module; Based on the first aggregated features, generating candidate box features; and The candidate box features are input into the RoI alignment module.

8. The method for detecting a small number of targets based on subspace foreground perception according to claim 7, characterized in that: The RoI alignment module is used to: Receiving the candidate box features input by the region proposal network module; Based on the candidate box features, generate query RoI features; Aligning the query RoI features; and The aligned query RoI features are input into the second subspace-based feature aggregation module.

9. The method for detecting a small number of targets based on subspace foreground perception according to claim 8, characterized in that: The second subspace-based feature aggregation module is used for: Receiving the query RoI feature input by the RoI alignment module and the support feature after pooling processing input by the pooling module; Aggregating the supporting features after the pooling process and the query RoI features to generate a second aggregated feature; and Based on the second aggregated features, the target detection result is generated.

10. A few-shot target detection system based on subspace foreground perception, characterized in that: include: An image acquisition module, used to acquire supporting images and query images; A target detection module is used to input the support image and the query image into a trained few-sample target detection model based on subspace foreground perception to obtain a target detection result output by the few-sample target detection model based on subspace foreground perception, wherein the few-sample target detection model based on subspace foreground perception is trained at least based on a bidirectional contrast loss function, and the target detection result includes at least a detection box, a target classification, a confidence level, and a regression parameter, wherein the few-sample target detection model based on subspace foreground perception is used to perform target detection on the query image based on at least a foreground perception module and multiple subspace-based feature aggregation modules to obtain the target detection result.

Images