A small sample target detection method based on attention mechanism and contrast learning

Abstract

This invention discloses a few-shot object detection method based on attention mechanism and contrastive learning, comprising: dividing the dataset into base class and new class datasets, and performing an N-way K-shot task partitioning on the base class dataset; processing selected support images and extracting features from the support images and query images; matching support features with query features using a collaborative attention module; performing pooling operations on regions of interest and support features; classifying and locating the pooled regions of interest, and performing contrastive learning using the pooled regions of interest and support features; combining the contrastive loss function with the loss function of Faster R-CNN to obtain an overall loss function for training; and fine-tuning the model trained on the base class dataset using the same process on the new class dataset to obtain the final few-shot object detection model. This invention can improve the detection accuracy of few-shot object detection.

Description

Technical Field

[0001] This invention relates to the field of computer vision object detection, and is a few-shot object detection method based on attention mechanism and contrastive learning. Background Technology

[0002] Object detection, which aims to classify and locate objects in images or videos, is a fundamental task in computer vision. Over the past decade, with the large-scale application of deep learning in object detection, the accuracy and speed of object detection techniques have been greatly improved. Currently, object detection, as an important component of scene understanding, is widely used in many areas of modern life, including face recognition, pedestrian detection, vehicle detection, and remote sensing detection.

[0003] However, deep learning-based object detection methods typically require a large amount of instance-level annotation information. When applying object detection to real-world scenarios, the following problems may arise: many object categories exist in reality where obtaining large amounts of images is difficult; the amount of data requiring annotation is large, consuming significant human and material resources, and may result in missed or incorrect annotations; generating accurate detection results requires substantial computational resources. Therefore, when there is only a small amount of labeled data for the target category, general object detection methods struggle to achieve satisfactory results. Since few-shot learning can achieve rapid classification and recognition of new target classes with a limited number of labeled samples, reducing reliance on large-scale labeled data, many researchers have integrated few-shot learning with traditional object detection algorithms, proposing the few-shot object detection task.

[0004] Few-shot object detection is generally defined as an N-way K-shot problem, meaning it involves data with N classes, each containing only K labeled samples. The number of classes is unlimited, and the value of K typically ranges from 1, 2, 3, 5, 10, 30, etc. Based on pre-training with abundant publicly available data, few-shot object detection can detect new categories of objects using only a very small number of labeled samples. This allows for the acquisition of detection models with good generalization performance, significantly improving model development efficiency, reducing the burden of annotating large amounts of data in the target domain, and making it more practical. Few-shot object detection not only requires extracting high-level semantic information relevant to classification tasks but also acquiring low-level pixel-level information for target localization, making it a very challenging problem.

[0005] Based on the ideas and model structures of few-shot object detection methods, existing methods can be summarized into two types: meta-learning-based methods and transfer learning-based methods. Most methods are based on the two-stage object detector Faster R-CNN with ResNet as the backbone network. For meta-learning-based few-shot object detection methods, meta-learners are first used to learn meta-knowledge from different base class tasks, and then tasks containing new classes are used to complete the detection of the new classes by adjusting the meta-knowledge. Transfer learning-based few-shot object detection methods transfer the knowledge learned from the base classes to the detection task of the new classes. The paper "Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector" (IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2020: 4013-4022) designs an aggregation method, Attention-RPN, which uses an attention mechanism to introduce information supporting features into the Region Proposal Network (RPN), guiding the RPN to generate class-related candidate regions to filter out most background boxes and boxes of non-matching classes. The paper "FSCE: Few-ShotObject Detection via Contrastive Proposal Encoding" (IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021: 7352-7362) proposes a few-shot object detection method based on contrastive candidate box encoding. It is based on Faster R-CNN and adds a contrastive branch in parallel with the classification and localization branches to the RoI Head. It also proposes CPE Loss to optimize the contrastive targets, promote the consistency between candidate targets of the same category, and expand the difference between candidate targets of different categories.

[0006] In existing meta-learning-based few-shot object detection methods, those fusing support image features and query image features often fail to consider the spatial correlation between the two, thus losing some fine-grained features of new object classes. Secondly, a common shortcoming of most methods incorporating contrastive learning into few-shot object detection is the addition of the contrastive branch to the RoI Head, running in parallel with the classification and bounding box regression branches. This flattens the pooled candidate region features before inputting them into the fully connected layer for feature extraction, causing each candidate region feature to lose spatial information and thus reducing the effectiveness of few-shot object detection. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings and defects of existing methods by providing a few-sample target detection method based on attention mechanism and contrastive learning. In few-sample target detection, attention mechanism and supervised contrastive learning are used to solve the problem of low accuracy in few-sample target detection.

[0008] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution.

[0009] A few-shot object detection method based on attention mechanism and contrastive learning includes the following steps:

[0010] S1. Faster R-CNN is used as the basic model architecture, and ResNet-101 is used as the backbone network. The object detection dataset is divided into a base class dataset and a new class dataset with disjoint categories. Based on the idea of ​​meta-learning, the base class dataset is divided into tasks. Each task contains N categories. Each category samples a support image set containing K images and a query image set containing Q images.

[0011] S2. Select a set of support images and a query image. Each support image contains three RGB channels and a mask channel representing the target location in the support image.

[0012] S3. Perform feature extraction on the query image and the support image after adding mask channels to obtain support features and query features;

[0013] S4. Using the collaborative attention module, based on the supporting features and query features obtained in S2, generate the relevance weights between the two, and then use the relevance weights to weight the supporting features and query features respectively, to obtain the weighted supporting features and weighted query features.

[0014] S5. Input the weighted query features obtained in S4 into the RPN to obtain candidate regions that may be foreground, and perform RoI Align operation on the corresponding regions of interest to obtain pooled candidate region features; at the same time, prune the weighted support features according to the target annotation location information, and perform RoI Align operation on the pruned support features to obtain pooled support region features.

[0015] S6. Input the pooled candidate region features obtained in S5 into the RoI Head module for target classification and localization. At the same time, input them together with the pooled support region features into the contrast branch for contrastive learning. Propose a similarity calculation method and a contrastive loss function, and combine the contrastive loss function with the loss function of Faster R-CNN to form the final loss function for training. After multiple rounds of training on the base class dataset, the base class model is obtained.

[0016] S7. Using the base class model trained on the base class dataset, fine-tune it on the new class dataset, in a way that synchronizes with the training process of S2-S6.

[0017] Specifically, the process of step S2 is as follows:

[0018] S21. Obtain the RGB three channels of the input supporting image;

[0019] S22. Add a mask channel to the support image: Obtain the position information of the target of the corresponding category in the support image according to the sampling category. If the image contains multiple objects belonging to the sampling category, select only one as the target. Assign a value of 1 to each pixel within the bounding box of the target of interest, and assign a value of 0 to the pixels of other image positions to obtain a mask channel indicating the position of the target of interest in the image.

[0020] S23. After concatenating the mask channel with the three RGB channels of the image, a supported image with 4 channels is obtained.

[0021] Specifically, step S4 includes the following process:

[0022] S41. The support features F obtained in S3 s and query feature F q A 1×1 convolutional layer is used to reduce the number of channels, resulting in support features with even fewer channels. Query features with fewer channels

[0023] S42. Support features with fewer computational channels Query features with fewer channels The cosine similarity between them yields the feature correlation matrix C(x). q x s ), where x s and x q They represent in and Spatial location within;

[0024] S43. Using two 3×3 convolutional layers to ensure the consistency of adjacent similarities in the feature correlation matrix, we obtain...

[0025] S44. Based on the eigencorrelation matrix Query attention map A is obtained q And Support Attention Map A s For query attention graph A q The calculation formula is:

[0026]

[0027] Where H and W represent the height and width of the support feature, respectively, and γ represents the temperature coefficient. The formula means that the support feature... All spatial locations and query features spatial position x q The degree of relevance; Supporting attention map A s Calculation method and query attention graph A s The formula is the same:

[0028]

[0029] S45. Using Support Attention Map A s And query attention graph A q For the support feature F respectively s and query feature F s We perform weighted analysis to obtain the weighted support features F′. s And weighted query features F′ q .

[0030] Specifically, step S5 includes the following process:

[0031] S51. Weight the query features F′ q The input to the RPN output may be a candidate region of the foreground, which is then mapped to F′. q The corresponding region of interest is obtained from the above.

[0032] S52. Perform RoI Align operation on all regions of interest to obtain pooled candidate region features. Where M represents the number of candidate regions output by the RPN;

[0033] S53. Map the labeled location information of the target of interest in the support image to the weighted support feature F′. s In the process, corresponding supporting features are cropped out based on the target location;

[0034] S54. Perform RoI Align operation on all supporting target features to obtain pooled supporting region features.

[0035] Specifically, the comparison branch in step S6 includes the following steps:

[0036] S61. Input the pooled candidate region features of the contrast branch. Support region features after pooling It constitutes all the regional characteristics

[0037] S62. Calculate the similarity between feature sample pairs:

[0038] For the regional feature f i The value matrix V is obtained by projecting convolutions of the values, keys, and queries, each of size 1×1. i Key-value matrix K i and query matrix Q i Similarly, the regional features f can be obtained. j Value matrix V j Key-value matrix K j and query matrix Q j ;

[0039] By using cross-attention calculation, the region features f are respectively... j Alignment region feature f i Regional characteristics f i For the homogeneous feature f of the region j ; will the regional features f j Alignment region feature f i When cross-attention is calculated, the method is as follows:

[0040]

[0041] Where d represents the number of channels in the value matrix, key matrix, and query matrix; feature f i Alignment feature f j The method for calculating cross-attention is the same as that for the other method, and its cross-attention calculation method is as follows:

[0042]

[0043] Finally, according to f i with f j|i and fj with f i|j The cosine similarity is used to calculate their expected value, which gives the similarity score between the two pairs, expressed by the formula:

[0044]

[0045] Where sim(·,·) represents cosine similarity;

[0046] S63. To shorten the distance between targets of the same category and widen the distance between targets of different categories, contrastive loss is used for calculation and optimization. Therefore, the contrastive loss function is expressed as:

[0047]

[0048] Where N represents the number of categories; K represents the number of supporting images in each category; and M represents the number of candidate regions output by the RPN. Represents the regional feature f i Besides itself, with f i The number of regional features of the same category; Representative regional characteristics f i The intersection-union ratio (IoU) of the corresponding candidate region and its matching ground truth bounding box is greater than a specified threshold. To ensure consistency of candidate regions used for comparison, Representing regional features f i Without comparing the loss with its own calculation, Representative regional characteristics f j The category needs to be related to the regional characteristics f i The same category, and The meanings of f and f are the same, representing regional characteristics. i The loss is not compared with its own calculation; τ is a temperature coefficient, which is used to adjust the degree of attention given to difficult samples.

[0049] Specifically, the final loss function in step S6 is expressed as:

[0050]

[0051] Where λ is the balancing factor, used to balance the scale of loss. This represents the loss function of Faster R-CNN.

[0052] Specifically, in step S7, when fine-tuning on the new class dataset, all layers in the base class model except for the last layer of RPN and the RoI Head module are frozen, and fine-tuning is performed according to the training process of S2-S6 to obtain the final small sample target detection system model.

[0053] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0054] 1. This invention addresses the problem of target detection with few samples by introducing an attention mechanism and contrastive learning on the basis of meta-learning to improve the accuracy of target detection with few samples;

[0055] 2. By introducing a collaborative attention mechanism, this invention takes into account the spatial correlation between supporting image features and query image features when fusing them, so that supporting features and query features are closely matched, providing more reasonable query features for subsequent candidate region generation, classification and bounding box regression.

[0056] 3. In introducing contrastive learning, this invention does not directly add the contrastive branch to the RoI Head module in parallel with its classification and bounding box regression branches. Instead, it places the contrastive branch after RoI Align. Considering the strong match between support features and query features, and that support features contain richer, more specific category information, making them suitable for contrastive learning, pooled support region features are also used as input to the contrastive branch. Simultaneously, spatial features are utilized—that is, global pooling is avoided—and a cross-attention mechanism is introduced to spatially align two candidate regions before comparing their similarity. This similarity is then used in the proposed contrastive loss function. This allows the object detection system model to learn more robust feature representations with a limited number of samples, thereby achieving better object detection results. Attached Figure Description

[0057] Figure 1 This is a flowchart of a method according to an embodiment of the present invention.

[0058] Figure 2 This is a schematic diagram of the collaborative attention module structure according to an embodiment of the present invention.

[0059] Figure 3 This is a schematic diagram of the system model structure according to an embodiment of the present invention.

[0060] Figure 4 This is a schematic diagram of the structure of a feature sample pair similarity calculation network according to an embodiment of the present invention. Detailed Implementation

[0061] This invention discloses a few-shot object detection method based on attention mechanism and contrastive learning, comprising: dividing the dataset into a base class dataset and a new class dataset, and performing N-way K-shot task partitioning on the base class dataset; adding a mask channel to the selected support images, and extracting features from the query image and the support images after adding the mask channel, respectively obtaining query features and support features; using a collaborative attention module to match the query features and support features to obtain weighted query features and weighted support features; inputting the weighted query features into an RPN to obtain the region of interest (ROI) of the query image, and then performing pooling operations on the ROI and the weighted support features of the query image to obtain pooled ROI and pooled support features; while using the pooled ROI as input to the RoI Head for classification and localization, performing contrastive learning using the pooled ROI and pooled support features, wherein the similarity calculation method is based on a cross-attention mechanism; and applying the contrastive loss function to the Faster... The loss function of R-CNN is combined to obtain an overall loss function for training. Based on the basic model for few-shot object detection trained on the base class dataset, fine-tuning is performed on the new class dataset using the same process to obtain the final few-shot object detection system model. This invention can improve the detection accuracy of few-shot object detection.

[0062] The present invention will now be described in further detail with reference to the accompanying drawings.

[0063] The present invention provides a few-shot object detection method based on attention mechanism and contrastive learning, the process of which is as follows: Figure 1 As shown, it includes the following steps:

[0064] S1: Faster R-CNN is used as the basic model architecture, with ResNet-101 as the backbone network. The MS COCO dataset is divided into a base class dataset and a new class dataset with disjoint categories. The new class dataset consists of the 20 intersecting categories of the MS COCO dataset and the PASCAL VOC dataset, while the remaining categories are classified as base classes. Based on the idea of ​​meta-learning, the base class dataset is divided into tasks. Each task contains N categories, and each category samples a support image set containing K images and a query image set containing Q images. In this embodiment, N = 1, 5, 10; K = 1, 2, 3, 5, 10, 30.

[0065] S2: For task t, select category as The system provides a set of support images and a query image, and performs the following processing on the support images: Based on the RGB three channels of the support images, a mask channel is added according to the location information of the target of interest (ROI) in the support images to indicate the location of the ROI in the support images. The specific implementation process includes the following steps:

[0066] S21: Obtain the RGB three channels of the input supporting image;

[0067] S22: Add a mask channel to support the image. First, based on the sampling category... Obtain the location information of the target of the corresponding category in the supporting image. If the image contains multiple objects belonging to the sampling category, select only one as the target. Then, assign 1 to each pixel within the bounding box of the target of interest and 0 to the pixels at other image locations to obtain the mask channel indicating the location of the target of interest in the image.

[0068] S23: After stitching the mask channel onto the three RGB channels of the image, a support image with a total of 4 channels is finally obtained.

[0069] S3: Input the query image and the support image after adding mask channels into the backbone network ResNet-101 for feature extraction to obtain the support features F. s and query feature F q ;

[0070] S4: Using the collaborative attention module, based on the support features F obtained in S3... s and query feature F q This generates correlation weights between the two, and then uses these correlation weights to apply them to the supporting features F. s and query feature F q We perform weighting to obtain weighted support features and weighted query features;

[0071] In step S4, as Figure 2 As shown, the collaborative attention module includes the following steps:

[0072] S41: The support features F obtained in S3 s and query feature F q A 1×1 convolutional layer is used to reduce the number of channels, resulting in support features with even fewer channels. Query features with fewer channels

[0073] S42: Support features with fewer computational channels Query features with fewer channels The cosine similarity between them yields the feature correlation matrix C(x). q ,x s ), where x s and x q They represent in and Spatial location within;

[0074] S43: Two 3×3 convolutional layers are used to ensure the consistency of adjacent similarities in the feature correlation matrix, resulting in...

[0075] S441: Based on the eigencorrelation matrix Query attention map A is obtained q And Support Attention Map A s Query attention map A q The calculation formula is:

[0076]

[0077] Among them, H q and W q They represent the supporting features F respectively. s The height and width, γ represents the temperature coefficient, and the formula means that support features with fewer channels. All spatial locations and fewer channels in the query features spatial position x q The degree of relevance.

[0078] S442: Support Attention Graph A s Calculation method and query attention graph A q The calculation method is the same, and the formula is:

[0079]

[0080] Among them, H s and W s They represent the supporting features F respectively. s The height and width.

[0081] S45: Utilizing Support Attention Map A s And query attention graph A q For the support feature F respectively s and query feature F s We perform weighted analysis to obtain the weighted support features F′. s And weighted query features F′ q .

[0082] S5: Weight the query features F′ q The input to the RPN output may be a candidate region of the foreground, which is then mapped to F′.q The corresponding regions of interest are obtained, and RoI Align operation is performed on all regions of interest to obtain pooled candidate region features. Where M represents the number of candidate regions output by the RPN; simultaneously, the labeled location information of the target of interest in the support image is mapped to the weighted support feature F′. s First, the weighted support features are cropped based on the labeled location information to obtain the support target features. Then, the RoI Align operation is performed on all the support target features to obtain the pooled support region features.

[0083] S6: Pooled candidate region features The input is fed into the RoI Head module for target classification and localization, and then compared with pooled support region features. The data are input together into the contrastive branch for contrastive learning; a similarity calculation method and a contrastive loss function are proposed, and the contrastive loss function is combined with the loss function of Faster R-CNN to form the final loss function for training; the resulting system model structure is as follows. Figure 3 As shown, it comprises five modules: First, the feature extraction module, which extracts features from the input image; second, the feature matching module, which uses a collaborative attention mechanism to match support features with query features; third, the region pooling module, which uses RoIAlign to pool region features; and finally, the detection and contrast modules. The detection module uses pooled candidate region features for classification and localization, while the contrast module performs comparative learning on the input region features to improve instance-level feature representation.

[0084] S61: Utilizing the features of candidate regions after pooling Support region features after pooling All regional features

[0085] S62: As Figure 4 As shown, the similarity between feature sample pairs is calculated.

[0086] For the regional feature f i The value matrix V is obtained by projecting convolutions of the values, keys, and queries, each of size 1×1. i Key-value matrix K i and query matrix Q i Similarly, the regional features f can be obtained. j Value matrix V j Key-value matrix K j and query matrix Q j ;

[0087] By using cross-attention calculation, the region features f are respectively... j Alignment region feature f i Regional characteristics f i Alignment region feature f j .

[0088] Region feature f j Alignment region feature f i When cross-attention is calculated, the method is as follows:

[0089]

[0090] Where d represents the number of channels in the value matrix, key matrix, and query matrix.

[0091] Region feature f i Alignment region feature f j When cross-attention is calculated, the method is as follows:

[0092]

[0093] Finally, according to f i with f j|i and f j with f i|j By calculating the cosine similarity of the two pairs and their arithmetic mean, we obtain f. i with f j The similarity score can be expressed by the formula:

[0094]

[0095] Where sim(·,·) represents cosine similarity.

[0096] S63: To shorten the distance between targets of the same category and widen the distance between targets of different categories, contrastive loss is used for calculation and optimization. Therefore, the contrastive loss function is expressed as:

[0097]

[0098] Where N represents the number of categories; K represents the number of supporting images in each category; and M represents the number of candidate regions output by the RPN. Represents the regional feature f i Besides itself, with f i The number of regional features of the same category; Representative regional characteristics f i The intersection-union ratio (IoU) of the corresponding candidate region and its matching ground truth bounding box is greater than a specified threshold. To ensure the consistency of candidate regions used for comparison, Representing regional features f i Without comparing the loss with its own calculation, Representative regional characteristics f j The category needs to be related to the regional characteristics f i The same category, and The meanings of f and f are the same, representing regional characteristics. i The loss is not compared with its own calculation; τ is a temperature coefficient, which is used to adjust the degree of attention given to difficult samples.

[0099] S64: Combining the contrastive loss function with the Faster R-CNN loss function, the overall loss function is obtained as follows:

[0100]

[0101] Where λ is the balancing factor, used to balance the scale of loss. This represents the loss function of Faster R-CNN.

[0102] S7: Fine-tune the base class model trained on the base class dataset on the new class dataset. When fine-tuning on the new class dataset, freeze all layers in the base class model except for the last layer of RPN and the RoI Head module, and then fine-tune it according to the training process of S2-S6 to obtain the final few-shot object detection system model.

[0103] The few-sample target detection method based on attention mechanism and contrastive learning described in this invention is implemented by a computer program.

Claims

1. A few-shot target detection method based on attention mechanism and contrastive learning, characterized in that, Includes the following steps: S1. Faster R-CNN is used as the basic model architecture, and ResNet-101 is used as the backbone network. The object detection dataset is divided into a base class dataset and a new class dataset with disjoint categories. Based on the idea of ​​meta-learning, the base class dataset is divided into tasks. Each task contains N categories. Each category samples a support image set containing K images and a query image set containing Q images. S2. Select a set of support images and a query image. Each support image contains three RGB channels and a mask channel representing the target location in the support image. S3. Perform feature extraction on the query image and the support image after adding mask channels to obtain support features and query features; S4. Using the collaborative attention module, based on the supporting features and query features obtained in S2, generate the relevance weights between the two, and then use the relevance weights to weight the supporting features and query features respectively, to obtain the weighted supporting features and weighted query features. S5. Input the weighted query features obtained in S4 into the RPN to obtain candidate regions that may be foreground, and perform RoI Align operation on the corresponding regions of interest to obtain pooled candidate region features; at the same time, prune the weighted support features according to the target annotation location information, and perform RoI Align operation on the pruned support features to obtain pooled support region features. S6. Input the pooled candidate region features obtained in S5 into the RoI Head module for target classification and localization. At the same time, input them together with the pooled support region features into the contrast branch for contrastive learning. Propose a similarity calculation method and a contrastive loss function, and combine the contrastive loss function with the loss function of Faster R-CNN to form the final loss function for training. After multiple rounds of training on the base class dataset, the base class model is obtained. S7. Using the base class model trained on the base class dataset, fine-tune it on the new class dataset, in a way that synchronizes with the training process of S2-S6.

2. The few-sample target detection method based on attention mechanism and contrastive learning according to claim 1, characterized in that, The specific process of step S2 is as follows: S21. Obtain the RGB three channels of the input supporting image; S22. Add a mask channel to the support image: Obtain the position information of the target of the corresponding category in the support image according to the sampling category. If the image contains multiple objects belonging to the sampling category, select only one as the target. Assign a value of 1 to each pixel within the bounding box of the target of interest, and assign a value of 0 to the pixels of other image positions to obtain a mask channel indicating the position of the target of interest in the image. S23. After concatenating the mask channel with the three RGB channels of the image, a supported image with 4 channels is obtained.

3. The few-sample target detection method based on attention mechanism and contrastive learning according to claim 1, characterized in that, Step S4 includes the following process: S41. The support features F obtained in S3 s and query feature F q A 1×1 convolutional layer is used to reduce the number of channels, resulting in support features with even fewer channels. Query features with fewer channels S42. Support features with fewer computational channels Query features with fewer channels The cosine similarity between them yields the feature correlation matrix C(x). q ,x s ), where x s and x q They represent in and Spatial location within; S43. Using two 3×3 convolutional layers to ensure the consistency of adjacent similarities in the feature correlation matrix, we obtain... S44. Based on the eigencorrelation matrix Query attention map A is obtained q And Support Attention Map A s For query attention graph A q The calculation formula is: Where H and W represent the height and width of the support feature, respectively, and γ represents the temperature coefficient. The formula means that the support feature... All spatial locations and query features spatial position x q The degree of relevance; Supporting attention map A s Calculation method and query attention graph A s The formula is the same: S45. Using Support Attention Map A s And query attention graph A q For the support feature F respectively s and query feature F s We perform weighted analysis to obtain the weighted support features F′. s And weighted query features F′ q .

4. The few-sample target detection method based on attention mechanism and contrastive learning according to claim 1, characterized in that, Step S5 Includes the following processes: S51. Weight the query features F′ q The input to the RPN output may be a candidate region of the foreground, which is then mapped to F′. q The corresponding region of interest is obtained from the above. S52. Perform RoI Align operation on all regions of interest to obtain pooled candidate region features. Where M represents the number of candidate regions output by the RPN; S53. Map the labeled location information of the target of interest in the support image to the weighted support feature F′. s In the process, corresponding supporting features are cropped out based on the target location; S54. Perform RoI Align operation on all supporting target features to obtain pooled supporting region features.

5. A few-sample target detection method based on attention mechanism and contrastive learning according to claim 1, characterized in that, The comparison branch in step S6 includes the following steps: S61. Input the pooled candidate region features of the contrast branch. Support region features after pooling It constitutes all the regional characteristics S62. Calculate the similarity between feature sample pairs: For the regional feature f i The value matrix V is obtained by projecting convolutions of the values, keys, and queries, each of size 1×1. i Key-value matrix K i and query matrix Q i Similarly, the regional features f can be obtained. j Value matrix V j Key-value matrix K j and query matrix Q j ; By using cross-attention calculation, the region features f are respectively... j Alignment region feature f i Regional characteristics f i For the homogeneous feature f of the region j ; will the regional features f j Alignment region feature f i When cross-attention is calculated, the method is as follows: Where d represents the number of channels in the value matrix, key matrix, and query matrix; feature f i Alignment feature f j The method for calculating cross-attention is the same as that for the other method, and its cross-attention calculation method is as follows: Finally, according to f i with f j|i and f j with f i|j The cosine similarity is used to calculate their expected value, which gives the similarity score between the two pairs, expressed by the formula: Where sim(·,·) represents cosine similarity; S63. To shorten the distance between targets of the same category and widen the distance between targets of different categories, contrastive loss is used for calculation and optimization. Therefore, the contrastive loss function is expressed as: Where N represents the number of categories; K represents the number of supporting images in each category; and M represents the number of candidate regions output by the RPN. Represents the regional feature f i Besides itself, with f i The number of regional features of the same category; Representative regional characteristics f i The intersection-union ratio (IoU) of the corresponding candidate region and its matching ground truth bounding box is greater than a specified threshold. To ensure consistency of candidate regions used for comparison, Representing regional features f i Without comparing the loss with its own calculation, Representative regional characteristics f j The category needs to be related to the regional features f i The same category, and The meanings of f and f are the same, representing regional characteristics. i The loss is not compared with its own calculation; τ is a temperature coefficient, which is used to adjust the degree of attention given to difficult samples.

6. The few-sample target detection method based on attention mechanism and contrastive learning according to claim 1, characterized in that, The final loss function in step S6 is expressed as follows: Where λ is the balancing factor, used to balance the scale of loss. This represents the loss function of Faster R-CNN.

7. A few-sample target detection method based on attention mechanism and contrastive learning according to claim 1, characterized in that, In step S7, when fine-tuning on the new class dataset, all layers in the base class model except for the last layer of RPN and the RoI Head module are frozen. Based on this, fine-tuning is performed according to the training process of S2-S6 to obtain the final small sample object detection system model.

Images