A multi-task based self-supervised small sample target detection method and system

Abstract

The application provides a kind of based on multi-task self-supervised small sample target detection method and system, it is related to computer vision technical field, including: constructing multiple different image detection meta-tasks, including support image set and query image set, input target detection model;Extract and aggregate the features of support image set and query image set, obtain overall aggregated features;Whole aggregated features and query image feature set are input into transformer, and target detection result is obtained, including fixed number of detection frame and detection feature;Detection frame and the region proposal generated by self-supervision module are compared, detection feature and the embedding feature of self-supervision module are compared, loss is calculated, the weight of transformer is adjusted based on loss calculation result, the target detection model is optimized, and the target detection result is optimized.The application can better capture the difference and change of target category, thereby effectively reducing the offset or difference in feature space, and improving the performance of small sample target detection model on new task or new category.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a self-supervised few-shot target detection method and system based on multi-task. Background Technology

[0002] Few-shot object detection refers to learning and generalizing detection models with limited training data to adapt to predictions of new categories. Few-shot object detection has significant theoretical and practical value because in many fields, such as medicine, military, and finance, obtaining large amounts of labeled data is difficult or expensive. Humans can quickly identify new concepts with just a few examples. Few-shot object detection can bridge the gap between human intelligence and artificial intelligence, potentially paving the way for the development of general artificial intelligence.

[0003] The core challenge of few-shot object detection is extracting effective feature representations from limited data and leveraging prior knowledge or meta-learning techniques to enhance the model's generalization ability. To address this, many researchers have proposed various methods from different perspectives, which can be broadly categorized as follows: The first category is generative model-based methods. These methods utilize generative adversarial networks (GANs) or variational autoencoders (VAEs) to generate additional samples, expanding the few-shot object detection dataset, increasing data diversity, and thus improving the model's generalization ability. The second category is metric learning-based methods. These methods aim to improve the model's generalization ability in few-shot situations by learning similarity or distance metrics between samples. Metric learning methods primarily learn appropriate feature representations and metric functions to capture the relationships between samples, thereby achieving object detection in few-shot situations. The third category is meta-learning-based methods. These methods mainly utilize meta-learning or meta-optimization techniques to learn how to quickly adapt to new tasks or new categories from a small number of samples. Meta-learning improves the model's performance on few-shot object detection tasks by iteratively updating model parameters, optimizing hyperparameters, or adjusting optimization algorithms.

[0004] In current few-sample object detection models, the intractable problem of intra-class bias is prevalent. Intra-class bias refers to the diversity and variability among objects of the same class in few-sample object detection. Because each class has only a small number of samples, the distribution of intra-class features is uneven or incomplete, thus affecting the model's ability to generalize to new samples.

[0005] How to alleviate or resolve intra-class bias issues, thereby ensuring the model's generalization ability to new samples, is an urgent problem to be solved in small sample object detection. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a self-supervised few-shot object detection method and system based on multi-tasks. This invention combines self-supervision with few-shot object detection in a multi-task manner, and introduces a denoising module and a team module to better align the self-supervised branch with the few-shot main branch. Through self-supervised learning, this invention utilizes auxiliary tasks that do not require manual annotation to pre-train the model, thereby learning feature representations with rich semantic information. These feature representations can better capture the differences and variations in target categories, effectively reducing bias or differences in the feature space, improving the model's performance on new tasks or new categories, increasing diversity, and mitigating intra-category bias problems.

[0007] To achieve the above objectives, the present invention provides a self-supervised few-shot object detection method based on multiple tasks, comprising: sampling with replacement based on a few-shot object detection dataset, constructing multiple different image detection meta-tasks, and inputting them into an object detection model, wherein the image detection meta-tasks include a support image set and a query image set;

[0008] Features of the support image set and the query image set are extracted using a convolutional neural network to obtain a support image feature set and a query image feature set.

[0009] The supporting image feature set is aggregated, and then concatenated with the feature sets of all query images to obtain the overall aggregated features;

[0010] The overall aggregated features and the query image feature set are input into the transformer to obtain the target detection result, which includes a fixed number of detection boxes and detection features;

[0011] The detection bounding box and the region proposal generated by the self-supervised module are compared, the detection features are compared with the embedded features of the self-supervised module, the loss is calculated, and the weights of the transformer are adjusted based on the loss calculation results to optimize the target detection model.

[0012] The optimized target detection model is then used to continue performing target detection and target detection model optimization on the remaining target detection meta-tasks.

[0013] As a further improvement of the present invention, the step of sampling with replacement based on a small sample target detection dataset to construct multiple different image detection meta-tasks includes:

[0014] In a small sample object detection dataset, N categories are randomly selected, and K samples are extracted from each category to form the support image set, represented as follows:

[0015]

[0016] in,

[0017] X represents a sample image in the small sample object detection dataset, Y represents the image pixel label information corresponding to the sample image in the small sample object detection dataset, S represents an image in the support image set, and i is a sample in the support image set;

[0018] In the small sample object detection dataset, samples from N disjoint categories of the supporting image set are randomly selected to form the query image set. Each category is represented as:

[0019] Q = {X} q ,Y q}

[0020] in,

[0021] X represents the sample images in the small sample object detection dataset, Y represents the image pixel label information corresponding to the sample images in the small sample object detection dataset, and q represents the image in the query image set.

[0022] As a further improvement of the present invention, the overall aggregated features and the query image feature set are input into the transformer to obtain the target detection result; including:

[0023] The overall aggregated features and the query image feature set are input into the transformer, then processed by the encoder for attention, and then sent to the decoder for prediction to obtain a fixed number of detection boxes and detection features.

[0024] As a further improvement of the present invention, the decoder end of the converter is provided with a denoising module, which generates positive and negative sample pairs of noise;

[0025] The positive and negative sample pairs, along with the ground truth (GT) labels and the initialized query boxes, are fed into the decoder to train the object detection model to reconstruct the GT labels and boxes.

[0026] As a further improvement of the present invention, the decoder end of the converter is also provided with a team module;

[0027] When the query box is sent to the decoder, the team module groups the query boxes according to different query targets and query regions, sets the number and proportion of query boxes in each group, and guides the target detection model to learn to capture target features on different proportions of query targets and query regions.

[0028] As a further improvement of the present invention, the query boxes grouped by the team module predict objects within a specific size range for each group of query boxes.

[0029] As a further improvement of the present invention, the self-supervised module uses a high recall algorithm to generate content labels and bounding boxes for the image detection meta-task, and inputs the image detection meta-task into the self-supervised image encoder to obtain the embedded features.

[0030] As a further improvement of the present invention, the detection box and the region proposal generated by the self-supervised module are compared, the detection features are compared with the embedded features of the self-supervised module, the loss is calculated, and the weights of the transformer are adjusted based on the loss calculation results; including:

[0031] Using pairwise matching loss, the optimal assignment between the prediction and the target is calculated using the following formula:

[0032]

[0033] in,

[0034] σ represents a permutation of N elements. This represents the optimal allocation between prediction and target, where y represents the true label data for the i-th image. The predicted label data represents the i-th image. Represents the matching loss;

[0035] The matching loss includes the classification loss and localization loss of the detected target. Therefore, the formula for calculating the matching loss is:

[0036]

[0037] in,

[0038] 'c' represents the category label, and 'b' represents the bounding box. This represents the classification loss for the category labels. This indicates the positioning loss of the detection box;

[0039] Introducing the loss L for reconstructing the object embedding descriptor emb (z i ,z j The formula is expressed as:

[0040]

[0041] in,

[0042] z represents an embedded object; This represents the embedded object obtained from the i-th image;

[0043] The final self-supervised auxiliary loss is:

[0044]

[0045] Where, λ f The weights representing the class loss, λ b The weights λ represent the loss of the predicted bounding box. e The weights represent the embedding loss.

[0046] λf , λ b , λ e These are respectively used as weights for the three prediction heads in the transformer: the bounding box of the decoder's predicted output, the predicted output category, and the predicted object embedding descriptor.

[0047] This invention also provides a self-supervised few-shot target detection system based on multiple tasks, including: a meta-task construction module, a feature extraction module, a feature aggregation module, a shared transformer module, and a self-supervised auxiliary optimization module;

[0048] The meta-task construction module is used for:

[0049] Based on a small sample object detection dataset, sampling with replacement is performed to construct multiple different image detection meta-tasks, which are then input into the object detection model. The image detection meta-tasks include a support image set and a query image set.

[0050] The feature extraction module is used for:

[0051] Features of the support image set and the query image set are extracted using a convolutional neural network to obtain a support image feature set and a query image feature set.

[0052] The feature aggregation module is used for:

[0053] The supporting image feature set is aggregated, and then concatenated with the feature sets of all query images to obtain the overall aggregated features;

[0054] The shared converter module is used for:

[0055] The overall aggregated features and the query image feature set are input into the transformer to obtain the target detection result, which includes a fixed number of detection boxes and detection features;

[0056] The self-supervised auxiliary optimization module is used for:

[0057] The detection bounding box and the region proposal generated by the self-supervised module are compared, the detection features are compared with the embedded features of the self-supervised module, the loss is calculated, and the weights of the transformer are adjusted based on the loss calculation results to optimize the target detection model.

[0058] The optimized target detection model is then used to continue performing target detection and target detection model optimization on the remaining target detection meta-tasks.

[0059] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0060] This invention can effectively utilize self-supervised learning to expand the sample coverage, providing the model with more observation angles to compensate for the problem that a small number of samples are difficult to cover the new category bias in deep models, thereby improving the accuracy, generalization ability and stability of small sample target detection.

[0061] This invention combines self-supervised learning with few-shot object detection in a multi-task manner. The self-supervision in this invention serves as an auxiliary task to improve the detection performance of the main task of few-shot object detection.

[0062] The self-supervised branch of this invention aims to predict the embeddings of a single self-supervised image encoder evaluated on an object region. Similar to the backbone network, the self-supervised image encoder learns transformation-invariant embeddings by utilizing the invariance of feature transformations; these embeddings are then extracted into the detector's embeddings.

[0063] To make self-supervised detection more suitable for target detection with few samples, this invention introduces a denoising module to expand positive and negative samples and a team module for precise localization. The denoising module expands the positive and negative samples and uses a contrastive denoising training method to accelerate model convergence. The team module utilizes positional constraints for precise localization to improve the accuracy of target detection. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of a multi-task-based self-supervised few-sample target detection method disclosed in one embodiment of the present invention;

[0065] Figure 2 This is a schematic diagram of a self-supervised small sample target detection system based on multi-tasks disclosed in one embodiment of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] The present invention will now be described in further detail with reference to the accompanying drawings:

[0068] In this invention, multi-task learning is a machine learning method aimed at simultaneously training a model to solve multiple related tasks. In object detection, multi-task learning can be applied to handle multiple related object detection tasks simultaneously, thereby improving the performance and generalization ability of the object detection model. Self-supervised learning is also a machine learning method that utilizes auxiliary tasks without manual annotation to pre-train the model, thereby learning feature representations with rich semantic information. These feature representations can better capture the differences and variations in target categories, effectively reducing the bias or difference in the feature space and improving the model's performance on new tasks or new categories. Few-sample object detection refers to object detection based on a very small number of samples, resulting in only a few samples per category and uneven or incomplete distribution of features within each category, which can affect the object detection model's generalization ability to new samples. Based on this, this invention uses self-supervised learning to alleviate the intra-category bias problem.

[0069] like Figure 1 As shown, the present invention provides a self-supervised few-sample target detection method based on multi-task, comprising the following steps:

[0070] S1. To mimic the human learning process, sampling with replacement is performed on a small sample object detection dataset. Each sampling yields different image samples to form an image detection meta-task, ultimately constructing a large number of different image detection meta-tasks. Each image detection meta-task includes a support image set and a query image set. The image detection meta-task is input into the object detection model. The support image set is used to extract the learned knowledge for each task, and the query image set is used to adjust the detection model parameters, transforming knowledge into highly adaptable parameters for the detection model.

[0071] in,

[0072] In a small sample object detection dataset, N classes are randomly selected, and K samples are drawn from each class to form a support image set, represented as:

[0073]

[0074] In the formula,

[0075] X represents a sample image in the small sample object detection dataset, Y represents the image pixel label information corresponding to the sample image in the small sample object detection dataset, S represents an image in the support image set, and i is a sample in the support image set;

[0076] In the small sample object detection dataset, samples from N disjoint categories of the supporting image set are randomly selected to form the query image set. Each category is represented as:

[0077] Q = {X} q ,Y q}

[0078] In the formula,

[0079] X represents the sample images in the small sample object detection dataset, Y represents the image pixel label information corresponding to the sample images in the small sample object detection dataset, and q represents the image in the query image set.

[0080] Furthermore,

[0081] The small sample target detection dataset of this invention can be selected from the most widely used COCO image detection dataset.

[0082] S2. Extract features from the support image set and the query image set using a convolutional neural network (CNN) to obtain the support image feature set and the query image feature set;

[0083] in,

[0084] like Figure 1 As shown, a shared convolutional neural network is used to extract features from the supporting image set and the query image set.

[0085] S3. Perform feature aggregation on the supporting image feature set, and then concatenate it with the feature sets of all query images to obtain the overall aggregated features;

[0086] in,

[0087] like Figure 1 As shown, the supporting image feature set is first input into the CAM feature aggregation module to obtain the supporting image set aggregated features that are independent of the category. Then, the supporting image set aggregated features and the query image feature set are input into the CAM feature aggregation module together to obtain the supporting image set aggregated features and the query features, which are called the overall aggregated features.

[0088] S4. Input the overall aggregated features and the query image feature set into the transformer to obtain the target detection results, including a fixed number of detection boxes and detection features;

[0089] in,

[0090] The overall aggregated features and the query image feature set are input into the transformer, then processed by the encoder for attention, and then sent to the decoder for prediction to obtain a fixed number of detection boxes and detection features.

[0091] Furthermore,

[0092] (1) The decoder of the converter is equipped with a denoising module. The denoising module uses a contrastive denoising method to generate positive and negative sample pairs of noise.

[0093] The positive and negative sample pairs, along with the ground truth (GT) labels and the initialized query boxes, are fed into the decoder to train the object detection model to reconstruct the GT labels and boxes.

[0094] (2) The decoder end of the converter is also equipped with a team module;

[0095] When the query box is fed into the decoder, the team module groups the query boxes according to different query targets and query regions, sets the number and proportion of query boxes in each group, and guides the object detection model to learn to capture target features on different proportions of query targets and query regions.

[0096] The query boxes, grouped by the team module, predict objects within a specific size range for each group. The loss function for the team module is expressed as:

[0097]

[0098] in, Represents the i-th query q i The prediction box, when The center point and When the distance exceeds the threshold η (where η is set to 0.25 in this invention), for To impose a penalty, σ represents the number of frames to be penalized.

[0099] Specifically,

[0100] The contrastive denoising method uses contrastive learning to learn the noise of positive and negative samples, making self-supervised learning more suitable for small sample object detection. The denoising module for positive and negative samples can suppress confusion and select high-quality anchors to predict bounding boxes, helping the object detection model learn to converge faster without object boxes while improving the model's performance.

[0101] S5. Compare the detection box with the region proposal generated by the self-supervised module, compare the detection features with the embedded features of the self-supervised module, calculate the loss, learn the invariance of feature transformation, adjust the weight of the transformer based on the loss calculation results, and optimize the target detection model.

[0102] In this process, a high-recall algorithm is used to generate region proposals, including content labels and bounding boxes, for the image detection meta-task. The image detection meta-task is then input into a self-supervised image encoder to obtain embedding features. The embedding features located by the self-supervised image encoder are aligned with the embedding features from the backbone network (RestNet-101), and the category is set to 1, representing the presence of a target.

[0103] The self-supervised module is mainly used for region localization and embedding feature components. Specifically, it uses an unsupervised region proposal method to output a set of M bounding boxes b1…b m The loss is optimized to minimize the difference between the detector bounding box predictions and these M detection boxes. The loss involves matching the predicted bounding boxes with these M detection boxes. The box generator's selection strategy is Top-K. Three prediction heads: f box The predicted bounding box, fcat Predict whether the output category is background or object, and reconstruct the object embedding descriptor using backbone. emb The outputs are: Where v i Representing the decoder output, the optimal assignment between the prediction and the target is calculated using pairwise matching loss, as shown in the formula:

[0104]

[0105] In the formula,

[0106] σ represents a permutation of N elements. This represents the optimal allocation between prediction and target, where y represents the true label data for the i-th image. The predicted label data represents the i-th image. Represents the matching loss;

[0107] The matching loss includes the classification loss and localization loss of the detected target. Therefore, the formula for calculating the matching loss is:

[0108]

[0109] In the formula,

[0110] 'c' represents the category label, and 'b' represents the bounding box. This represents the classification loss for the category labels. This indicates the positioning loss of the detection box;

[0111] Introducing the loss L for reconstructing the object embedding descriptor emb (z i ,z j The formula is expressed as:

[0112]

[0113] in,

[0114] z represents an embedded object; This represents the embedded object obtained from the i-th image;

[0115] The final self-supervised auxiliary loss is:

[0116]

[0117] Where, λ f The weights representing the class loss, λ b The weights λ represent the loss of the predicted bounding box. e The weights represent the embedding loss.

[0118] λ f , λb , λ e These are used as weights for the three prediction heads in the transformer: the bounding box of the decoder's predicted output, the predicted output category, and the predicted object embedding descriptor.

[0119] S6. Using the optimized target detection model, continue to perform target detection and target detection model optimization on the remaining target detection meta-tasks.

[0120] In this invention, the multi-task approach includes an auxiliary task (self-supervised task) and a main task (object detection task). The two tasks share the feature extraction and transformer components (i.e., steps S1-S4). During the pre-training of the object detection model, the object detection task is maintained (including steps S1-S4). During fine-tuning of the object detection model, only the self-supervised task is executed (including steps S1-S5). During object detection, the object detection task and the self-supervised task run simultaneously (including steps S1-S6). In other words, multi-task learning generates the final object detection result through the cooperation of the object detection task (main branch) and the self-supervised task (auxiliary branch). By using self-supervision as an auxiliary task to assist the main object detection task, more diverse perspectives are provided for object detection. The auxiliary task and the main task share features in both feature extraction and transformer components. Multi-task training improves the robustness of the model.

[0121] Furthermore,

[0122] Multi-task learning trains tasks together to overcome the inadequacy of annotated data. It provides inductive bias to each task to trigger regularization effects between them. It combines meta-DETR with self-supervised branches in a multi-task manner (e.g., Figure 2 (As shown). The loss function of the main object detection task (Meta-DETR) is similar to that of Deformable DETR, through... Each layer of the constraint transformer decoder:

[0123]

[0124] Where c′ i The category label representing the main task. The prediction category label representing the main task, b′ i The true detection box represents the main task. The bounding box label represents the main task, j represents one of the N samples, and the total training loss is:

[0125] L loss =λ ssl (L ssl +L dn +L pos )+L meta

[0126] λ ssl This represents the proportion of loss in the self-supervised auxiliary branch.

[0127] like Figure 2 As shown, the present invention also provides, for example, a system for a self-supervised few-shot target detection method based on multi-tasks, comprising: a meta-task construction module, a feature extraction module, a feature aggregation module, a shared transformer module, and a self-supervised auxiliary optimization module;

[0128] The meta-task building module is used for:

[0129] Based on sampling with replacement using a small sample object detection dataset, multiple different image detection meta-tasks are constructed and input into the object detection model. The image detection meta-tasks include a support image set and a query image set.

[0130] The feature extraction module is used for:

[0131] Features of the support image set and the query image set are extracted using a convolutional neural network to obtain the support image feature set and the query image feature set.

[0132] The feature aggregation module is used for:

[0133] The supporting image feature set is aggregated, and then concatenated with the feature sets of all query images to obtain the overall aggregated features;

[0134] Shared converter module, used for:

[0135] The overall aggregated features and the query image feature set are input into the transformer to obtain the target detection results, including a fixed number of detection boxes and detection features;

[0136] The self-supervised auxiliary optimization module is used for:

[0137] The detection bounding box and the region proposal generated by the self-supervised module are compared, the detection features are compared with the embedded features of the self-supervised module, the loss is calculated, and the weights of the transformer are adjusted based on the loss calculation results to optimize the target detection model.

[0138] The optimized target detection model is then used to continue performing target detection and target detection model optimization on the remaining target detection meta-tasks.

[0139] As shown in Table 1, the method proposed in this invention has better detection performance in small sample target detection compared with the existing Meta-DETR method with end-to-end DETR for small sample domains.

[0140]

[0141] Table 1 Comparison of the detection performance of this invention with other different models.

[0142] Advantages of this invention:

[0143] This invention proposes self-supervised learning to alleviate the problem of insufficient labeled data in small-sample object detection tasks. By labeling a small amount of data, self-supervision can provide the model with more perspectives on the samples, increasing sample diversity. Simultaneously, to better suit self-supervision with small samples, this invention introduces a denoising module for expanding positive and negative samples and a team module for precise localization. The denoising module generates positive and negative sample noise and uses a contrastive denoising training method to improve the model's convergence speed. The team module utilizes positional constraints to provide more precise localization of the query box, effectively improving the object detection model's detection performance for new categories.

[0144] This invention employs a multi-task approach, combining self-supervised learning with few-shot target detection. Self-supervision serves as an auxiliary task, primarily improving the performance of the main task of few-shot target detection.

[0145] This invention incorporates a denoising module and a team module into the self-supervised branch. This enhances the constraint of the self-supervised branch on the model, making it more suitable for small-sample object detection, and simultaneously accelerating its convergence speed.

[0146] This invention enables end-to-end training and inference without requiring additional parameters or computation.

[0147] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A self-supervised few-sample target detection method based on multi-task, characterized in that, include: Based on a small sample object detection dataset, sampling with replacement is performed to construct multiple different image detection meta-tasks, which are then input into the object detection model. The image detection meta-tasks include a support image set and a query image set. Features of the support image set and the query image set are extracted using a convolutional neural network to obtain a support image feature set and a query image feature set. The supporting image feature set is aggregated, and then concatenated with the feature sets of all query images to obtain the overall aggregated features; The overall aggregated features and the query image feature set are input into the transformer to obtain the target detection result, which includes a fixed number of detection boxes and detection features; The detection bounding box and the region proposal generated by the self-supervised module are compared, the detection features are compared with the embedded features of the self-supervised module, the loss is calculated, and the weights of the transformer are adjusted based on the loss calculation results to optimize the target detection model. The optimized target detection model is then used to continue performing target detection and target detection model optimization on the remaining target detection meta-tasks.

2. The self-supervised few-sample target detection method based on multi-tasks according to claim 1, characterized in that: The method involves sampling with replacement based on a small sample object detection dataset to construct multiple different image detection meta-tasks, including: In a small sample object detection dataset, N categories are randomly selected, and K samples are extracted from each category to form the support image set, represented as follows: in, X represents a sample image in the small sample object detection dataset, Y represents the image pixel label information corresponding to the sample image in the small sample object detection dataset, S represents an image in the support image set, and i is a sample in the support image set; In the small sample object detection dataset, samples from N disjoint categories of the supporting image set are randomly selected to form the query image set. Each category is represented as: Q={X q ,Y q } in, X represents the sample images in the small sample object detection dataset, Y represents the image pixel label information corresponding to the sample images in the small sample object detection dataset, and q represents the image in the query image set.

3. The self-supervised few-sample target detection method based on multi-tasks according to claim 1, characterized in that: The overall aggregated features and the query image feature set are input into the transformer to obtain the target detection result; including: The overall aggregated features and the query image feature set are input into the transformer, then processed by the encoder for attention, and then sent to the decoder for prediction to obtain a fixed number of detection boxes and detection features.

4. The self-supervised few-sample target detection method based on multi-tasks according to claim 1, characterized in that: The decoder of the converter is equipped with a denoising module, which generates positive and negative sample pairs of noise. The positive and negative sample pairs, along with the ground truth (GT) labels and the initialized query boxes, are fed into the decoder to train the object detection model to reconstruct the GT labels and boxes.

5. The self-supervised few-sample target detection method based on multi-tasks according to claim 4, characterized in that: The decoder end of the converter is also equipped with a team module; When the query box is sent to the decoder, the team module groups the query boxes according to different query targets and query regions, sets the number and proportion of query boxes in each group, and guides the target detection model to learn to capture target features on different proportions of query targets and query regions.

6. The self-supervised few-sample target detection method based on multi-tasks according to claim 5, characterized in that: The query boxes, grouped by the team module, predict objects within a specific size range for each group.

7. The self-supervised few-sample target detection method based on multi-tasks according to claim 1, characterized in that: The self-supervised module uses a high recall algorithm to generate content labels and bounding boxes for the image detection meta-task, and inputs the image detection meta-task into the self-supervised image encoder to obtain the embedded features.

8. The self-supervised few-sample target detection method based on multi-tasks according to claim 1, characterized in that: The detection bounding box and the region proposals generated by the self-supervised module are compared, the detected features are compared with the embedded features of the self-supervised module, the loss is calculated, and the weights of the transformer are adjusted based on the loss calculation results; including: Using pairwise matching loss, the optimal assignment between the prediction and the target is calculated using the following formula: in, σ represents a permutation of N elements. This represents the optimal allocation between prediction and target, where y represents the true label data for the i-th image. The predicted label data represents the i-th image. Represents the matching loss; The matching loss includes the classification loss and localization loss of the detected target. Therefore, the formula for calculating the matching loss is: in, 'c' represents the category label, and 'b' represents the bounding box. This represents the classification loss for the category labels. This indicates the positioning loss of the detection box; Introducing the loss L for reconstructing the object embedding descriptor emb (z i ,z j The formula is expressed as: in, z represents an embedded object; This represents the embedded object obtained from the i-th image; The final self-supervised auxiliary loss is: Where, λ f The weights representing the class loss, λ b The weights λ represent the loss of the predicted bounding box. e The weights represent the embedding loss; λ f , λ b and λ e These are respectively used as weights for the three prediction heads in the transformer: the bounding box of the decoder's predicted output, the predicted output category, and the predicted object embedding descriptor.

9. A system for implementing the self-supervised small-sample target detection method based on multiple tasks as described in any one of claims 1 to 8, characterized in that, include: The module includes a meta-task construction module, a feature extraction module, a feature aggregation module, a shared transformer module, and a self-supervised auxiliary optimization module. The meta-task construction module is used for: Based on a small sample object detection dataset, sampling with replacement is performed to construct multiple different image detection meta-tasks, which are then input into the object detection model. The image detection meta-tasks include a support image set and a query image set. The feature extraction module is used for: Features of the support image set and the query image set are extracted using a convolutional neural network to obtain a support image feature set and a query image feature set. The feature aggregation module is used for: The supporting image feature set is aggregated, and then concatenated with the feature sets of all query images to obtain the overall aggregated features; The shared converter module is used for: The overall aggregated features and the query image feature set are input into the transformer to obtain the target detection result, which includes a fixed number of detection boxes and detection features; The self-supervised auxiliary optimization module is used for: The detection bounding box and the region proposal generated by the self-supervised module are compared, the detection features are compared with the embedded features of the self-supervised module, the loss is calculated, and the weights of the transformer are adjusted based on the loss calculation results to optimize the target detection model. The optimized target detection model is then used to continue performing target detection and target detection model optimization on the remaining target detection meta-tasks.

Images