Small sample target detection method based on contrastive learning and multi-aspect distillation

Abstract

The application discloses a small sample target detection method based on contrast learning and multi-aspect distillation, constructs a small sample target detection network model based on contrast learning and multi-aspect distillation, carries out basic class basic training on the small sample target detection network model, obtains a small sample target detection basic class model, carries out new class fine-tuning training on the small sample target detection basic class model, obtains a final small sample target detection model, and finally detects small sample targets according to the final small sample target detection model. The method solves the problems that the traditional target detection method is prone to class confusion and weak information capturing capacity, greatly improves the capturing capacity of the detection model on target features, simultaneously strengthens foreground representation and reduces the interference of background information, and improves target detection performance.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a small-sample target detection method based on contrastive learning and multi-faceted distillation. Background Technology

[0002] In recent years, deep neural networks have achieved great success in various computer vision tasks, such as image recognition, object detection, and semantic segmentation. A major factor in their success is the use of large amounts of labeled data for training. However, in many practical applications, such as military, aerospace, medical, and agricultural pest and disease control, available data is scarce, making it difficult to obtain large amounts of accurately labeled data. Therefore, how to effectively learn for target tasks with insufficient sample data—that is, few-shot learning (FSL)—has become one of the main research topics in deep learning. Currently, most research on few-shot learning focuses on image classification, with relatively little research on few-shot object detection tasks.

[0003] Existing methods primarily rely on the traditional object detection framework Faster R-CNN. This framework is mainly designed for traditional object detection, thus limiting its performance in small-sample object detection. While Faster R-CNN can accurately locate new classes, it is prone to misclassifying detected new classes as easily confused base classes. This is mainly because the Faster R-CNN feature encoder contains almost no discriminative information for classifying most new classes. During the base class training phase, it easily treats objects containing new classes as "background," leading to class confusion. Summary of the Invention

[0004] This invention provides a small-sample target detection method based on contrastive learning and multi-faceted distillation to overcome the technical problem of class confusion in existing small-sample target detection methods.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A small-sample target detection method based on contrastive learning and multi-face distillation includes the following steps:

[0007] S1: Obtain a publicly available small sample dataset of object detection data, and randomly divide the dataset into a base class dataset and a new class dataset based on the object detection dataset;

[0008] The target detection dataset includes the PASCAL VOC dataset and the MS COCO2014 dataset;

[0009] And select either the PASCAL VOC dataset or the MS COCO2014 dataset;

[0010] S2: Construct a small-sample target detection network model based on contrastive learning and multi-faceted distillation;

[0011] S3: Based on the base class dataset, perform base class basic training on the few-shot object detection network model to obtain a few-shot object detection base class model;

[0012] S4: Based on the new class data sample set, fine-tune the small sample target detection base class model to obtain the final small sample target detection model;

[0013] S5: Detect small sample targets based on the final small sample target detection model.

[0014] Furthermore, the few-shot target detection network model based on contrastive learning and multi-faceted distillation described in S2 includes a Faster R-CNN backbone network, a region proposal network, a feature extractor module, a contrastive branch module, a target detection network branch, and a multi-faceted distillation module;

[0015] The input of the Faster R-CNN backbone network is connected to the image input layer, and the output of the Faster R-CNN backbone network is connected to the input of the region proposal network and the input of the feature extractor module, respectively.

[0016] Furthermore, the Faster R-CNN backbone network is used to extract features from the input image to obtain the target feature map, and the target feature map is input to the region proposal network and the feature extractor module respectively;

[0017] The output of the region proposal network is connected to the input of the feature extractor module; and the region proposal network is used to generate target region proposal boxes based on the input target feature map.

[0018] The output of the feature extractor module is connected to the input of the target detection network branch and the input of the comparison branch module, respectively; and the output of the comparison branch module is connected to the input of the target detection network branch.

[0019] The feature extractor module is used to map the region proposal box onto the feature map input through the Faster R-CNN backbone network to obtain the target feature map, and input the target feature map to the target detection network branch and the comparison branch module respectively;

[0020] The target detection network branch includes a first network branch and a second network branch;

[0021] Furthermore, the first network branch includes a first fully connected layer and a classifier module connected in sequence;

[0022] The second network branch includes a second fully connected layer and a regressor module connected in sequence;

[0023] The outputs of the classifier module and the regressor module are connected to the multi-faceted distillation module.

[0024] The comparison branch module is used to optimize the target feature map based on the cosine similarity score of the target feature map using a comparison loss function;

[0025] The first fully connected layer is used to perform a fully connected operation on the output of the feature extractor module and the output of the comparison branch module, and the output of the first fully connected layer is transmitted to the classifier module;

[0026] The classifier module is used to obtain the category of the target feature map based on the output of the first fully connected layer, and output the probability of each target feature map category;

[0027] The second fully connected layer is used to perform a fully connected operation on the output of the feature extractor module and the output of the comparison branch module, and the output of the second fully connected layer is transmitted to the regressor module;

[0028] The regressor module is used to obtain the position of the target detection box based on the output of the second fully connected layer, and output the position offset of the target detection box;

[0029] The multi-faceted distillation module is used to extract common target features by optimizing the extraction of the probability of each target feature map category and the position offset of the target detection box through distillation loss; and to retrain the classifier module and the regressor module using the extracted common target features.

[0030] Furthermore, in S3, the base class training of the few-sample object detection network model is performed, including the following steps:

[0031] S31: Input the samples in the base class dataset into the Faster R-CNN backbone network for feature extraction; and input the feature maps after feature extraction into the region proposal network and the feature extractor module respectively; the region proposal network generates target region proposal boxes based on the input target feature maps;

[0032] S32: The feature extractor module maps the region proposal box onto the input target feature map to obtain the target feature map, and inputs the target feature map to the first network branch and the second network branch respectively;

[0033] S33: Train the output of the feature extractor module through the first network branch to obtain the category of the target feature map and output the probability of each target feature map category;

[0034] The output of the feature extractor module is trained by the second network branch to obtain the position of the target detection box and output the position offset of the target detection box;

[0035] S34: Determine whether the output of the few-sample object detection network model trained on the base class has converged;

[0036] If the output of the trained few-shot object detection network model converges, then the trained few-shot object detection network model is confirmed as the base class model for few-shot object detection; otherwise, proceed to step S35.

[0037] S35: Reselect samples from the base class dataset and repeat S31 to S34.

[0038] Furthermore, the formula for calculating the loss function used to determine whether the output of the few-sample object detection network model trained on the base class converges is as follows:

[0039] L det =L rpn +L cls +L reg

[0040] In the formula: L det L represents the loss function used for base class training. rpn L represents the cross-entropy loss of the region proposal network; cls L represents the cross-entropy loss of the classifier; reg This represents the smoothing loss function of the regressor.

[0041] Furthermore, in S4, the small sample target detection base class model is fine-tuned and trained based on the new class data sample set, including the following steps:

[0042] S41: The samples in the new class data sample set are input into the Faster R-CNN backbone network for feature extraction; and the feature maps after feature extraction are input into the region proposal network and the feature extractor module respectively; the region proposal network generates target region proposal boxes according to the input target feature maps, and the target region proposal boxes are input into the feature extractor module;

[0043] S42: The feature extractor module maps the region proposal box onto the input target feature map to obtain the target feature map, and inputs the target feature map into the first network branch, the second network branch and the comparison branch module respectively;

[0044] The optimized target feature map is obtained by comparing the cosine similarity score of the target feature map by the branch module, and the optimized target feature map is input into the first network branch and the second network branch respectively.

[0045] S43: Train the output of the feature extractor module and the optimized target feature map through the first network branch to obtain the category of the target feature map and output the probability of each target feature map category;

[0046] The output of the feature extractor module is trained with the optimized target feature map by the second network branch to obtain the position of the target detection box and output the position offset of the target detection box.

[0047] S44: Input the probability of each target feature map category and the position offset of the target detection box into the multi-aspect distillation module for training;

[0048] S45: Determine whether the output of the small sample target detection base class model adjusted based on the new class of data sample set has converged;

[0049] If the output of the adjusted few-shot object detection base class model converges, then the trained few-shot object detection base class model is confirmed as the final few-shot object detection model; otherwise, proceed to step S46.

[0050] S46: Select samples from the new class dataset and repeat S41 to S45.

[0051] Furthermore, in S45, the loss function used to determine whether the output of the small sample object detection base class model adjusted based on the new class dataset has converged is:

[0052] L = L det +λ(L SupCon +L distill )

[0053]

[0054]

[0055] In the formula: L det L represents the loss function used for base class training; λ is a hyperparameter for balancing the loss; SupCon Represents the contrastive loss function; τ represents the temperature hyperparameter; 1 [k≠i] Z represents a value of 1 when k ≠ i and a value of 0 when k = i; i ·Z j Indicates the calculation of Z i and Z j Cosine similarity between them; Z i ·Z kIndicates the calculation of Z i and Z k The cosine similarity between them; j and i represent parameter variables; N represents the number of samples in the dataset; L distill The distillation loss function is represented by v; the sampling feature is represented by S; the set of sampling features for the new class of data sample c is represented by δ′. c μ represents the standard deviation of the new class of data sample c. c S represents the mean of the new class of data sample c; c N{μ} represents the set of the k closest base classes of the new class data sample c; c ,δ′ c The Gaussian distribution represents sampling from a new class of data, sample c; f θ (v) represents the classifier used to calculate the probability distribution of each class; CE represents the cross-entropy loss function; CE(c, f) θ (v) represents the calculation of the classifier's predicted value f for the feature. θ (v) The difference between the data and the real new class data sample c.

[0056] Furthermore, the contrast branch module includes a multilayer perceptron and a contrast loss function layer;

[0057] Furthermore, the multilayer perceptron includes two linear layers and one ReLU activation function layer.

[0058] Beneficial Effects: This invention discloses a few-sample object detection method based on contrastive learning and multi-faceted distillation. It constructs a few-sample object detection network model based on contrastive learning and multi-faceted distillation; performs base class training on this network model to obtain a few-sample object detection base class model; then fine-tunes this base class model with new classes to obtain a final few-sample object detection model; finally, it detects few-sample objects based on this final model. This method solves the problems of class confusion and weak information capture ability in traditional object detection methods, greatly improves the detection model's ability to capture target features, enhances foreground representation, reduces background interference, and improves object detection performance. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart of the small-sample target detection method based on contrastive learning and multi-faceted distillation of the present invention;

[0061] Figure 2 This is a schematic diagram illustrating the basic training of the small-sample target detection method based on contrastive learning and multi-faceted distillation of the present invention.

[0062] Figure 3 This is a schematic diagram illustrating the fine-tuning training of the small-sample target detection method based on contrastive learning and multi-faceted distillation of the present invention.

[0063] Figure 4 This is a diagram showing the final model structure of the small-sample target detection method based on contrastive learning and multi-faceted distillation of the present invention.

[0064] Figure 5 This is a schematic diagram of the comparative branch module structure described in this invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] This embodiment provides a small-sample target detection method based on contrastive learning and multi-faceted distillation, such as Figure 1 As shown, it includes the following steps:

[0067] S1: Obtain a publicly available small sample dataset of object detection data, and randomly divide the dataset into a base class dataset and a new class dataset based on the object detection dataset;

[0068] The target detection dataset includes the PASCAL VOC dataset and the MS COCO2014 dataset;

[0069] And select either the PASCAL VOC dataset or the MS COCO2014 dataset;

[0070] Specifically, the dataset uses two datasets, PASCAL VOC and MS COCO2014, and is divided into base class datasets and new class datasets with disjoint categories according to certain criteria. In this embodiment, the model is trained using the PASCAL VOC dataset and MS COCO2014 respectively.

[0071] The PASCAL VOC dataset is divided into 20 classes: {"bird", "bus", "cow", "mbike", "sofa", "aero", "bike", "boat", "bottle", "car", "cat", "chair", "table", "dog", "horse", "person", "plant", "sheep", "train", "tvmonitor"}. Any 15 classes are used as base classes, and the remaining 5 classes are used as new classes, each with K instances, where K takes the value {1, 2, 3, 5, 10}. In this embodiment, the new class dataset is divided into three types, labeled {1, 2, 3}, namely split1, split2, and split3. For easier comparison of model performance, the class distribution for the three divisions is fixed.

[0072] In split1 (the first group), novel1 (the new class 1) is {"bird", "bus", "cow", "mbike", "sofa"};

[0073] In split1, base1 (base class 1) is {"aero", "bike", "boat", "bottle", "car", "cat", "chair", "table", "dog", "horse", "person", "plant", "sheep", "train", "tvmonitor"};

[0074] In split2 (the second group), novel2 (new class 2) is {"aero", "cow", "sofa", "bottle", "horse"}, and base2 (base class 2) in split2 is {"bike", "bird", "boat", "bus", "car", "cat", "chair", "table", "dog", "mbike", "person", "plant", "sheep", "train", "tvmonitor"};

[0075] In split3 (the third group), novel3 (the new class 3) consists of {"sofa", "boat", "cat", "chair", "table"}, and base3 (the base class 3) consists of {"aero", "bike", "birt", "bottle", "bus", "car", "chair", "cow", "table", "dog", "horse", "person", "plant", "train", "tvmonitor"}.

[0076] The MS COCO2014 dataset is divided as follows: MS COC2014 has 80 classes, of which 60 classes are randomly selected as base classes and the remaining 20 classes are used as new classes. K takes the value {1,2,3,5,10,30}. The division rule of the MS COCO2014 dataset is the same as that of the PASCAL VOC dataset, and will not be repeated here.

[0077] S2: Construct a small-sample target detection network model based on contrastive learning and multi-faceted distillation;

[0078] Specifically, such as Figure 2 As shown, the constructed few-sample target detection network model based on contrastive learning and multi-faceted distillation includes a Faster R-CNN backbone network, a region proposal network, a feature extractor module, a contrastive branch module, a target detection network branch, and a multi-faceted distillation module;

[0079] The input of the Faster R-CNN backbone network is connected to the image input layer, and the output of the Faster R-CNN backbone network is connected to the input of the region proposal network and the input of the feature extractor module, respectively.

[0080] Furthermore, the Faster R-CNN backbone network is used to extract features from the input image to obtain the target feature map, and the target feature map is input to the region proposal network and the feature extractor module respectively;

[0081] The output of the region proposal network is connected to the input of the feature extractor module; and the region proposal network is used to generate target region proposal boxes based on the input target feature map.

[0082] The output of the feature extractor module is connected to the input of the target detection network branch and the input of the contrast branch module, respectively; and the output of the contrast branch module is connected to the input of the target detection network branch; and the contrast branch module includes a multilayer perceptron and a contrast loss function layer; and the multilayer perceptron includes two linear layers and a ReLU activation function layer.

[0083] The feature extractor module is used to map the region proposal box onto the feature map input through the Faster R-CNN backbone network to obtain the target feature map, and input the target feature map to the target detection network branch and the comparison branch module respectively;

[0084] The target detection network branch includes a first network branch and a second network branch;

[0085] Furthermore, the first network branch includes a first fully connected layer and a classifier module connected in sequence;

[0086] The second network branch includes a second fully connected layer and a regressor module connected in sequence;

[0087] The outputs of the classifier module and the regressor module are connected to the multi-faceted distillation module.

[0088] The comparison branch module is used to optimize the target feature map based on the cosine similarity score of the target feature map using a comparison loss function;

[0089] The first fully connected layer is used to perform a fully connected operation on the output of the feature extractor module and the output of the comparison branch module, and the output of the first fully connected layer is transmitted to the classifier module;

[0090] The classifier module is used to obtain the category of the target feature map based on the output of the first fully connected layer, and output the probability of each target feature map category;

[0091] The second fully connected layer is used to perform a fully connected operation on the output of the feature extractor module and the output of the comparison branch module, and the output of the second fully connected layer is transmitted to the regressor module;

[0092] The regressor module is used to obtain the position of the target detection box based on the output of the second fully connected layer, and output the position offset of the target detection box;

[0093] The multi-faceted distillation module is used to optimize the extraction of common features of each target feature map category and the position offset of the target detection box through distillation loss; and to retrain the classifier module and the regressor module through the extracted common features of the target to improve the classification accuracy of the classifier and the localization accuracy of the regressor.

[0094] S3: Based on the base class dataset, perform base class basic training on the few-shot object detection network model to obtain a few-shot object detection base class model;

[0095] S4: Based on the new class data sample set, fine-tune the small sample target detection base class model to obtain the final small sample target detection model;

[0096] S5: Detect small sample targets based on the final small sample target detection model.

[0097] The purpose of this invention is to address the existing class confusion problem in few-sample object detection and improve the accuracy of object detection with a small number of samples. To address this problem, this invention proposes a few-sample object detection method based on contrastive learning and multi-face distillation. It consists of two stages: training a Faster R-CNN network using samples from a base class dataset, and then fine-tuning it using samples from a new class dataset. By designing a contrastive branch module, supervised contrastive learning minimizes the distance between positive samples and maximizes the distance between negative samples, thereby enhancing the discriminative power between different objects and mitigating class confusion. Simultaneously, this invention also designs a multi-face distillation module to learn class-independent common features between the base class and the new class, and distills these common features to improve the performance of new class object detection. For the field of few-sample object detection with only a small number of samples, this invention can solve its class confusion problem and further improve the performance and detection accuracy of few-sample object detection. The proposed method is more lightweight, more accurate, and has already achieved good detection results on public datasets.

[0098] In a specific embodiment, during the base class training phase, the few-shot object detection network model based on contrastive learning and multi-faceted distillation is trained using either 15 base classes from the PASCAL VOC dataset or 60 base classes from the MS COCO2014 dataset; and in S3, the few-shot object detection network model undergoes basic base class training, such as... Figure 3 As shown, it includes the following steps:

[0099] S31: Input the samples in the base class dataset into the Faster R-CNN backbone network for feature extraction; and input the feature maps after feature extraction into the region proposal network and the feature extractor module respectively; the region proposal network generates target region proposal boxes based on the input target feature maps;

[0100] S32: The feature extractor module maps the region proposal box onto the input target feature map to obtain the target feature map, and inputs the target feature map to the first network branch and the second network branch respectively;

[0101] S33: Train the output of the feature extractor module through the first network branch to obtain the category of the target feature map and output the probability of each target feature map category;

[0102] The output of the feature extractor module is trained by the second network branch to obtain the position of the target detection box and output the position offset of the target detection box;

[0103] S34: Determine whether the output of the few-sample object detection network model trained on the base class has converged;

[0104] If the output of the trained few-shot object detection network model converges, then the trained few-shot object detection network model is confirmed as the base class model for few-shot object detection; otherwise, proceed to step S35.

[0105] S35: Reselect samples from the base class dataset and repeat S31 to S34.

[0106] In a specific embodiment, the formula for calculating the loss function used to determine whether the output of the few-sample object detection network model trained on the base class converges is as follows:

[0107] L det =L rpn +L cls +L reg

[0108] In the formula: L det L represents the loss function used for base class training. rpn L represents the cross-entropy loss of the region proposal network; cls L represents the cross-entropy loss of the classifier; reg This represents the smoothing loss function of the regressor.

[0109] In a specific embodiment, after the base class samples are trained, a base class model is obtained. Next, fine-tuning is performed on the new class samples based on the base class model. In S4, the small sample target detection base class model is trained using the new class data sample set for new class fine-tuning. Figure 4 As shown, it includes the following steps:

[0110] S41: The samples in the new class data sample set are input into the Faster R-CNN backbone network for feature extraction; and the feature maps after feature extraction are input into the region proposal network and the feature extractor module respectively; the region proposal network generates target region proposal boxes according to the input target feature maps, and the target region proposal boxes are input into the feature extractor module;

[0111] S42: The feature extractor module maps the region proposal box onto the input target feature map to obtain the target feature map, and inputs the target feature map into the first network branch, the second network branch and the comparison branch module respectively;

[0112] The optimized target feature map is obtained by comparing the cosine similarity score of the target feature map by the branch module, and the optimized target feature map is input into the first network branch and the second network branch respectively.

[0113] S43: Train the output of the feature extractor module and the optimized target feature map through the first network branch to obtain the category of the target feature map and output the probability of each target feature map category;

[0114] The output of the feature extractor module is trained with the optimized target feature map by the second network branch to obtain the position of the target detection box and output the position offset of the target detection box.

[0115] S44: Input the probability of each target feature map category and the position offset of the target detection box into the multi-aspect distillation module for training;

[0116] S45: Determine whether the output of the small sample target detection base class model adjusted based on the new class of data sample set has converged;

[0117] If the output of the adjusted few-shot object detection base class model converges, then the trained few-shot object detection base class model is confirmed as the final few-shot object detection model; otherwise, proceed to step S46.

[0118] S46: Select samples from the new class of data sample set and repeat S41 to S45.

[0119] Specifically, the fine-tuning stage for the new class involves fine-tuning the few-shot object detection base model on a new class dataset. This invention introduces a contrastive learning branch module and a distillation module into the classification and regression layers to retrain the classifier and regressor. During this fine-tuning stage, only the classifier and regressor modules are updated. That is, they are trained on the new class dataset and their weights are adjusted using a loss function to adapt to the object detection task on the new dataset. The region proposal network and feature extractor modules remain unchanged, maintaining the same weights as in the few-shot object detection base model. By fine-tuning only the classifier and regressor modules, the semantic information from the base model is transferred to the new class dataset. After training, the final few-shot object detection model is obtained. Figure 4 This is a schematic diagram illustrating the fine-tuning training of the small-sample target detection method based on contrastive learning and multi-faceted distillation of the present invention, and in Figure 3 Based on the base class model, the model is fine-tuned using a new class dataset.

[0120] In a specific embodiment, the loss function used in S45 to determine whether the output of the small sample object detection base class model adjusted based on the new class dataset has converged is:

[0121] L = Ldet +λ(L SupCon +L distill )

[0122]

[0123]

[0124] In the formula: L det L represents the loss function used for base class training; λ is a hyperparameter for balancing the loss; SupCon Represents the contrastive loss function; τ represents the temperature hyperparameter; 1 [k≠i] Z represents a value of 1 when k ≠ i and a value of 0 when k = i; i ·Z j Indicates the calculation of Z i and Z j Cosine similarity between them; Z i ·Z k Indicates the calculation of Z i and Z k The cosine similarity between them; j and i represent parameter variables; N represents the number of samples in the dataset; L distill The distillation loss function is represented by v; the sampling feature is represented by S; the set of sampling features for the new class of data sample c is represented by δ′. c μ represents the standard deviation of the new class of data sample c. c S represents the mean of the new class of data sample c; c N{μ} represents the set of the k closest base classes of the new class data sample c; c ,δ′ c The Gaussian distribution represents sampling from a new class of data, sample c; f θ (v) represents the classifier used to calculate the probability distribution of each class; CE represents the cross-entropy loss function; CE(c, f) θ (υ) represents the calculation of the classifier's predicted value f for the feature. θ (v) The difference between the data and the real new class data sample c.

[0125] Specifically, traditional object detection detectors can distinguish categories well when there is sufficient data. However, with limited data, they are prone to category confusion and weak information capture capabilities. To improve the model's ability to capture object features, enhance foreground representation, and reduce background interference, thereby improving object detection performance, this invention introduces a contrastive loss function at the beginning of the feature extraction module. The contrastive branch module is as follows: Figure 5As shown, it consists of a multilayer perceptron (MLP) and a contrastive loss function (SupConLoss). The MLP consists of two linear layers and a ReLU activation function. The introduction of this contrastive loss function helps the model better distinguish between different categories of targets; and the formula for the contrastive loss function is:

[0126]

[0127] In the formula: L SupCon The formula for the contrast loss function is given, where τ is the temperature hyperparameter, 1 [k≠i] The 1 in the text represents the numerical value 1. [k≠i] Z represents the value 1 when k ≠ i and the value 0 when k = i. i ·Z j Indicates the calculation of Z i and Z j Cosine similarity between them, Z i ·Z k Indicates the calculation of Z i and Z k The cosine similarity between them, where j and i represent variables, and N represents the number of samples; L SupCon The loss function can maximize the similarity between positive sample pairs and enhance the difference between negative sample pairs, thus learning more discriminative feature representations.

[0128] To better identify common features between the new class and the base class, this invention employs a multi-faceted distillation module to learn category-independent common features between the base class and the new class, thereby improving detection accuracy. The multi-faceted distillation module mainly calculates the feature set S sampled from a Gaussian distribution. c The performance of the model is measured by the average cross-entropy loss. Samples generated using a Gaussian distribution contain common features of the new class; and the distillation loss function is calculated as follows:

[0129]

[0130] L distill The formula representing the distillation loss function, where S is given below:

[0131] S={v|v~N(μ c ,δ′ c )}

[0132] In the formula: v represents the sampling feature, S represents the set of sampling features for the new class c, and δ′ c μ represents the standard deviation of the new class c. c S represents the mean of the new class c. c N{μ} represents the set of the k closest base classes of the new class c. c ,δ′c The Gaussian distribution represents sampling of a new class c samples, f θ (υ) represents the classifier used to calculate the probability distribution of each class, and CE represents the cross-entropy loss function, CE(c, f θ (υ) represents the calculation of the classifier's predicted value f for the feature. θ The difference between (υ) and the real sample c; the entire distillation loss function represents the further use of cross-entropy loss to train the classifier of the target detector and extract common features.

[0133] like Figure 4 The diagram shown illustrates the model structure of the small-sample target detection method based on contrastive learning and multi-faceted distillation according to the present invention. Figure 2 and Figure 3 Together constituting the present invention, such as Figure 4 The small sample target detection model shown in this invention is divided into base class training and new class training, and the loss function is different in each stage:

[0134] During the basic training phase, the standard Faster R-CNN loss is used, where L... rpn L represents the cross-entropy loss of the region proposal network, primarily used to obtain region proposal boxes from candidate boxes. cls L represents the cross-entropy loss of the classifier, primarily used for classifying region proposal boxes. reg This represents the smoothed L1 loss of the regressor, primarily used for region proposal box regression; L det The loss function for training the base classes is as follows:

[0135] L det =L rpn +L cls +L reg

[0136] During the fine-tuning training phase, a contrastive branch module is added to the head of the feature extractor, combining the contrastive loss function SupCon and the distillation loss with the Faster R-CNN loss to form the final loss function. Here, λ is a hyperparameter for balancing the loss, set to 0.5 in this paper. The final overall algorithm loss function is:

[0137] L = L det +λ(L SupCon +L distill )

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A small sample target detection method based on contrastive learning and multi-aspect distillation, characterized in that, The method comprises the following steps: S1: obtaining a target detection data set of a public small sample, and randomly dividing the target detection data set to obtain a base class data set and a new class data sample set; The target detection data set comprises a PASCAL VOC data set and an MS COCO2014 data set; And any one of the PASCAL VOC data set or the MS COCO2014 data set is selected; S2: constructing a small sample target detection network model based on contrast learning and multi-aspect distillation; The small sample target detection network model based on contrast learning and multi-aspect distillation comprises a Faster R-CNN backbone network, a region proposal network, a feature extractor module, a contrast branch module, a target detection network branch, and a multi-aspect distillation module; The input end of the Faster R-CNN backbone network is connected with an image input layer, and the output end of the Faster R-CNN backbone network is respectively connected with the input end of the region proposal network and the input end of the feature extractor module; And the Faster R-CNN backbone network is used for feature extraction of an input image to obtain a target feature map, and the target feature map is respectively input into the region proposal network and the feature extractor module; The output end of the region proposal network is connected with the input end of the feature extractor module; and the region proposal network is used for generating a target region proposal frame according to the input target feature map; The output end of the feature extractor module is respectively connected with the input end of the target detection network branch and the input end of the contrast branch module; and the output end of the contrast branch module is connected with the input end of the target detection network branch; The feature extractor module is used for mapping the region proposal frame to the feature map input through the Faster R-CNN backbone network, obtaining a target feature frame map, and respectively inputting the target feature frame map into the target detection network branch and the contrast branch module; The target detection network branch comprises a first network branch and a second network branch; And the first network branch comprises a first full connection layer and a classifier module connected in sequence; The second network branch comprises a second full connection layer and a regressor module connected in sequence; The output ends of the classifier module and the regressor module are connected with the multi-aspect distillation module; The contrast branch module is used for optimizing the target feature frame map through a contrast loss function according to a cosine similarity score of the target feature frame map; The first full connection layer is used for full connection operation of the output of the feature extractor module and the output of the contrast branch module, and the output of the first full connection layer is transmitted to the classifier module; The classifier module is used for obtaining the class of the target feature frame map according to the output of the first full connection layer, and outputting the probability of each target feature frame map class; The second full connection layer is used for full connection operation of the output of the feature extractor module and the output of the contrast branch module, and the output of the second full connection layer is transmitted to the regressor module; The regressor module is configured to obtain a position of a target detection frame according to an output of the second full connection layer, and output a position offset of the target detection frame; The multi-aspect distillation module is configured to extract target common features of probabilities of the target feature frame classes and the position offset of the target detection frame through distillation loss optimization; and retrain the classifier module and the regressor module through the extracted target common features; S3: performing base class basic training on the small sample target detection network model based on the base class data set, to obtain a small sample target detection base class model; S4: performing new class fine-tuning training on the small sample target detection base class model based on the new class data sample set, to obtain a final small sample target detection model; S5: detecting a small sample target based on the final small sample target detection model.

2. The small sample target detection method based on contrastive learning and multi-aspect distillation according to claim 1, characterized in that, The base class basic training on the small sample target detection network model in S3 includes the following steps: S31: inputting samples in the base class data set into a Faster R-CNN backbone network for feature extraction; and inputting feature maps after the feature extraction into a region proposal network and a feature extractor module respectively; the region proposal network generates target region proposal frames according to the input target feature maps; S32: mapping the region proposal frames to the input target feature maps through the feature extractor module, obtaining target feature frame graphs, and inputting the target feature frame graphs into a first network branch and a second network branch respectively; S33: training the output of the feature extractor module through the first network branch, obtaining the classes of the target feature frame graphs, and outputting probabilities of the target feature frame classes; training the output of the feature extractor module through the second network branch, obtaining the positions of the target detection frames, and outputting the position offsets of the target detection frames; S34: determining whether the output of the small sample target detection network model after the base class training converges; if the output of the small sample target detection network model after the training converges, confirming that the small sample target detection network model after the training is a small sample target detection base class model; otherwise, performing S35; S35: reselecting samples in the base class data set, and repeatedly performing S31 to S34.

3. The method of claim 2, wherein the method is based on contrastive learning and multi-aspect distillation. The calculation formula of the loss function for determining whether the output of the small sample target detection network model after the base class training converges is wherein: represents a loss function for base class training; represents a cross-entropy loss for the region proposal network; represents a cross-entropy loss for the classifier; represents a smooth loss function for the regressor.

4. The small sample target detection method based on contrastive learning and multi-aspect distillation according to claim 3, characterized in that, S4: performing new class fine-tuning training on the small sample target detection base class model based on the new class data sample set, to obtain a final small sample target detection model; S5: detecting a small sample target based on the final small sample target detection model. S31: inputting samples in the base class data set into a Faster R-CNN backbone network for feature extraction; and inputting feature maps after the feature extraction into a region proposal network and a feature extractor module respectively; the region proposal network generates target region proposal frames according to the input target feature maps; S32: mapping the region proposal frames to the input target feature maps through the feature extractor module, obtaining target feature frame graphs, and inputting the target feature frame graphs into a first network branch and a second network branch respectively; S33: training the output of the feature extractor module through the first network branch, obtaining the classes of the target feature frame graphs, and outputting probabilities of the target feature frame classes; training the output of the feature extractor module through the second network branch, obtaining the positions of the target detection frames, and outputting the position offsets of the target detection frames; S34: determining whether the output of the small sample target detection network model after the base class training converges; if the output of the small sample target detection network model after the training converges, confirming that the small sample target detection network model after the training is a small sample target detection base class model; otherwise, performing S35; S35: reselecting samples in the base class data set, and repeatedly performing S31 to S34. The calculation formula of the loss function for determining whether the output of the small sample target detection network model after the base class training converges is S4: performing new class fine-tuning training on the small sample target detection base class model based on the new class data sample set, to obtain a final small sample target detection model; S5: detecting a small sample target based on the final small sample target detection model. The comparison branch module acquires an optimized target feature frame diagram according to the cosine similarity score of the target feature frame diagram, and inputs the optimized target feature frame diagram into the first network branch and the second network branch respectively; S43: The first network branch is used to train the output of the feature extractor module and the optimized target feature frame diagram, acquire the category of the target feature frame diagram, and output the probability of each target feature frame category; The second network branch is used to train the output of the feature extractor module and the optimized target feature frame diagram, acquire the position of the target detection frame, and output the position offset of the target detection frame; S44: The probability of each target feature frame category and the position offset of the target detection frame are input into the multi-aspect distillation module for training; S45: It is judged whether the output of the small sample target detection base class model adjusted based on the new class data sample set converges or not; If the output of the small sample target detection base class model after adjustment converges, the small sample target detection base class model after training is confirmed as the final small sample target detection model; otherwise, S46 is executed; S46: The samples in the new class data sample set are reselected, and S41 to S45 are repeatedly executed.

5. The method of claim 4, wherein the method is based on contrastive learning and multi-aspect distillation. The loss function for judging whether the output of the small sample target detection base class model adjusted based on the new class data set converges or not in S45 is ， wherein: represents a loss function for base class training; is a hyperparameter for the balanced loss set; represents a contrastive loss function; represents a temperature hyperparameter; represents is 1 when is 0 when represents calculating the cosine similarity between and represents calculating the cosine similarity between and j,i each represents a parameter variable; N represents the number of samples in the dataset; represents a distillation loss function; represents a sampled feature; represents a set of sampled features for a new class data sample c; represents the standard deviation of the new class data sample c; represents the mean of the new class data sample c; represents a set of the top k base classes closest to the new class data sample c; Gaussian distribution represents sampling for the new class data sample c; represents a classifier for calculating the probability distribution of each class; represents a cross-entropy loss function; represents calculating the difference between the predicted value of the classifier for the feature and the real new class data sample c.

6. The method of claim 1, wherein the method is based on contrastive learning and multi-aspect distillation. The comparison branch module includes a multi-layer perceptron and a comparison loss function layer; And the multi-layer perceptron includes two linear layers and one ReLu activation function layer.

Images