Small sample image classification optimization method based on supervised contrast learning and multi-task setting

Abstract

The application discloses a small sample image classification optimization method based on supervised contrast learning and a multi-task setting, and specifically comprises the following stages: a feature extractor training stage, a proxy task stage and a small sample classification stage; the feature extractor training stage introduces a category supervision signal into contrast learning to participate in image representation training; the proxy task stage is replaced by a background confusion method which is more suitable for small sample learning; and the small sample classification stage adds a multi-task setting on the basis of a cosine classifier, thereby relieving the instability of parameter optimization caused by meta-learning training. Compared with the prior art, the application has the advantages that the feature extractor is guided to learn more semantic information, the generality of meta-knowledge is enhanced, the accuracy of small sample learning in general image classification tasks is improved, the method is simple, the use effect is good, and the application has high practical value and good development prospect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of small sample image classification, and particularly relates to an image classification optimization method based on supervised contrast learning and multi-task. BACKGROUND

[0002] Data-driven artificial intelligence has achieved very remarkable achievements, and almost covers multiple hot research fields such as computer vision, natural language processing, audio and speech, reinforcement learning and data analysis. However, in many scenarios, due to reasons such as privacy, security and cost, it is often impossible to obtain large-scale labeled training samples. When there are only limited samples, the conventional model is difficult to generalize in different data distributions. Taking industrial detection in the intelligent manufacturing scenario as an example, poor model generalization has become a key challenge affecting its production efficiency, especially in the case of frequent changes in lighting conditions and lack of defect samples. Similar problems also exist in some high-end industries. For example, in the task of finding high-energy cosmic rays, researchers hope to find cosmic rays carrying special energy from the dense cosmic rays. The whole process will be very time-consuming and costly. Today, small sample learning is very active in all walks of life, including credit card fraud, ticket identification, intent recognition, cold start recommendation and gesture recognition.

[0003] Small sample learning has very important practical significance for the development of artificial intelligence. In 2003, the concept of small sample learning was first proposed by Li Feifei, which triggered extensive discussion in the industry. It subverted the traditional machine learning for data and knowledge of subjective cognition. The new class, extremely limited data set is the biggest feature of small sample learning. Foreign research on small sample learning started early and achieved rich results. In 2017, the University of Toronto jointly published a groundbreaking research with the famous American social communication giant Twitter. Small sample learning does not need special design for the model, and the feature extractor and a classifier based on metric learning can achieve good results. This simple idea has influenced a large number of small sample learning works and made positive progress on this basis. In the same year, OpenAI proposed the meta-learning paradigm, trying to help small sample learning make breakthroughs at the task level. The proposal of meta-learning makes small sample learning no longer need a large number of base class data in the pre-training stage. Subsequently, at the first World Artificial Intelligence Conference held in China in 2018, Zhang Chi of the Chinese Academy of Sciences made a report on the theme of “Artificial Intelligence Based on Big Data”, which pointed out that “the utilization rate of deep learning for data is too low, which leads us to need a large amount of data to train it. We must increase the utilization rate of individual data to completely solve small sample learning”. Similarly, Hugo Larochelle, chief scientist of Google Brain, pointed out at the 2019 Deep Learning Summit that “small sample learning, with its unique learning method, will inevitably become the next hot spot in the field of deep learning”. Solving small sample learning is a necessary way for machine learning to become general artificial intelligence. The concept of small sample learning is inspired by the powerful reasoning and analysis ability of human beings, aiming to learn and generalize new concepts from a small number of samples.

[0004] Currently, the baseline methods of small sample learning can be roughly divided into metric learning and meta learning. Metric learning needs a flexible enough feature extractor, which is responsible for transferring the feature expression learned in the base class dataset to the downstream task to the maximum extent. The commonly used methods of metric learning include cosine similarity, Euclidean distance, relationship network and graph neural network. Meta learning proposes the training method of scenario reproduction. The same task organization form is used in the training stage and the test stage to help the model extract useful rules. Meta learning can well handle the differences between different tasks, thus ignoring the characteristics of specific tasks. There are many representative works on small sample learning in metric learning and meta learning at home and abroad. Among them, the prototype network is widely welcomed because of its simple idea and effective results. It calculates the average value of the feature vector in the support set as the prototype, and uses the Euclidean distance to compare the distance between the query sample and the prototype. Subsequently, Baseline++ further normalizes the feature vector and calculates the cosine similarity of the input feature and the weight matrix on the standard sphere. Relationship network further abstracts the similarity calculation and designs a special model to directly output the result. Similar work also includes graph neural network. In the graph neural network, prior knowledge is used to propagate the features and labels of the support set to the query set through the message propagation mechanism. In the aspect of meta learning, MAML proposes a training mechanism of scenario reproduction in the task scene. Some subsequent works have made a series of improvements on the basis of MAML, improving the ability of task space parameter initialization; MTFL uses meta learning strategy to reduce the number of tasks required for DNN fine-tuning to only 8000; MetaOptNet further uses linear support vector machine as inner loop to improve the representation of feature vector. Recently, Meta Baseline first comprehensively explores the advantages between meta learning and full classification learner, and proposes a new baseline model in the field of small sample learning.

[0005] The existing technology has different degrees of problems to solve small sample learning, firstly, the problem of feature extractor, the knowledge learned by traditional supervised learning is related to the label, and the feature expression of the underlying class which has not been seen is lack of generalization, which greatly restricts the development of small sample learning. Most of the work is based on transfer learning, trying to fix part of the parameters of the feature extractor, and fine-tune the parameters of the other part. Experimental results show that this method has obvious effect for data sets with small difference in domain. Another solution is to use multi-task learning to make up for the lack of small sample learning data. By collecting different types of related tasks, the loss function forces the optimization of different tasks to be synchronized, which will not make the parameters deliberately approximate any task, and the result will make the additional small sample task also perform well. Simple multi-task learning has limited effect on small sample learning, and combining meta-learning can better release this advantage. However, the existence of meta-learning will cause the problem of unstable parameter optimization, and the test set result will fluctuate greatly due to the sampling task. SUMMARY

[0006] The purpose of the present application is to provide a small sample image classification optimization method based on supervised contrast learning and multi-task setting to solve the problems of the prior art. The method uses label mask matrix and background confusion method, based on small sample meta baseline, takes supervised learning and multi-task as the breakthrough point, guides the feature extractor to learn more semantic information through multi-task learning, enhances the generality of meta knowledge, improves the accuracy of small sample learning in general image classification tasks, and has high practical value and good development prospect.

[0007] The specific technical solution to achieve the purpose of the present application is: a small sample image classification optimization method based on supervised contrast learning and multi-task setting, characterized in that the method mainly comprises:

[0008] 1) The label mask matrix and the background confusion method are used in instance contrast learning and its proxy task respectively. The biggest feature of supervised contrast learning is that the positive sample not only contains the target sample after background confusion, but also includes other samples of the same class. The label mask matrix and the background confusion method achieve the best result in the baseline method in the mini-imgeNet data set.

[0009] 2) Adopting the multi-task setting based on the small sample classifier, specifically including: predicting the image rotation angle, predicting the relative position of the image patch and the image embedding space clustering task, the auxiliary task can help the feature extractor learn more bottom-level semantic information to a certain extent. In the small sample classification stage, the relevant measurement method can better calculate the distance between the query sample and the support sample in the embedding space. A large number of experiments prove that the auxiliary task of image embedding clustering performs best in each benchmark data set.

[0010] The present application specifically includes the following three stages:

[0011] S1, feature extractor training stage

[0012] The label mask matrix M records the similarities and differences of the sample labels in each batch, and the label mask matrix M is introduced as a class supervision signal into instance contrast learning to participate in image representation training.

[0013] The feature extractor training of the step S1 specifically includes the following steps:

[0014] S1-1: Record the similarities and differences of the sample labels in each batch

[0015] The label mask matrix M records the similarities and differences of the sample labels in each batch, and the size of the label mask matrix M is [bsz, bsz], wherein bsz is the length of the data points in the batch, and the elements in the label mask matrix M are composed of 1 and 0. If the sample i and j have the same label, the value corresponding to the label mask matrix is 1, otherwise it is 0, that is, mask {i,j} = 1, mask {i,j} = 0.

[0016] S1-2: Supervised contrast learning

[0017] The range of positive sample pairs is expanded to other samples of the same class, and a supervised contrast loss is used to obtain the feature extractor F. In the calculation process of the loss function, only the number of the element value of 1 in the row or column where the benchmark sample is located needs to be found. The supervised contrast loss L sup is calculated by the following formula (a):

[0018]

[0019] In the formula, P(i) is the index set of other samples consistent with the label of the benchmark sample; F i is the feature vector corresponding to sample i, and F p and F a correspond to the feature vectors corresponding to samples p and a respectively; |P(i)| is the cardinality of the index set of other samples consistent with the label of the benchmark sample; A(i) is the index of other negative samples except the same class; γ is a scalar-temperature parameter;

[0020] By supervising the contrastive loss, a flexible enough feature extractor F is obtained. In the small sample classification stage, the support set C support will be sent into the feature extractor F to obtain the corresponding class prototype. If the 5-way 1-shot task, the feature vector of the support set C support is directly used as the class prototype, if the task is a 5-way 5-shot task, the average of the five feature vectors is calculated as the prototype according to the following formula (c):

[0021]

[0022] In the formula, x is a sample point in the support set C support ; |C support | is the cardinality of the support set; θ is the parameter of the feature extractor F; W n is the prototype vector in each task.

[0023] The feature extractor F and the cosine similarity classifier are trained together on the base class dataset. W b =[W1,W2,……,W n ] represents the d-dimensional prototype vector of each class in the support set. It should be noted that the calculation of the cosine similarity p(y=c|x) needs to normalize the feature vector, which is represented by the following formula (d):

[0024]

[0025] In the formula, < > represents the cosine similarity operation between two vectors after normalization; F θ (x) is the feature vector corresponding to the sample x, and similarly, f θ (x) is also its corresponding feature vector; w n is the element in the prototype matrix except the target prototype w n '.

[0026] In the small sample classification task, the similarity score of the query set sample and the prototype is between -1 and 1, and the closer the value is to 1, the more similar the two vectors are in the embedding space. On the contrary, the greater the difference between the two vectors. In the prediction stage, the images in the query set will be classified into the prototype closest to them. The label of the prototype is the label of the image in the query set.

[0027] S2: Agent task stage

[0028] Evaluate the contribution of traditional proxy tasks such as data augmentation, view transformation to instance contrast learning, design a background confusion method that is more suitable for small sample learning, and background cropping method mainly consists of two parts of saliency detection and background cropping. Background confusion method is different from traditional data augmentation and view transformation, and background confusion can enhance the foreground expression of the image. Assume that the entire base class dataset is C base During the entire training process, the dataset is organized in the form of N-way K-shot, where the support set C support has N categories, and each category has K samples. The query set C query represents the sample that needs to be classified. C support Each picture in C

[0029] 1) Saliency mapping module: The saliency mapping module freezes the pre-training weights of the pre-training model except the input layer, and calculates the gradient of the label score with respect to the input image. After obtaining the saliency score of each pixel in the input image, search for the highest scoring point among all pixels as the origin, and calculate the average value of the saliency score in the artificially set region. If it is greater than the foreground threshold, this region is considered to be found. Otherwise, if the average score is less than the threshold, the search needs to continue.

[0030] 2) Background confusion module: For each batch of data participating in the training of the base class dataset, sample an input data point and another data point randomly selected from other categories, and extract the foreground region using the saliency mapping module. With the help of matrix cropping operation, the foreground of the two data points is exchanged to obtain a new background confusion enhanced sample.

[0031] S3, small sample classification stage

[0032] On the basis of the cosine classifier, a multi-task setting is added to alleviate the instability of parameter optimization caused by meta-learning training. The small sample classification stage uses a multi-task setting to help the small sample classification task explore more rich semantic information.

[0033] Under the multi-task setting, small sample classification involves two optimization paths: one path is based on cosine similarity, and the parameter is represented as θ. The loss function is defined as L few . The other path is the auxiliary task, and the parameter is represented as The additional loss is L auxIt is particularly pointed out that in the task of predicting the image rotation angle and the relative position of the image, the data set quantity of the small sample classification task and the auxiliary task remains unchanged. In the auxiliary task of embedding space clustering, the data set of the support set is also subjected to a foreground enhancement module, and the sample quantity becomes twice that before. The loss function L of the whole multi-task setting is total is represented by the following formula (b):

[0034]

[0035] In the formula, alpha is L few and L aux The regularization hyperparameter weighting coefficient of the two loss functions. W b represents a set of prototype feature vectors. Theta and are the parameters of the feature extractor and the auxiliary task respectively.

[0036] Each parameter update of multi-task learning involves the parameters of the shared feature extractor F and the parameters of the respective classification head. The following specifically introduces the work of each multi-task.

[0037] 1) Predicting the image rotation angle task: In the auxiliary task, we preprocessed the images in the support set accordingly. The specific method is to rotate the image regularly and record the corresponding label. In order to facilitate training, each image is rotated by 0 degrees, 90 degrees, 180 degrees and 270 degrees respectively. Since no additional cost is required for labeling, the calculation amount generated by preprocessing can be ignored. The samples after rotation enhancement need to be converted into feature vectors by the feature extractor, and then input into the specially designed angle prediction network R to output the corresponding label. Considering the simplicity of the image rotation angle prediction task, a multi-layer fully connected neural network is used for implementation, and the hyperparameter alpha is set to 1.

[0038] 2) Predicting the relative position of the image patch: The conventional operation is to divide each support set image into 9 parts, and each patch is position-coded based on the center position, which is left upper, middle upper, right upper, left, right, left lower, middle lower and right lower. In the experiment, the relative position of the 8 possible positions except the center position can be predicted, and the possible results contain 8, but the correct result is only one. Considering that the task of predicting the relative position of the image block contains more possibilities, a higher-performance cosine classifier is used for prediction. The finer the image patch is divided, the less effective semantic information each block carries. The difficulty of the auxiliary task is also greater, and the 3x3 cutting is set in the present application, and the hyperparameter alpha is set to 1.

[0039] 3) Image embedding clustering: Since the above two types of auxiliary tasks do not participate in pre-training in the feature extractor stage, adding a new task requires adjusting the parameters from scratch. Inspired by the pre-training of stage one, we propose an image clustering auxiliary task using supervised contrast. In the embedding space, the image clustering task can help small sample learning to predict new categories well. From the perspective of the loss function, it is consistent with the optimization direction of the small sample classification task itself. The numerator of the supervised contrast loss is the distance between all positive sample pairs in the embedding space, and the denominator is the distance between all negative sample pairs in the embedding space. Only when the labels of the same category are constantly approaching and the labels of different categories are constantly diverging, the loss value of the image clustering task will become smaller and smaller, and the total loss value under the multi-task setting will become smaller and smaller. The hyperparameter alpha is set to 2.

[0040] The present application follows the learning framework of Meta Baseline, and mainly solves two significant problems existing in the pre-training stage and the classification stage of small sample learning. In the pre-training stage, instance contrast learning only focuses on its own attributes, resulting in a lack of global class boundary information for the feature extractor. By adding a label mask matrix, a supervised signal can be provided for instance supervised contrast learning. Secondly, in the small sample classifier stage, the cosine similarity under the meta-learning framework has the problem of unstable parameter optimization, and the test set result is greatly affected by the sampling task fluctuations. In order to alleviate this problem, a multi-task experimental setting is also proposed. In summary, the present application mainly studies the feature expression of the encoder facing supervised contrast learning in the direction of small samples, and carries out research from the perspective of background confusion and multi-task.

[0041] Compared with the prior art, the present application has the advantages of being based on a small sample meta baseline method, taking supervised learning and multi-task as the starting point, designing a label mask matrix, a background confusion method, and multi-task learning, guiding the feature extractor to learn more semantic information, enhancing the generality of meta-knowledge, and improving the accuracy of small sample learning in general image classification tasks. The method is simple, has good use effect, has high practical value and good development prospect. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a schematic diagram of the architecture of the present application;

[0043] Figure 2 is a schematic diagram of the prediction image rotation angle task;

[0044] Figure 3 is a schematic diagram of the prediction image patch relative position task;

[0045] Figure 4 is a schematic diagram of the image embedding clustering task. DETAILED DESCRIPTION

[0046] The application will be described in detail below with reference to the accompanying drawings and examples. Obviously, the examples listed are only used to explain the application, and are not used to limit the scope of the application.

[0047] Example 1

[0048] Referring to Figure 1 , the application specifically includes the following steps:

[0049] S1, a feature extractor training stage, records the similarities and differences of sample labels within each batch by means of a label mask matrix M. The elements in the label mask matrix M are composed of 1 and 0. If samples i and j have the same label, the corresponding value of the label mask matrix M is 1, and vice versa. The label mask matrix M is introduced as a class supervision signal into instance contrast learning to participate in image representation training.

[0050] The label mask matrix M expands the range of positive sample pairs to other samples of the same class in supervised contrast learning. In the calculation process of the loss function, only the number of the element value of 1 in the row or column where the reference sample is located needs to be found. Specifically, the supervised contrast loss is calculated by the following formula (a):

[0051]

[0052] In the formula, P(i) is the index set of other samples consistent with the label of the reference sample; |P(i)| is the cardinality of the index set of other samples consistent with the label of the reference sample; A(i) is the index of other negative samples except the same class; F i is the feature vector corresponding to sample i. Similarly, F p and F a correspond to the feature vectors corresponding to samples p and a respectively; γ is a scalar-temperature parameter.

[0053] Through the supervised contrast loss, a flexible enough feature extractor F is obtained. In the small sample classification stage, the support set C support will be sent to the feature extractor F to obtain the corresponding class prototype. If the task is a 5-way 1-shot task, the feature vector of the support set C support is directly used as the class prototype. If the task is a 5-way 5-shot task, the average of the five feature vectors is calculated as the prototype according to the following formula (c):

[0054]

[0055] In the formula, x is a sample point in the support set C support ; |C support | is the cardinality of the support set; θ is the parameter of the feature extractor F; W n is the prototype feature vector representing the task.

[0056] The feature extractor F and the cosine similarity classifier are trained together on the base dataset C base W b = [W1, W2, …, W n ] represents the d-dimensional prototype vector of each class in the support set C support It is worth noting that the calculation of cosine similarity requires normalization of the feature vector, and its formula is specifically represented by the following (d) formula:

[0057]

[0058] In the formula, < > represents the cosine similarity operation between two vectors after normalization. θ (x) and f θ (x) represent the feature vectors corresponding to x respectively; w n is an element in the prototype matrix except the target prototype w n '.

[0059] In the small sample classification task, the query set C query The similarity scores of the samples and the prototypes are both between -1 and 1. The closer the value is to 1, the more similar the two vectors are in the embedding space. Conversely, the greater the difference between the two vectors. In the prediction phase, the images in the query set C query will be classified into the prototype closest to it. The label of the prototype is the label of the image in the query set C query .

[0060] S2: Evaluate the contribution of traditional proxy tasks such as data augmentation and view transformation to instance contrast learning, and design a background confusion method that is more suitable for small sample learning. It is different from traditional data augmentation and view transformation. Background confusion can enhance the foreground expression of images.

[0061] Assume that the entire base dataset is C base During the entire training process, the dataset is organized in the form of N-way K-shot, where the support set C support has N classes, and each class has K samples. The query set C query represents the samples that need to be classified. Each picture in C support can transfer the foreground to the pictures of other classes to form augmented samples. Specifically, the background confusion method includes two main parts:

[0062] 1) saliency mapping module: the saliency mapping module freezes the pre-training weights of the pre-training model except the input layer, calculates the gradient of the label score with respect to the input image, and after obtaining the saliency score of each pixel in the input image, searches for the highest scoring point in all pixels as the origin, and calculates the average value of the saliency score in the artificially set region, if it is greater than the foreground threshold, this region is considered to be found; otherwise, if the average value of the score is less than the threshold, the search needs to be continued.

[0063] 2) background confusion module: for the base class dataset C base For each batch of data participating in training, sample an input data point and another data point randomly selected from other classes, and extract the foreground region by using the saliency mapping module. By means of matrix clipping operation, the foreground of the two data points is exchanged to obtain a new background confusion enhanced sample.

[0064] The pseudo code of the saliency mapping implementation background confusion algorithm is shown in Table 1 as follows:

[0065] Table 1 pseudo code description as follows:

[0066]

[0067] S3, a small sample classification stage, adds a multi-task setting on the basis of the cosine classifier, which alleviates the instability of parameter optimization caused by meta-learning training, and at the same time, assists the small sample classification task to explore more rich semantic information.

[0068] Referring to Figure 1 , the small sample classification under the multi-task setting involves two optimization paths, one path is based on cosine similarity, and the parameter is represented as θ, and the loss function is defined as L few . The other path is an auxiliary task, and the parameter is represented as The additional loss is L aux It is particularly pointed out that in the task of predicting the rotation angle of the image and the relative position of the image, the number of datasets of the small sample classification task and the auxiliary task remains unchanged. In the auxiliary task of embedding space clustering, the support set C support , the base class dataset is C base It will also pass through a foreground enhancement module, and the number of samples will be doubled. The loss function of the whole multi-task setting is represented by the following formula (b):

[0069]

[0070] In the formula, α is the regularization hyperparameter weighting coefficient of L few and L aux two loss functions; W b represents the set of prototype feature vectors; θ and These are the parameters for the feature extractor and the auxiliary task, respectively.

[0071] Each parameter update in multi-task learning involves the parameters of the shared feature extractor F and the parameters of each classifier head. The following section details the various tasks involved in multi-task learning.

[0072] 1) Image rotation angle prediction task

[0073] See also Figure 2 In auxiliary tasks, support set C support The images in the dataset underwent preprocessing. Specifically, the images were rotated regularly and their corresponding labels were recorded. For ease of training, each image was rotated by 0 degrees, 90 degrees, 180 degrees, and 270 degrees. The overall framework is as follows: Figure 2 As shown. Since no additional cost is required for labeling, the computational cost of preprocessing is negligible. The rotated samples need to be transformed into feature vectors by the feature extractor F, and then fed into a specially designed angle prediction network R to output the corresponding labels. Considering the simplicity of the image rotation angle prediction task, a multi-layer fully connected neural network is used for implementation, with the hyperparameter α set to 1.

[0074] 2) Predict the relative positions of image patches

[0075] See Figure 3 The figure illustrates the framework for predicting the relative positions of image patches. The conventional approach involves dividing each support set image into nine equal parts, with each patch encoded based on its center position: top left, top center, top right, left, right, bottom left, bottom center, and bottom right. In experiments, relative position prediction was performed on all eight possible positions other than the center, resulting in eight possible outcomes, but only one correct result. Considering the greater number of possibilities in predicting the relative positions of image patches, a higher-performance cosine classifier was used. The finer the image patch is divided, the less effective semantic information each patch carries, increasing the difficulty of the auxiliary task. This invention uses a 3x3 segmentation, with the hyperparameter α set to 1.

[0076] 3) Image Embedding Clustering: Since the above two types of auxiliary tasks did not participate in the pre-training of the feature extractor F stage, adding a new task requires adjusting the parameters from scratch. Inspired by the pre-training in stage one, a supervised contrastive approach is used to propose an image clustering auxiliary task.

[0077] See Figure 4In the embedding space, the image clustering task can help small sample learning to make good prediction of new categories. From the perspective of loss function, it is consistent with the optimization direction of the classification task itself. The numerator of the supervised contrast loss is the distance between all positive sample pairs in the embedding space, and the denominator is the distance between all negative sample pairs in the embedding space. Only when the labels of the same category are constantly close to each other and the labels of different categories are constantly far away from each other, the loss value of the image clustering task will become smaller and smaller, and the total loss value under the multi-task setting will become smaller and smaller. The hyperparameter a is set to 2.

[0078] The above specific implementation is only for further illustration of the present application, and is not used to limit the present application patent. Any equivalent implementation of the present application shall be included in the scope of the claims of the present application patent.

Claims

1. A few-sample image classification optimization method based on supervised contrastive learning and multi-task setting, characterized in that, This method specifically includes the following three stages: S1, Feature Extractor Training Phase The label mask matrix M is used to record the differences between the labels of samples in each batch, and the label mask matrix M is used as a category supervision signal to be introduced into instance contrastive learning to participate in image representation training. The elements in the label mask matrix M are all composed of 1 and 0. If samples i and j have the same label, the value of the label mask matrix is ​​1, and otherwise it is 0. S2: Agent Task Phase The label mask matrix M and the background obfuscation method are applied to instance comparison learning and its proxy task, respectively. S3, Small Sample Classification Stage A multi-task setting is added to the few-shot classifier for few-shot classification. The multi-task setting specifically includes: predicting the image rotation angle, predicting the relative position of the image patch, and image embedding space clustering. In step S3, the few-sample classification under the multi-task setting uses cosine similarity and an auxiliary task. The parameters based on cosine similarity are expressed as follows: The loss function is defined as The parameters of the auxiliary task are expressed as follows: The additional loss is The multi-task setup is configured such that in the tasks of predicting image rotation angles and relative image positions, the dataset size remains unchanged for the few-shot classification task and the auxiliary task. In the auxiliary task of embedding spatial clustering, the support set... and base class dataset The foreground enhancement module doubles the number of samples, and the loss function in the entire multi-task setup... It can be expressed by the following equation (b): （b）； In the formula, for and The weighting coefficients of the regularization hyperparameters for the two loss functions; The feature vector represented by each prototype; Each parameter update in multi-task learning involves the parameters of the shared feature extractor F and the parameters of each classifier head.

2. The few-sample image classification optimization method based on supervised contrastive learning and multi-task setting according to claim 1, characterized in that, The feature extractor training in step S1 specifically includes the following steps: S1-1: Record the differences in sample labels within each batch. The label mask matrix M records the differences in labels among samples within each batch. The size of the label mask matrix M is [bsz, bsz], where bsz is the length of the data points within the batch. If samples i and j have the same label, the corresponding value of the label mask matrix M is 1; otherwise, it is 0. , ; S1-2: Supervised Comparative Learning The range of positive sample pairs is expanded to other samples of the same category. Using supervised contrastive loss, a feature extractor F is obtained. In the calculation of the loss function, it is only necessary to find the index of the row or column where the element with a value of 1 is located, i.e., the supervised contrastive loss. Calculated by the following formula (a): （a）； In the formula, P(i) is the set of other sample indices that have the same label as the baseline sample; |P(i)| is the cardinality of the set of other sample indices that have the same label as the baseline sample; Indexes other negative samples besides those in the same category; It is a scalar-temperature parameter.

3. The few-sample image classification optimization method based on supervised contrastive learning and multi-task setting according to claim 1, characterized in that, The background obfuscation method in step S2 is to enhance the foreground representation of the image, assuming the entire base class dataset is... The query set is Throughout the training process, Organized into an N-way K-shot format, in which the support set There are N categories, each with K samples, and a query set. This represents the samples that need to be classified, and supports sets. Each image in the dataset has its foreground shifted onto images of other categories to create enhanced samples. The background obfuscation method includes two parts: saliency detection and background cropping. The saliency detection uses a saliency mapping module to freeze the pre-trained weights of the pre-trained model except for the input layer, calculates the gradient of the label score relative to the input image, and after obtaining the saliency score of each pixel in the input image, searches for the point with the highest score among all pixels as the origin. It then calculates the average saliency score within a manually set region. If the average score is greater than a foreground threshold, the region is considered found; otherwise, if the average score is less than the threshold, the search continues. The background cropping uses a background obfuscation module for the base class dataset. For each batch of data involved in training, the input data points are sampled and another data point is randomly selected from other categories. The foreground region is extracted using the saliency mapping module. Then, the foreground of the two data points is swapped using matrix clipping operations to obtain a new enhanced sample with background confusion.

4. The few-sample image classification optimization method based on supervised contrastive learning and multi-task setting according to claim 1, characterized in that, Step S3 supports the set of data in the few-sample classification stage. The data is fed into feature extractor F to obtain the corresponding category prototype; if it is a 5-way 1-shot task, the support set is... The feature vectors are directly used as the class prototypes; if it is a 5-way 5-shot task, the average of the 5 feature vectors is calculated according to the following formula (c). As a prototype: （c）； In the formula, x is the support set. A sample point; | |for The cardinality; These are the parameters of the feature extractor F; The feature extractor F and the cosine similarity classifier work together on the base class dataset. Training was conducted on [the platform]. Indicates support set The d-dimensional prototype vector of each category; The calculation of the cosine similarity requires normalization of the feature vectors, as expressed by the following equation (d): （d）； In the formula, < > represents the cosine similarity operation between two vectors after normalization; For the prototype matrix excluding the target prototype ' element; and These are the feature vectors corresponding to sample x and the query set. The similarity scores between the sample and the prototype range from -1 to 1. The closer the score is to 1, the more similar the two vectors are in the embedding space; conversely, the further away the score is, the greater the difference between the two vectors. During the prediction phase, the query set... Images in the set are grouped into the closest prototype, and the prototype's label is the query set. Labels for images in the middle.

5. The few-sample image classification optimization method based on supervised contrastive learning and multi-task setting according to claim 1 or claim 4, characterized in that, The task of predicting image rotation angle, in the auxiliary task, uses the support set... The images are preprocessed by rotating them at 0°, 90°, 180°, and 270° and recording the corresponding labels. The rotated and enhanced samples are then transformed into feature vectors by a feature extractor F, and the corresponding labels are output by an angle prediction network R. The angle prediction network R uses a multi-layer fully connected neural network with hyperparameters... Setting it to 1 predicts the relative position of the image patches for each support set. The image is divided into 9 parts, and each patch is coded based on the center position, namely top left, top center, top right, left, right, bottom left, bottom center, and bottom right. The relative position is predicted for the 8 positions other than the center position.

6. The few-sample image classification optimization method based on supervised contrastive learning and multi-task setting according to claim 1 or claim 4, characterized in that, The image embedding clustering proposed uses supervised contrastive analysis to assist in image clustering. The numerator of the supervised contrastive loss is the distance between all positive sample pairs in the embedding space, and the denominator is the distance between all negative sample pairs in the embedding space. Only when labels of the same category move closer together and labels of different categories move further apart will the loss value of the image clustering task decrease, and only then will the total loss value decrease under a multi-task setting. Its hyperparameters... Set to 2.

Images