Semi-supervised few-shot image classification learning method and system based on negative label learning

Abstract

The application discloses a semi-supervised few-shot image classification learning method and system based on anti-label learning, and the method comprises the following steps: constructing a meta task, using a pre-trained neural network as a feature extractor, extracting the features of corresponding data in the meta task, and training a classifier on a support set for subsequent classification tasks; an anti-label learning module gives no-label data an anti-label with a higher correct rate, and the classifier learns and updates on the anti-label, and iterates until the anti-label cannot be selected; after the iteration of the anti-label module ends, a positive label with balanced classes and a higher correct rate is obtained, and the classifier is used for learning and updating. The application fully and high-quality uses no-label data, and can obtain a higher few-shot learning image classification accuracy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to a semi-supervised few-shot image classification learning method and system based on reverse label learning. BACKGROUND

[0002] With the development of deep learning, convolutional neural networks have surpassed human level in many image tasks, but the training of these models relies on a large amount of data, and the collection of some data is difficult in real life, such as the collection of data on endangered birds and the like, and in addition, the labeling of these data also requires a large amount of manpower and financial resources. In contrast, the human visual system can quickly learn new concepts and features from a small number of examples, and then identify similar objects in new data. In order to imitate the ability of humans to quickly learn, and reduce the dependence of the method on data, few-shot learning has received more and more attention in recent years. Few-shot learning aims to combine prior knowledge to quickly generalize to new tasks that contain only a small amount of supervised information. In this setting, only a few labeled samples are needed to identify each class, so the cost of manual labeling can be greatly reduced.

[0003] Based on the few-shot learning setting with less data, a problem that needs to be faced is that it is difficult to make the model better fit the distribution of the data with very few labeled data. Therefore, in order to solve this problem, the research direction of combining semi-supervised learning has appeared in few-shot learning.

[0004] In addition, in order to solve the problem of difficult data labeling, the method of reverse label learning has also emerged. Reverse label, as the name implies, is to label the data with the opposite label, which is an indirect way to represent that the data does not belong to a certain class. This approach can greatly reduce the error of data labeling, for example, for a 5-class problem, the probability of error of labeling data with the true label (positive label) is 4 times the probability of error of labeling data with the reverse label. In addition, in semi-supervised few-shot learning, since there are very few labeled data, the model is difficult to have good results in the initial stage. Labeling pseudo-labels for unlabeled data with such a model will result in a large number of errors and class imbalance. In such a case, the method of combining reverse label learning can solve this problem. The semi-supervised few-shot learning method based on reverse label learning studied in the present application is designed for semi-supervised few-shot learning, and a suitable reverse label labeling method is designed, and the problems of insufficient use of unlabeled data in semi-supervised few-shot learning are solved by combining reverse label learning.

[0005] At present, there are many methods for studying semi-supervised few-shot learning, but there are still some problems: 1) the accuracy of labeling pseudo-labels for unlabeled data is low, and the incorrectly labeled samples will affect the final result; 2) the pseudo-labels labeled on the unlabeled data have the phenomenon of class imbalance; 3) the method is relatively complex. Summary of the Invention

[0006] The purpose of this invention is to provide a semi-supervised few-shot image classification learning method and system based on anti-label learning.

[0007] The technical solution to achieve the objective of this invention is as follows: Firstly, this invention provides a semi-supervised few-shot image classification learning method based on anti-label learning, comprising the following steps:

[0008] Step 1: Construct a meta-task. Use a pre-trained neural network as a feature extractor to extract image data. Extract features corresponding to the support set, query set, and unlabeled dataset in the meta-task. Train a classifier on the support set for subsequent image classification tasks.

[0009] Step 2: The anti-label learning module assigns anti-labels to the unlabeled image data with a high accuracy of 95%. The classifier learns and updates the anti-labels, and the process is iterated until no anti-labels can be selected.

[0010] Step 3: The positive label learning module obtains positive labels with balanced categories and an accuracy of up to 85%, and then uses a classifier to learn and update them.

[0011] Step 4: Use the trained classifier to predict the final image classification result on the query set.

[0012] Secondly, the present invention provides a semi-supervised few-shot image classification learning system based on anti-label learning, comprising:

[0013] The feature extraction module is used to construct the meta-task. It uses a pre-trained neural network as a feature extractor to extract image data, extracts features corresponding to the support set, query set, and unlabeled dataset in the meta-task, and trains a classifier on the support set for subsequent image classification tasks.

[0014] The anti-label learning module is used to assign anti-labels to unlabeled image data with the highest accuracy. The classifier learns and updates the anti-labels, and iterates until no anti-labels can be selected.

[0015] The positive label learning module is used to obtain positive labels that are class-balanced and meet the second accuracy requirement, and then uses a classifier to learn and update them.

[0016] The trained classifier is used to predict the final image classification category on the query set.

[0017] Thirdly, this application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect above.

[0018] In a fourth aspect, the present application also provides a computer readable storage medium, having stored thereon a computer program, which when executed by a processor implements the method of the first aspect.

[0019] In a fifth aspect, the present application also provides a computer program product comprising a computer program which when executed by a processor implements the method of the first aspect.

[0020] Compared with the prior art, the present application has the following advantages: (1) the anti-label learning module designed by the present application greatly reduces the error rate of labeling the unlabeled image data by labeling the anti-labels and learning in the initial stage when the model effect is not good; (2) after the anti-label learning module, the positive label learning module designed by the present application can obtain a positive label with high accuracy and balanced categories, and continue to train the model; (3) compared with the previous method, the method proposed by the present application is simple in process, can more fully and high-quality use of unlabeled image data for learning, and finally obtains better effect on image classification task. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The flow chart of the semi-supervised few-shot image classification learning method based on anti-label learning of the present application. DETAILED DESCRIPTION

[0022] In combination Figure 1 A semi-supervised few-shot image classification learning method based on anti-label learning, specifically comprising the following steps:

[0023] Step 1, constructing a meta-task, using a pre-trained neural network as a feature extractor for extracting image data, extracting the features corresponding to the support set, query set and unlabeled image data set in the meta-task, and training a classifier on the support set for subsequent image classification tasks;

[0024] First, the features of the corresponding data sets in the meta-task are extracted by a convolutional neural network:

[0025] x set∈{S,Q,U} =F(I;θ r ) (1)

[0026] Where I is the input data, F(·; θ r ) is a pre-trained convolutional neural network model, and θ r is the parameter of the model. x set is the feature extracted from the set, set can be S, Q or U, representing the support set, query set and unlabeled data set respectively.

[0027] Then initialize the classifier f(·; θ c ), where θ c is the parameter of the classifier. Map x S to the corresponding probability space using the classifier:

[0028] p S = f(x S ; θ c ) (2)

[0029] Then train using the cross-entropy loss, which is expressed as follows:

[0030]

[0031] Step 2, build the anti-label learning module to give the unlabeled image data an anti-label with 95% accuracy, and learn and update the classifier on the anti-label through continuous iteration until the anti-label cannot be selected;

[0032] First, use the classifier f(·; θ c ) to map the unlabeled data to the probability space of the class:

[0033] p U = f(x U ; θ c ) (4)

[0034] where x U is the feature of the unlabeled data obtained in formula (1), p U is the class posterior probability value obtained after the unlabeled data is mapped by the classifier, and the superscript U represents the unlabeled data. The first iteration of the anti-label learning module is as follows:

[0035]

[0036] where k represents the corresponding position of the smallest probability in p U , i.e., the class with the smallest probability, is the probability value of the corresponding class, δ is the threshold for selecting the anti-label, is the anti-label of the current input data, i.e., when the probability value of the class with the smallest probability is less than the threshold, the anti-label is 1, and the anti-label class is k at this time. When the condition is not met, the sample does not participate in this round of iteration.

[0037] After the first iteration, the set of unlabeled data marked with the anti-label is obtained where i represents the index of the selected anti-label data, and N j is the number of data selected according to formula (5) in the jth iteration, where j represents the number of iterations. and respectively represent the feature vector of the i-th selected data and the corresponding counter-label.

[0038] In the counter-label learning module, an iterative strategy of selecting counter-labels is designed according to the characteristics of few-shot learning. For an input data, the labeled counter-label category will not be considered in the subsequent iteration process. Here, a list is defined to record the counter-labels selected in each iteration for each data:

[0039]

[0040] wherein is the labeled counter-label of the sample that meets the condition in this iteration. Some samples cannot meet the condition. Here, in order to show all cases, some lists are set to empty. M is the data quantity of the unlabeled data set U, and the number of categories is C, that is

[0041] Next, the cross-entropy loss function of formula (2) is modified so that it can be used to train the classifier f(·; θ c ) on , which is as follows:

[0042]

[0043] Thus, the first round of iteration in the counter-label learning module ends. Then, the above steps are repeated until no counter-label data that meets the condition can be selected in the iteration, i.e., the iteration stops.

[0044] The counter-label learning module in step 2 labels the unlabeled image data through iteration. After the iteration ends, the labels that are extremely uncertain for the data are excluded, i.e., the counter-labels selected in the iteration. The positive label learning module judges the Position list. For an input data, if it only has one label that has not been labeled as a counter-label, the label is taken as the positive label of the data. If there are multiple labels that have not been labeled as counter-labels, the sample is discarded and does not participate in the subsequent process. The data set with positive labels is obtained wherein L is the number of data that meet the condition and are selected to label the positive label. Next, the classifier f(·; θ c ) is trained on using formula 2.

[0045] Step 4 uses the trained classifier to predict the final category result on the features of the query set.

[0046] First, use the classifier f(·; θ c ) to map the features of the query set to the probability space related to the category:

[0047] p Q =f(x Q ;θ c ) (8)

[0048] where x Q is the feature of the query set obtained by formula (1), then the output probability p Q is processed to obtain the class of the input data:

[0049]

[0050] where i represents the position of the data in the query set, and the query set data has E, so here i∈[0, 1, 2,..., E].

[0051] The semi-supervised few-shot image classification learning method based on the anti-label learning of the application can provide more high-quality image data labels by using the anti-label learning module and the positive label learning module, and finally improve the accuracy of the few-shot image classification task.

[0052] The application can be used in specific embodiments, for example, for the few-shot bird data image classification task, the following describes this embodiment.

[0053] Embodiment

[0054] Consistent with the above image classification step, first, the features of the corresponding few-shot bird data set in the meta-task are extracted by the convolutional neural network:

[0055] x set∈{S,Q,U} =F(I;θ r ) (1)

[0056] where I is the input bird image data, F(·; θ r ) is a pre-trained convolutional neural network model, and θ r is the parameter of the model. x set is the bird image feature extracted by the set, and set can be S, Q or U, representing the support set, the query set and the unlabeled data set.

[0057] Then, the classifier f(·; θ c ) is initialized for bird image classification, where θ c is the parameter of the classifier. The classifier is used to map x S to the corresponding probability space:

[0058] p S =f(x S ;θ c ) (2)

[0059] Then the training is performed using cross-entropy loss, which is expressed as follows:

[0060]

[0061] In step 2, the unlabeled bird image data is mapped to the probability space of the class using the classifier f(·; θ c ):

[0062] p U U ; θ c ) (4)

[0063] where x U is the feature of the unlabeled bird image obtained in formula (1), p U is the posterior probability value of the class obtained after the classifier is mapped to the unlabeled bird image data, and the superscript U represents the unlabeled bird image data. The first iteration of the anti-label learning module is performed as follows:

[0064]

[0065] where k represents the position corresponding to the minimum probability in p U , i.e., the class corresponding to the minimum probability, and represents that the class is the least likely class of the current bird image data, is the probability value of the corresponding class, and δ is the threshold for selecting the anti-label, is the anti-label of the current input bird data, i.e., when the probability value of the class with the minimum probability is less than the threshold, the anti-label is 1, and the anti-label class is k at this time, and when the condition is not met, the sample does not participate in the iteration of this round.

[0066] After the first iteration, the set of anti-labeled data in the unlabeled data is obtained where i represents the index of the selected anti-labeled data, and N j is the number of data selected according to formula (5) in the jth iteration, where j represents the number of iterations. and represent the feature vector of the i-th selected data and the anti-label of the corresponding bird data, respectively.

[0067] In the anti-label learning module, according to the above algorithm description, the anti-label of the bird data can be iteratively selected. For an input data, the anti-labeled class that has been labeled will not be considered in the subsequent iteration process. Here, a list is defined to record the anti-label selected in each iteration for each bird data:

[0068]

[0069] ​in This is the inverse label for the samples that meet the conditions in this iteration. Some samples do not meet the conditions; therefore, some lists are set to empty to represent all cases. M represents the amount of data in the unlabeled bird dataset U, and the number of categories is C, i.e.

[0070] Next, the cross-entropy loss function in formula (2) will be modified so that it can be used in... Train the classifier f(·θ) c This is used for subsequent few-sample bird classification tasks, in the following format:

[0071]

[0072] This concludes the first iteration of the anti-label learning module. The above steps are then repeated until no anti-label data that meets the conditions can be selected in the iteration, at which point the iteration stops.

[0073] The positive label learning module evaluates the Position list. For each input data point, if only one label remains unlabeled, that label is used as the positive label for that bird data point. If multiple labels remain unlabeled, the sample is discarded and not included in subsequent processes. This results in a dataset with positive labels. Where L represents the number of data points that meet the criteria and are selected to be labeled positively. Next, the classifier f(·θ) is used. c )exist Then use formula (2) for training.

[0074] In the next step, the classifier f(·θ) is used. c This maps the query set features to a probability space associated with bird categories:

[0075] p Q =f(x) Q ;θ c (8)

[0076] Where x Q The features of the query set obtained by formula (1) are then used to determine the output probability p. Q After processing, the category of the input bird data is obtained as follows:

[0077]

[0078] Where i represents the position of the data in the query set, and there are E data in the query set, then i∈[0,1,2,...,E].

[0079] Through the above steps, the specific category of the few-shot bird image can be obtained. In the specific implementation process, the few-shot bird data set is constructed. Compared with the past method, the correct rate of the bird data is greatly improved in the case of relatively simple method flow. Compared with the traditional method, the correct rate of 94% is 4% higher than the correct rate of 90% in the final bird image classification task.

[0080] Based on the same inventive concept, the embodiment of the present application also provides a semi-supervised few-shot image classification learning system based on counter-label learning, comprising:

[0081] A feature extraction module is configured to construct a meta-task, use a pre-trained neural network as a feature extractor to extract image data, extract features corresponding to a support set, a query set and a no-label data set in the meta-task, and train a classifier on the support set for subsequent image classification tasks.

[0082] A counter-label learning module is configured to label no-label image data with a counter-label at a first correct rate, learn and update the counter-label using the classifier, and continuously iterate until the counter-label cannot be selected.

[0083] A positive label learning module is configured to obtain a positive label that is balanced in categories and meets a second correct rate, and learn and update the positive label using the classifier.

[0084] A trained classifier is used to predict the category result of the final image classification on the query set.

[0085] The specific implementation of each module in the semi-supervised few-shot image classification learning system is the same as the steps of the real-time infrared imaging simulation method described above, and will not be repeated here. The first correct rate is 95%, and the second correct rate is 85%.

[0086] In one embodiment, an electronic device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the semi-supervised few-shot image classification learning method based on counter-label learning described above when executing the program.

[0087] In one embodiment, a computer-readable storage medium is provided, which stores a computer program executable by a processor to implement the semi-supervised few-shot image classification learning method based on counter-label learning described above.

[0088] In one embodiment, a computer program product is provided, comprising a computer program executable by a processor to implement the semi-supervised few-shot image classification learning method based on counter-label learning described above.

[0089] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A semi-supervised few-shot image classification learning method based on anti-label learning, characterized in that, Includes the following steps: Step 1: Construct a meta-task. Use a pre-trained neural network as a feature extractor to extract image data. Extract features corresponding to the support set, query set, and unlabeled dataset in the meta-task. Train a classifier on the support set for subsequent image classification tasks. Step 2: The anti-label learning module assigns anti-labels to the unlabeled image data with the first accuracy, and uses a classifier to learn and update the anti-labels. This process is repeated until no anti-labels can be selected. Construct an anti-label learning module. This module iteratively processes unlabeled data to select anti-labels and uses them for classifier learning until no more data that meets the criteria can be selected, at which point the iteration ends. First, use the classifier f(·θ) c ), where θ c For the classifier parameters, map unlabeled data to the probability space of categories: p U =f(x U ; i c ) (4) Where x U p represents the unlabeled data features obtained in formula (1). U The class posterior probability value is the class obtained after mapping unlabeled data through a classifier. The superscript U represents unlabeled data. For the first iteration of the anti-label learning module, the anti-label method was selected as follows: Where k represents p U The position corresponding to the lowest probability is the category with the lowest probability. δ represents the probability value for the corresponding category, and δ serves as the threshold for selecting the anti-label. The inverse label of the current input data is set to 1 when the probability value of the class with the lowest probability is less than the threshold. In this case, the inverse label class is k. If the condition is not met, the sample will not participate in the current iteration. After the first round of iterations, a set of unlabeled data with anti-labeled data was obtained. Where i represents the index of the selected inverse-label data, and N j The number of data selected according to formula (5) in the j-th iteration is given by the number of iterations. and These represent the feature vector of the selected i-th data and its corresponding inverse label, respectively. In the anti-label learning module, an iterative strategy for selecting anti-labels is designed. For an input data set, categories that have already been labeled with anti-labels will not be considered again in subsequent iterations. A list is defined to record the anti-labels selected for each data set in each iteration. in The inverse label is assigned to the samples that meet the conditions in this iteration. Some samples do not meet the conditions, so some lists are set to empty; M is the amount of data in the unlabeled dataset U, and the number of classes is C, i.e. The cross-entropy loss function is modified to be used in... Train the classifier f(·θ) c ), in the following form: This concludes the first iteration. The aforementioned steps are then repeated until no unlabeled data that meets the conditions can be selected during the iteration, at which point the iteration stops. Step 3: The positive label learning module obtains positive labels that are class-balanced and meet the second accuracy requirement, and then uses a classifier to learn and update them. Step 4: Use the trained classifier to predict the final image classification result on the query set.

2. The semi-supervised few-shot image classification learning method based on anti-label learning according to claim 1, characterized in that, In step 1, a pre-trained convolutional neural network model is used to extract features from the support set, query set, and unlabeled image dataset in the meta-task. Then, a classifier is trained on the features of the support set, as follows: Extracting features from the corresponding image dataset in the meta-task using a convolutional neural network: x set∈{S,Q,U} =F(I;θ r ) (1) Where I is the input data, F(·;θ) r ) represents a pre-trained convolutional neural network model, θ r The parameters of this model; x set Features are extracted from the set, where set can be S, Q, or U, representing the support set, query set, and unlabeled dataset, respectively. Initialize the classifier f(·; θ) c ); Use a classifier to classify x S Mapping to the corresponding probability space: p S =f(x S ;θ c ) (2) Training is performed using cross-entropy loss, which is represented as follows: L(f,y)=-∑ k y k p k (3)。 3. The semi-supervised few-shot image classification learning method based on anti-label learning according to claim 1, characterized in that, In step 2, the anti-label learning module iteratively assigns anti-labels to the unlabeled image data. After the iteration ends, it excludes labels that are highly unbelievable to the data, i.e., the anti-labels selected in the iteration. The positive label learning module judges the Position list. For an input data, if there is only one label that has not been assigned an anti-label, that label is taken as the positive label of the data. If there are multiple labels that have not been assigned anti-labels, the sample is discarded and will not participate in the subsequent process. Get a dataset with positive labels Where L is the number of data points that meet the criteria and are selected to be labeled positively; next, the classifier f(·θ) is used. c )exist Then use formula (2) for training.

4. The semi-supervised few-shot image classification learning method based on anti-label learning according to claim 1, characterized in that, Step 4 uses the trained classifier to predict the final category result based on the features of the query set, as detailed below: First, use the classifier f(·θ) c This maps the query set features to a probability space related to the category: p Q =f(x Q ;θ c ) (8) Where x Q The features of the query set obtained by formula (1); Next, the output probability p Q After processing, the categories of the input data are obtained as follows: Where i represents the position of the data in the query set, and there are E data in the query set, then i∈[0,1,2,...,E].

5. The semi-supervised few-shot image classification learning method based on anti-label learning according to claim 1, characterized in that, The first accuracy rate was 95%, and the second accuracy rate was 85%.

6. A semi-supervised few-shot image classification learning system based on anti-label learning, characterized in that, The system for implementing the method of claim 1 includes: The feature extraction module is used to construct the meta-task. It uses a pre-trained neural network as a feature extractor to extract image data, extracts features corresponding to the support set, query set, and unlabeled dataset in the meta-task, and trains a classifier on the support set for subsequent image classification tasks. The anti-label learning module is used to assign anti-labels to unlabeled image data with the highest accuracy. The classifier learns and updates the anti-labels, and iterates until no anti-labels can be selected. The positive label learning module is used to obtain positive labels that are class-balanced and meet the second accuracy requirement, and then uses a classifier to learn and update them. The trained classifier is used to predict the final image classification category on the query set.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1-5.

Images