A task-adaptive small sample image classification method based on meta-transfer learning

Through the task adaptive method based on metatransfer learning, the problem of insufficient and imbalance in the meta-learning method is solved, the accuracy and adaptability of small sample image classification are improved, and effective classification in real-life scenarios is realized.

CN114511739BActive Publication Date: 2024-11-05HARBIN ENG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210089412.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-25
Publication Date
2024-11-05
Estimated Expiration
2042-01-25

AI Technical Summary

Technical Problem

The existing meta-learning method uses shallow network to extract features inadequately and fails to effectively deal with the problems of task imbalance, category imbalance and distribution imbalance in small sample learning in real scenarios.

Method used

Using a task adaptive method based on metatransfer learning, the weights of feature extractors and classifiers are obtained through pre-training deep networks, the feature extractors are frozen, and the meta-training sets and meta-test sets are used for training. It is difficult to select samples with low accuracy. Task, update metatransfer learning parameters and balance parameters, and improve the performance of the model in difficult tasks.

Benefits of technology

The accuracy of the model in small sample image classification is improved, the problems of insufficient and imbalance of feature extraction are solved, and the adaptability and generalization ability of the model in real scenarios is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114511739B_ABST
    Figure CN114511739B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of computer technology, and specifically relates to a task-adaptive small sample image classification method based on meta-transfer learning. The present invention compensates for the problem of insufficient feature extraction of the MAML model using a 4Conv shallow network by combining meta-transfer learning; trainable parameters are added to learn to balance the use of meta-knowledge in each task, solving the problems of task imbalance, category imbalance, and distribution imbalance in small sample learning in real scenarios. The present invention selects samples with low accuracy in each task and recombine their data to make it a more difficult task, so that the meta-learner improves the accuracy of the model in the process of learning more difficult tasks. The difficult task mining algorithm proposed in the present invention collects samples with poor classification effects online to form difficult tasks, allowing the learner to learn faster and better in difficult tasks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a task-adaptive small sample image classification method based on meta-transfer learning. Background Art

[0002] Deep learning has achieved great success in various recognition tasks. However, deep neural networks have many parameters and require sufficient labeled samples to train the models, which severely limits their scalability. For many rare classes, it is not feasible to collect a large number of training samples. On the contrary, people often only need to see some examples of new things to quickly identify a new object class. Inspired by this learning ability of humans, researchers hope to enable machine learning models to learn quickly with few training samples. The key to this fast learning ability is the acquisition of prior knowledge, that is, a large number of tasks are required to train the model so that the model can continuously summarize prior knowledge while quickly learning new knowledge. This can help the machine learning model make correct classifications when encountering unrecognized categories with only a small number of samples. This is the concept of few-shot learning. In the few-shot learning problem, the training set consists of a set of base classes with sufficient training samples for each class and a set of new classes with only a small number of labeled samples (shots) for each class. The goal of few-shot learning is to transfer the knowledge of the base classes so that the model can achieve better generalization performance after only a small number of new class samples are trained.

[0003] Meta-learning is applied as a framework in the field of small sample learning, using a large number of similar tasks to learn how to adapt to new tasks with only a few labeled samples. However, meta-learning uses shallow networks as feature extractors to avoid model overfitting, which will lead to insufficient feature extraction. In addition, existing meta-learning methods for small sample classification assume that the number of instances for each task and category is fixed. Even if the number of each task and category varies greatly, the meta-learner still uses meta-knowledge equally for learning in all tasks. Moreover, they do not consider the distribution differences of unseen tasks. The meta-knowledge learned in the training set may not be very useful. These constitute the problems of classification imbalance, task imbalance and distribution imbalance in small sample learning. At present, there is relatively little research on unbalanced small sample learning. How to deal with this small sample learning problem that is closer to real-life scenarios is one of the current problems that need to be solved urgently. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of insufficient feature extraction when the meta-learning method uses shallow networks to extract features and the problem that the existing small sample learning method does not consider the imbalance problem in real scenarios, and to provide a task-adaptive small sample image classification method based on meta-transfer learning.

[0005] A task-adaptive small sample image classification method based on meta-transfer learning, comprising the following steps:

[0006] Step 1: Obtain a large-scale image dataset, use the samples in the large-scale image dataset to pre-train the deep network, and output a set of weight parameter vectors Θ and θ for the feature extractor and classifier;

[0007] Step 2: Freeze the weight parameter vector θ of the feature extractor and reset the weight parameter vector θ of the classifier;

[0008] Step 3: Obtain a small sample image dataset to be classified and divide it into a meta-training set and a meta-test set; the meta-training set includes a support set and a query set. The support set is used to update the basic learner parameters, and the query set is used to calculate the loss function to update the meta-learner.

[0009] Step 4: Initialize the parameters of the meta-learner and base learner, and set the maximum number of iterations; the base learner is used to quickly learn new knowledge from new tasks, and the meta-learner is used to summarize the experience of all learned knowledge; small sample learning is based on tasks, and each task includes N classes;

[0010] Step 5: Randomly extract samples from the meta-training set to form training tasks, and then randomly extract training tasks to train the basic learner, continuously update and optimize the meta-transfer learning parameters and balance parameters, and in this step, the class of samples with low recognition accuracy is obtained as the difficult class;

[0011] Step 6: Randomly sample from the difficult class set to form a difficult task;

[0012] Step 7: Randomly select difficult tasks to train the basic learner, and continuously update and optimize the learner parameters, meta-transfer learning parameters, and balance parameters until the loss is lower than the threshold;

[0013] Step 8: Determine whether the maximum number of iterations has been reached; if not, clear the difficult class set and return to step 5 for the next iteration; if the maximum number of iterations has been reached, output the meta-transfer learning parameters and balance parameters after the iteration is completed to obtain the trained basic learner;

[0014] Step 9: Input the meta-test set into the trained basic learner to obtain the small sample image classification result.

[0015] Furthermore, during the pre-training in step 1, the loss function is:

[0016]

[0017] Where D is the sample data; f [Θ;θ]represents the model equation containing the weight parameter vectors Θ and θ of the feature extractor and classifier; x, y represent samples and labels respectively;

[0018] The update formulas for the weight parameter vectors Θ and θ of the feature extractor and classifier are:

[0019]

[0020] Furthermore, the collection process of the difficult category in step 5 is as follows:

[0021] Step 5.1: Get the support set and query set of the current input task;

[0022] Step 5.2: Calculate the loss function of the samples in the support set and update the base learner parameters [Θ; θ] and meta-transfer learning parameters Balance parameter ω ni ,γ i ,z i ;

[0023] (θ,θ0)=g(z i )(Θ,θ)

[0024] Wherein, the subscript i represents the index of the task, i = 1, 2, ..., N; g(·) = softplus(·), g(·) is a non-negative activation function, and the above formula is the initialization equation of the model. By balancing the variable z τ To determine how much meta-knowledge to use;

[0025]

[0026] Where μ is the learning rate of the meta-transfer learning parameter; L T(query) is the loss function of the query set samples;

[0027]

[0028] Among them, f(·) = max(0, min(·, 1)) is the clipping function; α is the task learning rate; a series of scalars As a coefficient of the loss in each task gradient descent step to handle class imbalance;

[0029] Step 5.3: Extract samples from the query set, calculate their classification accuracy, sort them in ascending order of accuracy, and take the class of the lowest m samples as the difficult class, where m is a preset constant.

[0030] The beneficial effects of the present invention are:

[0031] The present invention combines meta-transfer learning to make up for the problem that the MAML model uses a 4Conv shallow network to extract insufficient features; it increases trainable parameters to learn to balance the use of meta-knowledge in each task, solving the problems of task imbalance, category imbalance, and distribution imbalance in small sample learning in real scenarios. The present invention selects samples with low accuracy in each task and recombines their data to make it a more difficult task, so that the meta-learner can improve the accuracy of the model in the process of learning more difficult tasks. The difficult task mining algorithm proposed in the present invention collects samples with poor classification effects online to form difficult tasks, allowing the learner to learn faster and better in difficult tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is the overall structural diagram of the model in the present invention.

[0033] Figure 2 This is a training flow chart of the model in the present invention.

[0034] Figure 3 This is a structural diagram of the feature extractor ResNet-12 in the present invention.

[0035] Figure 4 This is a structural diagram of the inference network in the present invention.

[0036] Figure 5 It is a pseudo code diagram of the specific algorithm in the present invention. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with the accompanying drawings.

[0038] In order to solve the problem that the meta-learning method uses shallow networks to extract features and the existing small sample learning method does not consider the imbalance problem in real scenarios, this paper proposes a small sample image classification method based on meta-transfer task-adaptive meta-learning (MT-TAML). The technical solutions adopted are as follows:

[0039] A task-adaptive small sample image classification method based on meta-transfer learning. Figure 1 As shown in Figure 2 As shown, including:

[0040] Step 1: Obtain a large-scale image dataset (such as ImageNet), use the samples in the large-scale image dataset to pre-train the deep network, and output a set of weight parameter vectors Θ and θ for the feature extractor and classifier;

[0041] During pre-training, the loss function is:

[0042]

[0043] Where D is the sample data; f [Θ;θ] represents the model equation containing the weight parameter vectors Θ and θ of the feature extractor and classifier; x, y represent samples and labels respectively;

[0044] The feature extractor uses ResNet-12, such as Figure 3 As shown; the update formulas of the weight parameter vectors Θ and θ of the feature extractor and classifier are:

[0045]

[0046] Step 2: Freeze the weight parameter vector θ of the feature extractor and reset the weight parameter vector θ of the classifier;

[0047] Step 3: Obtain a small sample image dataset to be classified and divide it into a meta-training set and a meta-test set; the meta-training set includes a support set and a query set. The support set is used to update the basic learner parameters, and the query set is used to calculate the loss function to update the meta-learner.

[0048] Step 4: Initialize the parameters of the meta-learner and base learner, and set the maximum number of iterations; the base learner is used to quickly learn new knowledge from new tasks, and the meta-learner is used to summarize the experience of all learned knowledge; small sample learning is based on tasks, and each task includes N classes;

[0049] Step 5: Randomly extract samples from the meta-training set to form training tasks, and then randomly extract training tasks to train the basic learner, continuously update and optimize the meta-transfer learning parameters and balance parameters. The inference network of the balance parameters is as follows: Figure 4 As shown, in this step, the class of samples with low recognition accuracy is taken as the difficult class;

[0050] The collection process of the difficult category is:

[0051] Step 5.1: Get the support set and query set of the current input task;

[0052] Step 5.2: Calculate the loss function of the samples in the support set and update the base learner parameters [Θ; θ] and meta-transfer learning parameters Balance parameter ω ni ,γ i ,z i ;

[0053] (θ,θ0)=g(z i )(Θ,θ)

[0054] Wherein, the subscript i represents the index of the task, i = 1, 2, ..., N; g(·) = softplus(·), g(·) is a non-negative activation function, and the above formula is the initialization equation of the model. By balancing the variable z τ To determine how much meta-knowledge to use;

[0055]

[0056] Where μ is the learning rate of the meta-transfer learning parameter; L T(query) is the loss function of the query set samples;

[0057]

[0058] Among them, f(·) = max(0, min(·, 1)) is the clipping function; α is the task learning rate; a series of scalars As a coefficient of the loss in each task gradient descent step to handle class imbalance;

[0059] Step 5.3: Extract samples from the query set, calculate their classification accuracy, sort them in ascending order of accuracy, and take the class of the lowest m samples as the difficult class, where m is a preset constant; the specific algorithm of the task-adaptive small sample image classification method based on meta-transfer learning of the present invention is as follows Figure 5 As shown;

[0060] Step 6: Randomly sample from the difficult class set to form a difficult task;

[0061] Step 7: Randomly select difficult tasks to train the basic learner, and continuously update and optimize the learner parameters, meta-transfer learning parameters, and balance parameters until the loss is lower than the threshold;

[0062] Step 8: Determine whether the maximum number of iterations has been reached; if not, clear the difficult class set and return to step 5 for the next iteration; if the maximum number of iterations has been reached, output the meta-transfer learning parameters and balance parameters after the iteration is completed to obtain the trained basic learner;

[0063] Step 9: Input the meta-test set into the trained basic learner to obtain the small sample image classification result.

[0064] The present invention proposes a meta-transfer task-adaptive meta-learning (MT-TAML) algorithm based on the MAML basic model. By combining meta-transfer learning, the problem that the MAML model uses a 4Conv shallow network to extract insufficient features is compensated. Trainable parameters are added to learn to balance the use of meta-knowledge in each task, solving the problems of task imbalance, category imbalance, and distribution imbalance in small sample learning in real scenarios.

[0065] The difficult task mining algorithm proposed in the present invention collects samples with poor classification effects online to form difficult tasks, allowing the learner to learn faster and better in difficult tasks. Traditional meta-batch processing consists of randomly sampled tasks, where randomness means random difficulty. In the algorithm of the present invention, samples with low accuracy in each task are intentionally selected and their data are recombined to make it a more difficult task, so that the meta-learner can improve the accuracy of the model in the process of learning more difficult tasks.

[0066] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A task-adaptive small sample image classification method based on meta-transfer learning, characterized in that: The following steps are involved: Step 1: Obtain a large-scale image dataset, use the samples in the large-scale image dataset to pre-train the deep network, and output a set of weight parameter vectors Θ and θ for the feature extractor and classifier; Step 2: Freeze the weight parameter vector θ of the feature extractor and reset the weight parameter vector θ of the classifier; Step 3: Obtain a small sample image dataset to be classified and divide it into a meta-training set and a meta-test set; the meta-training set includes a support set and a query set. The support set is used to update the basic learner parameters, and the query set is used to calculate the loss function to update the meta-learner. Step 4: Initialize the parameters of the meta-learner and base learner, and set the maximum number of iterations; the base learner is used to quickly learn new knowledge from new tasks, and the meta-learner is used to summarize the experience of all learned knowledge; small sample learning is based on tasks, and each task includes N classes; Step 5: Randomly extract samples from the meta-training set to form training tasks, and then randomly extract training tasks to train the basic learner, continuously update and optimize the meta-transfer learning parameters and balance parameters, and in this step, the class of samples with low recognition accuracy is obtained as the difficult class; Step 6: Randomly sample from the difficult class set to form a difficult task; Step 7: Randomly select difficult tasks to train the basic learner, and continuously update and optimize the learner parameters, meta-transfer learning parameters, and balance parameters until the loss is lower than the threshold; Step 8: Determine whether the maximum number of iterations has been reached; if not, clear the difficult class set and return to step 5 for the next iteration; if the maximum number of iterations has been reached, output the meta-transfer learning parameters and balance parameters after the iteration is completed to obtain the trained basic learner; Step 9: Input the meta-test set into the trained basic learner to obtain the small sample image classification result.

2. The task-adaptive small sample image classification method based on meta-transfer learning according to claim 1, characterized in that: During pre-training in step 1, the loss function is: Where D is the sample data; f [Θ;θ] represents the model equation containing the weight parameter vectors Θ and θ of the feature extractor and classifier; x, y represent samples and labels respectively; The update formulas for the weight parameter vectors Θ and θ of the feature extractor and classifier are:

3. The task-adaptive small sample image classification method based on meta-transfer learning according to claim 1, characterized in that: The collection process of the difficult category in step 5 is as follows: Step 5.1: Get the support set and query set of the current input task; Step 5.2: Calculate the loss function of the samples in the support set and update the base learner parameters [Θ; θ] and meta-transfer learning parameters Balance parameter ω ni ,γ i ,z i ; (Θ,θ0)=g(z i )(Θ,Θ) Wherein, the subscript i represents the index of the task, i = 1, 2, ..., N; g(·) = softplus(·), g(·) is a non-negative activation function, and the above formula is the initialization equation of the model. By balancing the variable z τ To determine how much meta-knowledge to use; Where μ is the learning rate of meta-transfer learning parameters; L T(query) is the loss function of the query set samples; Among them, f(·) = max(0, min(·, 1)) is the clipping function; α is the task learning rate; a series of scalars As a coefficient of the loss in each task gradient descent step to handle class imbalance; Step 5.3: Extract samples from the query set, calculate their classification accuracy, sort them in ascending order of accuracy, and take the class of the lowest m samples as the difficult class, where m is a preset constant.