A small sample image classification method and system based on adversarial training and meta learning

Abstract

This invention discloses a few-sample image classification method and system based on adversarial training and meta-learning. The method includes image data acquisition, adversarial sample generation, adversarial training of the meta-learning model, training on test tasks, and final result output. The meta-learning model comprises an inner loop structure and an outer loop structure. By continuously performing inner and outer loop operations on multiple training tasks, the model parameters are iteratively optimized. This allows the model to adapt to new tasks simply by using the model updated in the outer loop. Based on a meta-learning algorithm, this invention classifies image categories more efficiently and accurately with limited image sample data, improving the accuracy and generalization ability of image classification. Furthermore, based on adversarial training, it maintains robust accuracy in image classification even when image data is subjected to adversarial attacks, enhancing the robustness of the model's classification capabilities.

Description

Technical Field

[0001] This invention belongs to the technical field of few-shot image classification, and mainly relates to a few-shot image classification method and system based on adversarial training and meta-learning. Background Technology

[0002] In recent years, with the rapid development of computing devices, large datasets, advanced models, and algorithms, deep learning has achieved remarkable results in many fields such as image recognition and natural language processing. For example, deep learning can easily achieve an accuracy of over 90% in image classification. However, deep learning increasingly demands a large number of training samples. Most high-performing models rely on massive amounts of data, and common deep learning models struggle to achieve good results in tasks with limited data. Traditional deep learning models typically require large amounts of labeled data to achieve good performance, but in practical applications, acquiring large-scale labeled datasets is a time-consuming and expensive task, especially in specific domains or tasks where data may be relatively limited, such as in medicine, military, and security. In these fields, the lack of a large amount of labeled data reduces the model's ability to generalize to new test samples, thus affecting the accuracy of deep learning models. Taking classification tasks as an example, humans can identify unseen categories using only a small number of labeled samples. This ability mainly stems from the various kinds of knowledge accumulated by humans in daily life and their innate thinking ability. Meta-learning algorithms are a highly effective method for learning from small samples, developed on this basis.

[0003] Traditional meta-learning methods can be widely used for few-sample image classification tasks. However, existing meta-learning methods still face a practical problem: once the model is subjected to adversarial examples, its classification performance drops significantly or it may even fail to complete the classification task, lacking robustness against adversarial attacks. On the other hand, there is a problem in the adversarial training domain: robustness and accuracy are often mutually exclusive. Improving the model's robustness against adversarial examples often results in a decrease in accuracy against raw samples. Balancing robustness and accuracy is a key challenge. These issues highlight the shortcomings of existing meta-learning methods. Therefore, a new meta-learning method is needed that can resist adversarial attacks while balancing robustness and accuracy to complete few-sample image classification tasks under adversarial attacks. Summary of the Invention

[0004] This invention addresses the problem in existing technologies where models struggle to achieve accurate image classification under adversarial attacks. It provides a few-sample image classification method and system based on adversarial training and meta-learning. The method includes image data acquisition, adversarial sample generation, adversarial training of the meta-learning model, training on test tasks, and final result output. The meta-learning model comprises inner and outer loop structures. By continuously performing inner and outer loop operations on multiple training tasks, the model parameters are iteratively optimized. This allows the model to adapt to new tasks simply by using the model updated in the outer loop. Based on meta-learning algorithms, this invention classifies images more efficiently and accurately with limited image sample data, improving both accuracy and generalization ability. Furthermore, adversarial training ensures robust accuracy in image classification even under adversarial attacks, enhancing the model's robustness.

[0005] To achieve the above objectives, the technical solution adopted by this invention is: a few-sample image classification method based on adversarial training and meta-learning, comprising the following steps:

[0006] S1, Image Data Acquisition: Acquire the CIFAR-FS dataset for the few-sample image classification task from the CIFAR100 Few Shot dataset, and divide it into training set and test set for training task and test task respectively.

[0007] S2, Generate adversarial examples: Use the projective gradient descent method to generate adversarial examples from the image data collected in step S1 for the training task, as the dataset for adversarial training.

[0008] S3, Meta-learning model adversarial training: The model is trained using image data of a different category than the target classification task. The original natural data from the training task and the adversarial sample data generated by the adversarial attack are input into the meta-learning model for model training. The meta-learning model includes an inner loop structure and an outer loop structure. By continuously performing inner and outer loop operations on multiple training tasks, the model parameters are iteratively optimized, so that when a new task is encountered, only the model updated by the outer loop needs to be used to adapt to the task.

[0009] S4, Test Task Training: The test task is used to fine-tune the meta-testing process of the meta-learning model to obtain the final meta-model used for prediction.

[0010] S5, Output Results: Based on the meta-model trained in step S4, input the target image data after the adversarial attack, and output the category classification result of the target image under the adversarial attack.

[0011] As an improvement to the present invention, the method of generating adversarial examples using projection gradient descent in step S2 is specifically as follows:

[0012]

[0013] Where, x i,adv Representing adversarial examples, y i This represents the true label of the original sample, and sign represents the sigmoid function. Let L(·) represent gradient descent, φ represent model parameters, L(·) represent the loss function, η be the learning rate, ∈ be the constraint range of the PGD projection, t be the number of iterations, and Clip be the number of iterations. min,max The (·) function is used to restrict input data to a specified range. When the perturbation exceeds a certain range, it will be mapped back to the specified range to ensure that the adversarial sample is within a certain range; t is the number of iterations.

[0014] As another improvement of the present invention, the adversarial training process in step S3 is specifically as follows:

[0015]

[0016] Where θ represents the model parameters, N is the number of samples in the training set, γ is the weight for measuring the loss from adversarial examples, and δ represents the adversarial perturbation. and Let x represent the adversarial sample loss and the original sample loss, respectively. i Represents the original sample, y i The labels represent the true sample labels, and min and max represent the minimum and maximum functions, respectively.

[0017] As another improvement of the present invention, the inner loop structure of the meta-learning model in step S3 includes a method for calculating the loss of the original task. Given the updated adaptive coefficients ψ and the parameter ω used to adjust the adaptive coefficients, the adaptive loss function is expressed as:

[0018]

[0019] Among them, R γ (·) represents the regularization term.

[0020] As another improvement of the present invention, in the outer loop of the meta-learning model in step S3, the initial parameters of the model are improved by iterative learning on the inner loop of multiple few-sample tasks, and a weighted TRADES adversarial loss function L is used. origi (φ,x i ,y i )+γL advi (φ,x i,adv ,y i The loss value for the original sample task is calculated through the weighted outer loop. and loss value against adversarial examples As a new outer loop loss function.

[0021] As a further improvement of the present invention, the loss function in step S3 uses cross-entropy, specifically:

[0022]

[0023] Where N is the number of samples in the training set, p is the probability distribution of the model's predicted output, and each element p in p is a probability distribution of the model's predicted output. i Let q represent the probability that a sample belongs to class i; q is the one-hot encoded representation of the sample's true label, meaning that q is true only if i equals the corresponding true label class. i q equals 1, otherwise q i All are equal to 0.

[0024] To achieve the above objectives, the present invention also adopts the following technical solution: a few-sample image classification system based on adversarial training and meta-learning, comprising a computer program, characterized in that: when the computer program is executed by a processor, it implements the steps of any of the methods described above.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] (1) The method of this invention combines meta-learning algorithm with adversarial example training for few-sample image classification tasks. The model's robustness accuracy in the face of adversarial attacks is better than that of traditional meta-learning methods, indicating that the model has strong robustness in few-sample classification tasks and greatly improves the model's robustness against adversarial attacks;

[0027] (2) By using the adversarial sample training method under the TRADES strategy, this invention improves the robust accuracy of the model under adversarial attacks while still maintaining a high standard accuracy. That is, the robust meta-learning model trained still has a high classification accuracy in the classification task of original natural samples, thus achieving a balance between robustness and accuracy to a certain extent. Attached Figure Description

[0028] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0029] Figure 2 This is a diagram of the inner loop structure of the meta-learning model of this invention;

[0030] Figure 3 This is a diagram of the outer loop structure of the meta-learning model of this invention;

[0031] Figure 4This is a schematic diagram of the dataset used for meta-learning in Embodiment 2 of the present invention;

[0032] Figure 5 This is a diagram illustrating the training process of the meta-learning model upon which this invention is based. Detailed Implementation

[0033] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0034] Example 1

[0035] A few-shot image classification method based on adversarial training and meta-learning, such as Figure 1 As shown, the method includes the following steps:

[0036] Step S1: Collect the CIFAR-FS dataset for the few-shot image classification task from the CIFAR100 Few Shot dataset, and divide it into a training set and a test set for the training task and the test task, respectively.

[0037] Step S2: Use projective gradient descent (PGD attack) to generate adversarial examples from the image data collected in step S1 for the training task, which will serve as the dataset for adversarial training.

[0038] The PGD attack method is used to generate adversarial examples from training data, specifically:

[0039]

[0040] Where η is the learning rate, ∈ is the constraint range of the PGD projection, and Clip min,max The (·) function is used to restrict input data to a specified range. When the perturbation exceeds a certain range, it will be mapped back to the specified range to ensure that the adversarial sample is within a certain range and to prevent over-adjustment from making the sample unrealistic; t is the number of iterations.

[0041] PGD ​​attacks generate more challenging adversarial examples by iteratively adjusting the input data, helping the model better resist adversarial attacks. In each iteration step, the input data is gradually changed by adjusting the gradient direction to maximize the loss at each step.

[0042] Step S3: Use image data of different categories from the target classification task to train the model. Input the original natural data and adversarial sample data generated by adversarial attack from the training task into the meta-learning model for model training. The meta-learning model includes an inner loop structure and an outer loop structure. By continuously performing inner loop and outer loop operations on multiple training tasks, the model parameters are iteratively optimized, so that when a new task is encountered, only the model updated by the outer loop needs to be used to adapt to the task.

[0043] The inner loop structure of the meta-learning model is as follows: Figure 2 As shown, to achieve adaptive changes in the loss function, two parameters are used to update the loss function, where the adaptive coefficient ψ is used to adjust the original task loss. The update process involves updating another parameter, ω, which serves as the adaptive coefficient. This parameter is then updated through the meta-learning model to adjust the adaptive coefficient. This method enables adaptive changes to different tasks during meta-training. Furthermore, considering the need for model robustness, a regularization term R is added to the loss function. γ (·). Regularization of the adaptive loss function can prevent the model from overlearning. The new adaptive loss function is expressed as:

[0044]

[0045] In the inner loop, the meta-learning algorithm first initializes a model with a random parameter φ. This meta-model uses data of other types besides the target data as the training task for meta-learning. The training task was further divided into support sets. and query set These are used for training and testing tasks respectively within the inner loop. That is, within the inner loop, a small number of training samples are used to update the model's parameters, allowing the model to quickly adapt to the current target task. Support set Used in the training process within the inner loop, it performs one or more gradient descent operations, updating the model parameters φ to φ. i There are n support sets in the inner loop. Gradient updates on different support sets are performed using the original parameters. Finally, after the inner loop ends, n updated model parameters φ are obtained. i Specifically:

[0046]

[0047] External circulation structure such as Figure 3As shown, the adversarial training process balances robust accuracy and standard accuracy, enabling the model to perform well in adversarial example tasks while maintaining good performance in tasks with original samples. A weighted TRADES adversarial loss function is used, i.e. The loss value for the original sample task in the weighted outer loop. and loss value against adversarial examples This serves as a new outer loop loss function, attempting to achieve a balance.

[0048] The complete adversarial training process is as follows:

[0049]

[0050] Where φ represents the model parameters, N is the number of samples in the training set, γ is the weight for measuring the loss of adversarial examples, and δ represents the adversarial perturbation, i.e., the difference between the generated adversarial example and the original sample. and These represent the adversarial sample loss and the original sample loss, respectively, both using the cross-entropy loss function. In this minimax formula, the inner layer contains a maximization process, which involves adjusting the loss based on sample x. i Add a perturbation δ to the value, and after passing through the neural network, compare it with the true value y. i The comparison yields the adversarial example loss value. The maximization operation aims to find the perturbation that maximizes the adversarial example loss value, making the model more susceptible to being misled by adversarial examples and causing misclassification. The outer minimization process, on the other hand, minimizes the loss value of the trained neural network model on the training data while keeping the perturbation constant. This allows the model to learn certain robust features during adversarial training, improving its resistance to adversarial attacks and thus enhancing model security.

[0051] In the outer loop, the meta-learning algorithm improves the model's initial parameters through iterative learning on multiple few-sample tasks in the inner loop, enabling it to quickly adapt to new tasks. For each training task in the inner loop, the obtained parameters φ... i This will be used as the initial parameters for a new model that is only used in this training task, and then the corresponding query set will be used. The model was tested, yielding n loss values. These losses were then weighted and averaged to obtain the overall loss function L of the meta-model in the inner loop phase. sumThroughout the training process, the model's parameters φ are iteratively optimized by continuously performing inner and outer loop operations on multiple training tasks. This way, when encountering a new task, only the model updated in the outer loop needs to be used, allowing for rapid adaptation to the task with only a limited amount of training data. Finally, the meta-learning algorithm achieves rapid learning and adaptation to a small number of tasks using the updated initial model parameters, resulting in a effectively trained final model, specifically:

[0052]

[0053] Throughout the process, the model loss function uses cross-entropy, specifically:

[0054]

[0055] Where N is the number of samples in the training set, p is the probability distribution of the model's predicted output, and each element p in p is a probability distribution of the model's predicted output. i Let q represent the probability that a sample belongs to class i; q is the one-hot encoded representation of the sample's true label, meaning that q is true only if i equals the corresponding true label class. i q equals 1, otherwise q i All values ​​are equal to 0. The model evaluation metrics used are Standard Accuracy (SA) and Robust Accuracy (RA), which represent the model's classification accuracy on raw natural samples and adversarial samples, respectively.

[0056] Step S4: Use the test task to fine-tune the meta-testing process of the meta-learning model to obtain the final meta-model used for prediction.

[0057] In the stage of predicting the target task, a portion of the target data is used as training data. This data is then fed into the newly learned meta-model for fine-tuning. During the fine-tuning process using the target training data, the loss function is calculated, gradient descent is performed, and the model's initial parameters are updated from φ to φ0. T Finally, test data from the target data were used. The images are then classified to obtain the final image category labels.

[0058] Step S5: Based on the meta-model trained in step S4, input the target image data after the adversarial attack and output the category classification result of the target image under the adversarial attack.

[0059] The above steps have yielded a meta-model that meets the requirements. The target data of the small sample images is divided into a training set and a test set. The training set is used to train the meta-model. This meta-model can obtain an optimized meta-model that adapts to the new task with only a small amount of training data and a small number of gradient descent steps. Finally, the test set is used for testing, and it has high prediction accuracy.

[0060] Example 2

[0061] The dataset used in this invention is taken from the CIFAR100 dataset, a commonly used dataset in the field of few-shot learning. This dataset consists of 20 superclasses, further subdivided into 100 categories, with each category comprising 600 32*32 color images, totaling 60,000 images. The dataset includes multiple categories such as animals and vehicles, and is widely used in the field of deep learning image classification.

[0062] Figure 4 This image showcases a portion of the images from the CIFAR-FS dataset. Because each image is only 32x32 pixels, they appear somewhat blurry and difficult to discern with the naked eye. The image illustrates the five categories under the superclass "Fish": ornamental fish, flounder, ray, shark, and trout, corresponding to each row in the image. In the dataset usage, the 100 image classes are divided into training, validation, and test sets. Specifically, 64 classes are used for meta-training, 16 for meta-validation, and 20 for the meta-test task. It's important to note that this meta-validation is not used as a test set in the meta-training task, but rather to validate the model's performance during training and prevent overfitting. Furthermore, 500 images are used as training images and 100 as test images within each category.

[0063] This invention uses the Python deep learning framework PyTorch to build a meta-learning deep neural network model. The training process of the meta-learning model is as follows: Figure 5 As shown. In the entire training process, step 1 initializes the meta-learning model using random parameters φ; step 2 copies the initial meta-learning model as a temporary model for training tasks during the inner loop of the meta-learning process. All copied temporary models use the parameters initialized in step 1, and all temporary models have the exact same network structure as the initial model; step 3 samples multiple times from the training tasks to obtain tasks for training, and further divides them into support sets for training and testing. Query set data Step 4: Train the network in the inner loop task using the support set data. Calculate the loss on the support set and update the parameters of the temporary model for this task to φ. i It is important to note that an adaptive loss function is used here. The model parameters are updated instead of using the original fixed loss function; step 5 is the most crucial step, which involves updating the query set data from the training task. Adversarial examples are generated by iteratively maximizing the adversarial loss through PGD attacks. Next, in step 6, the weighted loss function is calculated to update the model parameters in the outer loop. First, the original sample data, i.e., the query set data, is used to test the fine-tuned temporary model to obtain the original sample loss. Additionally, using the adversarial example data generated in step 5 as a test set, adversarial example testing was performed on the same temporary model to obtain the adversarial example loss. To achieve a certain level of robustness and accuracy for the model, the two loss functions are weighted to obtain a new loss for updating model parameters in the outer loop. Finally, using this new loss function, the loss is calculated on several training tasks to obtain the sum of the losses for the original samples and adversarial samples in the inner loop. Gradient descent is then performed to update the initial model parameters in the outer loop. Specifically:

[0064]

[0065] In this embodiment, due to the small number of task samples and the potential for overfitting during model training due to excessive parameters, the network consists of a four-layer convolutional neural network. The first two convolutional layers have 16 filters, and the last two have 32. The kernel size is 3*3, the stride is 1, the ReLU activation function is used, and the Adam learner is used as the model optimizer. The loss function of both the meta-learning algorithm and the learner is calculated based on the cross-entropy function. The learning rate inside the meta-learning algorithm is 0.01, and the learning rate of the meta-learner is 0.001. The meta-training process iterates 5 times per task, and fine-tuning is used to fine-tune the model on the test set, with 10 updates per fine-tuning step. The number of samples in each training task in the inner loop is 4, and the outer loop iterates a total of 60,000 times. Every 2,000 steps, the model is evaluated on the test set, and the best-performing model is saved based on SA and robust accuracy RA.

[0066] This invention compares the improved MAML (Model-Agnostic Meta-learning) meta-learning algorithm with the original algorithm in image classification tasks through extensive comparative experiments. Other comparison models include meta-learning algorithms that currently offer superior performance in few-shot learning and exhibit strong robustness against adversarial attacks, such as AQ, ANIL, TRADES, and R-MAML. out Table 1 shows the comparative data of the 5-way 1-shot image classification experiment results:

[0067] Table 1 Performance comparison of 5-way 1-shot models on the CIFAR-FS dataset

[0068] MAML 51.07％ 0.24％ AQ 31.25％ 27.34％ AT-ANIL 36.46％ 25.79％ <![CDATA[R-MAML out ]]> 39.76％ 27.15％ TRADES-LAML (This invention) 42.60％ 30.23％

[0069] As shown in the table above, the proposed model achieves a standard accuracy of 42.60% and a robust accuracy of 30.23%. While the MAML model achieves a 51% SA in the classification task, its classification accuracy drops significantly under adversarial attacks, with an RA of only 0.24% in the experiment. In contrast, the proposed model demonstrates a significantly improved robust accuracy under adversarial attacks, with a classification performance improvement of approximately 30%. Compared to other robust meta-learning models, the proposed model achieves varying degrees of improvement in both SA and RA. Specifically, the robust accuracy is 2.89% higher than AQ, 4.44% higher than AT-ANIL, and higher than R-MAML. out The model's RA improved by 3.13%.

[0070] In summary, this invention proposes a robust few-shot image classification method based on adversarial training and meta-learning. Under the same adversarial attack conditions, it significantly improves the robust accuracy of few-shot image classification compared with other meta-learning methods.

[0071] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A small sample image classification method based on adversarial training and meta-learning, characterized in that It includes the following steps: S1, Image Data Acquisition: Acquire the CIFAR-FS dataset for the few-sample image classification task from the CIFAR100 Few Shot dataset, and divide it into training set and test set for training task and test task respectively. S2, Generate adversarial examples: Use the projective gradient descent method to generate adversarial examples from the image data collected in step S1 for the training task, as the dataset for adversarial training. S3, Meta-learning model adversarial training: The model is trained using image data of a different category than the target classification task. The original natural data from the training task and the adversarial sample data generated by the adversarial attack are input into the meta-learning model for model training. The meta-learning model includes an inner loop structure and an outer loop structure. By continuously performing inner and outer loop operations on multiple training tasks, the model parameters are iteratively optimized so that when a new task is encountered, only the model updated by the outer loop needs to be used to adapt to the task. The inner loop structure of the meta-learning model includes a function for calculating the loss of the original task. Adaptive coefficients to be updated and parameters used to adjust the adaptive coefficients The adaptive loss function is then expressed as: ； in, Regularization term; In the outer loop of the meta-learning model, the initial parameters of the model are improved through iterative learning on multiple small sample tasks in the inner loop. A weighted TRADES adversarial loss function is used, which is a weighted sum of the loss values ​​from the outer loop when dealing with the original sample tasks. and loss value against adversarial examples , as the new outer loop loss function; S4, Test Task Training: The test task is used to fine-tune the meta-testing process of the meta-learning model to obtain the final meta-model used for prediction. S5, Output Results: Based on the meta-model trained in step S4, input the target image data after the adversarial attack, and output the category classification result of the target image under the adversarial attack.

2. The method of claim 1, wherein the method is based on adversarial training and meta-learning. The specific steps for generating adversarial examples using the projected gradient descent method in step S2 are as follows: ； in, Indicates adversarial examples, Represents the true label of the original sample. This represents the sigmoid function. This indicates gradient descent. Indicates model parameters, Represents the loss function. It's the learning rate. It is the constraint range of PGD projection. Indicates the number of iterations. This function is used to restrict input data to a specified range. When the perturbation exceeds a certain range, it will be mapped back to the specified range to ensure that the adversarial examples are within a certain range. It represents the number of iterations.

3. The few-shot image classification method based on adversarial training and meta-learning as described in claim 1, characterized in that: In step S3, the adversarial training process specifically includes: ； in, Indicates model parameters, It is the number of samples in the training set. It is a weight that measures the loss of adversarial examples. This indicates resistance to disturbances. and These represent the adversarial sample loss and the original sample loss, respectively. Represents the original sample. Indicates the labels of real samples. These represent the minimum and maximum functions, respectively.

4. The method of claim 3, wherein the method is based on adversarial training and meta-learning. In the outer loop of the S3 meta-learning model, the weighted TRADES adversarial loss function is specifically used as follows: ； wherein, denotes an adversarial sample, denotes a model parameter.

5. The method of claim 1, wherein: In step S3, the loss function used is cross-entropy, specifically: ； in, It is the number of samples in the training set. It is a probability distribution of the model's predicted output. Each element in Indicates that the sample belongs to the first The probability of a class; It is a one-hot encoded representation of the sample's true label, meaning it is only valid when... When equal to the corresponding real label category, Equals 1, all other cases All are equal to 0.

6. A small sample image classification system based on adversarial training and meta-learning, comprising a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-5 above.

Images