An Adversarial Training Method Based on Learnable Meta-Evaluation Network

Abstract

The present invention discloses an adversarial training method based on a learnable meta-evaluation network. In one round of adversarial training, the learnable meta-evaluation network and the target model are updated alternately. During the entire adversarial training process, the learnable meta-evaluation network and the target model are updated simultaneously. In addition, in order to maintain fairness, metadata is only used to train the learnable meta-evaluation network. In each iteration, a target model is first copied as a meta-model, and then the learnable meta-evaluation network is updated. After learning the parameters of the meta-evaluation network, the meta-model is directly discarded, and the learnable meta-evaluation network with updated parameters is used to train the target model. The present invention assists adversarial training by constructing a meta-learning-based learnable evaluation network to evaluate the difficulty of adversarial samples and generate corresponding weights to reweight the loss function of adversarial training. In this way, the difficulty of adversarial samples is adjusted to adapt to the current state of the model, so that there is no need to overfit low-quality biased adversarial samples. Therefore, the present invention improves the robustness of the target model without reducing the quality of the adversarial text.

Description

Technical Field

[0001] The present invention belongs to the field of adversarial training technology, and more specifically, relates to an adversarial training method based on a learnable meta-evaluation network. Background Art

[0002] Adversarial training is a defense method applied to deep learning models, aiming to improve their robustness against adversarial attacks. Existing adversarial training methods are mainly divided into two types.

[0003] The first type of approach uses a single attack strategy to generate adversarial examples and uses them alongside clean examples to train the target model. The Projected Gradient Descent (PGD) adversarial attack method is commonly used to generate adversarial examples. The PGD attack uses a set of hyperparameters, including the number of iterations, step size, and perturbation value, which together form the attack strategy for generating adversarial examples. Traditional adversarial training methods use a fixed attack strategy to generate adversarial examples and use them for adversarial training to improve model robustness.

[0004] The second type of adversarial training method uses multiple attack strategies to generate adversarial examples. This method first uses artificially formulated rules or policy networks to learn and generate the current attack strategy. Then, based on these attack strategies, corresponding adversarial examples are generated for adversarial training to improve the robustness of the model.

[0005] Most existing adversarial attack methods use fixed or artificially formulated attack strategies to generate adversarial samples. (1) Using fixed attack strategies, difficult adversarial samples are generated in the early stages of model training, causing the target model to collapse. The adversarial samples generated in the later stages of training are not difficult enough, resulting in limited improvement in the robustness of the target model. (2) Using artificially formulated rules to generate attack strategies requires a lot of domain experience and has limited improvement in model robustness, making them unsuitable for practical applications.

[0006] Existing methods, based on policy networks derived from rules or reinforcement learning, generate multiple attack strategies at different stages of target model training. While these methods achieve good results in improving model robustness, they rarely focus on the quality of generated adversarial examples. Adversarial examples generated using multiple attack strategies are not necessarily high-quality. Deep learning models tend to overfit to low-quality, biased adversarial examples, which often leads to decreased accuracy on clean samples and limited improvements in robustness. Summary of the Invention

[0007] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide an adversarial training method based on a learnable meta-evaluation network, which improves the robustness of the target model without reducing the quality of the adversarial text.

[0008] To achieve the above-mentioned object of the invention, the present invention is based on an adversarial training method of a learnable meta-evaluation network, which is characterized by comprising the following steps:

[0009] (1) Set the iteration threshold to T and initialize the iteration number to t = 1;

[0010] (2) Update the learnable meta-evaluation network

[0011] 2.1) Select n images from the metadata set As a clean sample, the clean sample is then subjected to a projected gradient descent attack, i.e., PGD attack, to generate adversarial perturbations that are added to the clean sample to form an adversarial sample.

[0012] 2.2) Copy the target model f θ And as a meta-model, the adversarial sample Send it into the meta-model to get the classification prediction value And calculate the meta-loss Among them, f θ represents the target model with parameters θ, It is an adversarial example The true classification label of

[0013] 2.3) Loss of Yuan Send it to the learnable meta-evaluation network to get adversarial samples The difficulty is the weight value

[0014]

[0015] Among them, h ω represents a learnable meta-evaluation network with parameters ω;

[0016] 2.4) Update the learnable meta-evaluation network parameters ω:

[0017]

[0018] Where ω* is the updated learnable meta-evaluation network parameter, and β is the learning rate;

[0019] (3) Update the target model

[0020] 3.1) Select n images x from the training set i , i=1,2,…,n as clean samples, and then perform projected gradient descent, i.e. PGD attack, on the clean samples to generate adversarial perturbations and add them to the clean samples to form adversarial samples x adv_i ;

[0021] 3.2) The adversarial sample x adv_i Send it to the target model to get the classification prediction value f θ (x adv_i ), and calculate the training loss Among them, represents the target model with parameters θ, y i is an adversarial example x adv_i The true classification label of

[0022] 3.3) The training loss Send it to the learnable meta-evaluation network after parameter update to obtain the adversarial sample x adv_i The difficulty is the weight value w i :

[0023]

[0024] in, The parameter is ω * Learnable meta-evaluation network;

[0025] 3.4) Update the parameters θ of the target model:

[0026]

[0027] Among them, θ * is the parameter of the updated target model, α is the learning rate;

[0028] (4) Determine whether the number of iterations t is equal to the iteration threshold T. If so, end the training of the target model. Otherwise, t = t + 1, ω = ω * , θ=θ * , return to step (2).

[0029] The object of the present invention is achieved like this:

[0030] The present invention is based on an adversarial training method of a learnable meta-evaluation network. In one round of adversarial training, the learnable meta-evaluation network and the target model are updated alternately. During the entire adversarial training process, the learnable meta-evaluation network and the target model are updated simultaneously. In addition, in order to maintain fairness, metadata is only used to train the learnable meta-evaluation network. In each iteration, a target model is first copied as a meta-model, and then the learnable meta-evaluation network is updated. After learning the parameters of the meta-evaluation network, the meta-model parameters are frozen, and the learnable meta-evaluation network with updated parameters is used to train the target model. The present invention assists adversarial training by constructing a meta-learning-based learnable evaluation network to evaluate the difficulty of adversarial samples and generate corresponding weights to reweight the loss function of adversarial training. In this way, the difficulty of adversarial samples is adjusted to adapt to the current state of the model, so that there is no need to overfit low-quality biased adversarial samples. Therefore, the present invention improves the robustness of the target model without reducing the quality of the adversarial text. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flowchart of a specific implementation of the adversarial training method based on a learnable meta-evaluation network of the present invention;

[0032] Figure 2 2 is a schematic diagram of a specific embodiment of the adversarial training method based on a learnable meta-evaluation network of the present invention;

[0033] Figure 3 This is a schematic diagram of adversarial sample recognition. DETAILED DESCRIPTION

[0034] The following describes the specific embodiments of the present invention in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when detailed descriptions of known functions and designs may dilute the main content of the present invention, such descriptions will be omitted here.

[0035] In order to effectively learn the parameter ω in the learnable meta-evaluation network, the present invention introduces a meta-learning mechanism that uses an inner loop to update the learnable meta-evaluation network and an outer loop to update the target model. Specifically, in this embodiment, Figure 1 、 2 As shown, the adversarial training method based on the learnable meta-evaluation network of the present invention includes the following steps:

[0036] Step S1: Initialization

[0037] Set the iteration threshold to T and initialize the iteration number to t=1.

[0038] Step S2: Update the learnable meta-evaluation network

[0039] In order to evaluate the difficulty of adversarial examples and adjust the difficulty of adversarial examples to adapt to the current state of the model, this paper proposes a new adversarial training framework to learn a meta-evaluation network, namely the learnable meta-evaluation network. The learning problem of this learnable meta-evaluation network is formalized as a meta-learning problem. The learnable meta-evaluation network is used to update at each iteration to readjust the attack strength of the adversarial examples to adapt to the current state of the target model. Specifically, the update of the learnable meta-evaluation network is:

[0040] Step S2.1: Generate adversarial examples

[0041] Select n images from the metadata set As a clean sample, the clean sample is then subjected to a projected gradient descent attack, i.e., PGD attack, to generate adversarial perturbations that are added to the clean sample to form an adversarial sample.

[0042] Step S2.2: Copy the target model as a meta-model to obtain adversarial examples The loss of

[0043] The target model f will be copied θ And as a meta-model, the adversarial sample Send it into the meta-model to get the classification prediction value And calculate the meta-loss Among them, f θ represents the target model with parameters θ, It is an adversarial example The true classification label.

[0044] Step S2.3: Adversarial Examples The meta-loss is fed into the learnable meta-evaluation network to obtain its difficulty

[0045] Loss of yuan Send it to the learnable meta-evaluation network to get adversarial samples The difficulty is the weight value

[0046]

[0047] Among them, h ω represents a learnable meta-evaluation network with parameters ω;

[0048] In this paper, a meta-loss is introduced to learn The optimal parameter ω* of the learnable meta-evaluation network is obtained. The learnable meta-evaluation network takes the meta-loss of the adversarial example as input and outputs a weight value to reflect the difficulty of the adversarial example. When the target model is subsequently updated, the attack strength of the adversarial example is readjusted to adapt to the current state of the target model.

[0049] In this embodiment, a fully connected layer is used as a learnable meta-evaluation network, and the parameters of the meta-learnable evaluation network are updated while the target network is trained through meta-learning.

[0050] Step S2.4: Update the learnable meta-evaluation network parameters ω:

[0051]

[0052] Where ω* is the updated learnable meta-evaluation network parameter and β is the learning rate.

[0053] Step S3: Update target model

[0054] Step S3.1: Generate adversarial example x adv_i

[0055] Select n images x from the training set i , i=1,2,…,n as clean samples, and then perform projected gradient descent, i.e. PGD attack, on the clean samples to generate adversarial perturbations and add them to the clean samples to form adversarial samples x adv_i ;

[0056] Step S3.2: Calculate training loss

[0057] The adversarial sample x adv_i Send it to the target model to get the classification prediction value f θ (x adv_i ), and calculate the training loss Among them, represents the target model with parameters θ, y i is an adversarial example x adv_i The true classification label of

[0058] Step S3.3: Set the training loss Send it to the learnable meta-evaluation network after parameter update to obtain the adversarial sample x adv_i The difficulty is the weight value w i :

[0059]

[0060] in, The parameter is ω * Learnable meta-evaluation network;

[0061] Step S3.4: Update the parameters θ of the target model:

[0062]

[0063] Among them, θ * is the parameter of the updated target model, and α is the learning rate.

[0064] Step S4: Determine whether the number of iterations t is equal to the iteration threshold T. If so, end the training of the target model. Otherwise, t=t+1, ω=ω * , θ=θ * , return to step S2.

[0065] Adversarial Example Identification

[0066] like Figure 3 As shown in the figure, after the target model is trained, the process of identifying adversarial samples generated by adversarial attack methods (such as PGD, C&W, and AA) based on the target model is as follows:

[0067] 1. For a given image x, generate an adversarial sample x using adversarial attack methods (such as PGD, C&W, and AA) based on the trained target model parameters. adv .

[0068] 2. Generate the adversarial sample x adv Input into the target model and output the logits value.

[0069] 3. After the logits value is fed into the softmax function, the largest value among the C class values ​​is the category to which the model determines the adversarial sample belongs.

[0070]

[0071] 4) f θ (x adv ) is compared with the original image x and label y. If they are equal, it means that the target model successfully defends against the adversarial sample generated by the adversarial attack method, otherwise it fails.

[0072]

[0073] Examples

[0074] In this example, the target model trained using the present invention is compared with the traditional PGD adversarial training method using the accuracy of clean samples and the robust accuracy of the adversarial attack method. PGD, C&W, and AA are the most commonly used adversarial attack methods. To evaluate the robustness of the present invention, extensive experiments were conducted on three widely used datasets: CIFAR-10, CIFAR-100, and Tiny ImageNet. The CIFAR-10 dataset consists of 60,000 32×32 color images divided into 10 classes, with 6,000 images per class. This dataset consists of 50,000 training images and 10,000 test images. CIFAR-100 consists of 100 classes, with 600 images per class. Each class has 500 training images and 100 test images. Tiny ImageNet is a scaled-down version of the ImageNet dataset with more complex data. It consists of 200 classes, with 500 training images and 50 validation images per class, each of which is 64×64. During testing, different numbers of iterations in the PGD method are selected to generate adversarial sample training models, which can reflect the use of the present invention in different adversarial sample generation methods.

[0075] The test results under the two settings are shown in Tables 1, 2, and 3.

[0076]

[0077] Table 1

[0078]

[0079] Table 2

[0080]

[0081] Table 3

[0082] It can be seen from the above three tables that the model trained using the method of the present invention (Ours) on top of the traditional PGD adversarial training method outperforms the previous method on all test sets.

[0083] On the CIFAR-10, CIFAR-100, and Tiny ImageNet datasets, our method outperforms the baseline method PGD-AT in all cases for different perturbation budgets, for example, when the number of iterations is 2, 4, 6, 8, and 10. Specifically, on the CIFAR-10 dataset, when the number of iterations is 10, our method surpasses PGD-AT in clean accuracy and improves robust accuracy by 1.62%, 1.24%, and 1.48% under PGD-10, C&W, and AA attacks, respectively. On the CIFAR-100 dataset, when the number of iterations is 10, our method outperforms the baseline model PGD-AT by approximately 1.74%, 1.03%, and 1.26% under PGD-10, C&W, and AA attacks, respectively. Furthermore, our method outperforms the current state-of-the-art method LAS-AT, which uses reinforcement learning to adaptively set its attack strategy, across different attack scenarios. On the Tiny ImageNet dataset, our method surpasses PGD-AT by approximately 2.26%, 2.27%, 2.17%, 1.96%, and 3.06% in robust accuracy under PGD-10, PGD-20, PGD-50, C&W, and AA attacks, respectively. Furthermore, our method outperforms the current state-of-the-art model, LAS-AT. These results demonstrate the effectiveness and superiority of our method.

[0084] Although the above describes the illustrative specific embodiments of the present invention to facilitate understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concepts of the present invention are protected.

Claims

1. An adversarial training method based on a learnable meta-evaluation network, characterized in that The following steps are involved: (1) Set the iteration threshold to T and initialize the iteration number to t = 1; (2) Update the learnable meta-evaluation network 2.1) Select n images from the metadata set As a clean sample, the clean sample is then subjected to a projected gradient descent attack, i.e., PGD attack, to generate adversarial perturbations that are added to the clean sample to form an adversarial sample. 2.2) Copy the target model f θ And as a meta-model, the adversarial sample Send it into the meta-model to get the classification prediction value And calculate the meta-loss in, f θ represents the target model with parameters θ, It is an adversarial example The true classification label of 2.3) Loss of Yuan Send it to the learnable meta-evaluation network to get adversarial samples The difficulty is the weight value Among them, h ω represents a learnable meta-evaluation network with parameters ω; 2.4) Update the learnable meta-evaluation network parameters ω: Where ω* is the updated learnable meta-evaluation network parameter, and β is the learning rate; (3) Update the target model 3.1) Select n images x from the training set i , i=1,2,…,n as clean samples, and then perform projected gradient descent, i.e. PGD attack, on the clean samples to generate adversarial perturbations and add them to the clean samples to form adversarial samples x adv_i ; 3.2) The adversarial sample x adv_i Send it to the target model to get the classification prediction value f θ (x adv_i ), and calculate the training loss in, represents the target model with parameters θ, y i is an adversarial example x adv_i The true classification label of 3.3) The training loss Send it to the learnable meta-evaluation network after parameter update to obtain the adversarial sample x adv_i The difficulty is the weight value w i : in, The parameter is ω * Learnable meta-evaluation network; 3.4) Update the parameters θ of the target model: Among them, θ * is the parameter of the updated target model, α is the learning rate; (4) Determine whether the number of iterations t is equal to the iteration threshold T. If so, end the training of the target model. Otherwise, t = t + 1, ω = ω * , θ=θ * , return to step (2).

Images