An adversarial training method and system based on predicted attack step length
Patent Information
- Application Number
- CN202310481128.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-28
- Publication Date
- 2026-09-11
- Estimated Expiration
- 2043-04-28
AI Technical Summary
[0002]深度学习模型在图像分类领域已经取得了巨大成成功，但研究发现，深度学习模型在对抗样本面前非常脆弱，为了解决深度学习模型的这一巨大安全隐患，相继出现了各种防御手段，其中对抗训练(AT)是公认最有效也是应用最为广泛的防御方法，现有对抗训练，主要分为PGD-AT和FGSM-AT，传统的对抗训练(AT)方法，主要采用投影梯度下降(PGD)攻击算法来生成对抗样本用于训练，需要手动设计攻击算法的一系列参数，如迭代次数、最大扰动大小、迭代的攻击步长等参数，即攻击策略，其存在的一个主要的问题，由于PGD攻击算法需要迭代多次，即需要多次前向与反向传播计算模型梯度，这使得基于投影梯度下降攻击算法的对抗训练(PGD-AT)的训练速度非常缓慢，即对抗训练的时间代价太大，为了解决这一问题，有人提出了使用快速梯度符号(FGSM)攻击算法来生成对抗样本用于训练，即基于快速梯度符号攻击算法的对抗训练(FGSM-AT)，该方法由于只需要一次大攻击步长的迭代，即只需要一次前向与反向传播计算模型梯度，因此该方法解决了PGD-AT训练速度缓慢的问题，但是出现了另外两个问题，一是FGSM-AT的对抗鲁棒性远低于PGD-AT，二是FGSM-AT还会出现灾难性的过拟合，即通过FGSM-AT训练出来的模型，对FGSM对抗攻击产生的对抗样本的预测达到了100％的精度，而对PGD等对抗攻击产生的对抗样本的预测几乎全部失败，即预测成功率为0
[0057] The beneficial effects of the method and system of this invention are as follows: This invention obtains a predicted scale value vector through a prediction model, and then adds a perturbation budget step size to the predicted scale value vector. Instead of using a fixed large attack step size, the prediction model is used to predict the attack step size of the sample, thereby solving the catastrophic overfitting problem in the existing FGSM-AT. At the same time, it greatly improves the adversarial robustness of the model. Furthermore, the target model used to generate adversarial examples only needs to perform one forward and backward propagation gradient calculation to generate good adversarial examples, which can effectively reduce the time cost of adversarial training.
Smart Images
Figure CN116484917B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence security technology, and in particular to an adversarial training method and system based on predicting attack step size. Background Technology
[0002] Deep learning models have achieved great success in image classification, but research has found that they are very vulnerable to adversarial examples. To address this significant security vulnerability, various defense methods have emerged, among which adversarial training (AT) is widely recognized as the most effective and widely used. Current adversarial training methods are mainly divided into PGD-AT and FGSM-AT. Traditional adversarial training (AT) methods primarily use the Projected Gradient Descent (PGD) attack algorithm to generate adversarial examples for training. This requires manually designing a series of parameters for the attack algorithm, such as the number of iterations, the maximum perturbation size, and the attack step size, i.e., the attack strategy. A major problem is that the PGD attack algorithm requires multiple iterations, i.e., multiple forward and backward propagation calculations of the model gradient. This makes adversarial training (PGD-AT) based on the Projected Gradient Descent attack algorithm... The training speed of PGD-AT is very slow, meaning that the time cost of adversarial training is too high. To solve this problem, some researchers have proposed using the Fast Gradient Sign (FGSM) attack algorithm to generate adversarial examples for training, namely, adversarial training based on the Fast Gradient Sign (FGSM-AT). This method only requires one iteration with a large attack step size, that is, only one forward and backward propagation to calculate the model gradient. Therefore, this method solves the problem of slow training speed of PGD-AT. However, two other problems arise: first, the adversarial robustness of FGSM-AT is much lower than that of PGD-AT; second, FGSM-AT can also suffer from catastrophic overfitting. That is, the model trained by FGSM-AT achieves 100% accuracy in predicting adversarial examples generated by FGSM adversarial attacks, but almost all predictions of adversarial examples generated by PGD and other adversarial attacks fail, that is, the prediction success rate is 0%. Summary of the Invention
[0003] To address the aforementioned technical problems, the present invention aims to provide an adversarial training method and system based on predicted attack step size. By employing a prediction model to predict the attack step size of samples, the adversarial robustness of adversarial training is further improved.
[0004] The first technical solution adopted in this invention is: an adversarial training method based on predicting attack step size, comprising the following steps:
[0005] Obtain a batch of clean samples and input them into the prediction model for training, and output a vector of predicted proportion values.
[0006] Add a perturbation budget step to the predicted scale value vector to obtain the predicted attack step size;
[0007] A batch of clean samples is input into the target model for training to generate adversarial examples;
[0008] Based on the predicted attack step size and adversarial examples, the prediction model and the target model are jointly trained to obtain the trained prediction model and the trained target model.
[0009] Furthermore, the step of obtaining a batch of clean samples and inputting them into the prediction model for training to obtain the predicted proportion vector specifically includes:
[0010] Obtain a batch of clean samples;
[0011] A prediction model is constructed, which includes a residual block, a ReLU activation layer, an average pooling layer, a Flatten layer, a fully connected layer, and a Sigmoid activation layer.
[0012] Input a batch of clean samples into the prediction model;
[0013] Based on the residual blocks of the prediction model, feature extraction is performed on a batch of clean samples to obtain the feature vector of the batch of clean samples.
[0014] Based on the ReLU activation layer of the prediction model, a nonlinear factor is applied to the feature vector of the batch clean samples to increase the nonlinear relationship between the feature vectors of the batch clean samples, thus obtaining the optimized feature vector of the batch clean samples.
[0015] Based on the average pooling layer of the prediction model, the optimized batch clean sample feature vector is compressed to obtain the compressed batch clean sample feature vector.
[0016] Based on the Flatten layer of the prediction model, the compressed batch clean sample feature vector is processed into one dimension to obtain a one-dimensional batch clean sample feature vector.
[0017] Based on the fully connected layer and sigmoid activation layer of the prediction model, the feature vector of a batch of one-dimensional clean samples is classified to obtain the predicted proportion value vector.
[0018] Furthermore, the step of extracting features from a batch of clean samples using the residual block based on the prediction model to obtain the feature vector of the batch of clean samples specifically includes:
[0019] Construct a residual block for the prediction model, wherein the residual block of the prediction model includes a 1×1 convolutional layer, a 3×3 convolutional layer and a batch normalization layer;
[0020] The 1×1 convolutional layer based on the residual block of the prediction model is used to expand the number of channels of the batch of clean samples to obtain the expanded features of the batch of clean samples.
[0021] The 3×3 convolutional layer based on the residual block of the prediction model performs convolution operations on a batch of clean samples to obtain the features of the batch of clean samples.
[0022] The batch normalization layer based on the residual block of the prediction model is used to standardize the features of the batch clean samples to obtain the standardized features of the batch clean samples.
[0023] The expanded batch clean sample features are added to the standardized batch clean sample features to obtain the batch clean sample feature vector.
[0024] Furthermore, the expression for predicting the attack step size is as follows:
[0025] α(θ)=1.25∈×PM θ (x)
[0026] In the above formula, α represents the predicted attack step size, 1.25∈ represents a constant value, ∈ represents the perturbation budget (hyperparameter), θ represents the parameters of the prediction model, x represents the batch of clean samples, and PM θ (x) represents the prediction model.
[0027] Furthermore, the step of inputting a batch of clean samples into the target model for training to generate adversarial examples specifically includes:
[0028] Random initialization perturbation is added to a batch of clean samples within the range [-∈~∈] to obtain random initialization results;
[0029] The random initialization results are input into the target model for one forward propagation training to obtain the forward propagation training results;
[0030] Backpropagation training is performed based on the forward propagation training results to calculate the gradient of the target model parameters;
[0031] The gradient sign of the extracted target model parameters is multiplied with the predicted attack step size to obtain the multiplication result;
[0032] The calculated multiplication result is added to the random initialization perturbation to obtain the summation result;
[0033] Projecting the summation result onto the range [-∈~∈] yields adversarial examples.
[0034] Furthermore, the expression for generating adversarial examples is specifically as follows:
[0035]
[0036] In the above formula, x+δ represents generating adversarial examples, and η represents random initialization values. The gradient sign represents the target model parameters, and x+η represents the result obtained by randomly initializing a batch of clean samples x.
[0037] Furthermore, the step of jointly training the prediction model and the target model based on the obtained predicted attack step size and adversarial examples to obtain the trained prediction model and the trained target model specifically includes:
[0038] Iterate through N, i = 0, ..., N-1, where i represents the (i+1)th iteration of the training set, and N represents the number of times the training set is used for training.
[0039] Iterate through B, j = 0, ..., B-1, where j represents training the (j+1)th batch, and B represents the size of a batch, i.e., the minimum number of samples for one training session.
[0040] Calculate η ij , where η ij This represents the random initialization value for the (j+1)th batch during the (i+1)th iteration of the training set;
[0041] If imodk = 0, then;
[0042] Through α(θ)=1.25∈×PM θ (x ij ), calculate x ij The attack step size α(θ);
[0043] pass Calculate the counter-perturbation δ(θ);
[0044] pass Update the parameters θ of the prediction model;
[0045] Through α=1.25∈×PM θ (x ij ), calculate x ij The attack step size α;
[0046] pass Calculate the counter-disturbance δ;
[0047] pass Update the parameters w of the target model;
[0048] Based on the updated prediction model parameters and the updated target model parameters, the trained prediction model and the trained target model are constructed.
[0049] Furthermore, the expression for the preset joint training objective of the prediction model and the target model is as follows:
[0050]
[0051] In the above formula, w represents the parameters of the target model, L(·) represents the loss function of the target model, D represents the training set constructed from a batch of clean samples, and E (x,y)～D Let represent the expected value of the target model loss under the distribution (x,y)~D, where y represents the label corresponding to the clean sample.
[0052] The second technical solution adopted in this invention is: an adversarial training system based on predicted attack step size, comprising:
[0053] The prediction training module is used to acquire a batch of clean samples and input them into the prediction model for training, and output a vector of predicted scale values.
[0054] Add a perturbation module to add a perturbation budget step size to the predicted scale value vector to obtain the predicted attack step size;
[0055] The target training module is used to input a batch of clean samples into the target model for training and to generate adversarial examples.
[0056] The joint training module performs joint training on the prediction model and the target model based on the obtained predicted attack step size and adversarial examples, resulting in the trained prediction model and the trained target model.
[0057] The beneficial effects of the method and system of this invention are as follows: This invention obtains a predicted scale value vector through a prediction model, and then adds a perturbation budget step size to the predicted scale value vector. Instead of using a fixed large attack step size, the prediction model is used to predict the attack step size of the sample, thereby solving the catastrophic overfitting problem in the existing FGSM-AT. At the same time, it greatly improves the adversarial robustness of the model. Furthermore, the target model used to generate adversarial examples only needs to perform one forward and backward propagation gradient calculation to generate good adversarial examples, which can effectively reduce the time cost of adversarial training. Attached Figure Description
[0058] Figure 1 This is a flowchart of the steps of an adversarial training method based on predicted attack step size according to the present invention;
[0059] Figure 2 This is a structural block diagram of an adversarial training system based on predicted attack step size according to the present invention;
[0060] Figure 3 This is a schematic diagram of the attack step size prediction process based on the prediction model of the present invention;
[0061] Figure 4 This is a schematic diagram of the adversarial sample generation process of the present invention. Detailed Implementation
[0062] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
[0063] Reference Figure 1 This invention provides an adversarial training method based on predicted attack step size, the method comprising the following steps:
[0064] S1. Obtain a batch of clean samples and input them into the prediction model for training, and output the predicted proportion vector.
[0065] Specifically, the prediction model (PM) consists of a residual block and a fully connected layer. The residual block consists of a 3×3 convolutional layer, a batch normalization layer, and a 1×1 convolutional layer. After the residual block, it needs to pass through ReLU activation, average pooling, and a Flatten layer, then through a fully connected layer, and finally through a Sigmoid activation layer to obtain the prediction scale value r.
[0066] Based on the residual blocks of the prediction model, feature extraction is performed on a batch of clean samples to obtain a batch clean sample feature vector. Then, based on the ReLU activation layer of the prediction model, a nonlinear factor is applied to the batch clean sample feature vector to increase the nonlinear relationship between the feature vectors, resulting in an optimized batch clean sample feature vector. Next, based on the average pooling layer of the prediction model, the optimized batch clean sample feature vector is compressed to obtain a compressed batch clean sample feature vector. Finally, based on the Flatten layer of the prediction model, the compressed batch clean sample feature vector is one-dimensionalized to obtain a one-dimensional batch clean sample feature vector. Finally, based on the fully connected layer and the Sigmoid activation layer of the prediction model, the one-dimensional batch clean sample feature vector is classified to obtain a predicted scale vector.
[0067] The residual block of the prediction model includes a 1×1 convolutional layer, a 3×3 convolutional layer, and a batch normalization layer. The 1×1 convolutional layer of the prediction model residual block is used to expand the number of channels in the batch of clean samples, resulting in expanded batch clean sample features. The 3×3 convolutional layer of the prediction model residual block is used to perform convolution operations on the batch of clean samples, resulting in batch clean sample features. The batch normalization layer of the prediction model residual block is used to standardize the batch clean sample features, resulting in standardized batch clean sample features. The expanded batch clean sample features and the standardized batch clean sample features are then added together to obtain the batch clean sample feature vector.
[0068] S2. Add a perturbation budget step to the predicted scale vector to obtain the predicted attack step.
[0069] Specifically, the process of obtaining the corresponding predicted attack step size from a batch of clean samples x through the prediction model is as follows: Figure 3 As shown, the output of the prediction model is r, and then r is multiplied by the constant value 1.25 to obtain the predicted attack step size α. The resulting α is a vector whose length is the batch size, that is, each sample will predict a different attack step size. And because the parameters of the target model and the prediction model will be continuously updated as training progresses, the attack step size predicted by the prediction model for the same sample will change as training progresses.
[0070] The specific expression for predicting the attack step size is as follows:
[0071] α(θ)=1.25∈×PM θ (x)
[0072] In the above formula, α represents the predicted attack step size, 1.25∈ represents a constant value, ∈ represents the perturbation budget (hyperparameter), θ represents the parameters of the prediction model, x represents the batch of clean samples, and PM θ (x) represents the prediction model;
[0073] This invention proposes a prediction model that uses clean samples as input to obtain a scale value (range 0-1), which is then multiplied by 1.25 times the perturbation budget (∈) to obtain the predicted attack step size (α). The attack step size of FGSM-AT is fixed at 1.25 times ∈. By using the prediction model to predict the attack step size of the samples instead of using a fixed large attack step size, the catastrophic overfitting problem in FGSM-AT is solved, and the adversarial robustness of the model is also greatly improved.
[0074] S3. Input a batch of clean samples into the target model for training to generate adversarial examples;
[0075] Specifically, the target model can be any deep convolutional neural network model for image classification, such as VGGNet, GoogLeNet, ResNet, and MobileNet.
[0076] Reference Figure 4 A batch of clean samples x is randomly initialized in the range [-∈ to ∈] to obtain x+η. Then, x+η is input into the target model (TM) for one forward propagation, followed by back propagation (BP) to calculate the gradient sign of the target model parameters, denoted as . Then multiply it with the attack step size α predicted by the prediction model, add the result of the multiplication to the random initial value η, and then project (restrict) the result of the addition to [-∈~∈] to obtain the adversarial perturbation δ, and x+δ is the adversarial sample;
[0077] The specific expression for generating adversarial examples is as follows:
[0078]
[0079] In the above formula, x+δ represents generating adversarial examples, and η represents random initialization values. The gradient sign represents the target model parameters, and x+η represents the result obtained by randomly initializing a batch of clean samples x.
[0080] S4. Based on the obtained predicted attack step size and adversarial examples, jointly train the prediction model and the target model to obtain the trained prediction model and the trained target model.
[0081] Specifically, the target model and the prediction model are jointly trained. The purpose of training the prediction model is to update its parameters (θ) to increase the loss of the target model, while the purpose of training the target model is to update its parameters (w) to minimize its loss. The objective of the joint training can be defined as:
[0082]
[0083] In the above formula, w represents the parameters of the target model, L(·) represents the loss function of the target model, D represents the training set constructed from a batch of clean samples, and E (x,y)～D Let represent the expected value of the target model loss under the distribution (x,y)~D, where y represents the label corresponding to the clean sample.
[0084] Further explanation of the parameters used for joint training: Hyperparameters: N represents the number of times the training set is used for training; B represents the batch size (i.e., the minimum number of samples in one training iteration); ∈ represents the perturbation budget; k represents the frequency of joint training of the prediction model and the target model, meaning that after training the prediction model once, the target model needs to be trained k times before the next training iteration of the prediction model. Other descriptors: PM θ (·) represents the prediction model, θ represents its parameters; x represents a sample in the training set, and y represents its corresponding label; ∏ [-∈,∈] [·] indicates projecting the calculation result within the brackets [] onto the range [-∈~∈]; L(x+η,y;w) indicates calculating the loss of the target model;
[0085] The specific steps of the algorithm logic for the adversarial training method based on predicting attack step size are as follows:
[0086] S41. Use a for loop to iterate through N, i = 0, ..., N-1, where i represents the (i+1)th iteration of the training set.
[0087] S42. Use a for loop to iterate through B, j = 0, ..., B-1, where j represents training the (j+1)th batch;
[0088] S43, Calculate η ij η ij This represents the random initialization value for the (j+1)th batch in the (i+1)th iteration of the training set. If imodk = 0, then...
[0089] S44, via α(θ)=1.25∈×PM θ (x ij ), calculate x ij The attack step size α(θ);
[0090] S45, Through Calculate the counter-perturbation δ(θ);
[0091] S46, Through Update the parameters θ of the prediction model;
[0092] S47, via α=1.25∈×PM θ (x ij ), calculate x ij The attack step size α;
[0093] S48, Through Calculate the counter-disturbance δ;
[0094] S49, Through Update the parameters w of the target model;
[0095] The clean sample training set (D) is trained in batches, which are divided into M batches for adversarial training. The adversarial training scans the sample set N times, where n = 1, ..., N represents the nth scan of D, i = 0, ..., M-1 represents the i-th batch participating in the training, k represents the k times the target model is trained after each training of the prediction model, and L is the loss function of the target model. Steps S44 to S46 train the prediction model and use the gradient ascent algorithm to update the parameters of the prediction model, thereby increasing the loss of the target model, i.e., increasing L. Steps S47 to S49 train the target model and use the gradient descent algorithm to update the parameters of the target model, thereby decreasing the loss of the target model, i.e., decreasing L. Under this game mechanism, the trained target model has a very strong ability to resist adversarial attacks. It can not only resist low adversarial attacks (such as FGSM) but also resist various strong adversarial attacks (such as PGD, AutoAttack, etc.).
[0096] This invention proposes a novel adversarial training method for models. The target model and the prediction model are jointly trained. A hyperparameter k is designed, and the target model is trained k times. Each time, a batch (default 128) of clean samples is used. The attack step size obtained by the prediction model is applied to the FGSM adversarial attack. The resulting batch of adversarial samples is used to train the target model. The parameters of the target model are updated to reduce the loss of the target model. After updating the parameters of the target model k times, the parameters of the prediction model are updated once to increase the loss of the target model. Under this adversarial training mechanism, the attack step size predicted by the prediction model will become more and more effective, and the robustness of the target model will become stronger and stronger. That is, it can not only resist FGSM attacks, but also resist various strong adversarial attack algorithms, such as PGD, Auto Attack, etc.
[0097] The simulation experiment process of this invention is as follows:
[0098] The dataset is CIFAR-10, which has 50,000 training images and 10,000 test images. The image size is 32×32×3. This invention uses the CIFAR-10 training set for training and the test set for validation.
[0099] The target model uses ResNet18, and training is performed for 110 epochs with a batch size of 128. The initial learning rate of the target model is 0.1, the momentum value is 0.9, and the weight decay is 0.0005. The learning rate is multiplied by 0.1 at the 100th and 105th epochs, respectively, and ∈ is set to 8 / 255.
[0100] The attack step size α of FGSM-AT is set to 10 / 255, the α of PGD-AT is set to 2 / 255, the number of iterations is 10, the hyperparameter k in the method proposed in this invention is set to 20, and the learning rate of the prediction model is a fixed value of 0.0003.
[0101] The prediction results of the target model trained by the method of this invention on the test set are compared with those of the existing FGSM-AT and PGD-AT. The data is shown in the table below. All experiments were conducted using two Tesla P100 chips.
[0102] Table 1. Comparison of the prediction results of the method of the present invention with existing FGSM-AT and PGD-AT data.
[0103] FGSM-AT 79.53 0.19 0.06 0.02 0.00 1.26 PGD-AT 82.78 50.63 49.56 49.34 45.47 5.34 Ours 86.31 50.84 49.99 49.57 46.44 1.05
[0104] The experimental results in the table above show that the target model trained by the method proposed in this invention has the best adversarial robustness and the least time consumption.
[0105] Reference Figure 2 An adversarial training system based on predicting attack step size includes:
[0106] The prediction training module is used to acquire a batch of clean samples and input them into the prediction model for training, and output a vector of predicted scale values.
[0107] Add a perturbation module to add a perturbation budget step size to the predicted scale value vector to obtain the predicted attack step size;
[0108] The target training module is used to input a batch of clean samples into the target model for training and to generate adversarial examples.
[0109] The joint training module performs joint training on the prediction model and the target model based on the obtained predicted attack step size and adversarial examples, resulting in the trained prediction model and the trained target model.
[0110] Image classification models are now widely used. For example, the Baidu app embeds an image classification model to identify objects in photos. However, general image classification models, i.e., those not trained with adversarial examples, can lead to significant deviations in classification results when subtle perturbations are added to the image. For instance, adding a small perturbation to a panda image might cause it to be classified as a gibbon. The target model obtained in this solution can replace general image classification models to improve the robustness of image classification software. The prediction model in this solution is used to assist in the training of the target model. The trained target model can then be used in image classification software to achieve more robust image classification capabilities.
[0111] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.
[0112] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.
Claims
1. An adversarial training method based on predicted attack step size, characterized in that, Includes the following steps: A batch of clean samples is acquired and input into the prediction model for training, and the predicted ratio vector is output; the clean samples are images. Add a perturbation budget step to the predicted scale value vector to obtain the predicted attack step size; A batch of clean samples is input into the target model for training to generate adversarial examples; Based on the predicted attack step size and adversarial examples, the prediction model and the target model are jointly trained to obtain the trained prediction model and the trained target model. The step of inputting a batch of clean samples into the target model for training to generate adversarial examples specifically includes: For batches of clean samples Random initialization perturbation is added between them to obtain random initialization results; in, Indicates the budget for disturbances; The random initialization results are input into the target model for one forward propagation training to obtain the forward propagation training results; Backpropagation training is performed based on the forward propagation training results to calculate the gradient of the target model parameters; The gradient sign of the extracted target model parameters is multiplied with the predicted attack step size to obtain the multiplication result; The calculated multiplication result is added to the random initialization perturbation to obtain the summation result; Project the summation result onto Between these, adversarial examples are obtained.
2. The adversarial training method based on predicted attack step size according to claim 1, characterized in that, The step of obtaining a batch of clean samples and inputting them into the prediction model for training to obtain the predicted proportion vector specifically includes: Obtain a batch of clean samples; A prediction model is constructed, which includes a residual block, a ReLU activation layer, an average pooling layer, a Flatten layer, a fully connected layer, and a Sigmoid activation layer. Input a batch of clean samples into the prediction model; Based on the residual blocks of the prediction model, feature extraction is performed on a batch of clean samples to obtain the feature vector of the batch of clean samples. Based on the ReLU activation layer of the prediction model, a nonlinear factor is applied to the feature vector of the batch clean samples to increase the nonlinear relationship between the feature vectors of the batch clean samples, thus obtaining the optimized feature vector of the batch clean samples. Based on the average pooling layer of the prediction model, the optimized batch clean sample feature vector is compressed to obtain the compressed batch clean sample feature vector. Based on the Flatten layer of the prediction model, the compressed batch clean sample feature vector is processed into one dimension to obtain a one-dimensional batch clean sample feature vector. Based on the fully connected layer and sigmoid activation layer of the prediction model, the feature vector of a batch of one-dimensional clean samples is classified to obtain the predicted proportion value vector.
3. The adversarial training method based on predicted attack step size according to claim 2, characterized in that, The step of extracting features from a batch of clean samples using the residual block based on the prediction model to obtain the feature vector of the batch of clean samples specifically includes: Construct a residual block for the prediction model, wherein the residual block of the prediction model includes a 1×1 convolutional layer, a 3×3 convolutional layer and a batch normalization layer; The 1×1 convolutional layer based on the residual block of the prediction model is used to expand the number of channels of the batch of clean samples to obtain the expanded features of the batch of clean samples. A 3×3 convolutional layer based on the residual block of the prediction model is used to perform convolution operations on a batch of clean samples to obtain the features of the batch of clean samples. The batch normalization layer based on the residual block of the prediction model is used to standardize the features of the batch clean samples to obtain the standardized features of the batch clean samples. The expanded batch clean sample features are added to the standardized batch clean sample features to obtain the batch clean sample feature vector.
4. The adversarial training method based on predicted attack step size according to claim 3, characterized in that, The specific expression for predicting the attack step size is as follows: In the above formula, This represents the predicted attack step size. Indicates a constant value. This represents the perturbation budget, or hyperparameter. The parameters of the prediction model, Indicates a batch of clean samples. This represents the prediction model.
5. The adversarial training method based on predicted attack step size according to claim 4, characterized in that, The specific expression for generating adversarial examples is as follows: In the above formula, This indicates the generation of adversarial examples. Indicates a random initialization value. The gradient sign representing the target model parameters. Indicates a batch of clean samples The result obtained by random initialization.
6. The adversarial training method based on predicted attack step size according to claim 5, characterized in that, The step of jointly training the prediction model and the target model based on the obtained predicted attack step size and adversarial examples to obtain the trained prediction model and the trained target model specifically includes: Loop traversal , ,in, Indicates the first term of the training set. Second traversal, Indicates the number of times the training set was used for training; Loop traversal , ,in, Indicates the first Train in batches. This represents the size of a batch, i.e., the minimum number of samples required for a single training session. calculate ,in, Indicates the training set of the first... The th traversal A batch of random initialization values; like ,but: pass ,calculate attack stride ; pass Calculate counter-disturbance ; pass Update the parameters of the prediction model ; pass ,calculate attack stride ; pass Calculate counter-disturbance ; pass Update the parameters of the target model ; Based on the updated prediction model parameters and the updated target model parameters, the trained prediction model and the trained target model are constructed.
7. The adversarial training method based on predicted attack step size according to claim 6, characterized in that, The expression for the preset joint training objective of the prediction model and the target model is as follows: In the above formula, These represent the parameters of the target model. Represents the loss function of the target model. This represents the training set constructed from a batch of clean samples. Indicates in The expected value of the target model loss under this distribution. The label represents the clean sample.
8. An adversarial training system based on predicted attack step size, characterized in that, A method for performing adversarial training based on predicted attack step size as described in claim 1 includes the following modules: The prediction training module is used to acquire a batch of clean samples and input them into the prediction model for training, and output a vector of predicted scale values. Add a perturbation module to add a perturbation budget step size to the predicted scale value vector to obtain the predicted attack step size; The target training module is used to input a batch of clean samples into the target model for training and to generate adversarial examples. The joint training module performs joint training on the prediction model and the target model based on the obtained predicted attack step size and adversarial examples, resulting in the trained prediction model and the trained target model.