Adversarial training method based on prior category enhancement
Through an adversarial training method based on prior category enhancement, adversarial samples are generated and neural networks are adjusted, which solves the problem of low robustness and low accuracy of important categories in the prior art, significantly improves the robustness of key categories and maintains overall robustness. .
Patent Information
- Application Number
- CN202110490295.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-06
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2041-05-06
AI Technical Summary
There are significant differences in the robust accuracy performance of existing adversarial training methods under different categories, especially in the problem of low robustness for important categories, which may result in serious consequences and significant losses in key scenarios.
Adopt the adversarial training method based on prior category enhancement, by obtaining training data and neural networks, we determine the prediction tag after the sample is attacked by FGSM, and generate adversarial samples based on the prediction tags, adjust the neural network to improve the robustness of important categories. Accuracy.
It effectively improves the robust accuracy for important categories, while maintaining the overall average robust accuracy. It is suitable for a variety of adversarial training systems, significantly improving the classification stability under key categories.
Smart Images
Figure CN113240080B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence, and in particular to an adversarial training method based on priori category enhancement. Background Art
[0002] Deep neural networks (DNNs) have been widely used in various fields of life, such as image recognition, text processing, and speech recognition. However, some scholars have pointed out that by adding artificially designed subtle noise, attackers can easily mislead the trained DNN with the added noise samples, causing it to misclassify. Depending on the attacker's noise design method, the attacker can achieve his own purpose. This attack is called adversarial attack, the added noise is called adversarial noise, the added noise samples are called adversarial samples, and the model's test accuracy for adversarial samples is called robust accuracy. There are many ways to defend against such attacks, and one of the most widely recognized defense methods is called adversarial training. The approach is to add adversarial samples as training samples during the training process to improve the generalization and robustness of the model. However, although the existing adversarial training methods can improve the robust accuracy of the model for the overall test set, its performance in different categories is quite different. In reality, there are many scenarios that have high requirements for the classification robustness of specific categories, such as the recognition of pornographic and violent images in network security and the recognition of pathological images in medical image classification. Existing methods cannot adapt to these importance priors, which in the worst case will have serious consequences and significant losses if the important classes happen to be the classes with the worst robust accuracy.
[0003] Therefore, the prior art still needs to be improved and developed. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide an adversarial training method based on prior category enhancement in view of the above-mentioned defects of the prior art, aiming to solve the problem of low robust accuracy of important categories in the prior art.
[0005] The technical solution adopted by the present invention to solve the technical problem is as follows:
[0006] A method for adversarial training based on prior category enhancement, comprising the steps of:
[0007] Acquire training data and a neural network, and determine the predicted label of the sample after being attacked by FGSM according to the training data and the neural network; the training data includes: a training sample and a true label corresponding to the training sample;
[0008] Determining, according to the training data, the neural network, and the predicted label, an adversarial sample corresponding to the training data;
[0009] According to the adversarial sample and the true label, the parameters of the neural network are adjusted to obtain a trained neural network.
[0010] The adversarial training method based on prior category enhancement, wherein the step of determining the predicted label of the sample after being attacked by FGSM based on the training data and the neural network, comprises:
[0011] Performing an FGSM attack on the training data to obtain weak adversarial samples corresponding to the training data;
[0012] The weak adversarial sample is input into the neural network to obtain a predicted label of the sample after being attacked by FGSM.
[0013] The adversarial training method based on prior category enhancement, wherein the weak adversarial sample is:
[0014]
[0015] in, represents a weak adversarial sample, x i represents the training sample, y i represents the true label, f θ (·) represents the neural network, l(·) represents the loss function corresponding to the adversarial attack, ε represents the maximum perturbation range, sgn(·) represents the sign function, Represents the gradient operator.
[0016] The adversarial training method based on prior category enhancement, wherein the step of determining the adversarial sample corresponding to the training data according to the training data, the neural network and the predicted label, comprises:
[0017] When the predicted label is a preset label, determining the adversarial noise corresponding to the training data according to the training data, the neural network and the predicted label;
[0018] When the predicted label is not a preset label, determining the adversarial noise corresponding to the training data according to the training data and the neural network;
[0019] An adversarial sample corresponding to the training data is determined according to the training sample and the adversarial noise.
[0020] In the adversarial training method based on prior category enhancement, when the predicted label is a preset label, the adversarial noise is:
[0021]
[0022] Among them, δ irepresents the adversarial noise corresponding to the training data, max(·) represents the maximum value function, ε represents the maximum perturbation range, and B p (0,ε) represents the L of the generated adversarial noise p The norm is limited to ε, λ represents a hyperparameter, l(·) represents the loss function corresponding to the adversarial attack, and f θ (·) represents the neural network, x i represents the training sample, δ represents the generated adversarial noise, y i represents the true label, y * Represents the predicted label.
[0023] In the adversarial training method based on prior category enhancement, when the predicted label is not a preset label, the adversarial noise is:
[0024]
[0025] Among them, δ i represents the adversarial noise corresponding to the training data, max(·) represents the maximum value function, ε represents the maximum perturbation range, and B p (0,ε) represents the L of the generated adversarial noise p The norm is limited to ε, l(·) represents the loss function corresponding to the adversarial attack, and f θ (·) represents the neural network, x i represents the training sample, δ represents the generated adversarial noise, y i represents the true label.
[0026] The adversarial training method based on prior category enhancement, wherein the adversarial sample is:
[0027]
[0028] in, represents adversarial samples, x i represents the training sample, δ i Represents the adversarial noise corresponding to the training data.
[0029] The adversarial training method based on prior category enhancement, wherein the parameters of the neural network are adjusted according to the adversarial sample and the true label to obtain the trained neural network, comprises:
[0030] Inputting the adversarial sample into the neural network, and obtaining a generated label corresponding to the adversarial sample through the neural network;
[0031] According to the generated label and the true label, the parameters of the neural network are adjusted, and the step of determining the adversarial sample corresponding to the training data according to the training data, the neural network and the predicted label is continued until the neural network meets the preset conditions to obtain a trained neural network.
[0032] A computer device comprises a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of any one of the above methods when executing the computer program.
[0033] A computer-readable storage medium stores a computer program, wherein the computer program implements the steps of any of the above methods when executed by a processor.
[0034] Beneficial effect: By first determining the predicted label of the sample after being attacked by FGSM, and determining the adversarial sample corresponding to the training data based on the training data, the neural network and the predicted label, and then using the adversarial sample for training to obtain the trained neural network, the robustness accuracy of the neural network to the preset label can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is the distribution diagram of robust accuracy of each category after adversarial training of CIFAR10 using the training method distribution in the prior art.
[0036] Figure 2 It is the distribution diagram of robust accuracy of each category after adversarial training of CIFAR100 using the training method distribution in the prior art.
[0037] Figure 3 It is a flowchart of the adversarial training method based on prior category enhancement in the present invention. DETAILED DESCRIPTION
[0038] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0039] Please also see Figure 1-Figure 3 , the present invention provides some embodiments of an adversarial training method based on prior category enhancement.
[0040] Adversarial training is a widely recognized adversarial defense method. Its basic idea is to input adversarial samples as training samples into the model for training. Taking the image classification problem as an example, the adversarial noise is generated as follows:
[0041]
[0042] Where (x i ,y i ) represents the input training samples and their corresponding true labels, f θ is a deep neural network model, l is the loss function corresponding to the adversarial attack, and cross entropy loss is usually used in classification problems. p (0,ε) represents the L of the generated adversarial noise p The norm must be limited to the maximum perturbation range ε, p∈{0,1,2,∞}, δ i This is the final generated adversarial noise.
[0043] Actual calculation of δ i The process usually uses the iterative gradient ascent method, which is calculated as follows:
[0044]
[0045] in Represents sample x i The adversarial sample obtained after the tth iteration, α is the step size of each iteration, which is usually much smaller than ε.
[0046] like Figure 1 and Figure 2 As shown in the figure, adversarial training is performed on CIFAR10 and CIFAR100 through the training method distribution of the background technology, and the robust accuracy distribution diagram of each category is drawn. It can be seen that there is an obvious imbalance in the robust accuracy between classes. On the one hand, this imbalance is not desirable; on the other hand, the importance of robust accuracy of different categories is different in many tasks, but the existing technology does not consider this a priori. Considering the worst case, if the most important category happens to be trained as the category with the worst robust accuracy due to the defects of the original technology, it will inevitably cause significant losses.
[0047] Before training, you need to pre-specify a category (preset label) as the important category that needs to be enhanced. Here, we assume that the important category is y * , where y * ∈{1,2,...,K}, K represents the number of categories, that is, the number of tags. It can be understood that the preset tags can be one or more.
[0048] like Figure 3 As shown, an adversarial training method based on prior category enhancement of the present invention comprises the following steps:
[0049] Step S100, obtaining training data and a neural network, and determining a predicted label of a sample after being attacked by FGSM based on the training data and the neural network; the training data includes: a training sample and a true label corresponding to the training sample.
[0050] Specifically, during adversarial training, some samples that are not important categories are easily misclassified as important categories after being attacked by adversarial attacks. These samples are considered to be on the boundary of important categories, so they are called boundary samples. These boundary samples will have a significant impact on the classification accuracy of important categories, so we need to find these boundary samples first.
[0051] The training data can be a training dataset In each training, a batch of samples is used Training is performed, where M is less than N.
[0052] Step S100 specifically includes:
[0053] Step S110: Perform an FGSM attack on the training data to obtain weak adversarial samples corresponding to the training data.
[0054] Step S120: input the weak adversarial sample into the neural network to obtain a predicted label of the sample after being attacked by FGSM.
[0055] First, a Fast Gradient Sign Method (FGSM) attack is performed on the input sample. This is a relatively weak attack, but its advantage is that the computational complexity is low. The weak adversarial sample is:
[0056]
[0057] in, represents a weak adversarial sample, x i represents the training sample, y i represents the true label, f θ (·) represents the neural network, l(·) represents the loss function corresponding to the adversarial attack, ε represents the maximum perturbation range, sgn(·) represents the sign function, Represents the gradient operator.
[0058] Then the weak adversarial sample is input into the neural network model to obtain the predicted label corresponding to the weak adversarial sample. if This indicates that the sample is extremely vulnerable to being misclassified as an important category, which is the boundary sample we are looking for.
[0059] Step S200: determining an adversarial sample corresponding to the training data according to the training data, the neural network and the predicted label.
[0060] Specifically, the predicted label of the sample after being attacked by FGSM is used to determine the adversarial sample corresponding to the training data. The training data is distinguished according to whether the predicted label is a preset label, so as to obtain different adversarial samples. Therefore, when the parameters of the neural network are adjusted according to the adversarial sample, the robust accuracy of the neural network for important categories is improved.
[0061] Step S200 specifically includes:
[0062] Step S210: When the predicted label is a preset label, determine the adversarial noise corresponding to the training data according to the training data, the neural network and the predicted label.
[0063] Step S220: When the predicted label is not a preset label, determine the adversarial noise corresponding to the training data according to the training data and the neural network.
[0064] Step S230: Determine an adversarial sample corresponding to the training data according to the training sample and the adversarial noise.
[0065] Specifically, when the predicted label is a preset label, the adversarial noise corresponding to the training data is determined based on the training data, the neural network and the predicted label. When the predicted label is not a preset label, the adversarial noise corresponding to the training data is determined based on the training data and the neural network. In other words, if the predicted label of the sample after being attacked by FGSM is the preset label, then the training sample is a boundary sample, and the boundary sample will affect the robust accuracy of the neural network for important categories. Therefore, it is necessary to introduce the predicted label to obtain the adversarial noise; if the predicted label of the sample after being attacked by FGSM is not the preset label, then the training sample is not a boundary sample, and the adversarial noise can be obtained only based on the training data and the neural network.
[0066] When the predicted label is a preset label, the adversarial noise is:
[0067]
[0068] Among them, δ i represents the adversarial noise corresponding to the training data, max(·) represents the maximum value function, ε represents the maximum perturbation range, and B p (0,ε) represents the L of the generated adversarial noise p The norm is limited to ε, λ represents a hyperparameter, l(·) represents the loss function corresponding to the adversarial attack, and f θ (·) represents the neural network, x irepresents the training sample, δ represents the generated adversarial noise, y i represents the true label, y * Represents the predicted label.
[0069] When the predicted label is the preset label and the training sample is a boundary sample, a new internal maximization loss function is proposed so that the boundary sample not only deviates away from the true label, but also away from the important category label. In this way, the trained neural network will not classify the training sample into an important category, and the robust accuracy of the important category is improved.
[0070] When the predicted label is not a preset label, the adversarial noise is:
[0071]
[0072] Among them, δ i represents the adversarial noise corresponding to the training data, max(·) represents the maximum value function, ε represents the maximum perturbation range, and B p (0,ε) represents the L of the generated adversarial noise p The norm is limited to ε, l(·) represents the loss function corresponding to the adversarial attack, and f θ (·) represents the neural network, x i represents the training sample, δ represents the generated adversarial noise, y i represents the true label.
[0073] If the predicted label is not the preset label and the training sample is not a boundary sample, the adversarial sample generation method of the existing technology is used without changing the internal maximization loss function.
[0074] After obtaining the adversarial noise, an adversarial sample is obtained based on the adversarial noise and the training sample. The adversarial sample is:
[0075]
[0076] in, represents adversarial samples, x i represents the training sample, δ i Represents the adversarial noise corresponding to the training data.
[0077] Step S300: According to the adversarial sample and the true label, the parameters of the neural network are adjusted to obtain a trained neural network.
[0078] Specifically, after obtaining the adversarial sample, the parameters of the neural network are adjusted according to the adversarial sample and the true label to obtain the trained neural network. Since the adversarial sample is obtained based on the training data, the neural network and the predicted label, when the adversarial sample is used to train the neural network, the trained neural network has a higher robust accuracy for important categories.
[0079] Step S300 specifically includes:
[0080] Step S310: input the adversarial sample into the neural network, and obtain a generated label corresponding to the adversarial sample through the neural network.
[0081] Step S320: adjusting the parameters of the neural network according to the generated label and the true label, and continuing to execute the step of determining the adversarial sample corresponding to the training data according to the training data, the neural network and the predicted label, until the neural network meets the preset conditions, thereby obtaining a trained neural network.
[0082] Specifically, the adversarial sample After inputting into the neural network, the generated label is obtained Then generate labels based on and the true label y i Calculate the overall loss L of the neural network, for example, Then, according to the gradient descent method, the parameters of the neural network are updated through back propagation. If the neural network meets the preset conditions, a trained neural network is obtained; if the neural network does not meet the preset conditions, step S200 is continued.
[0083] The preset condition may be that the neural network converges, for example, the overall loss is less than a preset threshold. The preset condition may also be that the neural network training times reach a preset number of times, for example, after training 1000 times, the training is stopped to obtain a trained neural network.
[0084] The present invention is tested on CIFAR10, CIFAR100 and SVHN datasets. The first class in the dataset is taken as the important category. The average robust accuracy (Avg) and the robust accuracy (Imp) of the important category of the model under different white-box attacks are compared. The experimental results of this method nested in the current mainstream framework in the industry are compared:
[0085] Table 1. Comparison of robust accuracy of two training methods under different attack modes for different datasets
[0086]
[0087] Table 2. Robust accuracy of different adversarial training methods on CIFAR10 dataset under different adversarial attacks
[0088]
[0089] It can be seen from the results in Table 1 and Table 2 that under different attack modes or different adversarial training methods, the robust accuracy (Imp) of important categories of CART-AT, CART-TRADES, and CART-MART using the method of the present invention is improved, and the overall average robust accuracy remains basically unchanged.
[0090] The present invention has the following technical effects:
[0091] [1] This scheme can effectively utilize the prior knowledge of category importance and significantly improve the robust accuracy of the target (important) category while maintaining the average robust accuracy almost unchanged.
[0092] [2] This solution is universally applicable to systems that include adversarial training and can be embedded in almost any system that includes adversarial training.
[0093] Based on the adversarial training method based on prior category enhancement in the above embodiment, the present invention also provides an embodiment of a computer device.
[0094] The computer device, memory and processor of the present invention, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:
[0095] S100, obtaining training data and a neural network, and determining a predicted label of a sample after being attacked by FGSM according to the training data and the neural network; the training data includes: a training sample and a true label corresponding to the training sample;
[0096] S200, determining an adversarial sample corresponding to the training data according to the training data, the neural network and the predicted label;
[0097] S300: training the neural network according to the adversarial sample and the true label to obtain a trained neural network.
[0098] Based on the adversarial training method based on prior category enhancement in the above embodiment, the present invention also provides an embodiment of a computer-readable storage medium.
[0099] The computer-readable storage medium of the present invention stores a computer program thereon, and when the computer program is executed by a processor, the following steps are implemented:
[0100] S100, obtaining training data and a neural network, and determining a predicted label of a sample after being attacked by FGSM according to the training data and the neural network; the training data includes: a training sample and a true label corresponding to the training sample;
[0101] S200, determining an adversarial sample corresponding to the training data according to the training data, the neural network and the predicted label;
[0102] S300: training the neural network according to the adversarial sample and the true label to obtain a trained neural network.
[0103] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.
Claims
1. An adversarial training method based on prior category enhancement, It is characterized in that Includes steps: Acquire training data and a neural network, and determine a predicted label of a sample after being attacked by FGSM based on the training data and the neural network; The training data includes: training samples and real labels corresponding to the training samples; Determining, according to the training data, the neural network, and the predicted label, an adversarial sample corresponding to the training data; According to the adversarial sample and the true label, adjusting the parameters of the neural network to obtain a trained neural network; The determining, according to the training data, the neural network, and the predicted label, an adversarial sample corresponding to the training data includes: When the predicted label is a preset label, determining the adversarial noise corresponding to the training data according to the training data, the neural network and the predicted label; wherein the preset label is a label that is pre-specified and needs to be enhanced; When the predicted label is not a preset label, determining the adversarial noise corresponding to the training data according to the training data and the neural network; Determining, according to the training sample and the adversarial noise, an adversarial sample corresponding to the training data; When the predicted label is a preset label, the adversarial noise is: in, represents the adversarial noise corresponding to the training data, represents the maximum value function, represents the maximum disturbance range, represents the generated adversarial noise The norm is restricted to within, represents the hyperparameter, represents the loss function corresponding to the adversarial attack, represents a neural network, represents the training sample, represents the generated adversarial noise, represents the true label, represents the predicted label; The adversarial example is: in, represents adversarial samples, represents the training sample, represents the adversarial noise corresponding to the training data; When the predicted label is not a preset label, the adversarial noise is: in, represents the adversarial noise corresponding to the training data, represents the maximum value function, represents the maximum disturbance range, represents the generated adversarial noise The norm is restricted to within, represents the loss function corresponding to the adversarial attack, represents a neural network, represents the training sample, represents the generated adversarial noise, represents the true label.
2. The adversarial training method based on prior category enhancement according to claim 1, It is characterized in that Determining the predicted label of the sample after being attacked by FGSM according to the training data and the neural network includes: Performing an FGSM attack on the training data to obtain weak adversarial samples corresponding to the training data; The weak adversarial sample is input into the neural network to obtain a predicted label of the sample after being attacked by FGSM.
3. The adversarial training method based on prior category enhancement according to claim 2, It is characterized in that The weak adversarial example is: in, represents a weak adversarial example, represents the training sample, represents the true label, represents a neural network, represents the loss function corresponding to the adversarial attack, represents the maximum disturbance range, represents the symbolic function, Represents the gradient operator.
4. The adversarial training method based on prior category enhancement according to any one of claims 1 to 3, It is characterized in that The step of adjusting the parameters of the neural network according to the adversarial sample and the true label to obtain a trained neural network includes: Inputting the adversarial sample into the neural network, and obtaining a generated label corresponding to the adversarial sample through the neural network; According to the generated label and the true label, the parameters of the neural network are adjusted, and the step of determining the adversarial sample corresponding to the training data according to the training data, the neural network and the predicted label is continued until the neural network meets the preset conditions to obtain a trained neural network.
5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.
6. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.