Adversarial sample optimization method based on decision boundary and trust region
Through a multi-level optimization method based on decision boundaries and trust domains, the problem of large perturbation in the existing adversarial sample generation method is solved. The generated adversarial sample perturbation is reduced by more than 40%, and it is more in line with human visual cognition.
Patent Information
- Application Number
- CN202210225326.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-09
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing adversarial sample generation methods have large perturbations, and it is not guaranteed that the perturbations are minimal and conform to human visual effects.
A multi-level optimization method based on decision boundaries and trust domains is adopted to generate adversarial samples through white box attacks, and the fitness function of the cross entropy loss function and Euclidean distance is used for iterative updates to ensure the minimum perturbation and the visual effect is optimized.
The perturbation of the generated adversarial samples was significantly reduced, which was more in line with human visual cognition, and the perturbation was reduced by more than 40%.
Abstract
Description
Technical Field
[0001] The present invention relates to an adversarial sample optimization method based on decision boundary and trust region, and belongs to the field of deep learning and image recognition. Background Art
[0002] With the development of artificial intelligence technology, deep neural networks are currently widely used in various fields such as image recognition, speech recognition, natural language processing, bioinformatics processing, robotics, chemistry, computer games, search engines, online advertising, medical automatic diagnosis and finance. Especially in the field of computer vision, deep neural networks perform very well, and the accuracy of image recognition tasks has far exceeded the accuracy of human recognition. However, in some specific occasions, adversarial samples of images are needed to cause neural networks to misidentify in order to protect the original objects. Adversarial attacks refer to adding carefully designed slight perturbations to the original samples to interfere with the neural network model, causing the neural network model to misclassify the image without affecting the human visual effect.
[0003] Adversarial samples were first proposed by Szegedy et al., who intentionally added subtle interference to the input image to form adversarial samples. The interfered input will cause the classification model to give a high-confidence error output, which means that adversarial samples have become a blind spot for neural network training. Later, Goodfellow et al. pointed out that the main reason why deep neural networks are attacked by adversarial samples is due to the existence of linear parts, and proposed a method to quickly generate adversarial samples for white-box attacks. The discovery of adversarial samples has promoted the study of the robustness of neural networks.
[0004] At present, the white-box attack methods proposed by the academic community are mainly divided into gradient-based attack methods and optimization algorithm-based attack methods. Among them, the gradient-based attack method has a better effect. The gradient-based attack method mainly uses the LP norm to measure the difference between the original sample and the adversarial sample, but this method can only ensure that the maximum pixel difference between the images meets certain conditions, which is not the optimal value for human vision. At present, the most adversarial method of gradient-based white-box attack is the optimization algorithm-based attack method (referred to as the C&W attack method) proposed by Carlini and Wagner in the paper "Towards Evaluating the Robustness of Neural Networks", which can effectively break various deep neural networks. The C&W attack method includes three distance measurement methods: L0, L2, and L∞. Among them, the L2 distance measurement method has the best human visual effect. However, whether it is a gradient-based or optimization algorithm-based attack method, the main measurement standard is the misclassification measurement between the recognition classification and the correct classification, which cannot guarantee the minimum disturbance. Summary of the invention
[0005] The purpose of the present invention is to solve the problem of large disturbance added when generating adversarial samples by existing attack methods, and proposes an adversarial sample optimization method based on decision boundary and trust region.
[0006] The objective of the present invention is achieved through the following technical solutions:
[0007] An adversarial sample optimization method based on decision boundary and trust region: adversarial samples are generated using a white-box attack method, and then multi-level optimization of the adversarial samples is performed based on the decision boundary and trust region to guide the generation of adversarial samples with minimal perturbation.
[0008] Preferably, the multi-level optimization of adversarial samples based on decision boundaries and trust regions includes the following contents:
[0009] 1. Determine the fitness function used to select elite samples;
[0010] 2. Iteratively updating the samples generated based on the adversarial samples until the samples are optimal, that is, the disturbance is minimized;
[0011] 3. Adjust the hyperparameters in the iterative update process of the sample.
[0012] Preferably, the fitness function is the sum of the cross entropy loss function and the distance between the original sample and the current sample.
[0013] Preferably, the distance between the original sample and the current sample is a Euclidean distance.
[0014] Preferably, the cross entropy loss function value is the cross entropy of the output vector of the last fully connected layer of the sample query attack model and the one-hot encoding of the true label.
[0015] Preferably, the iterative updating process is:
[0016] ① Let sample t be the adversarial sample and the number of iterations k be 0;
[0017] ②Add adversarial noise to t to generate a temporary sample group;
[0018] ③ Use the fitness function to score all samples in the temporary sample group, and select the best sample as the elite sample;
[0019] ④ Project the elite sample onto the original sample to obtain a temporary adversarial sample;
[0020] ⑤ Let t be the temporary adversarial sample, k = k + 1;
[0021] ⑥If k is equal to the preset maximum number of iterations maxIter, then t is output as the adversarial sample with the least disturbance; end;
[0022] ⑦ If the mean square error values between the temporary adversarial sample and the original sample obtained in the last 10 iterations are the same, then t is output as the adversarial sample with the minimum disturbance; end;
[0023] ⑧Return to ② and continue iterating.
[0024] Preferably, the adversarial noise is adversarial noise under L∞ restriction; wherein L∞ represents the similarity between the adversarial sample and the original sample.
[0025] Preferably, during the projection, it is ensured that the visual noise effect of the generated temporary adversarial sample always converges within the L∞ limit range.
[0026] Preferably, the step size of the projection is adjusted according to the following principle: the proportion λ of temporary samples in the temporary sample group that meets the following conditions is counted: for targeted attacks, the query label of the temporary sample should be equal to the target label; for non-target attacks, the query label of the temporary sample is not equal to the original sample; if 0.3≤λ≤0.7, the step size does not need to be adjusted; if λ is less than 0.3, the step size is increased; if λ is greater than 0.7, the step size is reduced.
[0027] Beneficial Effects
[0028] Compared with the existing adversarial sample direct generation method, the adversarial sample optimization method based on decision boundary and trust region of the present invention has the following beneficial effects:
[0029] 1. Compared with the existing adversarial sample direct generation method, the adversarial sample optimization method based on decision boundary and trust region of the present invention can generate adversarial sample disturbances significantly reduced.
[0030] 2. Compared with the existing adversarial sample direct generation method, the adversarial sample optimization method based on decision boundary and trust region of the present invention is more consistent with human visual cognition. DETAILED DESCRIPTION
[0031] The present invention is described in detail below with reference to specific embodiments.
[0032] Embodiment 1
[0033] An adversarial sample optimization method based on decision boundary and trust region is to generate adversarial samples using white-box attack method, and then perform multi-level optimization on the adversarial samples based on decision boundary and trust region to guide the generation of adversarial samples with minimal perturbation.
[0034] Preferably, multi-level optimization of adversarial samples is performed based on decision boundaries and trust regions, including the following:
[0035] 1. Determine the fitness function used to select elite samples;
[0036] Preferably, the fitness function selected by the present invention is the sum of the cross entropy loss function and the distance between the original sample and the current sample. Further, the distance between the original sample and the current sample adopts the Euclidean distance. Further, the cross entropy loss function value is the cross entropy of the output vector of the last fully connected layer of the sample query attack model and the true label unique hot encoding.
[0037] 2. Iteratively updating the samples generated based on the adversarial samples until the samples are optimal, that is, the disturbance is minimized;
[0038] Preferably, the iterative update process is:
[0039] ① Let sample t be the adversarial sample and the number of iterations k be 0;
[0040] ②Add adversarial noise to t to generate a temporary sample group;
[0041] Preferably, the adversarial noise uses adversarial noise under the L∞ limit; where L∞ represents the similarity between the adversarial sample and the original sample.
[0042] ③ Use the fitness function to score all samples in the temporary sample group, and select the best sample as the elite sample;
[0043] ④ Project the elite sample onto the original sample to obtain a temporary adversarial sample;
[0044] Preferably, during projection, the visual noise effect of the generated temporary adversarial samples is ensured to always converge within the L∞ limit.
[0045] Furthermore, the step size of the projection as a hyperparameter should not remain unchanged during the implementation of the method of the present invention, but should be adjusted according to the following principles: the proportion λ of temporary samples in the temporary sample group that meets the following conditions is counted: for targeted attacks, the query label of the temporary sample should be equal to the target label; for non-target attacks, the query label of the temporary sample is not equal to the original sample; if 0.3≤λ≤0.7, the step size does not need to be adjusted; if λ is less than 0.3, increase the step size; if λ is greater than 0.7, reduce the step size.
[0046] ⑤ Let t be the temporary adversarial sample, k = k + 1;
[0047] ⑥If k is equal to the preset maximum number of iterations maxIter, then t is output as the adversarial sample with the least disturbance; end;
[0048] ⑦ If the mean square error values between the temporary adversarial sample and the original sample obtained in the last 10 iterations are the same, then t is output as the adversarial sample with the minimum disturbance; end;
[0049] ⑧Return to ② and continue iterating.
[0050] Embodiment 2
[0051] Taking image recognition as an example, the process of generating adversarial samples with minimal disturbance using the present invention is as follows:
[0052] 1. Adversarial Sample Initialization
[0053] Based on the ImageNet dataset, 100 original images, i.e. 100 original samples, are selected, and the initial adversarial samples of the corresponding images are generated for each original sample through the white-box attack method.
[0054] 2. Formulate fitness function
[0055] The sample fitness function is set. The fitness function is mainly divided into two parts. At present, most deep neural networks for classification tasks use the labels obtained by model query and the real labels of the training set to calculate the cross entropy to obtain the loss function. Therefore, the first part of the fitness function is set as the cross entropy between the output vector of the last fully connected layer of the sample query attack model and the real label one-hot encoding. In addition, the basic requirement of adversarial samples is that they do not affect human visual recognition, so the second part of the fitness function is set as the Euclidean distance between the sample image and the original image.
[0056] 3. Sample iterative update
[0057] Set the maximum number of iterations maxIter, and perform the following iterative operations on samples within the maxIter range:
[0058] The initial adversarial sample adds adversarial noise under the L∞ limit, where L∞ represents the similarity between the adversarial sample and the original image, generating a temporary sample group with a certain population size;
[0059] Then, the fitness function is used to score each sample, and the sample with the smallest score is selected as the elite sample of the population;
[0060] Then, the elite sample is projected onto the original image to obtain a temporary adversarial sample. At the same time, the projection operation ensures that the visual noise effect of the generated temporary adversarial sample always converges within the L∞ limit.
[0061] When the mean square error value between the updated sample, that is, the temporary adversarial sample, and the original image is the same for more than 10 times, we believe that the sample has been updated to the optimal state, that is, the adversarial sample with the least perturbation is obtained, and the iteration operation is terminated; or the number of iterations reaches maxIter, the iteration operation is terminated, and the temporary adversarial sample at this time is used as the adversarial sample with the least perturbation.
[0062] 4. Hyperparameter Tuning
[0063] The present invention includes two hyperparameters, one is the L∞ limit of the adversarial noise added each time, and the other is the step size of the elite sample projected onto the original sample. The two parameters can be preset according to the specific application scenario. For this embodiment, L∞=8 is preset and the step size is 1. During the iteration process, L∞ is no longer adjusted, but the step size should be continuously adjusted according to the iteration status of the adversarial sample to make the perturbation converge quickly. The specific adjustment method is to count the proportion λ of samples in the temporary sample group that meet the following conditions: for target attacks, the query label of the temporary sample should be equal to the target label; for non-target attacks, the query label of the temporary sample is not equal to the original sample. According to experience, the threshold here is set to 0.3~0.7. That is, when 0.3≤λ≤0.7, we believe that the step size is reasonable and does not need to be updated. When the step size is outside this range and is less than 0.3, it proves that there are too few samples that meet the standard, and the step size should be increased; when it is greater than 0.7, the proportion is too high, it is easy to fall into the local optimum, and the step size needs to be reduced.
[0064] The present invention integrates the gradient algorithm with the optimization algorithm, and uses a multi-level guided generation method based on decision boundaries and trust regions to optimize the initial adversarial samples generated by the white-box attack method, thereby eliminating invalid perturbations in the adversarial samples and reducing the perturbation size in the adversarial samples by more than 40% compared with the initial adversarial samples, making the perturbation more covert.
[0065] The above is only a preferred embodiment of the present invention, and the present invention should not be limited to the contents disclosed in the embodiment. Any equivalent or modification completed without departing from the spirit disclosed in the present invention shall fall within the scope of protection of the present invention.
Claims
1. A method for optimizing adversarial samples based on decision boundaries and trust regions, characterized by: Generate adversarial samples using white-box attack methods, and then perform multi-level optimization on adversarial samples based on decision boundaries and trust regions to guide the generation of adversarial samples with minimal perturbations; The multi-level optimization of adversarial samples based on decision boundaries and trust regions includes the following:
1. Determine the fitness function used to select elite samples; 2. Iteratively updating the samples generated based on the adversarial samples until the samples are optimal, that is, the disturbance is minimized; 3. Adjusting the hyperparameters in the iterative update process of the sample; The fitness function is the sum of the cross entropy loss function and the distance between the original sample and the current sample; The distance between the original sample and the current sample is the Euclidean distance; The cross entropy loss function value is the cross entropy of the output vector of the last fully connected layer of the sample query attack model and the real label one-hot encoding; The iterative update process is: ① Let sample t be the adversarial sample and the number of iterations k be 0; ②Add adversarial noise to t to generate a temporary sample group; ③ Use the fitness function to score all samples in the temporary sample group, and select the best sample as the elite sample; ④ Project the elite sample onto the original sample to obtain a temporary adversarial sample; ⑤ Let t be the temporary adversarial sample, k = k + 1; ⑥If k is equal to the preset maximum number of iterations maxIter, then t is output as the adversarial sample with the least disturbance; end; ⑦ If the mean square error values between the temporary adversarial sample and the original sample obtained in the last 10 iterations are the same, then t is output as the adversarial sample with the minimum disturbance; end; ⑧Return to ② and continue iterating; The adversarial noise is adversarial noise under the L∞ limit; wherein L∞ represents the similarity between the adversarial sample and the original sample; During the projection, it is ensured that the visual noise effect of the generated temporary adversarial sample always converges within the L∞ limit range; The step size of the projection is adjusted according to the following principle: the proportion λ of the temporary samples in the temporary sample group that meets the following conditions is counted: for targeted attacks, the query label of the temporary sample should be equal to the target label; for non-target attacks, the query label of the temporary sample is not equal to the original sample; if 0.3≤λ≤0.7, the step size does not need to be adjusted; if λ is less than 0.3, the step size is increased; if λ is greater than 0.7, the step size is reduced.
