A method for generating face adversarial samples based on black-box attack
By preprocessing images and using adaptive evolution strategies, the method addresses the limitations of existing algorithms, enhancing the generation of adversarial solutions, the method effectively addresses the inefficiency and inefficiency of existing technologies, enhancing the generation of adversarial samples for decision-based attacks, thereby enhancing the application of the search for the technical problem of the technical problem.
Patent Information
- Application Number
- CN202210393983.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-15
- Publication Date
- 2024-12-24
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing black-box attack algorithms require a large number of queries and generate relatively large perturbations, making them inefficient for generating adversarial samples in real-world scenarios.
A method that modifies a small part of image pixels, uses image-based differential evolution to preprocess input images, and incorporates adaptive evolution strategies to reduce the solution space and minimize perturbations, optimizing the update iteration process to generate adversarial samples with fewer queries.
The method effectively generates adversarial samples with smaller disturbances, reducing the number of model accesses and improving search efficiency, making them undetectable to the human eye while maintaining classification errors in deep neural networks.
Smart Images
Figure CN114998952B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a sample generation method, and in particular to a face adversarial sample generation method based on black box attack. Background Art
[0002] As artificial intelligence technology becomes more mature, the boundaries between humans and machines begin to blur. In particular, deep neural networks have achieved great success in many tasks in the field of machine learning, such as face recognition, image recognition and classification, speech recognition, and autonomous driving. At present, deep neural network models have become the primary solution for solving tasks such as visual recognition, surveillance, drones, and robots, but these technologies often cause some security issues, so whether the deep neural network model is robust is crucial in these fields.
[0003] At present, research in this field can be divided into two aspects: attack and defense against neural networks. Attack refers to studying how to construct or generate adversarial samples to deceive neural networks, while defense refers to taking certain measures to restore adversarial samples to correctly recognized input data or train neural networks that are more robust and not interfered by adversarial samples. Attacks on neural networks can usually be divided into white-box attacks and black-box attacks according to different threat models. White-box attacks require the attacker to fully understand all the internal structures and training parameters of the target model, while black-box attacks know nothing about the internal structure of the target model and can only generate adversarial samples by observing the input and output of the model. Nowadays, there are many white-box attack algorithms with extremely high success rates, but people rarely pay attention to black-box attack algorithms that are closer to real-life scenarios. However, in real life, the confidentiality of many systems is very reliable, and there is almost no complete leakage of model information. Therefore, the application of white-box attacks is far less than that of black-box attacks.
[0004] Since black-box attack algorithms know nothing or very little about model information, they usually require a large number of queries to converge, or obtain relatively large perturbations under a limited query budget. In a sense, they lack efficiency. Therefore, this topic aims to improve existing black-box attack algorithms. To overcome the shortcomings of existing algorithms, consider using fewer queries to make smaller perturbations to each sample and reducing the dimensionality of the search space, thereby further improving the search efficiency and effectively generating adversarial samples for decision-based black-box attacks.
[0005] This can not only promote the generation of more effective defense algorithms, but also contribute to the protection of personal privacy. There are many malwares that use artificial intelligence systems to collect and analyze image data uploaded by users to the Internet to commit illegal and criminal acts. At this time, adversarial sample technology can be used to disguise or hide image data to deceive the image recognition system based on neural networks, thereby protecting user privacy and preventing information leakage. Adversarial samples can also be applied to the generation of verification codes. Currently, neural networks are used to quickly and efficiently identify verification codes, which makes websites face threats such as malicious password cracking and malicious large-scale registration. Generating adversarial samples for verification code recognition systems to build secure verification codes has positive theoretical research significance and important application value.
[0006] Black box attack is that the attacker can only obtain the input and output information of the target model. The main implementation methods are migration-based attack, score-based attack and access-based attack. Migration-based attack does not rely on model information, but requires information about training data, which is used to train a fully observable alternative model, and the model can synthesize adversarial interference. They rely on empirical observations, that is, adversarial samples often migrate between models, and are not practical. Score-based attacks require a continuous prediction score, but this is not applicable in most real-world scenarios. Decision-based attacks mostly increase the success rate of individual attacks at the cost of high access volume. Although it is more suitable for attacking the physical world, there are generally many queries and difficult selection of initial sensitive points. Therefore, the present invention proposes a new algorithm to solve the problems of black box attacks, while ensuring the success rate of the attack, making the added disturbance as small as possible and reducing the number of times the black box attack algorithm accesses the network. Summary of the invention
[0007] The purpose of the present invention is to solve the problems of large number of queries and difficulty in selecting initial sensitive points in existing black-box attacks, and to propose a method for generating face adversarial samples based on black-box attacks.
[0008] A method for generating face adversarial samples based on black box attack is implemented by the following steps:
[0009] Step 1: Modify a small part of the image pixels to generate efficient adversarial samples, improve the initialization samples, use the image-based differential evolution method to preprocess the input image, search for pixels that are easy to attack while reducing the dimension of the solution space, and then add random noise to the selected sensitive pixels as the initial input of the optimization method;
[0010] Step 2: Optimize the update iteration process:
[0011] On the basis of the adaptive evolution strategy based on the covariance matrix, when the search direction is determined by modeling the local geometric structure, multiple candidate solutions are generated through various methods to ensure that the best solution is selected from the current solution and the candidate solutions to select a better solution for the next iteration;
[0012] Step 3: Use image scaling technology to calculate a smaller disturbance vector and then amplify the noise distribution without increasing the noise value, so as to maximize the impact of the minimum disturbance and reduce the dimension of the solution space.
[0013] Preferably, step one also includes the step of selecting an initial adversarial sample, that is, directly using the target image as the adversarial sample for the target attack.
[0014] Preferably, in step 2, when determining the search direction by modeling the local geometric structure based on the covariance matrix adaptive evolution strategy, multiple candidate solutions are generated by multiple methods to ensure that the best solution is selected from the current solution and the candidate solutions to select a better solution for the next iteration, specifically:
[0015] Based on the update iteration in the evolutionary attack algorithm, when the parent generation (current solution) generates a new offspring (candidate solution), the noise is changed and multiple offspring are generated. The goals of these solutions are evaluated together and the optimal solution is selected.
[0016] Preferably, the step of using the image scaling technology in step 3 to calculate the smaller disturbance vector and then amplify the noise distribution without increasing the noise value, so as to maximize the influence of the minimum disturbance and reduce the dimension of the solution space is:
[0017] The first step of dimensionality reduction is performed when preprocessing the initial adversarial sample, and then further dimensionality reduction is performed in random coordinate sampling and dimensionality downsampling, that is, a part of pixels are selected at uniform intervals as the search subspace; when a candidate solution is found in the subspace, bilinear interpolation is performed according to the spatial coordinates of the pixels to obtain a candidate solution for the entire space.
[0018] Preferably, the step of modifying a small portion of image pixels to generate an efficient adversarial sample and improving the initialization sample in step 1 is specifically as follows:
[0019] Modeling the attack problem of the face matching model; the attack task is to construct an adversarial face image based on the target face image, so that the comparison result with the reference face image changes; for face verification, the model relies on another face image to identify whether the pair of images belong to the same identity and outputs a binary label. For face recognition, the model compares the input image with a collection of face images and then classifies the input image as a specific identity. This process is considered a multi-class classification task; given a real face image, the goal of the attack is to generate an adversarial face image near the original image, but it is misclassified by the model, which is solved by solving a constrained optimization problem
[0020] The beneficial effects of the present invention are:
[0021] The present invention studies the evolutionary attack algorithm and genetic algorithm of the black-box attack algorithm, understands the means of achieving spatial dimensionality reduction, and conducts in-depth research on the adaptive evolution strategy of the covariance matrix. An improved algorithm for generating samples based on decision-based black-box attacks is proposed, which effectively generates adversarial samples of decision-based black-box attacks by introducing smaller disturbances to each sample with fewer queries, limits the number of accesses to the model, improves the efficiency of the search, and makes deep neural network classification errors when the human eye cannot perceive the difference between the adversarial sample and the original image. Finally, a suitable face data set is selected and the improved algorithm is used for further processing.
[0022] Specifically,
[0023] When designing the initial adversarial sample, the present invention considers various parameters of the differential evolution algorithm during mutation, crossover, and selection, and selects the optimal parameters to ensure that the generated sample is adversarial.
[0024] The selection of noise when the parent solution generates the child solution can quickly and efficiently generate multiple child solutions and select the best one from them. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 is a flow chart of the method of the present invention;
[0026] Figure 2 The diagram is used to introduce the principle idea of the method of the present invention. DETAILED DESCRIPTION Specific implementation method one:
[0028] A method for generating face adversarial samples based on black box attack in this embodiment is as follows: Figure 1 As shown, the method is implemented by the following steps:
[0029] Step 1. Achieve the smallest possible interference. Many existing attack algorithms add tiny disturbances to all pixels of the entire image in order to deceive the model, which is difficult to be detected by the human eye. The present invention proposes to generate efficient adversarial samples by modifying only a small part of the image pixels, improve the initialization samples, use image-based differential evolution methods to pre-process the input images, search for pixels that are easy to attack while reducing the dimension of the solution space, and then add random noise to the selected sensitive pixels as the initial input of the optimization method; the improved adversarial samples maintain adversarial properties in subsequent iterative updates, and in the subsequent processing process, the distance between the adversarial samples and the original image is continuously reduced, so that the distance between the original image and the adversarial samples is minimized, achieving disturbances that are not easily detected by the human eye;
[0030] Step 2: Optimize the update iteration process:
[0031] On the basis of the adaptive evolution strategy based on the covariance matrix, when the search direction is determined by modeling the local geometric structure, multiple candidate solutions are generated through various methods to ensure that the best solution is selected from the current solution and the candidate solutions to select a better solution for the next iteration;
[0032] Step 3: For black box adversarial attacks, the dimension of the search space is very large, so reducing the dimension of the search space is a direct way to improve search efficiency. The present invention incorporates the idea of genetic algorithm into the idea of evolutionary attack method, combining the advantages of evolutionary attack method and genetic algorithm, namely:
[0033] Image scaling technology is used to calculate a smaller disturbance vector and then amplify the noise distribution without increasing the noise value, so as to maximize the impact of the minimum disturbance, reduce the dimension of the solution space, and improve the search efficiency. Specific 5 implementation method 2:
[0035] Different from the first specific implementation mode, in this implementation mode, a method for generating face adversarial samples based on black-box attack, step one also includes the step of selecting the initial adversarial sample. Since the deep neural network has strong robustness to random noise, when selecting the initial sample, that is, for the target attack, the target image is directly used as the adversarial sample. Specific implementation method three:
[0037] Different from the first or second specific implementation mode, in the method for generating face adversarial samples based on black box attack in the present implementation mode, when determining the search direction by modeling the local geometric structure based on the covariance matrix adaptive evolution strategy, multiple candidate solutions are generated by multiple methods to ensure that the optimal solution is selected from the current solution and the candidate solutions to select a better solution for the next iteration, specifically:
[0038] Based on the update iteration in the evolutionary attack algorithm, when the parent generation (current solution) generates a new offspring (candidate solution), the noise is changed and multiple offspring are generated. The goals of these solutions are evaluated together and the optimal solution is selected. Specific implementation method four:
[0040] Different from the third specific implementation mode, in the method for generating face adversarial samples based on black box attack in this implementation mode, the image scaling technology described in step 3 is used to calculate a smaller disturbance vector and then amplify the noise distribution, and the noise value does not increase, so as to maximize the influence of the minimum disturbance and reduce the dimension of the solution space. The steps are:
[0041] The first step of dimensionality reduction is performed when preprocessing the initial adversarial sample, and then further dimensionality reduction is performed in random coordinate sampling and dimensionality downsampling, that is, a part of pixels are selected at uniform intervals as the search subspace; when a candidate solution (that is, candidate adversarial noise) is found in the subspace, bilinear interpolation is performed according to the spatial coordinates of the pixels to obtain a candidate solution for the entire space. Specific implementation method five:
[0043] Different from the fourth embodiment, the face adversarial sample generation method based on black-box attack in this embodiment modifies a small part of the image pixels to generate efficient adversarial samples in step 1, and improves the initialization sample. Specifically, in the black-box attack, the attacker cannot access the model parameters and gradients, and can only obtain hard label predictions by sending queries to the target model. This attack setting is more practical in real-world face recognition systems. Figure 1 It is a black box attack model based on decision making. It can be seen that the adversarial samples generated are constantly optimized by increasing the number of visits to the black box model. However, due to the large number of queries, the attack efficiency is not high. Therefore, this study improves the algorithm based on evolutionary attack to reduce the number of queries and generate better adversarial samples.
[0044] Modeling the attack problem of the face matching model; the attack task is to construct an adversarial face image based on the target face image, so that the comparison result with the reference face image changes; for face verification, the model relies on another face image to identify whether the pair of images belong to the same identity and outputs a binary label. For face recognition, the model compares the input image with a collection of face images and then classifies the input image into a specific identity. This process is considered a multi-class classification task; given a real face image, the goal of the attack is to generate an adversarial face image near the original image, but it is misclassified by the model, which is solved by solving a constrained optimization problem.
[0045] The implementation principle of the present invention:
[0046] The basic theory of the evolutionary attack algorithm is that since we cannot access the configuration and parameters of the model, we can only send queries to probe the model, so we can use a black box optimization algorithm to minimize the objective function. The evolutionary attack method is based on a simple and effective variant of the covariance matrix adaptive evolution strategy (CMA-ES), namely (1+1)-CMA-ES. In each update iteration of (1+1)-CMA-ES, new offspring (candidate solutions) are generated from their parents (current solutions) by adding random noise, the two solutions are evaluated, and the better solution is selected for the next iteration.
[0047] like Figure 2 As shown in the figure, the large circle represents the target image; the curve I represents the decision boundary (this curve is actually invisible), the area above the curve is the infeasible domain, and the area below the curve is the feasible domain; the × points represent the infeasible solutions that have been queried, the small circles represent the feasible solutions that have been queried, and the solid dots represent the current feasible solutions. The basic steps are as follows: In order to explore the next feasible solution, sampling is required. The ellipse represents the sampling probability distribution, which obeys the Gaussian distribution. The center point of the distribution is the current feasible solution, and the covariance matrix is estimated based on the historical feasible solutions. The basic idea is: based on the historical exploration points, the exploration success rate in each direction can be calculated; continuing to explore in the direction with a large success rate makes it easier to find the next feasible solution. Therefore, the first step of sampling is performed according to the distribution, such as the first orange arrow starting from the green dot, reaching the initial candidate solution (circle 1). After the first step of sampling, the next feasible solution may be found, but it cannot be guaranteed to decrease. Therefore, the second step of exploration is performed, that is, starting from the initial candidate solution, moving a small step along the target image x (see the second arrow) to reach the second candidate solution (i.e. dot 2). Then, the second candidate solution is queried to determine whether it satisfies the constraints: if so, it is taken as the latest feasible solution, such as Figure 2 As shown, the sampling probability distribution (i.e., the ellipse) is updated; if it is not satisfied, the current solution is kept unchanged and the above sampling process is repeated.
[0048] And an appropriate distribution is designed to sample random noise in each iteration, which can simulate the local geometry of the search direction. In the above search algorithm, since each pixel of the image is regarded as a dimension, the dimension of the entire search space is very high, which often means that the search efficiency is very low. To this end, two acceleration methods are added to the search: (1) Random coordinate sampling: that is, each sampling is only performed in some dimensions (that is, some pixels), rather than all dimensions. The selection of dimensions is based on the sampling probability distribution, and the probability of dimensions with large variance being selected is also higher. (2) Dimension downsampling: It is assumed that the adversarial noise is also spatially smooth in the pixel coordinate system, that is, the adversarial noise of adjacent pixels is similar. Therefore, a part of the pixels are selected at uniform intervals as the search subspace; but when a candidate solution (that is, candidate adversarial noise) is found in the subspace, bilinear interpolation is performed according to the spatial coordinates of the pixel to obtain a candidate solution for the entire space.
[0049] The present invention uses Windows 10 64-bit for running experiments, the CPU is Intel i7-6700 3.4 GHz, the deep learning framework is TensorFlow 2.2, and the compilation environment is pycharm 3.8 / python language. The verification results of the method of the present invention are feasible and the effect is ideal.
[0050] The embodiments of the present invention disclose preferred embodiments, but are not limited thereto. A person skilled in the art can easily understand the spirit of the present invention based on the above embodiments and make different extensions and changes. However, as long as they do not deviate from the spirit of the present invention, they are all within the protection scope of the present invention.
Claims
1. A method for generating face adversarial samples based on black box attack, characterized by: The method is implemented by the following steps: Step 1: Modify a small part of the image pixels to generate efficient adversarial samples, improve the initialization samples, use the image-based differential evolution method to preprocess the input image, search for pixels that are easy to attack while reducing the dimension of the solution space, and then add random noise to the selected sensitive pixels as the initial input of the optimization method; Step 2: Optimize the update iteration process: On the basis of the adaptive evolution strategy based on the covariance matrix, when the search direction is determined by modeling the local geometric structure, multiple candidate solutions are generated through various methods to ensure that the best solution is selected from the current solution and the candidate solutions for the next iteration to select a better solution. Specifically, Based on the update iteration in the evolutionary attack algorithm, when the parent generates a new offspring, the noise is changed and multiple offspring are generated. The goals of these solutions are evaluated together and the optimal solution is selected. Step 3: Use image scaling technology to calculate the smaller disturbance vector and then amplify the noise distribution without increasing the noise value, so as to maximize the impact of the minimum disturbance and reduce the dimension of the solution space. The specific steps are: The first step of dimensionality reduction is performed when preprocessing the initial adversarial sample, and then further dimensionality reduction is performed in random coordinate sampling and dimensionality downsampling, that is, a part of pixels are selected at uniform intervals as the search subspace; when a candidate solution is found in the subspace, bilinear interpolation is performed according to the spatial coordinates of the pixels to obtain a candidate solution for the entire space.
2. According to claim 1, a method for generating face adversarial samples based on black box attack is characterized in that: Step one also includes the step of selecting the initial adversarial sample, that is, directly using the target image as the adversarial sample for the target attack.
3. According to claim 2, a method for generating face adversarial samples based on black box attack is characterized in that: The steps described in step 1 to modify a small part of the image pixels to generate an efficient adversarial sample and improve the initialization sample are as follows: Modeling the attack problem of the face comparison model; the attack task is to construct an adversarial face image based on the target face image, so that the comparison result with the reference face image changes; for face verification, the model relies on another face image to identify whether the pair of images belong to the same identity and outputs a binary label; For face recognition, the model compares the input image with a collection of face images and then classifies the input image into a specific identity. This process is considered a multi-class classification task. Given a real face image, the goal of the attack is to generate an adversarial face image near the original image, but it is misclassified by the model, which is solved by solving a constrained optimization problem.