Adversarial sample generation method and system based on sparse perturbation
The iterative selection and perturbation optimization strategy based on the loss function of gradient value and the perturbation optimization strategy generate sparse perturbation samples, which solves the problem of large and unreal perturbation in the existing technology, and realizes efficient adversarial sample generation, which improves the robustness evaluation of DNN.
Patent Information
- Application Number
- CN202210829831.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-15
- Publication Date
- 2025-08-08
- Estimated Expiration
- 2042-07-15
AI Technical Summary
The existing adversarial sample generation methods mainly focus on dense perturbations, and the lack of research on sparse perturbations has led to the large perturbations of the generated adversarial sample and is not real enough, making it difficult to effectively evaluate the robustness of DNNs.
By iteratively selecting perturbation points based on the gradient value of the loss function, generating initial adversarial samples, and improving sparseness and authenticity through perturbation optimization strategies, deleting redundant perturbation and reducing redundant perturbation amplitude, attacking with multiple image data sets.
A 100% attack success rate is achieved under CIFAR-10 and ImageNet datasets, and the generated perturbation sparsity and authenticity are better than the existing technology, and the perturbation amplitude is lower.
Smart Images
Figure CN115311513B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image adversarial sample generation, and specifically relates to constructing adversarial samples by generating sparse perturbations to carry out adversarial attacks on DNN models. Background Art
[0002] In recent years, deep neural networks (DNNs) have made tremendous progress in the field of imaging, and have been widely applied to tasks such as object detection, image semantic segmentation, and image classification. As DNN structures become increasingly complex, their ability to extract image features continues to improve, leading to new breakthroughs in areas with high safety requirements, such as facial recognition and autonomous driving.
[0003] Although DNNs have high recognition accuracy across various tasks, they may produce diametrically opposed conclusions when presented with specially modified samples, demonstrating poor robustness. Szegedy et al. proposed that by creating adversarial examples, attackers can create subtle perturbations to images that are imperceptible to the human eye. This can lead to DNNs misjudging these images. This is a critical flaw in areas where safety is paramount. For example, in autonomous driving, an attacker could create subtle, artificial perturbations to roadside billboards. This could lead to incorrect recognition results, resulting in erroneous decisions and potentially serious accidents. To ensure the safety of related applications, it is necessary to research methods for generating adversarial examples and use them to evaluate the robustness of DNNs.
[0004] Current adversarial sample generation methods for images can be divided into two categories: one is dense adversarial sample generation methods, which do not limit the number of perturbed pixels, but only optimize the overall perturbation amplitude added to the image, usually adding l2 or l ∞ Norm constraints modify all pixels in the image to varying degrees. Another type of adversarial example generation method is sparse adversarial example generation, which limits the number of pixels perturbed. The added perturbations are typically subject to the l0-norm constraint, striving to modify as few image pixels as possible to achieve a successful attack. While most current research focuses on dense adversarial example generation, adding sparse perturbations to images can make the added perturbations even smaller, making the adversarial examples more realistic. Therefore, sparse adversarial example generation methods are also worthy of attention and research. Furthermore, sparse perturbations generated by sparse adversarial example generation methods can reveal the relationship between the original image and the adversarial example, and can further elucidate the mechanism by which adversarial examples attack DNNs. Summary of the Invention
[0005] Purpose of the invention: The purpose of the present invention is to provide a method and system for generating adversarial samples based on sparse perturbation, which can use a variety of image data sets to improve the perturbation sparsity and attack success rate of the generated adversarial samples, and improve the authenticity of the adversarial samples.
[0006] Technical solution: To achieve the above-mentioned purpose, the present invention provides a method for generating adversarial samples based on sparse perturbations, comprising the following steps:
[0007] Step 1: Iteratively select perturbation points based on the gradient value of the loss function with respect to the input image to generate the initial adversarial sample. Each iteration determines the candidate set of new perturbation points in descending order of gradient value. The perturbation that minimizes the loss function value is selected and added to the image until the perturbation image can successfully attack the DNN.
[0008] Step 2: For the initial adversarial sample, calculate the importance of the perturbation of each perturbation point relative to the overall perturbation, judge and delete redundant perturbations one by one in order of perturbation importance from small to large, and after calculating the perturbation importance again, remove the perturbations of the set number or proportion before sorting them in order of perturbation importance from small to large; re-execute step 1 to add perturbations so that the adversarial sample can successfully attack the DNN until the set number of cycles is exceeded or the sparsity of the perturbation no longer increases for the set number of consecutive times.
[0009] As a preference, in order to make the generated adversarial samples more realistic, while ensuring the success of the attack, the importance of the perturbation of each perturbation point in the generated adversarial sample is further calculated, and redundant perturbations are judged and deleted one by one in order of perturbation importance from small to large; the perturbation amplitude is reduced for the remaining perturbation points to obtain the final adversarial sample that can successfully attack the DNN.
[0010] As in step 1 above, the method for selecting the disturbance point and adding the disturbance in the t+1th iteration is as follows:
[0011] Step 11: Input the image generated after the first t iterations into the DNN and calculate the gradient of the loss function with respect to the input image. The loss function formula used is as follows:
[0012] f(x,y)=max(Z(x) y -max{Z(x) i :i≠y},-h)
[0013] Where x and y are the images and their corresponding true labels, respectively, Z(x) i is the predicted probability of DNN for image x about label i, h is the confidence parameter, which defaults to 0;
[0014] Step 12: For the perturbation points determined in the previous t iterations in the image, continue to add perturbations based on the gradient values calculated in this iteration under the limit of the perturbation threshold. The specific formula for adding perturbations to the image is as follows:
[0015]
[0016] in are the images before and after the perturbation is added, Indicates that the perturbation added to each pixel in the image x is limited to ε, g is the gradient value matrix of the loss function with respect to the input image, v is the matrix recording the perturbation position, the value of the perturbation point position is 1, and the rest are 0, g·v represents the logical product of the element values at the corresponding positions of the two matrices, and α is a fixed value parameter set to control the amplitude of the added perturbation;
[0017] Step 13: Among the remaining undisturbed pixels in the image, the first k pixels p1, p2, ..., p k As a candidate perturbation point set, add perturbation to one of the pixels. If i The added perturbation causes the loss function value to drop the most, then p i As the new perturbation point in the t+1th iteration, add its perturbation to the image and modify p in v accordingly. i The value of the corresponding pixel position is 1;
[0018] Step 14: Determine whether the image generated by the current iteration can successfully attack the DNN. If it can, jump to step 2. Otherwise, increase the number of iterations by 1 and jump to step 11 to continue adding perturbations to the image.
[0019] Preferably, the method for calculating the disturbance importance in step 2 is as follows:
[0020] First, obtain the adversarial sample x adv The perturbation δ added in , assuming that the perturbation of the adversarial sample consists of the perturbation δ of q pixels j (j=1,2,3,...,q), for each disturbance δ in the disturbance δ j Calculate its importance;
[0021] From the original adversarial sample x adv Remove the disturbance δ j , and get the new perturbation image
[0022] calculate The loss function value δ j The perturbation importance is calculated by Compared with the original adversarial sample x advThe loss function change is measured and the calculation formula is as follows:
[0023]
[0024] Preferably, the step 2 specifically includes:
[0025] Step 21: Get the perturbation δ added to the adversarial sample and calculate the perturbation importance;
[0026] Step 22: Determine whether the perturbations are redundant or not in ascending order of importance, and delete the redundant perturbations. If the sample after removing the perturbation still causes the DNN to make an incorrect judgment, the perturbation is redundant.
[0027] Step 23: Calculate the perturbation importance again after removing redundant perturbations;
[0028] Step 24: Sort the perturbations by importance, removing the set number or proportion of perturbations with less importance in the adversarial sample, and jump to step 1 to continue adding perturbations to the image until the perturbed image successfully attacks the DNN;
[0029] Step 25: Continuously iterate steps 21-24 to optimize the sparsity of the perturbation. If the above steps are iterated more than the set number of times, or the sparsity of the perturbation does not improve for the set number of consecutive times, the iteration ends and the perturbation with the highest sparsity during the iteration process is output.
[0030] Preferably, the step 3 specifically includes:
[0031] Step 31: Get the perturbation δ added to the current adversarial example and calculate the perturbation importance;
[0032] Step 32: Determine and remove redundant perturbations in the adversarial examples one by one in order of perturbation importance from small to large;
[0033] Step 33: Recalculate the perturbation importance for the adversarial sample after removing redundant perturbations;
[0034] Step 34: Optimize the disturbance amplitude for the remaining disturbance points. j , the calculation formula for the reduced disturbance amplitude at the corresponding disturbance point is as follows:
[0035]
[0036] The parameter λ is a fixed value parameter used to control the reduced disturbance amplitude, σ j is the importance of the disturbance. j The reduction value at the corresponding disturbance point is If the sample obtained after the perturbation can still successfully attack the DNN, the adjustment of the perturbation amplitude is retained; otherwise, the adjustment of the perturbation amplitude is canceled.
[0037] Based on the same inventive concept, the present invention provides an adversarial sample generation system based on sparse perturbation, comprising:
[0038] The initial perturbation generation module is used to iteratively select perturbation points based on the gradient value of the loss function with respect to the input image to generate initial adversarial samples. In each iteration, the candidate set of new perturbation points is determined in descending order of gradient value. The perturbation that minimizes the loss function value is selected and added to the image until the perturbed image can successfully attack the DNN.
[0039] The perturbation optimization module is used to calculate the importance of the perturbation of each perturbation point relative to the overall perturbation for the initial adversarial sample, judge and delete redundant perturbations one by one in order of perturbation importance from small to large, and after recalculating the perturbation importance, sort the perturbations from small to large according to the perturbation importance and remove the set number or proportion of perturbations before.
[0040] And a loop control module is used to cyclically call the initial perturbation generation module and the perturbation optimization module, add perturbations to the image and optimize them so that the adversarial sample can successfully attack the DNN until the set number of cycles is exceeded or the sparsity of the perturbation no longer increases for the set number of consecutive times.
[0041] Preferably, the system also includes a perturbation amplitude optimization module, which is used to calculate the importance of the perturbation of each perturbation point in the generated adversarial sample, judge and delete redundant perturbations one by one in order of perturbation importance from small to large; reduce the perturbation amplitude for the remaining perturbation points, and obtain the final adversarial sample that can successfully attack the DNN.
[0042] Based on the same inventive concept, the present invention provides a computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the steps of the sparse perturbation-based adversarial sample generation method are implemented.
[0043] Based on the same inventive concept, the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the sparse perturbation-based adversarial sample generation method.
[0044] Beneficial effects: The present invention provides a method for generating adversarial samples based on sparse perturbations. This method iteratively selects perturbation points based on the gradient value of the loss function with respect to the input image to generate initial adversarial samples. Each iteration determines the candidate set of newly added perturbation points in descending order of the gradient value, and selects the perturbation that minimizes the loss function value to be added to the image. Secondly, for the initial perturbation scheme, a perturbation optimization strategy is used to improve the sparsity and authenticity of the adversarial samples. The perturbation is improved based on the importance of each perturbation to escape the local optimum, and the redundant perturbation and the redundant perturbation amplitude are further reduced. Because the present invention adopts the above-mentioned technical solution, it can achieve a 100% attack success rate in the CIFAR-10 and ImageNet datasets, as well as in non-target attack scenarios and target attack scenarios. Compared with the prior art, the perturbation generated by the present invention can achieve better perturbation sparsity while having a lower perturbation amplitude. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 is a flow chart of a method according to an embodiment of the present invention;
[0046] Figure 2 A flowchart for generating an initial disturbance according to an embodiment of the present invention;
[0047] Figure 3 This is a disturbance optimization flow chart of an embodiment of the present invention;
[0048] Figure 4 Generate a visualization graph for adversarial examples according to embodiments of the present invention;
[0049] Figure 5 Schematic diagram of the processed samples used in the embodiment of the present invention, where (a) is the original sample, (b) is the generated adversarial sample, and (c) is the perturbed visual image. DETAILED DESCRIPTION
[0050] The present invention is further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.
[0051] like Figure 1 As shown, the embodiment of the present invention discloses a method for generating adversarial samples based on sparse perturbations, which mainly includes the following steps:
[0052] Step 1: Generate initial perturbations. Initial adversarial examples are generated by iteratively selecting perturbation points based on the gradient of the loss function with respect to the input image. Each iteration determines a candidate set of new perturbation points in descending order of gradient value. The perturbation that minimizes the loss function is added to the image until the perturbed image can successfully attack the DNN.
[0053] Step 2: Optimize the perturbation. A perturbation optimization strategy is used to improve the sparsity and authenticity of the adversarial examples based on the initial perturbation scheme. To escape the local optimum, the importance of each perturbation relative to the overall perturbation is calculated. Based on the perturbation importance, redundant and less important perturbations are removed. The perturbation is then re-added to Step 1 to ensure that the adversarial examples successfully attack the DNN. This process continues until the set number of iterations is exceeded or the sparsity of the perturbation stops improving after the set number of iterations.
[0054] Figure 2 The process of generating the initial perturbation scheme in step 1 of the present invention is illustrated. Specifically, the method of selecting the perturbation point and adding the perturbation in the t+1th iteration is further described as follows:
[0055] Step 11: Input the image generated after the first t iterations into the DNN and calculate the gradient of the loss function with respect to the input image. The gradient of the input image can be calculated using the backward() function in the PyTorch framework. The loss function formula used is as follows:
[0056] f(x,y)=max(Z(x) y -max{Z(x) i :i≠y},-h)
[0057] Where x and y are the images and their corresponding true labels, respectively, Z(x) i is the predicted probability of DNN for image x about label i, h is the confidence parameter, h≥0, and is set to 0 by default.
[0058] Step 12: For the perturbation points determined in the previous t iterations in the image, continue to add perturbations based on the gradient values calculated in this iteration under the limit of the perturbation threshold; the specific formula for adding perturbations to the image is as follows:
[0059]
[0060] in are the images before and after the perturbation is added, Indicates that the perturbation added to each pixel in the image x is limited to ε, g is the gradient value matrix of the loss function with respect to the input image, v is the matrix recording the perturbation position, the value of the perturbation point position is 1, and the rest are 0, g·v represents the logical product of the element values at the corresponding positions of the two matrices, and α is a fixed value parameter set to control the amplitude of the added perturbation.
[0061] Step 13: Among the remaining undisturbed pixels in the image, the first k pixels p1, p2, ..., p k As a candidate perturbation point set, add perturbation to one of the pixels. If i The added perturbation causes the loss function value to drop the most, then p i As the new disturbance point in the t+1th iteration, its disturbance is added to the image;
[0062] Step 14: Determine whether the image generated by the current iteration can successfully attack the DNN. If it can, jump to step 2. Otherwise, increase the number of iterations by 1 and jump to step 11 to continue adding perturbations to the image.
[0063] Specifically, the method for calculating the disturbance importance in step 2 of this embodiment is as follows:
[0064] First, obtain the perturbation δ added to the adversarial sample;
[0065] Calculate the importance of each perturbation in perturbation δ, assuming that the adversarial sample x adv The perturbation of is composed of the perturbation δ of q pixels j (j=1,2,3,...,q), from the original adversarial sample x adv Remove the disturbance δ j , and get the new perturbation image
[0066] calculate The loss function value δ j The perturbation importance is calculated by Compared with the original adversarial sample x adv The loss function change is measured and the calculation formula is as follows:
[0067]
[0068] The importance of perturbations can be used to assess which perturbations are less aggressive to DNN, and the weakest perturbations may be redundant.
[0069] like Figure 3 As shown, the specific implementation method of jumping out of the local optimum in step 2 is as follows:
[0070] Step 21: First, obtain the initial adversarial sample generated in step 1 and calculate the initial perturbation δ. The perturbation calculation formula is as follows:
[0071] δ=xx adv
[0072] Step 22: Calculate the disturbance importance for the initial disturbance δ;
[0073] Step 23: Determine whether the perturbation point is redundant one by one in the order of importance from small to large. If the sample after removing the perturbation still causes the DNN to make an incorrect judgment, the perturbation is redundant.
[0074] Step 24: After removing redundant disturbance points, the importance of each disturbance may also change due to the change in the overall disturbance, so the disturbance importance needs to be calculated again;
[0075] Step 25: Remove the top n perturbations with less importance in the adversarial sample according to the perturbation importance ranking, and jump to step 1 to continue adding perturbations to the image until the perturbed image successfully attacks the DNN;
[0076] Step 26: Continuously iterate steps 21-25 to optimize the sparsity of the perturbation. If the above steps are iterated more than m times, or the sparsity of the perturbation does not improve for p consecutive times, the iteration ends and the perturbation with the highest sparsity during the iteration is output.
[0077] In order to make the generated adversarial samples more realistic, while ensuring the success of the attack, we can further optimize the perturbation scheme by reducing redundant perturbations and the redundant perturbation amplitudes of the remaining perturbation points to obtain the final adversarial samples.
[0078] Specifically, if Figure 4 As shown, another embodiment of the present invention discloses a method for generating adversarial samples based on sparse perturbations, which, in addition to the above steps 1 and 2, also includes step 3: calculating the importance of the perturbation of each perturbation point in the generated adversarial sample, judging and deleting redundant perturbations one by one in order of perturbation importance from small to large; reducing the perturbation amplitude for the remaining perturbation points to obtain the final adversarial sample that can successfully attack the DNN.
[0079] The method for reducing redundant disturbances and redundant disturbance amplitudes in step 3 is as follows:
[0080] Step 31: First, obtain the adversarial sample generated after escaping the local optimum in step 2 and calculate the corresponding perturbation δ;
[0081] Step 32: Calculate the disturbance importance of the disturbance δ;
[0082] Step 33: Sort by perturbation importance to remove redundant perturbations in the adversarial sample;
[0083] Step 34: Recalculate the perturbation importance for the adversarial sample after removing redundant perturbations;
[0084] Step 35: Optimize the perturbation amplitude for the remaining perturbation points. According to the order of perturbation importance, the perturbation amplitude should be reduced in the following way: for perturbation points with higher importance, the perturbation amplitude should be reduced to a smaller extent to retain higher aggressiveness; conversely, for perturbation points with lower importance, the perturbation amplitude should be reduced to a larger extent. j , the calculation formula for the reduced disturbance amplitude at the corresponding disturbance point is as follows:
[0085]
[0086] The parameter λ is a fixed value parameter used to control the reduced disturbance amplitude, σ j is the importance of the disturbance. j The reduction value at the corresponding disturbance point is If the sample obtained after the perturbation can still successfully attack the DNN, the perturbation optimization operation is retained; otherwise, the adjustment of the perturbation amplitude is canceled.
[0087] For the convenience of description, we evaluate the effect of the present invention using the ImageNet dataset for the Inception-v3 model. The process is as follows:
[0088] Step 1: Obtain the original image sample dataset required by the adversarial sample generation method and preprocess the image. The specific process is as follows:
[0089] Step 11: Download the ILSVRC2012 dataset from the ImageNet image dataset website (https: / / image-net.org / );
[0090] Step 12: Perform normalization on the image samples in the acquired data set, normalizing each pixel value of [0,255] in the image to the interval [0,1].
[0091] Step 2: Input the preprocessed image into the DNN, and iteratively select the perturbation point based on the gradient value of the loss function with respect to the input image to generate the initial adversarial sample. The specific process is as follows: Figure 2 As shown, the specific steps in the t+1th iteration are as follows:
[0092] Step 21: The image generated after the first t iterations Input DNN and calculate the gradient of loss function with respect to input image. The running result is Inception-v3 relative to The gradient value g;
[0093] Step 22: For the disturbance points determined in the first t iterations in the image, under the limit of the disturbance threshold, the gradient value calculated based on this iteration is Continue adding disturbances;
[0094] Step 23: Determine the candidate set of new perturbation points in descending order of gradient values, select the perturbation that minimizes the loss function value and add it to the image. The result of the operation is the image after the new perturbation point is added.
[0095] Step 24: Repeat steps 21-24 until the image can successfully attack the DNN, and output the initial adversarial sample x generated in step 2 adv .
[0096] Step 3: If Figure 4 As shown in the figure, a perturbation optimization strategy is used to improve the sparsity and authenticity of adversarial samples. The perturbation is improved based on the importance of each perturbation to escape the local optimum and further reduce redundant perturbations and the amplitude of redundant perturbations. The specific steps are as follows:
[0097] Step 31: First obtain the adversarial sample x generated in step 2 adv , and calculate the initial perturbation δ;
[0098] Step 32: Calculate the importance of each disturbance in the disturbance δ. The result of this operation is the importance of each disturbance δ in the disturbance δ. j The importance of σ j .
[0099] Step 33: Determine whether the perturbations are redundant or not in ascending order of importance, and remove the redundant perturbations. The result is the adversarial sample after removing the redundant perturbations.
[0100] Step 34: Calculate the perturbation importance again;
[0101] Step 35: Remove n perturbations with lower perturbation importance in the adversarial sample;
[0102] Step 36: Jump to step 2 and continue adding perturbations to the image until the image can successfully attack the DNN;
[0103] Step 37: Determine whether the local optimum exit stage in step 3 meets the termination condition. If so, exit the loop and output the perturbation with the highest sparsity; otherwise, return to step 31 and continue to exit the local optimum.
[0104] Step 38: Calculate the disturbance importance σ;
[0105] Step 38: Reduce the redundant perturbation points in the adversarial sample in order of perturbation importance from small to large;
[0106] Step 39: Calculate the perturbation importance σ;
[0107] Step 310: reducing the redundant disturbance amplitude in order of disturbance importance from small to large;
[0108] Step 311: Output the final constructed adversarial sample x adv .
[0109] In order to further describe the specific process of the present invention, Figure 5 The original image sample in is used to generate adversarial samples by adding perturbations to the original image using the adversarial sample generation method based on sparse perturbations. The specific process is as follows:
[0110] (1) First, normalize the original image and normalize the pixel value of each [0,255] in the image to the interval [0,1];
[0111] (2) Iteratively select perturbation points based on the gradient value of the loss function with respect to the input image to generate the initial adversarial sample;
[0112] (3) For the initial perturbation scheme, the perturbation optimization strategy is used to improve the sparsity and authenticity of the adversarial sample, and the final adversarial sample is obtained, such as Figure 5 As shown in (b), it can be seen that the adversarial samples and the original samples are almost indistinguishable to the naked eye. Figure 5 (c) in the figure visualizes the perturbations added to the adversarial sample. From the perturbation visualization results, it can be seen that the perturbations generated by the present invention are relatively sparse, and the added perturbations are mainly concentrated in the key recognition positions of the image.
[0113] To further demonstrate the effectiveness of our method, we applied it to the ImageNet dataset to attack the Inception-v3 model and compared it with the GreedyFool and Homotopy-attack methods. The results are shown in Table 1. ASR represents the attack success rate of the adversarial example; the l0-norm represents the number of perturbed pixels in the image, which is used to assess the sparsity of the generated perturbation; the l1-norm refers to the sum of the absolute values of the elements in the perturbation matrix, and the l2-norm refers to the square root of the sum of the squares of the elements in the perturbation matrix, which is used to measure the overall magnitude of the perturbation. The results show that the adversarial example generation method proposed in this paper achieves a 100% attack success rate and performs optimally in terms of both the sparsity of the generated perturbation (l0) and the overall perturbation magnitude (l1, l2).
[0114] Table 1 Comparison of experimental results
[0115]
[0116] Based on the same inventive concept, an embodiment of the present invention provides an adversarial sample generation system based on sparse perturbation, including: an initial perturbation generation module, used to iteratively select perturbation points based on the gradient value of the loss function with respect to the input image to generate an initial adversarial sample, wherein in each iteration, a candidate set of new perturbation points is determined in descending order of the gradient value, and the perturbation that minimizes the loss function value is selected and added to the image until the perturbated image can successfully attack the DNN; a perturbation optimization module, used to calculate the importance of the perturbation of each perturbation point relative to the overall perturbation for the initial adversarial sample, judge and delete redundant perturbations one by one in ascending order of perturbation importance, and after recalculating the perturbation importance, sort the perturbations in ascending order according to the perturbation importance to remove a set number or proportion of perturbations before; and a loop control module, used to cyclically call the initial perturbation generation module and the perturbation optimization module, add perturbations to the image and optimize them so that the adversarial sample can successfully attack the DNN, until the set number of loops is exceeded or the sparsity of the perturbation no longer increases for a set number of consecutive times.
[0117] To further improve the authenticity of adversarial samples, the system also includes a perturbation amplitude optimization module, which is used to calculate the importance of the perturbation of each perturbation point in the generated adversarial sample, and judge and delete redundant perturbations one by one in order of perturbation importance from small to large; the perturbation amplitude is reduced for the remaining perturbation points to obtain the final adversarial sample that can successfully attack the DNN.
[0118] The specific working process of each module described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here. The division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as combining multiple modules or integrating them into another system.
[0119] Based on the same inventive concept, the present invention provides a computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the steps of the sparse perturbation-based adversarial sample generation method are implemented.
[0120] Based on the same inventive concept, the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the sparse perturbation-based adversarial sample generation method.
[0121] Those skilled in the art will appreciate that the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes instructions for causing a computer system (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present invention. Storage media include various media capable of storing computer programs, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random-access memories (RAMs), magnetic disks, or optical disks.
Claims
1. A method for generating adversarial samples based on sparse perturbations, characterized in that: The steps include: Step 1: Iteratively select perturbation points based on the gradient value of the loss function with respect to the input image to generate the initial adversarial sample. In each iteration, the candidate set of new perturbation points is determined in descending order of gradient value. The perturbation that minimizes the loss function value is selected and added to the image until the perturbation image can successfully attack the DNN. The method for selecting perturbation points and adding perturbations in the t+1th iteration is as follows: Step 11: Input the image generated after the first t iterations into the DNN and calculate the gradient of the loss function with respect to the input image. The loss function formula used is as follows: f(x,y)=max(Z(x) y -max{Z(x) i :i≠y},-h) Where x and y are the images and their corresponding true labels, respectively, Z(x) i is the predicted probability of DNN for image x about label i, and h is the confidence parameter; Step 12: For the perturbation points determined in the first t iterations in the image, continue to add perturbations based on the gradient values calculated in step 11 under the limit of the perturbation threshold. The specific formula for adding perturbations to the image is as follows: in are the images before and after the perturbation is added, Indicates that the perturbation added to each pixel in the image x is limited to ε, g is the gradient value matrix of the loss function with respect to the input image, v is the matrix recording the perturbation position, the value of the perturbation point position is 1, and the rest are 0, g·v represents the logical product of the element values at the corresponding positions of the two matrices, and α is a fixed value parameter set to control the amplitude of the added perturbation; Step 13: Among the remaining undisturbed pixels in the image, the first k pixels p1, p2, ..., p k As a candidate perturbation point set, add perturbation to one of the pixels. If i The added perturbation causes the loss function value to drop the most, then p i As the new perturbation point in the t+1th iteration, add its perturbation to the image and modify p in v accordingly. i The value of the corresponding pixel position is 1; Step 14: Determine whether the image generated by the current iteration can successfully attack the DNN. If it can, jump to step 2. Otherwise, increase the number of iterations by 1 and jump to step 11 to continue adding perturbations to the image. Step 2: For the initial adversarial sample, calculate the importance of the perturbation of each perturbation point relative to the overall perturbation, judge and delete redundant perturbations one by one in order of perturbation importance from small to large, and after calculating the perturbation importance again, remove the perturbations of the set number or proportion set before in order of perturbation importance from small to large; re-execute step 1 to add perturbations so that the adversarial sample can successfully attack the DNN until the set number of cycles is exceeded or the sparsity of the perturbation does not increase for the set number of consecutive times; the method for calculating the perturbation importance is as follows: Get adversarial sample x adv The perturbation δ added in , assuming that the perturbation of the adversarial sample is composed of the perturbation δ of q pixels j Composition, j = 1, 2, 3, ..., q, for each disturbance δ in the disturbance δ j Calculate its importance: From the original adversarial sample x adv Remove the disturbance δ j , get the new perturbation image x adv* ; Calculate x adv* The loss function value f(x adv* ,y), y is the true label of image x, δ j The perturbation importance σ j By x adv* Compared with the original adversarial sample x adv The loss function value f(x adv ,y) changes, the calculation formula is: σ j =f(x adv* ,y)-f(x adv ,y).
2. The method for generating adversarial samples based on sparse perturbation according to claim 1, characterized in that: The method further comprises: Step 3: Calculate the importance of the perturbation of each perturbation point in the generated adversarial sample, judge and delete redundant perturbations one by one in order of perturbation importance from small to large; reduce the perturbation amplitude for the remaining perturbation points to obtain the final adversarial sample that can successfully attack the DNN.
3. The method for generating adversarial samples based on sparse perturbation according to claim 1, characterized in that: The step 2 specifically includes: Step 21: Get the perturbations added to the adversarial sample and calculate the perturbation importance; Step 22: Determine whether the perturbations are redundant or not one by one in ascending order of importance, and delete the redundant perturbations. If the adversarial example after removing one perturbation still causes the DNN to make an incorrect judgment, then the perturbation is redundant. Step 23: Calculate the perturbation importance again after removing redundant perturbations; Step 24: Sort the perturbations by importance, removing the set number or proportion of perturbations from the adversarial sample, and jump to step 1 to continue adding perturbations to the image until the perturbed image successfully attacks the DNN. Step 25: Continuously iterate steps 21-24 to optimize the sparsity of the perturbation. If the above steps are iterated more than the set number of times, or the sparsity of the perturbation does not improve for the set number of consecutive times, the iteration ends and the perturbation with the highest sparsity during the iteration process is output.
4. The method for generating adversarial samples based on sparse perturbation according to claim 2, characterized in that: The step 3 specifically includes: Step 31: Get the perturbations added to the adversarial sample and calculate the perturbation importance; Step 32: Determine and remove redundant perturbations in the adversarial examples one by one in order of perturbation importance from small to large; Step 33: Recalculate the perturbation importance for the adversarial sample after removing redundant perturbations; Step 34: Optimize the disturbance amplitude for the remaining disturbance points; for the specific disturbance δ j , the disturbance amplitude is reduced at the corresponding disturbance point The calculation formula is as follows: The parameter λ is a fixed value parameter used to control the reduced disturbance amplitude, σ j is the importance of the disturbance; if in δ j The reduction value at the corresponding disturbance point is If the sample obtained after the perturbation can still successfully attack the DNN, the adjustment of the perturbation amplitude is retained; otherwise, the adjustment of the perturbation amplitude is canceled.
5. A sparse perturbation-based adversarial sample generation system, characterized in that: include: The initial perturbation generation module is used to iteratively select perturbation points based on the gradient value of the loss function with respect to the input image to generate initial adversarial samples. In each iteration, the candidate set of new perturbation points is determined in descending order of gradient value. The perturbation that minimizes the loss function value is selected and added to the image until the perturbation image can successfully attack the DNN. The method for selecting perturbation points and adding perturbations in the t+1th iteration is as follows: Step 11: Input the image generated after the first t iterations into the DNN and calculate the gradient of the loss function with respect to the input image. The loss function formula used is as follows: f(x,y)=max(Z(x) y -max{Z(x) i :i≠y},-h) Where x and y are the images and their corresponding true labels, respectively, Z(x) i is the predicted probability of DNN for image x about label i, and h is the confidence parameter; Step 12: For the perturbation points determined in the first t iterations in the image, continue to add perturbations based on the gradient values calculated in step 11 under the limit of the perturbation threshold. The specific formula for adding perturbations to the image is as follows: in are the images before and after the perturbation is added, Indicates that the perturbation added to each pixel in the image x is limited to ε, g is the gradient value matrix of the loss function with respect to the input image, v is the matrix recording the perturbation position, the value of the perturbation point position is 1, and the rest are 0, g·v represents the logical product of the element values at the corresponding positions of the two matrices, and α is a fixed value parameter set to control the amplitude of the added perturbation; Step 13: Among the remaining undisturbed pixels in the image, the first k pixels p1, p2, ..., p k As a candidate perturbation point set, add perturbation to one of the pixels. If i The added perturbation causes the loss function value to drop the most, then p i As the new perturbation point in the t+1th iteration, add its perturbation to the image and modify p in v accordingly. i The value of the corresponding pixel position is 1; Step 14: Determine whether the image generated by the current iteration can successfully attack the DNN. If it cannot successfully attack the DNN, the number of iterations is increased by 1, and jump to step 11 to continue adding perturbations to the image; The perturbation optimization module is used to calculate the importance of each perturbation point relative to the overall perturbation for the initial adversarial sample. It then determines and removes redundant perturbations one by one in ascending order of perturbation importance. After recalculating the perturbation importance, it sorts the perturbations in ascending order of importance and removes the set number or proportion of perturbations. The method for calculating perturbation importance is as follows: Get adversarial sample x adv The perturbation δ added in , assuming that the perturbation of the adversarial sample is composed of the perturbation δ of q pixels j Composition, j = 1, 2, 3, ..., q, for each disturbance δ in the disturbance δ j Calculate its importance: From the original adversarial sample x adv Remove the disturbance δ j , and get the new perturbation image calculate The loss function value y is the true label of image x, δ j The perturbation importance σ j pass Compared with the original adversarial sample x adv The loss function value f(x adv ,y) changes, and the calculation formula is: And a loop control module is used to cyclically call the initial perturbation generation module and the perturbation optimization module, add perturbations to the image and optimize them so that the adversarial sample can successfully attack the DNN until the set number of cycles is exceeded or the sparsity of the perturbation no longer increases for the set number of consecutive times.
6. The adversarial sample generation system based on sparse perturbation according to claim 5, characterized in that: It also includes a perturbation amplitude optimization module, which is used to calculate the importance of the perturbation of each perturbation point in the generated adversarial sample, judge and delete redundant perturbations one by one in order of perturbation importance from small to large; reduce the perturbation amplitude for the remaining perturbation points, and obtain the final adversarial sample that can successfully attack DNN.
7. A computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is loaded into a processor, the steps of the adversarial sample generation method based on sparse perturbation are implemented.
8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the adversarial sample generation method based on sparse perturbation are implemented.