An adversarial attack method and system using class activation map
By using class activation maps (CAMs) in a weighted manner across different regions of an image, combined with gradient attack methods, final adversarial examples are generated, solving the problem of large perturbation in existing technologies and achieving attack effects with low perturbation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2023-02-10
- Publication Date
- 2026-06-23
AI Technical Summary
Existing gradient-based adversarial attack methods suffer from large image perturbations and low concealment, making it difficult to reduce the perturbation amount without affecting the success rate of the attack.
By using class activation maps (CAM) to weight the perturbations and limit the attack intensity in different regions of the image, and combining this with gradient attack methods, the final adversarial examples are generated.
Without affecting the success rate of the attack, it significantly reduces the amount of perturbation and improves the concealment of the perturbation, making it suitable for any gradient attack method.
Smart Images
Figure CN116109886B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning technology, and in particular to an adversarial attack method and system utilizing category activation graphs. Background Technology
[0002] With the rapid development of deep learning, deep neural networks have been widely used in computer vision in recent years. However, deep learning has also brought some security challenges. Researchers have found that deep neural networks are vulnerable to adversarial examples. By adding carefully designed small perturbations, the generated adversarial examples can cause the model to misclassify. This method of generating adversarial examples is called an adversarial attack.
[0003] Adversarial attacks can be categorized into white-box attacks and black-box attacks. A white-box attack allows the attacker to access all information about the target model, such as its network architecture and parameters, while a black-box attack prevents the attacker from obtaining this information. This invention provides an attack method specifically designed for white-box attacks. In white-box attacks, two key performance indicators reflect the attack method's effectiveness: attack success rate and perturbation amount. Attack success rate refers to the percentage of misclassified adversarial samples out of all adversarial samples. Perturbation amount refers to the difference between the original image and the adversarial sample in terms of perturbation level. p Distance under norms, commonly used norms include L2 (representing the square root of the sum of squares of the elements; in adversarial examples, it represents the square root of the sum of squares of the elements of the perturbation; for image data, the smaller the L2 norm, the more difficult it is for the human eye to recognize the adversarial example) and l. ∞ (This represents the maximum absolute value of each element; in adversarial examples, it represents the maximum value of the perturbation elements). The higher the attack success rate and the lower the perturbation amount, the stronger the attack method.
[0004] Researchers have proposed a series of white-box attack methods, which can be broadly categorized into two types: gradient-based adversarial attacks and optimization-based adversarial attacks. The former aims to constrain the perturbation amount within a certain range, striving for a higher attack success rate; representative methods include FGSM, PGD, MIM, and TIM. The latter optimizes the perturbation amount while ensuring the generated adversarial examples are minimized.
[0005] Compared to optimization methods, gradient-based adversarial attacks are more widely used in practice due to their faster computation speed. However, their drawback is the large amount of perturbation in the adversarial examples. Existing gradient-based attack methods, including FGSM, PGD, MIM, and TIM, all add perturbations globally to the image, meaning that the value of each pixel in the image may change. However, the problem with this approach is that it modifies the image too much, especially the perturbation at the L2 norm, resulting in low stealth and easy detection by the human eye. Summary of the Invention
[0006] To address the aforementioned problems, the present invention aims to provide an adversarial attack method and system utilizing Classification Activation Maps (CAMs). By using CAMs to limit the attack intensity in different regions of an image, the perturbation can be reduced without affecting the attack success rate. Furthermore, it can be combined with any gradient-based attack method.
[0007] The above-mentioned objective of this invention is achieved through the following technical solutions:
[0008] An adversarial attack method utilizing class activation graphs includes the following steps:
[0009] S1: Load the trained deep learning module for generating adversarial perturbations, set the number of adversarial attacks, and read the original image data to be attacked.
[0010] S2: Attack the original image data according to the number of attacks, wherein the perturbation added to the original image data in each round is obtained by weighting the initial perturbation calculated from the adversarial sample generated in the previous round and the CAM image;
[0011] S3: After performing the number of attacks mentioned above, generate the final adversarial sample.
[0012] Furthermore, step S1 also includes:
[0013] Choose any gradient attack method, including FGSM, PGD, MIM, and TIM.
[0014] The maximum number of iterations is set according to the type of the selected gradient attack method.
[0015] Furthermore, step S2 also includes: the adversarial sample generated in each round is generated by adding the perturbation of the current round to the adversarial sample generated in the previous round, specifically:
[0016] Let the original image data be x, and the perturbation added in the t-th round be p. t The adversarial sample generated in round t is The adversarial sample generated in round t+1 is:
[0017]
[0018] in, That is, the adversarial sample in the first round is the input original image data.
[0019] Furthermore, step S2 also includes: calculating the initial perturbation for the current round based on the adversarial examples generated in the previous round, specifically:
[0020] The adversarial sample generated in the previous round is input into the deep learning model. Using the backpropagation of the deep learning model, the gradient of the deep learning model relative to the adversarial sample generated in the previous round is calculated, which is the initial perturbation of the current round.
[0021] Furthermore, when the gradient attack method is PGD, perturbations are added to the image iteratively along the direction of gradient increase, specifically as follows:
[0022] Let the original image be x, the category be y, and the model be... θ The loss function is L(θ,x,y), and the gradient of the loss function with respect to the original image is...
[0023] Let coefficient α represent the constraint value of each round of perturbation. The initial perturbation is obtained by processing the gradient with the sign function and multiplying it by α.
[0024]
[0025] in, The initial disturbance in round t+1 is... This is the gradient relative to the adversarial sample generated in the previous round.
[0026] Furthermore, step S2 also includes: calculating the CAM graph for the current round based on the adversarial examples generated in the previous round, specifically:
[0027] The CAM graph is defined using an image of the same size as the input image data from the previous round of adversarial examples;
[0028] In the CAM image, the contribution distribution of different regions to a specified category is distinguished by the pixel value. The larger the pixel value, the higher the significance score, and the greater the contribution of the corresponding region to the prediction result of the specified category. In the (t+1)th round, the CAM image generated based on the adversarial examples from the previous round is represented as C. t+1 .
[0029] Further, in step S2, the perturbation added to the original image data in each round is obtained by weighting the initial perturbation calculated from the adversarial sample generated in the previous round and the CAM image, specifically:
[0030] The input is the initial perturbation. The weight is the CAM graph C t+1 The output is a weighted perturbation, denoted as p. t+1The initial perturbation is directly multiplied by the CAM image, which means that the initial perturbation is weighted pixel by pixel;
[0031] For a pixel at position (i,j) in the image, the perturbation value of pixel (i,j) in round t+1 is equal to the initial perturbation corresponding to pixel (i,j) multiplied by the score of the position corresponding to pixel (i,j) in the CAM image of this round, that is:
[0032] Finally, the adversarial samples generated in each round are as follows:
[0033] Where ⊙ represents the Hadamard product, indicating the multiplication of elements at corresponding positions;
[0034] After the number of attacks mentioned above, the obtained This is the final adversarial sample.
[0035] An adversarial attack system for performing the adversarial attack method using class activation graphs as described above, comprising:
[0036] The adversarial preparation module is used to load the trained deep learning module for generating adversarial perturbations, set the number of adversarial attacks, and read the original image data to be attacked.
[0037] The adversarial attack module is used to attack the original image data according to the number of attacks, wherein the perturbation added to the original image data in each round is obtained by weighting the initial perturbation calculated from the adversarial sample generated in the previous round and the CAM image.
[0038] The final sample generation module is used to generate the final adversarial sample after performing the number of attacks described above.
[0039] A computer device includes a memory and one or more processors, the memory storing computer code that, when executed by the one or more processors, causes the one or more processors to perform the method described above.
[0040] A computer-readable storage medium storing computer code that, when executed, performs the method described above.
[0041] Compared with the prior art, the present invention has at least one of the following beneficial effects:
[0042] (1) A method for adversarial attacks using class activation maps (CAMs) is provided, comprising the following steps: S1: Loading a trained deep learning module for generating adversarial perturbations, setting the number of attacks for the adversarial attack, and reading the original image data to be attacked; S2: Attacking the original image data according to the number of attacks, wherein the perturbation added to the original image data in each round is obtained by weighting the initial perturbation calculated from the adversarial sample generated in the previous round and the CAM map; S3: After performing the number of attacks, generating the final adversarial sample. The above technical solution uses class activation maps (CAMs) to limit the attack intensity in different regions of the image, reducing the amount of perturbation without affecting the attack success rate.
[0043] (2) The adversarial attack method using category activation graphs provided by this invention can be combined with all gradient attack methods, including single-step attack methods and iterative attack methods. Attached Figure Description
[0044] Figure 1 This is a schematic diagram of the original aerial image of a river, which is a category of the present invention.
[0045] Figure 2 This is a schematic diagram of a CAM image of a river, which is a category of the present invention.
[0046] Figure 3 This is a schematic diagram of an aerial original image of a beach, which is classified as a beach in this invention.
[0047] Figure 4 This is a schematic diagram of a CAM image of a beach, which is a category of the present invention.
[0048] Figure 5 This is an overall flowchart of an adversarial attack method utilizing category activation graphs according to the present invention;
[0049] Figure 6 This is a detailed flowchart of an adversarial attack method utilizing category activation graphs according to the present invention;
[0050] Figure 7 This is a structural diagram of an adversarial attack system utilizing category activation graphs according to the present invention. Detailed Implementation
[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0052] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.
[0053] Gradient-based adversarial attack methods are more widely used in practice due to their faster computation speed, but their drawback is the large amount of perturbation in the adversarial examples. Therefore, the goal of this invention is to design a novel gradient-based attack method that reduces the amount of perturbation compared to existing methods.
[0054] With the development of deep learning interpretability theory, researchers have proposed Classification Activation Maps (CAMs) to analyze the decision-making principles of models using feature visualization. Research has found that, for a given class, deep neural networks focus on certain regions of an image when making decisions, rather than the entire image. Figure 1-4 As shown, two images and their corresponding class activation maps (CAMs) from the AI D dataset (a large aerial image dataset) were selected. Figure 1 and 2 As can be seen, the image category is river. From the category activation map (CAM), it can be seen that the model focuses more on the river channel. Figure 3 and 4 As can be seen, the image category is beach, and the category activation map (CAM) shows that the model pays more attention to the junction of the coastline.
[0055] Therefore, this invention fully utilizes this characteristic of deep neural networks, using Class Activation Maps (CAMs) as weights to weight the perturbations, ensuring that the perturbations are added to the regions of greatest interest to the model. This reduces the amount of perturbation while maintaining the success rate of the attack. As a plug-and-play module, this invention can be combined with any gradient-based method.
[0056] The following is an illustration through specific examples:
[0057] First Embodiment
[0058] like Figure 5 and 6 As shown, this embodiment provides an adversarial attack method utilizing class activation graphs, including the following steps:
[0059] S1: Load the trained deep learning module for generating adversarial perturbations, set the number of adversarial attacks, and read the original image data to be attacked.
[0060] Specifically, in this embodiment, it is first necessary to load a pre-trained deep learning model for generating subsequent adversarial perturbations. This deep learning model can be any deep learning model for adversarial perturbations trained using existing technologies. Since it is not the core inventive point of this invention, it will not be described in detail in this embodiment.
[0061] Furthermore, this embodiment does not limit the gradient attack method used in this invention, and the technical points of this invention can be combined with any gradient-based method. The gradient attack method can be any of the following: FGSM, PGD, MIM, TIM, etc. The maximum number of iterations (i.e., the number of attacks) is set according to the type of the selected gradient attack method. For example, FGSM is a single-step attack method with a maximum of 1 iteration; PGD, MIM, and TIM are iterative attack methods, and the maximum number of iterations is set according to actual needs. In this embodiment, taking PGD as an example, the maximum number of iterations, max_iter, can be set to 5.
[0062] Next, the raw image data is read; this can be a single image or a batch of images.
[0063] S2: Attack the original image data according to the number of attacks, wherein the perturbation added to the original image data in each round is obtained by weighting the initial perturbation calculated from the adversarial sample generated in the previous round and the CAM image.
[0064] Specifically, in the attack section, a max_iter round attack is performed on the original image data. If counting starts from 1, then the starting round is 1 and the ending round is max_iter. Let the original image data be x, and the perturbation added in the t-th round be p. t The adversarial sample generated in round t is Therefore, the adversarial sample in each round is generated by adding the perturbation of the current round to the adversarial sample in the previous round. Thus, the adversarial sample generated in the (t+1)th round is:
[0065]
[0066] in, That is, the input of the adversarial sample in the first round is the original image data.
[0067] For each round of attack, the initial perturbation for the current round is first calculated based on the adversarial sample generated in the previous round. Here, the initial perturbation refers to the perturbation obtained directly using the gradient attack method, corresponding to the weighted perturbation mentioned later. The specific calculation method for the initial perturbation is as follows: input the adversarial sample generated in the previous round into the deep learning model, and use the backpropagation of the deep learning model to calculate the gradient of the deep learning model relative to the adversarial sample generated in the previous round, which is the initial perturbation for the current round.
[0068] Taking the gradient attack method PGD as an example, perturbations are added to the image iteratively along the direction of gradient increase. Specifically, let the original image be x, the category be y, the model be θ, and the loss function be L(θ,x,y). The gradient of the loss function with respect to the original image is... Let coefficient α represent the constraint value of each round of perturbation. The initial perturbation is obtained by processing the gradient with the sign function and multiplying it by α.
[0069]
[0070] in, The initial disturbance in round t+1 is... This is the gradient relative to the adversarial sample generated in the previous round.
[0071] Then, based on the adversarial examples generated in the previous round, the CAM graph for the current round is calculated. Specifically, the CAM graph is defined using an image of the same size as the input image data, based on the adversarial examples from the previous round. In the CAM graph, the contribution distribution of different regions to a specified category is distinguished by the pixel values. The larger the pixel value, the higher the significance score, and the greater the contribution of the corresponding region to the prediction result of the specified category. The range of pixel values is [0,1]. There are a series of algorithms for CAM graphs, such as CAM, Grad-CAM, Grad-CAM++, etc. Since CAM requires replacing the fully connected layers in the network with GAP layers, retraining is required when the model structure is incompatible, which is quite troublesome in practical applications. Therefore, subsequent optimization algorithms for CAM, such as Grad-CAM and Grad-CAM++, are more commonly used. Specific algorithms are not detailed here. In round t+1, the CAM graph generated based on the adversarial examples from the previous round is represented as C. t+1 .
[0072] It should be noted that for each round of the iterative attack (except the first round), the CAM map of the adversarial sample from the previous round is calculated. In the first round of the iterative attack and in the single-step attack, the CAM map of the original image is calculated.
[0073] The following is a key step in this invention: weighting the initial adversarial perturbation. The input is the initial perturbation. The weight is the CAM graph C t+1 The output is a weighted perturbation, denoted as p. t+1 The initial adversarial perturbation and the CAM image are the same size as the original image, therefore they are identical in size. The initial perturbation is directly multiplied by the CAM image, which represents a pixel-wise weighting of the initial perturbation.
[0074] For a pixel at position (i,j) in the image, the perturbation value of pixel (i,j) in round t+1 is equal to the initial perturbation corresponding to pixel (i,j) multiplied by the score of the position corresponding to pixel (i,j) in the CAM image of this round, that is:
[0075]
[0076] Finally, the adversarial samples generated in each round are as follows:
[0077]
[0078] Where ⊙ represents the Hadamard product, indicating the multiplication of elements at corresponding positions;
[0079] After the number of attacks mentioned above, the obtained
[0080] Furthermore, it should be noted that if an attack succeeds during the iteration process, the attack can be terminated early. If an attack succeeds before reaching the `max_iter` round, the iteration can be stopped.
[0081] In another embodiment, the CAM image can also be binarized and then used as a mask to combat disturbances. The difference from this embodiment is that this embodiment does not perform binarization, but retains floating-point numbers, which makes the weighting method more refined.
[0082] S3: After performing the stated number of attacks, generate the final adversarial sample, i.e. Second Embodiment
[0083] like Figure 7 As shown, this embodiment provides a class activation graph adversarial attack system for executing the class activation graph adversarial attack method as described in the first embodiment, comprising:
[0084] Adversarial preparation module 1 is used to load the trained deep learning module for generating adversarial perturbations, set the number of adversarial attacks, and read the original image data to be attacked.
[0085] The adversarial attack module 2 is used to attack the original image data according to the number of attacks, wherein the perturbation added to the original image data in each round is obtained by weighting the initial perturbation calculated from the adversarial sample generated in the previous round and the CAM image.
[0086] The final sample generation module 3 is used to generate the final adversarial sample after performing the number of attacks.
[0087] A computer-readable storage medium stores computer code that, when executed, performs the methods described above. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.
[0088] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.
[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0090] It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. An adversarial attack method utilizing class activation graphs, characterized in that, Includes the following steps: S1: Load the trained deep learning model for generating adversarial perturbations, set the number of adversarial attacks, and read the original image data to be attacked. S2: Attack the original image data according to the number of attacks, wherein the perturbation added to the original image data in each round is obtained by weighting the initial perturbation calculated from the adversarial sample generated in the previous round and the CAM image; Step S2 further includes: calculating the CAM graph for the current round based on the adversarial examples generated in the previous round, specifically: The CAM graph is defined using an image of the same size as the input image data from the previous round of adversarial examples; In the CAM image, the contribution distribution of different regions to a specified category is distinguished by the size of the pixel value. The larger the pixel value, the higher the significance score, and the greater the contribution of the corresponding region to the prediction result of the specified category. +1 round, the CAM graph generated based on the adversarial examples from the previous round is represented as follows: ; S3: After performing the number of attacks mentioned above, generate the final adversarial sample.
2. The adversarial attack method using category activation graphs according to claim 1, characterized in that, Step S1 also includes: Choose any gradient attack method, including FGSM, PGD, MIM, and TIM. The maximum number of iterations is set according to the type of gradient attack method selected.
3. The adversarial attack method using category activation graphs according to claim 1, characterized in that, Step S2 further includes: the adversarial sample generated in each round is generated by adding the perturbation of the current round to the adversarial sample generated in the previous round, specifically: Let the original image data be , No. The disturbance added by the wheel is , No. The adversarial examples generated in the round are Then the first The adversarial sample generated in round +1 is: in, That is, the input of the adversarial example in the first round is the original image data.
4. The adversarial attack method using category activation graphs according to claim 1, characterized in that, Step S2 further includes: calculating the initial perturbation for the current round based on the adversarial examples generated in the previous round, specifically: The adversarial sample generated in the previous round is input into the deep learning model. Using the backpropagation of the deep learning model, the gradient of the deep learning model relative to the adversarial sample generated in the previous round is calculated, which is the initial perturbation of the current round.
5. The adversarial attack method using category activation graphs according to claim 4, characterized in that, Also includes: When the gradient attack method is PGD, perturbations are added to the image iteratively along the direction of gradient increase, specifically: Let the original image be Category The model is The loss function is The gradient of the loss function with respect to the original image is ; Let coefficients This represents the constraint value for each round of perturbation, and the gradient is processed by the sign function and multiplied by... Thus, the initial perturbation was obtained: in, For the first The initial disturbance of +1 round, This is the gradient relative to the adversarial sample generated in the previous round.
6. The adversarial attack method using category activation graphs according to claim 5, characterized in that, In step S2, the perturbation added to the original image data in each round is obtained by weighting the initial perturbation calculated from the adversarial sample generated in the previous round and the CAM image, specifically: The input is the initial perturbation. The weights are those of the CAM graph. The output is a weighted perturbation, denoted as The initial perturbation is directly multiplied by the CAM image, which means that the initial perturbation is weighted pixel by pixel; For the position in the image is The pixels, in +1 round pixel The perturbation value is equal to the pixel The corresponding initial disturbance is multiplied by the number of pixels in the CAM image in this round. The score at the corresponding position, i.e.: Finally, the adversarial samples generated in each round are as follows: in, The Hadamard product represents the multiplication of elements at corresponding positions; After the number of attacks described above, the obtained This is the final adversarial sample.
7. A system for performing adversarial attacks using class activation graphs as described in any one of claims 1-6, characterized in that, include: The adversarial preparation module is used to load the trained deep learning model for generating adversarial perturbations, set the number of adversarial attacks, and read the original image data to be attacked. The adversarial attack module is used to attack the original image data according to the number of attacks, wherein the perturbation added to the original image data in each round is obtained by weighting the initial perturbation calculated from the adversarial sample generated in the previous round and the CAM image. The final sample generation module is used to generate the final adversarial sample after performing the number of attacks described above.
8. A computer device comprising a memory and one or more processors, the memory storing computer code that, when executed by the one or more processors, causes the one or more processors to perform the method as described in any one of claims 1 to 6.
9. A computer-readable storage medium storing computer code, wherein when the computer code is executed, the method of any one of claims 1 to 6 is performed.