A method and device for generating an adversarial sample based on a forward class activation map

Abstract

The application discloses a kind of based on forward class activation map's adversarial sample generation method and device, it is related to artificial intelligence technical field.The method includes: initial adversarial sample is based on the aggregated momentum gradient of the tth round to obtain predicted adversarial sample and the corresponding forward class activation map of predicted adversarial sample;Important area is selected from forward class activation map and local image transformation is carried out to the forward class activation map to obtain local enhancement image;Local enhancement image is based on SIM method to obtain a group of copy images;Copy image obtains the average gradient of the t+1th round by the back propagation of loss function, according to the average gradient of the t+1th round and the aggregated momentum gradient of the tth round, obtains the aggregated momentum gradient of the t+1th round;According to the aggregated momentum gradient of the t+1th round, the initial adversarial sample of the tth round is updated, and updated adversarial sample is obtained, if iteration number reaches training number, update adversarial sample is determined as final adversarial sample.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and more particularly to a method and device for generating an adversarial sample based on a forward class activation map. BACKGROUND

[0002] The wide application of deep learning models has greatly improved the performance of many computer vision tasks, such as image classification, object detection, image segmentation, etc. However, recent research has found that deep neural networks (DNNs) are vulnerable to carefully designed adversarial samples. In 2014, Szegedy et al. first discovered the existence of adversarial samples in the field of image classification, i.e., adding a small perturbation to the original image can cause the DNN model to make a wrong decision. More seriously, these small perturbations are very subtle and not easily perceived by the human visual system. This discovery reveals that DNN models have many security problems, and if not addressed, it will cause uncontrollable security risks. Therefore, generating adversarial samples with high attackability not only can detect the vulnerabilities of DNN models, but also can promote the robustness of DNN models.

[0003] Existing adversarial sample generation methods can be divided into two categories: white-box attack methods and black-box attack methods. Among them, the white-box attack method assumes that all information of the target model is known and uses this information to generate adversarial samples. On the contrary, the black-box attack method can generate adversarial samples with attackability without knowing any information of the target model. Since most model information is difficult to obtain in the real world, black-box attacks are more meaningful. Existing black-box attack models can be further divided into two categories: query-based attack methods and transfer-based attack methods. Query-based methods query the target model by using some input images, and then use the queried information to generate adversarial samples to attack the black-box model. Although the query-based attack method can achieve good black-box attack effect, this method inevitably introduces high computational overhead due to the need for a large number of query operations. Transfer-based methods attack the black-box target model by generating adversarial samples on the white-box model, mostly gradient-based attack methods.

[0004] Scholars in related fields have proposed many transfer-based adversarial attack methods based on the entire process of deep learning, among which gradient-based attack methods are the most common: using the gradient backpropagation of neural networks to gradually change the pixel values of samples, and maximizing the network loss to make the neural network decision error.

[0005] However, due to the different structures and parameters of different models, the migration-based method is poor in attacking the black-box target model. Similar to the overfitting problem of the data set during the training of the DNN model, there is also an overfitting problem of the white-box model during the generation of the adversarial sample. Therefore, researchers introduce the data enhancement method into the field of adversarial sample generation and propose a series of input transformation attacks, such as scaling, translation, brightness transformation and image mixing of images, which significantly improve the migration of the adversarial sample.

[0006] The existing input transformation attack method is to transform the whole image without distinction, however, different regions of the image have different influences on the model decision, and the same transformation processing of all regions will limit the improvement of the migration. SUMMARY

[0007] The embodiment of the application provides a kind of based on the generation method and device of adversarial sample of forward class activation map, can utilize improved forward class activation map, identify the important area of greater influence on model decision while relieving the overfitting problem of white-box model, simultaneously by local input transformation to important area, effectively improve the migration of adversarial sample.

[0008] The embodiment of the application provides a kind of based on the generation method of adversarial sample of forward class activation map, comprising:

[0009] According to the initial image, the initial adversarial sample of the tth round is obtained, the initial adversarial sample is based on the aggregated momentum gradient of the tth round to obtain a predicted adversarial sample and a forward class activation map corresponding to the predicted adversarial sample;

[0010] An important region is selected from the forward class activation map, and a local image transformation is performed on the forward class activation map to obtain a local enhanced image; the local enhanced image is based on the SIM method to obtain a group of copy images;

[0011] The copy image obtains the average gradient of the t+1th round through the back propagation of the loss function, and the aggregated momentum gradient of the t+1th round is obtained according to the average gradient of the t+1th round and the aggregated momentum gradient of the tth round.

[0012] The initial adversarial sample of the tth round is updated according to the aggregated momentum gradient of the t+1th round to obtain an updated adversarial sample, and if the iteration number reaches the training number, the updated adversarial sample is determined as the final adversarial sample.

[0013] Preferably, the forward class activation map corresponding to the predicted adversarial sample is determined by the following formula:

[0014]

[0015] Wherein, denotes the predicted adversarial sample, g denotes the aggregated momentum gradient of the t-th round, a denotes the hyperparameter, x denotes the initial adversarial sample, y c denotes the predicted score of the DNN for class c without softmax function activation, denotes the data of position (i, j) in channel k of feature layer A, Z is equal to the product of the width and height of the feature layer, denotes the weight of A k , H(·) denotes the bilinear interpolation function, A k denotes the data in channel k of feature layer A, c denotes the class of the input data, and k denotes the k-th channel of feature layer A.

[0016] Preferably, the important region is determined by the following formula:

[0017]

[0018] The local image transformation is determined by the following formula:

[0019] LT q (x) = x · (β · S q (x) + 1 - S q (x)) + ξ

[0020] wherein S q (x) denotes the important region, denotes the forward class activation map, β is a hyperparameter, ξ denotes a random noise, and LT q (x) denotes the locally enhanced image.

[0021] Preferably, the average gradient of the t+1-th round is determined by the following formula:

[0022]

[0023] The aggregated momentum gradient of the t+1-th round is determined by the following formula:

[0024]

[0025] wherein, denotes the initial adversarial sample of the t-th round, Q = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100] denotes the value range of q, m1 denotes the number of copy images, m2 denotes the number of locally enhanced images, J(x, y; θ) denotes the loss function, θ denotes the parameters of the white-box model, g t denotes the aggregated momentum gradient of the t-th round, g t+1 denotes the aggregated momentum gradient of the t+1-th round, denotes the average gradient of the t+1-th round.

[0026] Preferably, the initial adversarial sample of the tth round is updated by the following formula:

[0027]

[0028] wherein, denotes the initial adversarial sample of the tth round, denotes the updated adversarial sample.

[0029] The embodiment of the present application also provides an adversarial sample generation device based on a forward class activation map, comprising:

[0030] A first obtaining unit is configured to obtain an initial adversarial sample of a tth round according to an initial image, wherein the initial adversarial sample is based on a predicted adversarial sample obtained by an aggregated momentum gradient of the tth round and a forward class activation map corresponding to the predicted adversarial sample;

[0031] A second obtaining unit is configured to select an important region from the forward class activation map and perform a local image transformation on the forward class activation map to obtain a local enhanced image, wherein the local enhanced image is based on a SIM method to obtain a set of copy images;

[0032] A third obtaining unit is configured to obtain an average gradient of a t+1th round by back propagation of the copy images through a loss function, and obtain an aggregated momentum gradient of the t+1th round according to the average gradient of the t+1th round and the aggregated momentum gradient of the tth round;

[0033] A determining unit is configured to update the initial adversarial sample of the tth round according to the aggregated momentum gradient of the t+1th round to obtain an updated adversarial sample, and determine the updated adversarial sample as a final adversarial sample if the number of iterations reaches a training number.

[0034] Preferably, the forward class activation map corresponding to the predicted adversarial sample is determined by the following formula:

[0035]

[0036] wherein, denotes the predicted adversarial sample, g denotes the aggregated momentum gradient of the tth round, and a denotes a hyperparameter, x denotes the initial adversarial sample, y c denotes a prediction score of the DNN for the class c without softmax function activation, denotes data of a position (i, j) in a channel k of a feature layer A, and Z is equal to the product of the width and the height of the feature layer, denotes the weight of A k , and H(·) denotes a bilinear interpolation function, kc k represents the data in the k-th channel of the feature layer A, c represents the category of the input data, and k represents the k-th channel of the feature layer A.

[0037] Preferably, the important region is determined by the following formula:

[0038]

[0039] The local image transformation is determined by the following formula:

[0040] LT q (x) = x (β S q (x) + 1 - S q (x)) + ξ

[0041] wherein S q (x) represents the important region, represents the forward class activation map, β is a hyperparameter, ξ represents a random noise, and LT q (x) represents the local enhanced image.

[0042] The embodiment of the present application also provides a computer device, which comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to enable the processor to execute the forward class activation map-based adversarial sample generation method.

[0043] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to enable the processor to execute the forward class activation map-based adversarial sample generation method.

[0044] The embodiment of the present application provides a kind of based on forward class activation map's adversarial sample generation method and device, the method comprises: obtaining the initial adversarial sample of the tth round according to initial image, the initial adversarial sample is based on the aggregated momentum gradient of the tth round to obtain predicted adversarial sample and the forward class activation map corresponding to the predicted adversarial sample;Important region is selected from the forward class activation map, and local image transformation is carried out to the forward class activation map to obtain local enhanced image;The local enhanced image is based on SIM method to obtain a group of copy images;The copy image is obtained by the back propagation of loss function to obtain the average gradient of the t+1th round, the aggregated momentum gradient of the t+1th round is obtained according to the average gradient of the t+1th round and the aggregated momentum gradient of the tth round;The initial adversarial sample of the tth round is updated according to the aggregated momentum gradient of the t+1th round, and the updated adversarial sample is obtained, if the iteration number reaches training number, the updated adversarial sample is determined as final adversarial sample.The method is based on the gradient-based adversarial sample generation method, and the improved forward class activation map is used to identify the important region that has greater influence on model decision, while the overfitting problem of white-box model is relieved, and the migration of adversarial sample is effectively improved by local input transformation to important region. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0046] Figure 1 A flowchart of the adversarial sample generation method based on forward class activation map provided by the embodiment of the present application is shown.

[0047] Figure 2A A shark clean sample diagram provided by the embodiment of the present application is shown.

[0048] Figure 2B A forward class activation diagram calculated on the clean sample provided by the embodiment of the present application is shown.

[0049] Figure 2C An important region diagram screened based on forward class activation map provided by the embodiment of the present application is shown.

[0050] Figure 2D A copy image diagram generated by local transformation provided by the embodiment of the present application is shown.

[0051] Figure 2E A classification diagram of adversarial sample on target model Inc-v4 provided by the embodiment of the present application is shown.

[0052] Figure 3 A structure schematic diagram of an adversarial sample generation device based on a forward class activation map is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0054] Figure 1 A flowchart of an adversarial sample generation method based on a forward class activation map is provided for an embodiment of the present application, as shown in the figure, the method comprises the following steps: Figure 1

[0055] Step 101, obtaining an initial adversarial sample of the tth round according to an initial image, the initial adversarial sample is based on a predicted adversarial sample obtained by an aggregated momentum gradient of the tth round and a forward class activation map corresponding to the predicted adversarial sample;

[0056] Step 102, selecting an important region from the forward class activation map and performing local image transformation on the forward class activation map to obtain a local enhanced image; the local enhanced image is based on a SIM method to obtain a set of copy images;

[0057] Step 103, the copy images are obtained by the back propagation of a loss function to obtain an average gradient of the t+1th round, and the average gradient of the t+1th round and the aggregated momentum gradient of the tth round are used to obtain an aggregated momentum gradient of the t+1th round;

[0058] Step 104, updating the initial adversarial sample of the tth round according to the aggregated momentum gradient of the t+1th round to obtain an updated adversarial sample, and if the number of iterations reaches the training number, the updated adversarial sample is determined as a final adversarial sample.

[0059] Before step 101, an original image x and a corresponding true label y are selected and obtained from a data set; an adversarial sample in a first round of iteration process is initialized as x; an initial aggregated momentum gradient g0 is set as 0; a white box model with a parameter θ is selected, and a loss function of the white box model is J(x, y; θ).

[0060] In this embodiment, the data set selected for use is 1000 images respectively belonging to 1000 categories randomly selected from ILSVRC 2012 validation set.

[0061] ​​In step 101, adversarial samples for round t are obtained based on the selected initial image. To distinguish them from other adversarial samples described later, these adversarial samples for round t are referred to as the initial adversarial samples. Further, predicted adversarial samples are obtained based on the initial adversarial samples and the pooling momentum gradient for round t. Furthermore, the forward class activation map corresponding to the predicted adversarial samples can be obtained based on the predicted adversarial samples.

[0062] Specifically, before introducing the forward class activation graph, let's first introduce a few concepts:

[0063] Class Activation Maps (CAMs) are a method for visualizing class-specific information within deep learning convolutional neural networks (DNNs). By analyzing the feature maps and weights of the DNN, CAMs visualize regions in an image that are relevant to the target class as heatmaps. The generated CAMs allow us to see which image regions the DNN focuses on during the classification process.

[0064] In this embodiment of the invention, the expression for the class activation graph is as shown in formula (1):

[0065]

[0066] In formula (1), x represents the input image, A represents the feature layer output by the last convolutional layer of the DNN, k represents the k-th channel of feature layer A, c represents the category of the input data, and A k This represents the data in channel k of feature layer A, and H(·) represents the bilinear interpolation function. A represents k The weight, The calculation method is expressed by the following formula (2):

[0067]

[0068] In formula (2), y c This represents the prediction score of the DNN for class c without activation by the softmax function. Z represents the data at position (i,j) in channel k of feature layer A, and Z is equal to the product of the width and height of the feature layer.

[0069] Based on formulas (1) and (2) above, in this embodiment of the invention, after obtaining the predicted adversarial sample, the forward class activation map can be further obtained. Specifically,

[0070] The predicted adversarial example is expressed by the following formula (3):

[0071]

[0072] Further, the forward class activation map is represented by the following formula (4):

[0073]

[0074] wherein, represents the predicted adversarial sample, x represents the initial adversarial sample, g represents the t-th round of aggregated momentum gradient, and a is a hyperparameter, represents the forward class activation map.

[0075] In the implementation of the present application, since different regions of the image have different influences on the model decision, in the embodiment of the present application, the image transformation of the local region can be performed according to the forward class activation map, specifically:

[0076] In step 102, the important region is selected from the forward class activation map, wherein the selection of the important region can be represented according to the following formula (5):

[0077]

[0078] wherein, Q q (·) represents the data ranked in the qth percentile among all input data, represents the forward class activation map.

[0079] Further, after determining the important region, the forward class activation map can be subjected to local image transformation, and the local image transformation is determined by the following formula:

[0080] LT q (x)=x·(β·S q (x)+1-S q (x))+ξ (6)

[0081] wherein, β is a hyperparameter, in the embodiment of the present application, β is used to control the degree of pixel value scaling, ξ∈[-∈,∈] is a random noise with the same shape as the input image, which is used to enhance the effect of image transformation, ∈=16 is the maximum value of the difference between the adversarial sample and the input image, and LT q (x) represents the local enhanced image.

[0082] In the embodiment of the present application, in order to further enhance the migration of the generated adversarial samples, the SIM (Scale-Invariant (attack) Method) method is combined, and the pixel values of the locally enhanced image are respectively changed to 1 / 1, 1 / 2, 1 / 4, 1 / 8, 1 / 16 of the locally enhanced image, thereby obtaining five copy images. In the embodiment of the present application, the set of copy images obtained by the SIM method for the locally enhanced image, the set here can include five, or can include three, and the specific number of the set of copy images is not limited here.

[0083] In step 103, the transformed set of copy images is sent into the white box model, and the average gradient of the t+1th round is obtained through the back propagation of the loss function, and the average gradient of the t+1th round is determined by the following formula (7):

[0084]

[0085] , wherein, x t represents the tth round of adversarial samples, that is, the initial adversarial samples, Q = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100] represents the value range of q, m1 represents the number of copy images generated by the SIM method, and m2 represents the number of locally enhanced images generated by the local input transformation method, , wherein, g t+1 represents the average gradient of the t+1th round, J(x, y; θ) represents the loss function of the white box model, and θ represents the parameters of the white box model. In the embodiment of the present application, m1 = 5 and m2 = 10.

[0086] Further, according to the determined average gradient of the t+1th round and the aggregated momentum gradient of the tth round, the aggregated momentum gradient of the t+1th round can be obtained through the momentum algorithm, wherein the aggregated momentum gradient of the t+1th round is represented by the following formula (8):

[0087]

[0088] , wherein, g t+1 represents the average gradient of the t+1th round, J(x, y; θ) represents the loss function of the white box model, and θ represents the parameters of the white box model. In the embodiment of the present application, m1 = 5 and m2 = 10. t , wherein, g t represents the aggregated momentum gradient of the tth round, g t+1 represents the average gradient of the t+1th round, and g t+1 represents the aggregated momentum gradient of the t+1th round. t+1 , wherein, g t represents the aggregated momentum gradient of the tth round, g t+1 represents the average gradient of the t+1th round, and g t+1 represents the aggregated momentum gradient of the t+1th round. , wherein, g t represents the aggregated momentum gradient of the tth round, g t+1 represents the average gradient of the t+1th round, and g t+1 represents the aggregated momentum gradient of the t+1th round.

[0089] In step 104, after the aggregated momentum gradient of the t+1th round is determined, the initial adversarial samples of the tth round can be updated according to the aggregated momentum gradient of the t+1th round to obtain updated adversarial samples, and the updated adversarial samples are determined by the following formula (9):

[0090]

[0091] ​wherein, representing updating the adversarial sample.

[0092] In the embodiments of the present application, after obtaining the updated adversarial sample, if it is determined that the current iteration number has reached the training number, the updated adversarial sample is determined as the final adversarial sample; further, if it is determined that the current iteration number has not reached the training number, the above steps are re-executed, that is, starting from step 101 until the updated adversarial sample is obtained again and the iteration number reaches the requirement of the training number.

[0093] It should be noted that in the embodiments of the present application, the training number can be set to 10 times.

[0094] In order for those skilled in the art to better understand the technical effects of the embodiments of the present application, the image adversarial sample generation method provided by the embodiments of the present application is further described below in combination with specific experimental analysis.

[0095] 1. Dataset

[0096] The selected dataset is an ImageNet-compatible dataset, which contains 1000 pictures from different categories, and the picture size is uniformly adjusted to 299x299x3.

[0097] 2. Model

[0098] The adopted model includes four normal training models Inception-v3 (Inc-v3), Inception-v4 (Inc-v4), Inception-ResNet-v2 (IncRes), and ResNet-v2-101 (Res-v2), and three adversarial training models Inc ens3 , Inc ens4 , and IncRes ens .

[0099] The adversarial sample generation method selects DIM, TIM, SIM, Admix, SCM-P, and the method provided by the embodiments of the present application (denoted as LTA). The proxy model selects Inc-v3, and the target model selects Inc-v3 (white-box attack), Inc-v4, IncRes, Res-v2, Inc ens3 , Inc ens4 , and IncRes ens .

[0100] Figure 2A A clean sample of a shark is shown; Figure 2B The forward class activation map calculated on the sample is shown; Figure 2C The important regions selected by us according to the forward class activation map are shown; Figure 2DThe generated copy images of local transformation are shown; the classification of the corresponding adversarial samples generated by the method provided by the embodiment of the application and our method on the target model Inc-v4 is respectively as shown in Figure 2E It can be seen that the originally clean samples are correctly classified as sharks, but the adversarial samples generated by the method provided by the embodiment of the application on the proxy model Inc-v3 are incorrectly classified as sticks, indicating that the attack is successful.

[0101] Table 1 shows the attack success rate of the adversarial samples generated by various methods on the target model, and it can be seen that the performance of the method provided by the embodiment of the application exceeds all other existing methods, further improving the transferability of the adversarial samples. The adversarial samples generated on the proxy model can achieve a very high attack success rate when attacking unknown target models, sufficiently alleviating the overfitting of the adversarial samples to the proxy model, and effectively improving the attack success rate of the generated adversarial samples in the transfer attack scene.

[0102] The adversarial sample generation method provided by the embodiment of the application is very important for the research of artificial intelligence security, and also has an important role in the security technology system.

[0103]

[0104] In summary, the embodiment of the application provides an adversarial sample generation method and device based on a forward class activation map, which comprises: obtaining an initial adversarial sample of the tth round according to an initial image, the initial adversarial sample being based on a predicted adversarial sample obtained from an aggregated momentum gradient of the tth round and a forward class activation map corresponding to the predicted adversarial sample; selecting an important region from the forward class activation map and performing local image transformation on the forward class activation map to obtain a local enhanced image; the local enhanced image is based on a SIM method to obtain a group of copy images; the copy images are obtained through the back propagation of a loss function to obtain an average gradient of the t+1th round, and the average gradient of the t+1th round and the aggregated momentum gradient of the tth round are used to obtain an aggregated momentum gradient of the t+1th round; the initial adversarial sample of the tth round is updated according to the aggregated momentum gradient of the t+1th round to obtain an updated adversarial sample, and if the number of iterations reaches a training number, the updated adversarial sample is determined as a final adversarial sample. Based on the gradient-based adversarial sample generation method, the improved forward class activation map is used to identify the important region that has a greater impact on the model decision, and the overfitting problem of the white-box model is alleviated, the local input transformation of the important region is performed, and the transferability of the adversarial sample is effectively improved.

[0105] Based on the same inventive concept, the embodiment of the present application provides a forward class activation map-based adversarial sample generation device. Since the principle of the device for solving the technical problem is similar to a forward class activation map-based adversarial sample generation method, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described herein.

[0106] As shown in Figure 3 , the device mainly comprises a first obtaining unit 201, a second obtaining unit 202, a third obtaining unit 203 and a determining unit 204.

[0107] The first obtaining unit 201 is configured to obtain an initial adversarial sample of the tth round according to an initial image, wherein the initial adversarial sample is based on a predicted adversarial sample obtained by a tth round of aggregated momentum gradient and a forward class activation map corresponding to the predicted adversarial sample.

[0108] The second obtaining unit 202 is configured to select an important region from the forward class activation map and perform local image transformation on the forward class activation map to obtain a local enhanced image, wherein the local enhanced image is based on a SIM method to obtain a set of copy images.

[0109] The third obtaining unit 203 is configured to obtain an average gradient of the t+1th round by back propagation of the copy images through a loss function, and obtain an aggregated momentum gradient of the t+1th round according to the average gradient of the t+1th round and the aggregated momentum gradient of the tth round.

[0110] The determining unit 204 is configured to update the initial adversarial sample of the tth round according to the aggregated momentum gradient of the t+1th round to obtain an updated adversarial sample, and determine the updated adversarial sample as a final adversarial sample if the number of iterations reaches a training number.

[0111] Preferably, the forward class activation map corresponding to the predicted adversarial sample is determined by the following formula:

[0112]

[0113] wherein, represents the predicted adversarial sample, g represents the aggregated momentum gradient of the tth round, and a represents a hyperparameter, x represents the initial adversarial sample, y c represents a prediction score of the DNN for the class c without softmax function activation, represents data of position (i, j) in channel k of feature layer A, and Z is equal to the product of the width and height of the feature layer, represents the weight of A k , and H(·) represents a bilinear interpolation function, kdenotes the data in the k-th channel of feature layer A, c denotes the class of input data, and k denotes the k-th channel of feature layer A.

[0114] Preferably, the important region is determined by the following formula:

[0115]

[0116] The local image transformation is determined by the following formula:

[0117] LT q (x) = x · (β · S q (x) + 1 - S q (x)) + ξ

[0118] wherein S q (x) denotes the important region, denotes the forward class activation map, β is a hyperparameter, ξ denotes a random noise, and LT q (x) denotes the locally enhanced image.

[0119] Preferably, the average gradient of the t+1-th round is determined by the following formula:

[0120]

[0121] The aggregated momentum gradient of the t+1-th round is determined by the following formula:

[0122]

[0123] wherein denotes the initial adversarial sample of the t-th round, Q = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100] denotes the value range of q, m1 denotes the number of copy images, m2 denotes the number of locally enhanced images, J(x, y; θ) denotes a loss function, θ denotes the parameters of the white-box model, and g t denotes the aggregated momentum gradient of the t-th round, g t+1 denotes the aggregated momentum gradient of the t+1-th round, denotes the average gradient of the t+1-th round.

[0124] Preferably, the initial adversarial sample of the t-th round is updated by the following formula:

[0125]

[0126] wherein denotes the initial adversarial sample of the t-th round, denotes the updated adversarial sample.

[0127] It should be understood that the above one kind of forward class activation map based adversarial sample generation device includes units only for logical division according to the functions implemented by the device, and in actual application, the above units can be stacked or split. And the function implemented by the one kind of forward class activation map based adversarial sample generation device provided in the embodiment corresponds to the one kind of forward class activation map based adversarial sample generation method provided in the above embodiment. For the more detailed processing flow of the device, the above method embodiment one has been described in detail, and will not be described in detail here.

[0128] Another embodiment of the present application also provides a computer device, which comprises a processor and a memory; the memory is used to store computer program code, the computer program code comprises computer instructions; when the processor executes the computer instructions, the electronic device executes each step of the one kind of forward class activation map based adversarial sample generation method in the method flow shown in the above method embodiment.

[0129] Another embodiment of the present application also provides a computer readable storage medium, which stores computer instructions, when the computer instructions run on the computer device, the computer device executes each step of the one kind of forward class activation map based adversarial sample generation method in the method flow shown in the above method embodiment.

[0130] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0131] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A method for generating adversarial samples based on forward class activation map, characterized in that, The method comprises the following steps: According to the initial image, the initial adversarial sample of the tth round is obtained, the initial adversarial sample is used to obtain the predicted adversarial sample based on the aggregated momentum gradient of the tth round, and the forward class activation map corresponding to the predicted adversarial sample is obtained; An important region is selected from the forward class activation map, and a local enhanced image is obtained by performing local image transformation on the forward class activation map; the local enhanced image is used to obtain a group of copy images based on the SIM method; The copy images are used to obtain the average gradient of the (t+1) th round through the back propagation of the loss function, and the aggregated momentum gradient of the (t+1) th round is obtained according to the average gradient of the (t+1) th round and the aggregated momentum gradient of the tth round; The initial adversarial sample of the tth round is updated according to the aggregated momentum gradient of the (t+1) th round to obtain an updated adversarial sample, and if the number of iterations reaches the training number, the updated adversarial sample is determined as the final adversarial sample; The forward class activation map corresponding to the predicted adversarial sample is determined by the following formula: The important region is determined by the following formula: The local image transformation is determined by the following formula: LT q (x) = x · (β · S q (x) + 1 - S q (x)) + ξ wherein, denotes the predicted adversarial sample, g denotes the aggregated momentum gradient of the t-th round, and a denotes a hyperparameter, x denotes the initial adversarial sample, y c denotes the predicted score of the DNN for class c without softmax function activation, denotes the data of position (i, j) in channel k of feature layer A, and Z is equal to the product of the width and height of the feature layer, denotes the weight of A k , H(·) denotes a bilinear interpolation function, and A k denotes the data in channel k of feature layer A, c denotes the class of the input data, k denotes the k-th channel of feature layer A, S q (x) denotes an important region, denotes a forward class activation map, β is a hyperparameter, and ξ denotes a random noise, LT q (x) denotes a locally enhanced image, denotes a forward class activation map, Q q (·) denotes the data ranked in the q-th percentile among all input data.

2. The method of claim 1, wherein, The average gradient of the (t+1) th round is determined by the following formula: The aggregated momentum gradient of the (t+1) th round is determined by the following formula: wherein, denotes the initial adversarial sample for the t-th round, Q = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100] represents the value range of q, m1 represents the number of copy images, m2 represents the number of local enhancement images, J(x, y; θ) represents a loss function, θ represents the parameters of the white box model, g t represents the aggregated momentum gradient of the tth round, g t+1 represents the aggregated momentum gradient of the t+1th round, represents the average gradient of the t+1th round.

3. The method of claim 1, wherein, The initial adversarial sample of the tth round is updated by the following formula: wherein, denotes the initial adversarial sample of the t-th round, denotes the updated adversarial sample, g t+1 denotes the aggregated momentum gradient of the t+1-th round.

4. A forward class activation map-based adversarial sample generation apparatus characterized by comprising: The method comprises the following steps: A first obtaining unit is configured to obtain the initial adversarial sample of the tth round according to the initial image, the initial adversarial sample is used to obtain the predicted adversarial sample based on the aggregated momentum gradient of the tth round, and the forward class activation map corresponding to the predicted adversarial sample is obtained; A second obtaining unit is configured to select an important region from the forward class activation map and obtain a local enhanced image by performing local image transformation on the forward class activation map; the local enhanced image is used to obtain a group of copy images based on the SIM method; A third obtaining unit is configured to obtain the average gradient of the (t+1) th round through the back propagation of the loss function by using the copy images, and obtain the aggregated momentum gradient of the (t+1) th round according to the average gradient of the (t+1) th round and the aggregated momentum gradient of the tth round; A determining unit is configured to update the initial adversarial sample of the tth round according to the aggregated momentum gradient of the (t+1) th round to obtain an updated adversarial sample, and if the number of iterations reaches the training number, the updated adversarial sample is determined as the final adversarial sample; The forward class activation map corresponding to the predicted adversarial sample is determined by the following formula: The important region is determined by the following formula: The local image transformation is determined by the following formula: LT q (x) = x · (β · S q (x) + 1 - S q (x)) + ξ wherein, denotes the predicted adversarial sample, g denotes the aggregated momentum gradient of the t-th round, and a denotes a hyperparameter, x denotes the initial adversarial sample, y c denotes the predicted score of the DNN for class c without softmax function activation, denotes the data at position (i, j) in channel k of feature layer A, and Z is equal to the product of the width and height of the feature layer, denotes the weight of A k , h(·) denotes the bilinear interpolation function, and A k denotes the data in channel k of feature layer A, c denotes the class of the input data, k denotes the k-th channel of feature layer A, and S q (x) denotes the important region, denotes the forward class activation map, b is a hyperparameter, and c denotes a random noise, q (x) denotes the locally enhanced image, denotes the forward class activation map, Q q (·) denotes the data ranked in the q-th percentile among all input data.

5. A computer device, comprising: The computer device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to enable the processor to execute the forward class activation map-based adversarial sample generation method in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer program is executed by the processor to enable the processor to execute the forward class activation map-based adversarial sample generation method in any one of claims 1-3.

Images