Adversarial sample generation method and system based on multi-mode image

Abstract

The application provides a multi-mode image-based adversarial sample generation method and system. In order to make the generated adversarial sample more effectively misjudge the DNN under the limitation of slight disturbance, the application first uses multiple modes of the image to determine the key area, limit the disturbance position, and obtain an image mask. Then, multiple rounds of iteration are performed to add disturbance to the image under the limitation of the disturbance pixels. In each iteration process, the importance of the current candidate pixels is first calculated, and the top-k pixel points are selected. Then, the loss function is calculated, and the gradient direction of the loss function is used as the disturbance direction. Instead of using fixed parameters as the disturbance amplitude in each iteration, the mean and standard deviation of the pixels around each pixel are calculated, similar noise is generated around each pixel, and the noise is added to the image of the last iteration as disturbance. The application can reduce the disturbance amplitude and the number of disturbance pixels, and improve the imperceptibility and attack success rate of the adversarial sample.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of based on multi-mode image's adversarial sample generation method and system, belong to artificial intelligence test field. BACKGROUND

[0002] In recent years, deep learning has been widely concerned by academia and industry, and has achieved remarkable results in computer vision, natural language processing, speech and video and other fields. While deep learning is widely concerned, the quality problem of deep neural network (DNN) system is also focused on. In safety-related fields and other key fields, the erroneous behavior of deep neural network can lead to serious consequences that cannot be retrieved. Adversarial samples can cause the system to be chaotic, form a missed judgment or a false judgment, and even cause the system to crash or be hijacked, so they can be used as effective test inputs to evaluate the robustness of the current model and find defects in the deep neural network. In addition, by studying adversarial samples, we can have a deeper understanding of some properties of the model and explain the working principle of the neural network to some extent. It can play a positive role in the security defense of the model. On the one hand, it can propose corresponding defense methods according to the attack methods and characteristics; on the other hand, it can promote the birth of more robust models based on the understanding of the defects of the neural network.

[0003] For image classification tasks, there are some pixels in the image that are beneficial to the classification task, and there are also some pixels that are focused by DNN, that is, the neural network response value is higher. These pixels are the part that needs to be focused on in the classification task and are the part that the DNN model responds to, which can play an important role in the classification stage and directly or indirectly affect the classification result. Most of the existing adversarial sample generation methods do not focus on the key pixels in the input image during the processing of the initial sample, or only focus on the gradient information of the pixels in the image sample. If the multi-dimensional information of the image can be used to locate and divide the key pixels before perturbation, the perturbation will have better effect and higher efficiency. SUMMARY

[0004] The present application aims to provide a kind of based on multi-mode image's adversarial sample generation method and system, reduce the perturbation amplitude and the number of perturbation pixel points, improve the imperceptibility and attack success rate of adversarial sample.

[0005] Technical scheme: in order to achieve the above-mentioned application purpose, the present application provides a kind of based on multi-mode image's adversarial sample generation method, including the following steps:

[0006] Step 1: Obtain the original image dataset, and transform the images to generate corresponding grayscale images and black-and-white two-color images;

[0007] Step 2: Obtain the key regions that determine the classification of the neural network using the class activation mapping method for RGB images, obtain the high-frequency regions for grayscale images, and obtain the edge regions for black-and-white two-color images. Take the intersection of the corresponding regions obtained from the three modes of images to limit the number and position of pixels of each sample image, and generate an image mask.

[0008] Step 3: Calculate the importance of pixels, select top-k pixels, and iteratively add perturbations according to the gradient direction of the loss function until the adversarial sample attack is successful or the maximum number of iterations is reached.

[0009] Preferably, the method for generating corresponding grayscale images and black-and-white two-color images in step 1 includes the following steps:

[0010] Step 11: Obtain the RGB images in the dataset and generate corresponding grayscale images for calculating the frequency information of the images.

[0011] Step 12: Further obtain black-and-white two-color images based on the grayscale images for edge extraction of the main part of the images.

[0012] Preferably, the method for generating an image mask in step 2 includes the following steps:

[0013] Step 21: Obtain the key regions that determine the classification of the DNN in the RGB images using the gradient-based class activation mapping method, and binarize them by thresholding to obtain the first layer image mask.

[0014] Step 22: Divide the edges of the main image according to the black-and-white two-color image obtained by transforming the RGB image, which is the second layer image mask.

[0015] Step 23: Perform DCT transformation and quantization on the grayscale image to obtain the frequency information of the image, and obtain the third layer image mask according to the division of the frequency.

[0016] Step 24: Generate the final image mask by integrating the image information obtained in the above steps, i.e., limit the perturbation by limiting the pixel points.

[0017] Preferably, the loss function in step 3 includes cross-entropy loss and vulnerability loss, and the formula is as follows:

[0018]

[0019] where with as the base, For the output of the last layer of the network, each value in the vector is a classification probability value of a different category, is the number of categories; is the incoming label, indicating the real category corresponding to the sample;

[0020]

[0021] wherein is the descending order obtained by sorting the output values of the last layer of the network.

[0022] As a preferred, the pixel importance calculation method in step 3 is to calculate the gradient value of the loss function on the image respectively , and combine the gradient directions of the two as the disturbance addition direction, the formula is as follows:

[0023]

[0024] wherein.sign() is a sign function, taking the sign of , returning 1 if greater than 0, -1 if less than 0, and 0 if equal to 0;

[0025] In step 2, the top-k pixels are selected as the limited pixel points in the current round of iteration according to the pixel importance in the candidate pixel points, and the importance weight calculation formula is as follows:

[0026]

[0027] wherein is the pixel classification importance value obtained by using the class activation mapping method, is the frequency value obtained by using the gray scale image to calculate the pixel frequency information, is the gradient value obtained by calculating the gradient of the loss function.

[0028] As a preferred, the method of iterative disturbance addition in step 3 is to generate disturbance continuously by multiple iterations, without using fixed parameters as the disturbance amplitude in each iteration, but using the pixel values around each pixel to obtain the disturbance value in each step, calculating the mean and standard deviation of the pixels around each pixel, and generating noise similar to the surrounding pixels. The specific formula for adding disturbance to the image is as follows:

[0029]

[0030] wherein , are the images before and after disturbance addition respectively, indicates that the disturbance added to each pixel in the image x is limited within , Noise similar to the surrounding pixels is sought, For the gradient direction, The mask of the selected top-k pixel points, the value of the perturbation point position is 1, and the rest is 0.

[0031] Based on the same inventive concept, the present application provides a kind of based on multi-mode image's adversarial sample generation system, comprising:

[0032] Image mode conversion module, for converting original RGB image into another two modes, gray scale chart and black and white binary chart.Gray scale chart can be used for the calculation of image frequency information, and black and white binary chart is used for the edge extraction of image main part;

[0033] Perturbation limiting module, for using class activation mapping method to obtain key area of decision neural network classification for RGB image, high-frequency area is obtained for gray scale chart, and edge area is obtained for black and white two colors, the corresponding area of three mode images is intersected, to limit the number and position of each sample image pixel, and generate image mask;

[0034] Perturbation adding module, for generating perturbation in multiple iterations, top-k pixels are selected based on pixel importance in each round, and perturbation is iteratively added on limited pixel points according to loss function gradient direction until adversarial sample attack succeeds or maximum iteration number is reached.

[0035] Based on the same inventive concept, the present application provides a kind of computer system, comprising memory, processor and computer program stored in memory and can be run on processor, when the computer program is loaded to processor, the based on multi-mode image's adversarial sample generation method is realized.

[0036] Based on the same inventive concept, the present application provides a kind of computer readable storage medium, the computer readable storage medium stores computer program, when the computer program is executed by processor, the based on multi-mode image's adversarial sample generation method is realized.

[0037] Beneficial effects: The application provides a multi-mode image-based adversarial sample generation method and system. For each initial image sample, a corresponding class activation map is calculated in an RGB mode, image frequency information is calculated in a grayscale mode, and an image main body edge is obtained in a black-and-white two-color mode. The three kinds of information are combined to limit the disturbance of the image pixels. Multiple iterations are performed to continuously add disturbance to the limited pixels until the DNN is successfully attacked or the maximum number of iterations is reached. In order to make the disturbance less noticeable and improve the imperceptibility of the adversarial sample, a fixed parameter is not used as the disturbance amplitude size in each iteration, but the mean and standard deviation of the pixels around each pixel are calculated to generate noise similar to the surrounding pixels. In each iteration, the pixel importance is calculated, the top-k pixels are selected from the candidate pixel points, and the disturbance is added in the gradient direction of the loss function. The application fully utilizes the color information, frequency information and edge information of the image, improves the general loss function, reduces the number and amplitude of the disturbed pixels, and improves the authenticity of the adversarial sample. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The overall step diagram of the embodiment of the application is shown in the figure.

[0039] Figure 2 The method flowchart of a specific example of the application is shown in the figure. DETAILED DESCRIPTION

[0040] The application will be further illustrated below in conjunction with specific embodiments, which should be understood as merely illustrating the application and not limiting the scope of the application. After reading the application, those skilled in the art can make various modifications to the application, which fall within the scope defined by the claims.

[0041] As shown in the figure, the application discloses a multi-mode image-based adversarial sample generation method, which mainly includes the following steps: Figure 1

[0042] Step 1: Obtain the original image dataset, and convert the image to generate a corresponding grayscale image and a black-and-white two-color image.

[0043] Step 2: Obtain different image information from the images in different modes in step 1. According to the threshold, it is determined whether the corresponding pixels of the image need to be disturbed, i.e. for the RGB image, the class activation mapping method is used to obtain the key area of the neural network classification, for the grayscale image, the high-frequency area is obtained, and for the black-and-white two-color, the edge area is obtained. The corresponding areas obtained from the three mode images are intersected to limit the number and position of the pixels of each sample image and generate an image mask.

[0044] ​Step 3: Calculate the importance of pixels, select top-k pixels, and iteratively add perturbations according to the gradient direction of the loss function until the perturbed adversarial sample is successful or the maximum number of iterations is reached.

[0045] As shown in Figure 2 Resnet50 classification network, taking Cifar10 dataset as initial sample for adversarial sample generation, the detailed steps of the adversarial sample generation method based on multi-mode image disclosed by the embodiment of the application are illustrated, as follows:

[0046] Step 1: Obtain the original image dataset, and generate corresponding images of another two gray scale and black and white two-color modes, load the classification network model, and the specific steps are as follows:

[0047] Step 11: Load the Cifar10 dataset from torchvision.datasets, divide it into training set and test set, and take the test set as initial sample;

[0048] Step 12: Obtain the RGB image in the dataset, generate the corresponding gray scale image, and use it to calculate the frequency information of the image;

[0049] Step 13: Further obtain the black and white two-color image based on the gray scale image, which is used for edge extraction of the main part of the image;

[0050] Step 14: Load the Resnet50 network from the saved.pth model file.

[0051] Step 2: Obtain the image information of different dimensions from the images of different modes in step 1, and determine whether the image needs to be perturbed according to the threshold value, that is, limit the number and position of pixels of each sample image, generate image mask, and the specific steps are as follows:

[0052] Step 21: Use the gradient-based class activation mapping method to obtain the key area of the RGB image that determines the DNN classification, and set the threshold value to the average of all pixel weight values in this example, and the class activation mapping Figure Two value is converted to the first layer image mask mask1;

[0053] Step 22: Because the black and white two-color image obtained by converting the RGB image has only two values of 0 and 1, the pixels with different adjacent values in the image are divided into the edge of the main image and set to 1, and the other pixels are set to 0, that is, the second layer image mask mask2;

[0054] Step 23: DCT transform and quantization of the grayscale image to obtain the frequency information of the image. Since the human eye is more sensitive to changes in low-frequency information in the image, in order to make the changes of the adversarial samples not be perceived by the human eye, high-frequency information is selected, and the third layer image mask mask3 is obtained by binarization.

[0055] Step 24: Generate the final image mask Mask by integrating the image information obtained in the above steps, i.e. pixel point restriction to constrain perturbation.

[0056] Step 3: Calculate the importance of the pixels, select top-k pixels, and iteratively add perturbations according to the gradient direction of the loss function until the adversarial sample attack succeeds or the maximum number of iterations is reached. The specific steps are as follows:

[0057] Step 31: The image obtained in each iteration is output by the classification network, and the loss function value is calculated. The cross-entropy loss and vulnerability loss are calculated respectively, and the formulas are as follows:

[0058]

[0059] where is the base, is the output of the last layer of the network, and each value in the vector is the classification probability value of a different category, is the number of categories; is the input label, indicating the real category corresponding to the sample.

[0060]

[0061] where is the descending order result obtained by sorting the output values of the last layer of the network, is the weight of the difference between each pair of probability values.

[0062] The gradient values of the loss function with respect to the image are calculated respectively , , and the gradient directions of the two are combined as the perturbation addition direction, and the formula is as follows:

[0063]

[0064] where.sign() is the sign function, taking the sign, returning 1 if greater than 0, -1 if less than 0, and 0 if equal to 0.

[0065] Step 32: Calculate the importance of the pixels, select top-k (such as 10%) pixels from the candidate pixels obtained in step 2 as the restricted pixel points in the current round of iteration, and the importance weight calculation formula is as follows:

[0066] ​

[0067] wherein is the pixel classification importance value obtained by using the class activation mapping method, is the frequency value obtained by using the gray scale image to calculate the pixel frequency information, is the gradient value obtained by calculating the gradient of the loss function in step 31.

[0068] Step 33: multiple iterations are performed to continuously generate perturbations, and instead of using fixed parameters as the perturbation amplitude in each iteration, the perturbation value in each step is obtained by using the pixel values around each pixel, the mean and standard deviation of the pixels around each pixel are calculated, and noise similar to the surrounding pixels is generated. Based on the gradient direction obtained in step 31 and combined with the image mask obtained in step 2, the perturbation is added, and the specific formula for adding perturbation to the image is as follows:

[0069]

[0070] wherein , are the images before and after the perturbation is added, indicates that the perturbation added to each pixel in the image x is limited to , is the noise similar to the surrounding pixels obtained, is the gradient direction obtained in step 31, is the mask of the top-k pixel points selected in step 32, the value of the perturbation point position is 1, and the rest is 0.

[0071] In order to further illustrate the effectiveness of the method of the present application, the method of the present application is applied to the Cifar10 data set, the Resnet50 model is attacked, and the BIM, Pixle and JSMA methods are compared, and the comparison results in the norm aspect are shown in Table 1. Among them, ASR represents the attack success rate of the adversarial sample; The norm can measure the difference between two vectors, and represents the sum of the absolute values of each element in the perturbation matrix; The norm refers to the square sum of the change amount of the modified pixels of the adversarial sample relative to the original picture, and is used to measure the overall amplitude of the perturbation; The norm refers to the maximum value of the absolute value of the change amount of the modified pixels of the adversarial sample relative to the original picture. The results show that the adversarial sample generated by the adversarial sample generation method proposed in the present application can achieve 100% attack success rate, and the generated perturbation is optimal in norm, norm, norm.

[0072] Table 1 Comparison of experimental results

[0073]

[0074] The comparison results of the image quality evaluation indexes are shown in Table 2. The PSNR refers to the peak signal-to-noise ratio, which represents the ratio of the peak signal energy to the average noise energy, and the average noise energy can be represented as the mean square error of the real image and the noisy image. The SSIM refers to the structural similarity, which is an index for measuring the similarity of two images, and it comprehensively evaluates the image quality from three aspects of brightness similarity, contrast similarity and structural similarity. The MSE refers to the mean square error, which is calculated by taking the mean square error of the adversarial sample and the original picture, and can evaluate the difference between images. The larger the PSNR is, the less the distortion is, and the better the quality of the generated image is. The closer the value of the SSIM to 1 is, the more similar the images are. The smaller the value of the MSE is, the more similar the images are. The experimental results show that the adversarial sample generated by the adversarial sample generation method proposed in the application has the optimal image quality and the highest similarity with the original picture.

[0075] Table 2 Comparison of image quality of adversarial samples

[0076]

[0077] Based on the same inventive concept, the application provides an adversarial sample generation system based on multi-mode images, which comprises an image mode conversion module for converting an original RGB image into two other modes, a grayscale image and a black-and-white binary image. The grayscale image can be used for calculating image frequency information, and the black-and-white binary image can be used for edge extraction of the main part of the image. A disturbance limiting module is used to limit the disturbance of each image sample. For the RGB image, a class activation mapping method is used to obtain the key area for determining the classification of the neural network. For the grayscale image, a high-frequency area is obtained. For the black-and-white binary image, an edge area is obtained. The intersection of the corresponding areas obtained from the three mode images is taken to limit the number and position of pixels of each sample image, and an image mask is generated. A disturbance adding module is used to generate a disturbance in multiple iterations. In each round, top-k pixels are selected based on the importance of the pixels, and then the disturbance is iteratively added to the limited pixel points according to the gradient direction of the loss function until the adversarial sample after adding the disturbance attacks successfully or the maximum number of iterations is reached.

[0078] Based on the same inventive concept, the application provides a computer system, which comprises a memory, a processor and a computer program stored on the memory and executable on the processor. When the computer program is loaded into the processor, the steps of the adversarial sample generation method based on multi-mode images are implemented.

[0079] Based on the same inventive concept, the application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the multi-mode image based adversarial sample generation method.

[0080] Those skilled in the art can understand that the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer system (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application. The storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage media that can store computer programs.

Claims

1. A method for generating adversarial examples based on multimodal images, characterized in that, Includes the following steps: Step 1: Obtain the original image dataset and convert the images to generate corresponding grayscale and black-and-white images; Step 2: For RGB images, use the class activation mapping method to obtain the key regions that determine the classification of the neural network; for grayscale images, obtain the high-frequency regions; for black and white images, obtain the edge regions. Take the intersection of the corresponding regions obtained from the three modes of images to limit the number and position of pixels in each sample image and generate an image mask. Step 3: Calculate pixel importance, select the top-k pixels and iteratively add perturbations according to the gradient direction of the loss function until the adversarial example attack after adding perturbations is successful or the maximum number of iterations is reached; The method for generating the corresponding grayscale image and black-and-white image in step 1 includes the following steps: Step 11: Obtain the RGB images from the dataset and generate the corresponding grayscale images for calculating the image frequency information; Step 12: Based on the grayscale image, a black and white image is further obtained, which is used for edge extraction of the main part of the image; The method for generating the image mask in step 2 includes the following steps: Step 21: Use gradient-based class activation mapping to obtain the key regions in the RGB image that determine the DNN classification, and binarize them by thresholding to obtain the first layer image mask; Step 22: Delineate the edges of the main image based on the black and white image obtained from the RGB image conversion, which is the second layer of image mask; Step 23: Perform DCT transformation and quantization on the grayscale image to obtain the image frequency information, and obtain the third layer image mask according to the frequency level. Step 24: Combine the first-layer image mask, the second-layer image mask, and the third-layer image mask obtained in steps 21, 22, and 23 to generate the final image mask, i.e., pixel point limitation to constrain the perturbation; The loss function in step 3 includes cross-entropy loss and vulnerability loss, as shown in the following formula: ； in by As the base, This is the output of the last layer of the network, where each value in the vector is the classification probability value for a different class. It is the number of categories; The input label represents the true category of this sample. ； in It is the descending order result obtained by sorting the output values ​​of the last layer of the network; The pixel importance calculation method in step 3 is as follows: Calculate the gradient values ​​of cross-entropy loss and vulnerability loss with respect to the image, respectively. , Combining the gradient directions of both as the direction of perturbation addition, the formula is as follows: ； Where .sign() is the sign function, which takes... The sign indicates whether the value is greater than 0, returns 1, less than 0, or equal to 0; From the candidate pixels obtained in step 2, the top-k pixels are selected as the limited pixels in the current iteration based on pixel importance. The formula for calculating the importance weight is as follows: ； in It is a pixel classification importance value obtained using the class activation mapping method. The frequency value is obtained by calculating pixel frequency information using a grayscale image. It is the gradient value obtained by calculating the gradient of the loss function.

2. The adversarial example generation method based on multi-modal images according to claim 1, characterized in that, The method for iteratively adding perturbation in step 3 is as follows: Multiple iterations are performed to continuously generate perturbations. Instead of using fixed parameters as the perturbation amplitude in each iteration, the perturbation value in each step is obtained from the values ​​of the pixels surrounding each pixel. The mean and standard deviation of the pixels surrounding each pixel are calculated to generate noise similar to that of the surrounding pixels. The specific formula for adding perturbation to the image is as follows: ； in , Add images before and after the perturbation, respectively. This means limiting the perturbation added to each pixel in image x to a certain extent. Inside, To obtain noise similar to that of surrounding pixels, For the gradient direction, This is a mask for the top-k selected pixels, with the value of 1 for the perturbation point and 0 for the rest.

3. An adversarial example generation system based on multi-modal images, characterized in that, The system for implementing the adversarial example generation method based on multimodal images according to claim 1 includes: The image mode conversion module is used to convert the original RGB image into a grayscale image and a black and white binary image; The perturbation constraint module is used to obtain the key regions that determine the classification of the neural network by using the activation mapping method for RGB images, high-frequency regions for grayscale images, and edge regions for black and white images. The intersection of the corresponding regions obtained from the three modes of images is used to constrain the number and position of pixels in each sample image and generate an image mask. The perturbation addition module is used to generate perturbations in multiple iterations. In each round, the top-k pixels are selected based on pixel importance, and perturbations are iteratively added to the limited pixels according to the gradient direction of the loss function until the adversarial example attack after adding perturbations is successful or the maximum number of iterations is reached.

4. A computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements the steps of the adversarial example generation method based on multimodal images according to claim 1 or 2.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the adversarial example generation method based on multimodal images according to claim 1 or 2.

Images