Black-box targeted adversarial sample generation method for image salient regions
By thresholding the image and fusion of local label feature maps, and optimizing the codec to generate adversarial samples with the adversarial transformation module, the problem of insufficient migration ability of black box targeted adversarial samples in deep learning models is solved, and the perturbation range is reduced At the same time, it improves the attack success rate and robustness of the samples.
Patent Information
- Application Number
- CN202411226146.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-09-03
AI Technical Summary
The prior art is difficult to improve the migration ability of black box targeted adversarial samples while reducing the disturbance range. Especially in deep learning models, the attack effect is relatively poor and the disturbance is likely to lead to the loss of input image information.
By thresholding the image, the heat map of the significant area is retained, the encoder is constructed using local tag feature maps, the mixed feature map is extracted, and the thresholded heat map is fused in the decoder to limit the perturbation in the significant area, combined with the adversarial transformation module Perform random affine transformation and optimize the codec to generate adversarial samples.
The migration and robustness of the adversarial samples were improved. The experimental results showed that the attack success rate increased by 5.4% to 7.5%, the disturbance area was reduced by 30% to 18%, and the disturbance amplitude was reduced by 18% to 11%. At the same time, ensure the effectiveness of the adversarial sample.
Smart Images

Figure CN118762251B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of methods for generating adversarial samples of artificial intelligence systems, and relates to a method for generating black-box targeted adversarial samples for salient areas of images. Background Art
[0002] In recent years, deep learning has made significant progress in the field of computer vision and has been widely used in tasks such as image classification, object detection, and semantic segmentation. Although deep learning has shown excellent performance in various tasks, studies have shown that it has security issues and is vulnerable to adversarial samples. Adversarial samples are carefully designed images that are generated by adding tiny and imperceptible non-random perturbations to the input image, which will cause the deep learning model to produce incorrect prediction results, posing a serious threat to the application of deep learning models. For example, in the field of autonomous driving, if carefully designed perturbations are artificially added to pedestrians or road signs, the autonomous driving system will misidentify or even fail to recognize the target category, resulting in incorrect system operation and causing serious traffic accidents. Therefore, in order to explore the potential vulnerabilities of deep learning models, it is necessary to use adversarial samples to evaluate the model, take corresponding defense methods for the vulnerabilities exposed by the model, and further improve the robustness and security of the model.
[0003] In the study of adversarial attacks, the attack methods can be divided into two types: white-box attacks and black-box attacks, according to the attacker's understanding of the internal structure of the target model. White-box attacks require the attacker to fully understand the model structure and parameters to generate adversarial samples, but their limitation is that the attacker must have a comprehensive understanding of the model. If the model structure or parameters change, the attacker needs to regenerate adversarial samples that are adapted to it, which increases the difficulty and complexity of the attack. Black-box attacks refer to attacks in which the attacker cannot directly access the internal structure or parameters of the target model and can only generate adversarial samples through limited input and output, resulting in high difficulty and relatively poor attack effects. In order to overcome this challenge, white-box proxy models can be used to generate adversarial samples, and the generated adversarial samples can be applied to unknown black-box models to create cross-model black-box attacks. This attack is called transfer attack. The key to transfer attacks lies in the transferability of adversarial samples. This feature enables adversarial samples to be transferred across models and even across tasks. It has received widespread attention and has become an important direction for adversarial sample research in recent years.
[0004] According to the different attack targets of adversarial samples, transfer-based attack methods can be divided into two categories: untargeted transfer attacks and targeted transfer attacks. The goal of untargeted attacks is to reduce the accuracy of the target model, while targeted attacks expect the generated adversarial samples to guide the model to incorrectly predict it as a specific category. Compared with untargeted attacks, targeted attacks are inherently more challenging. They not only need to deceive the deep learning model to mislead the input image to the gradient direction of the target category, but also ensure that its aggressiveness can be transferred to different black-box models. Therefore, in order to improve the transferability of targeted attacks, many studies have considered it from different angles, including advanced gradient calculation, input transformation, feature information learning, integrated model attacks, etc. Among them, input transformation (such as random resizing and padding, conversion, scaling, etc.) is an extremely effective method. By applying different transformations to the input image, multiple input modes are constructed to improve the transferability of adversarial samples, but input transformation may cause the loss of important information of the input image, thereby reducing the effectiveness of adversarial samples. On the other hand, since the hidden layers of different deep neural networks have similar activation patterns, the feature information obtained from the hidden layers of the classification network has a certain degree of transferability, which can be used to improve the transferability of adversarial samples. However, it is difficult for feature information alone to contain enough target class information to trigger misclassification of the target model, and the generated global perturbation lacks specificity. Summary of the invention
[0005] The purpose of the present invention is to provide a black-box targeted adversarial sample generation method for image salient areas, which can improve the migration ability of adversarial samples while reducing the disturbance range.
[0006] The technical solution adopted by the present invention is a method for generating black-box targeted adversarial samples for salient areas of an image, which is specifically implemented in the following steps:
[0007] Step 1: Threshold the heat map, retain the significant areas in the heat map that have a greater impact on the classification results, and obtain the thresholded heat map; then, based on the thresholded heat map, locally map the label information to the significant areas of the image, and use the generated local label feature map to construct an encoder to extract the mixed feature map;
[0008] Step 2: Fuse the thresholded heatmap in the decoder so that the perturbation generated by the codec does not exceed the salient areas of the image;
[0009] Step 3: Construct an adversarial transformation module and introduce it between the codec and the classification network. Perform a random affine transformation on the adversarial samples generated by the codec to destroy the perturbations in the image and promote the model to generate new adversarial samples that can resist such transformations.
[0010] The present invention is also characterized in that:
[0011] Step 1 is as follows:
[0012] Step 1.1, obtain the heat map of the target image by using the gradient weighted class activation mapping method, perform thresholding on the heat map, retain the significant areas in the heat map, and obtain the thresholded heat map;
[0013] Step 1.2: Use the mapping network to map the target label information into a label feature map, and then fuse the label feature map with the thresholded heat map obtained in step 1.1 to generate a local label feature map of the target category;
[0014] Step 1.3: Use the local label feature map generated in step 1.2 to build an encoder and extract a mixed feature map containing target label information and image features.
[0015] The thresholding process in step 1.1 is defined as:
[0016]
[0017] in, For heatmap, Indicates the specific pixel position in the image. is the heat map after thresholding, T Set the threshold for the thermal threshold of the key area filter condition. T In the range of 0.5-0.9;
[0018] The pixels in the heat map that exceed the threshold retain their original values, while the pixels below the threshold are set to 0, thereby filtering the heat map.
[0019] In step 1.2, the label feature map is converted into Heatmap with thresholding Fusion, get the local label feature map :
[0020]
[0021] in, , , Respectively represent the number of channels, height and width of the input image, is the heat map after thresholding.
[0022] In step 1.3, the encoder first calculates the input image Feature map , and then the fused local label feature map With image feature map Concatenate them to get a mixed feature map , which contains the original image information and target label information; the above and Indicates the number of channels of the input image, , Respectively represent the height and width of the input image;
[0023] The feature extraction process of the encoder is expressed as:
[0024]
[0025] Among them, Conv is the convolution operation, BN is batch normalization, is the activation function, K is the convolution kernel, k represents the convolution kernel size, s is the step length, p To fill in the parameters, i Indicates the coding layer number, when i =1, F 0 represents the feature map of the input image; when i =2,3, F i-1 Represents the output of the previous convolution layer;
[0026] The encoder is represented as:
[0027]
[0028] After completing feature extraction, the encoder feeds its output into the subsequent network.
[0029] Step 2 is as follows:
[0030] The feature map extracted by the encoder is sent to the residual block, and then the residual features obtained by residual learning are decoded to generate perturbation features; the thresholded heat map is fused to limit the perturbation to the salient areas of the image; finally, the perturbation limited by smooth projection is added to the image to obtain the adversarial sample.
[0031] Step 2 is implemented as follows:
[0032] The feature map extracted by the encoder F It is fed into the residual block and expressed as:
[0033]
[0034] Among them, ResBlock represents the residual block;
[0035] The residual features obtained by residual learning are decoded to generate perturbation features. At this time, the codec is updated as follows:
[0036]
[0037] in, UpSample(·) indicates the decoding process, represents the input image, is the local label feature map, The process of extracting features for the encoder;
[0038] Fusion thresholded heatmap After that, the output of the decoder is expressed as:
[0039]
[0040] Use smooth projection Output feature map of the decoder Perform variable restrictions, recorded as:
[0041]
[0042] in, is the disturbance intensity, δ For disturbance, is the hyperbolic tangent function;
[0043] The disturbance will be handled δ Add to Image In the output, we get the adversarial sample ; Thus, through the codec , completing the input image The latent vector with the target label Convert to output image ; The whole process is expressed as:
[0044] .
[0045] In step 3, use the adversarial transformation module The adversarial samples generated by the encoder and decoder are processed as follows:
[0046]
[0047] in, For adversarial samples, To counter the transformation module, δ for disturbance;
[0048] Random affine transformations include random flipping (-15°, 15°), random translation (0.2, 0.2), random cropping (0.8, 1.2), and random scaling (-10°, 10°).
[0049] Step 3 also includes the optimization of the parameters in the codec. After multiple rounds of optimization learning, the final adversarial sample is obtained. The specific optimization process is as follows:
[0050] The transformed adversarial sample is input into the classifier to obtain the classification result of the adversarial sample, and the cross entropy loss between the classification result and the target category is calculated:
[0051]
[0052] in, i For multiple target categories T The serial number in represents the cross entropy loss, represents a classification network, For adversarial examples;
[0053] Then, by continuously reducing the cross entropy loss, the adversarial samples are constantly moving closer to the target category, and the parameters in the codec are gradually optimized; the optimization during the training process is shown in the following formula:
[0054]
[0055] in, represents the input image, Represents the parameters in the generator, Represents the parameter for finding the minimum cross entropy loss value between the classification result and the target category , represents the adversarial transformation module, Represents smooth projection Restrictions on the perturbations generated by the encoder and decoder. The target category in the above formula is a set of multiple categories that can be set to attack at the same time. T .
[0056] The beneficial effects of the present invention are:
[0057] (1) The method of the present invention constructs an encoder based on local mapping, fuses the thresholded heat map with the label feature map, and locally maps different encoding layers in the encoder to ensure that the encoder accurately extracts the target category features in the salient area of the image;
[0058] (2) The method of the present invention integrates the threshold heat map to optimize the decoder, and uses the threshold heat map to limit the output layer of the decoder to ensure that the perturbation is generated in the salient area of the image;
[0059] (3) The method of the present invention designs an adversarial transformation module between the codec and the classification network, performs random affine transformation on the generated adversarial samples, and diversifies the generated adversarial samples to improve their mobility and robustness;
[0060] (4) In order to evaluate the performance of the proposed method, targeted black-box attacks were performed using Inception-v3 and Resnet-152 as source models. The experimental results show that on the ImageNet dataset, the average attack success rates of the proposed method on 6 different black-box models for 8 target categories are 5.4% and 7.5% higher than those of mainstream methods, respectively; on the MSCOCO dataset, the average attack success rates are increased by 8% and 7%, respectively. In addition, the perturbation area of the generated adversarial samples is reduced by 30% and 18%, respectively, and the overall amplitude of the perturbation is reduced by 18% and 11%, respectively. The proposed method can effectively improve the transferability of adversarial samples while reducing the perturbation area. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] Figure 1 The overall structure diagram of the condition generation model of the present invention;
[0062] Figure 2 is a structural diagram of a codec in the method of the present invention;
[0063] Figure 3 The loss change of the Inception-v3 classification network during the training process of the embodiment of the present invention;
[0064] Figure 4 The loss change of the ResNet-152 classification network during the training process of the embodiment of the present invention;
[0065] Figure 5 The change in the success rate of targeted attacks in different methods of migrating from the source classification network ResNet-152 to the target classification network Inception-v3 in the embodiments of the present invention;
[0066] Figure 6 The change of the target attack success rate in different methods in the embodiments of the present invention from the source classification network ResNet-152 to the target classification network GoogleNet;
[0067] Figure 7 The disturbance effects of different target categories in the embodiment of the present invention;
[0068] Figure 8 Comparison of disturbance effects of different methods in the embodiments of the present invention;
[0069] Fig. 9 It is an image superimposed with the heat map in an embodiment of the present invention;
[0070] Fig.10 Comparison of adversarial samples generated by models under different training sets in the embodiments of the present invention;
[0071] Fig.11This is a comparison of the generalization of disturbances in the embodiments of the present invention. DETAILED DESCRIPTION
[0072] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0073] The method of the present invention uses the designed conditional generation model to learn and generate adversarial samples of the target category. The overall structure of the conditional generation model is as follows: Figure 1 As shown, by the perturbation codec , Adversarial Transformation Module With classifier Generating adversarial samples through the encoder-decoder structure helps to effectively learn the distribution of important features of images in low-dimensional latent space, while achieving local adjustments to the input data. The adversarial transformation module performs random affine transformations on the generated adversarial samples, which promotes the model to further generate more robust adversarial samples by destroying the generated perturbations. The classifier is used to calculate the cross entropy loss between the adversarial sample and the target category, and back-propagates to update the model parameters by continuously reducing the loss.
[0074] The problem is described as: A picture from the training set The input image is is the true category of the image, is the set of target categories a specific category in The purpose of the entire model is to: On top, train the codec Learning to generate a sense of satisfaction Perturbation of the norm , thereby generating adversarial samples , so that it can be classified by the network Identify as target category ,Right now In the codec and classifier, the method of the present invention introduces a countermeasure transformation module , perform random affine processing on adversarial samples , diversifying adversarial samples to improve their robustness and transferability. The entire model finally generates the target adversarial sample by solving equation (1):
[0075] (1)
[0076] By calculating formula (1), we can obtain a generative model for a specific category in the dataset by minimizing the loss of that category. In this process, we only need to optimize the parameters , in order to optimize the codec, and the optimized codec can generate adversarial samples for the target category on any test set.
[0077] The present invention provides a method for generating black-box targeted adversarial samples for salient areas of an image, which is specifically implemented according to the following steps:
[0078] Step 1: Threshold the heat map, retain the significant areas in the heat map that have a greater impact on the classification results, and obtain the thresholded heat map; then, based on the thresholded heat map, locally map the label information to the significant areas of the image, and use the generated local label feature map to build an encoder to extract the mixed feature map. Specifically:
[0079] Step 1.1: Obtain the heat map of the target image by using the Gradient Weighted Class Activation Mapping (Grad-CAM) method on the images in the dataset, perform thresholding on the heat map, retain the significant areas in the heat map, and obtain the thresholded heat map.
[0080] For any input image, we can use the Grad-CAM method to visualize the CNN-based classification network and obtain a heat map of the target image:
[0081] (2)
[0082] It is worth noting that the heat map obtained at this time is the same size as the last layer of convolutional feature map. The heat map is upsampled to the resolution of the input image through bilinear interpolation to obtain the heat map ,in, , , Respectively represent the number of channels, height and width of the input image.
[0083] The obtained heat map is thresholded to retain the significant areas in the heat map. The thresholding process is defined as:
[0084] (3)
[0085] in, For heatmap, Indicates the specific pixel position in the image. is the heat map after thresholding, T The thermal threshold for the key area screening condition. The method of the present invention sets the threshold T In the range of 0.5-0.9, the pixels in the heat map that exceed the threshold retain their original values, while those below the threshold are set to 0, thereby filtering the heat map.
[0086] Step 1.2: Use the mapping network to map the target label information into a label feature map, and then fuse the label feature map with the thresholded heat map obtained in step 1.1 to generate a local label feature map of the target category.
[0087] The mapping network is used to map the target label information into a label feature map, and all the information of the entire image is composed of the category label of the class. Specifically, given a target category One-hot encoding of , first generate the target category through a multi-layer perceptron The latent vector The target latent vector By randomly sampling different categories , retaining valid target label information, whose vector length is M , corresponding to the number of channels in multiple dimensions (the number of channels in the method of the present invention is as follows Figure 2 Second, the potential vector is expanded to obtain a label feature map containing global label information. .
[0088] Label feature map Heatmap with thresholding After fusion, the final local label feature map is obtained:
[0089] (4)
[0090] The local label feature map will directionally add the target label information contained in the label feature map to the salient area segmented by the thresholded heat map. According to the different sizes of the encoding layer, it will be regenerated and superimposed with the input image features extracted by the encoder. Through continuous iteration, it will eventually be optimized into a perturbation in the salient area.
[0091] Step 1.3: Use the local label feature map generated in step 1.2 to build an encoder and extract a mixed feature map containing target label information and image features.
[0092] The encoder first calculates the input image Feature map , and then the fused local label feature map With image feature map Concatenate them to get a mixed feature map , which contains the original image information and the target label information. The height of the mixed feature map ( H ) and width ( W ) will change with the size of each layer of feature map in the encoder. The purpose is to embed label information into image features at different levels so that the classification network can still extract feature information of the target category when extracting deep features of the image. The feature extraction process of the encoder can be expressed as:
[0093] (5)
[0094] Among them, Conv is the convolution operation, BN is batch normalization, is the activation function, K is the convolution kernel, k represents the convolution kernel size, s is the step length, p To fill in the parameters, i represents the coding layer number (detailed parameters are shown in Table 1). It is worth noting that when i =1, F 0 represents the feature map of the input image; when i =2,3, F i-1 Represents the output of the previous convolution layer. The encoder can be expressed as:
[0095] (6)
[0096] After feature extraction, the encoder outputs F Send to the subsequent network.
[0097] Step 2: The thresholded heat map is fused in the decoder so that the perturbation generated by the codec does not exceed the salient area of the image, thereby improving the perturbation targeting.
[0098] The feature map extracted by the encoder is first fed into the residual block to learn the feature representation of the data. This process is expressed as:
[0099] (7)
[0100] ResBlock represents the residual block. The residual features obtained through residual learning are decoded to generate perturbation features. At this time, the codec can be updated as:
[0101] (8)
[0102] in, UpSample (·) indicates the decoding process. The decoder corresponds to the encoder and uses three upsampling layers to restore the image features. Each sampling layer includes deconvolution, normalization and activation processing. The first two activation layers in the upsampling use the ReLU activation function, and the last layer uses the Tanh activation function. The specific parameters of the codec are shown in Table 1:
[0103] Table 1 Network structure parameters of codec
[0104]
[0105] When ResNet152 is used as the source model, the corresponding codec detailed structure is as follows Figure 2 As shown. The numbers represent the height of each layer feature map (H ),Width( W ) and the number of channels ( M ). The image generated by the encoder-decoder can be added as a perturbation to the image. However, during the training process, the underlying channels (input image features) may contain information related to the global features of the image, which will cause the generated perturbation to spread outside the salient area. The number of channels is set to 3 to match the number of perturbation channels and to limit the range of generated perturbations. The output of the decoder can be expressed as:
[0106] (9)
[0107] For the decoder output feature map , the present invention adopts smooth projection right To restrict the variables, we can write it as:
[0108] (10)
[0109] in, is the disturbance intensity (set in the present invention =16 / 255), δ For disturbance, is the hyperbolic tangent function. , δ Can automatically satisfy The perturbation boundary of , the disturbance from Linear scaling to [0,255] does not exceed the valid range of image pixels, ensuring that the generated adversarial samples are visually similar to the original images. δ Add to Image In the output, we get the adversarial sample . Thus, through the codec , we can transform the input image The latent vector with the target label Convert to output image . This process can be expressed as:
[0110] (11)
[0111] From this, the encoder-decoder can generate a series of images that can be attacked as the target category through the test set.
[0112] Step 3: Construct an adversarial transformation module and introduce it between the codec and the classification network. Perform a random affine transformation on the adversarial samples generated by the codec to destroy the perturbations in the image and promote the model to generate adversarial samples that can resist such transformations, so as to improve the transferability and robustness of adversarial samples.
[0113] The method of the present invention constructs an adversarial transformation module between the codec and the classification network. After the codec completes the generation of adversarial samples, it does not directly input them into the classification network for classification, but first passes through the constructed adversarial transformation module. Final processing:
[0114] (12)
[0115] in, For adversarial samples.
[0116] In transformation In terms of the selection of , the method of the present invention adopts some image processing methods commonly used in input transformation, such as rotation, translation, scaling, and shearing. However, excessive transformation will cause the image to lose too many original features. Therefore, the method of the present invention adds the following to each transformation: Figure 1 The restrictions shown in the example include random flipping (-15°, 15°), random translation (0.2, 0.2), random cropping (0.8, 1.2), and random scaling (-10°, 10°). Through these restricted transformations, the diversity of perturbations is increased while destroying them, the generation performance of the encoder and decoder is enhanced, and new adversarial samples that can resist these transformations are generated. Finally, the transformed adversarial samples are input into the classifier to obtain the classification results of the adversarial samples, and the cross entropy loss between the classification results and the target categories is calculated:
[0117] (13)
[0118] in, i For multiple target categories T The serial number in represents the cross entropy loss, Represents the classification network. By continuously reducing the cross entropy loss between the classification result and the target category, the adversarial sample is continuously moved closer to the target category, and the parameters in the encoder and decoder are gradually optimized. The optimization during the training process is shown in formula (14):
[0119] (14)
[0120] In formula (14), represents the input image, Represents the parameters in the generator, Represents the parameter for finding the minimum cross entropy loss value between the classification result and the target category , represents the adversarial transformation module, Represents smooth projection Restrictions on the perturbations generated by the encoder-decoder, where the difference from Eq. (1) is that the target class is not a single class , but you can set the attack to a collection of multiple categories at the same time T .
[0121] Embodiment 1:
[0122] (1) Datasets and performance indicators
[0123] Dataset: This embodiment considers the widely used ImageNet ILSVRC2012 dataset and MS-COCO target detection dataset as the training sets of the model. The ImageNet ILSVRC2012 dataset contains more than 1 million high-resolution images covering 1,000 categories; the MS-COCO dataset contains images in various scenes, covering 80 different categories of targets. These two datasets can provide rich and diverse training samples, which can help the model learn more universal and accurate feature representations, and can better adapt to the data distribution of different fields and scenes, and improve the generalization ability of perturbations. In order to evaluate the performance of the model in a standard and comprehensive manner, this embodiment uses the Image Net-Neur IPS (1k) test set to test the transferability of adversarial samples, and also uses the MS-COCO test set to verify the generalization ability of perturbations.
[0124] Performance indicators: In order to better evaluate the effectiveness of the attack, this embodiment uses the targeted attack success rate as the main evaluation criterion and uses l 0 and l The 2 norm is used to measure the range and amplitude of disturbance respectively. The success rate of targeted attack is defined as p adv = N adv_target / N .in, p adv is the attack success rate, is the total number of samples in the test set, N adv_target is the number of adversarial examples in the test set that are identified as target categories by the model. l The 0-norm represents the number of non-zero elements in the matrix, and in adversarial samples, it represents the number of perturbed pixels in the image. l The range of generated disturbances is evaluated by the ratio of the number of non-zero elements obtained by the 0 norm to the total pixels of the image. lThe 2-norm refers to the square root of the sum of the squares of the elements in the perturbation matrix, which is used to measure the overall amplitude of the perturbation.
[0125] (2) Model training
[0126] This example uses a pre-trained classification network widely used on the ImageNet ILSVRC2012 dataset. Inception-v3 (Inv3) and ResNet-152 (RN152) are used as the source classification networks (white-box models) of the experiment, and Inception-v4 (Inv4), GoogleNet (GN), DenseNet-121 (DN), VGG-16_bn (VGG16) and other models are used as target classification networks (black-box models) to verify the transferability of adversarial samples.
[0127] In the selection of comparative experiments, this embodiment selected a variety of baseline methods with targeted attacks such as MIM (MI-FGSM), SI, TI, DIM (DI-FGSM) and their combined methods such as SI-MIM, TI-MIM, DI-MIM, etc., and compared them with the mainstream methods ODI, CFM, C-GSP, and CD-AP in recent years. The same benchmark experiment TI-MI-VT (combined by TI, MIM, and VT methods) was used for simulation in the ODI and CFM methods.
[0128] During the training process, each method maintained consistency in parameter setting and image preprocessing. In order to facilitate image input, both white box models first scaled the image and then performed center cropping. The final input image and output size of the Inception-v3 model were 299×299, and the input image and output image size of the ResNet-152 model were 224×224. In terms of model parameter setting: the initial learning rate was 2.5e-5, the batch size was 8, and the Adam optimizer was used to optimize the model parameters during training. The perturbation intensity was set to 16 / 255. In terms of the number of iterations, in order to achieve the best effect, MIM and DIM were iterated 30 and 50 times respectively, ODI and CFM were iterated 300 times, and the training rounds of C-GSP, CD-AP and the method of the present invention were all set to 20 times. The loss changes during training are as follows Figure 3 and Figure 4As shown in the figure, Inception-v3 and ResNet-152 are used as source classification networks respectively. By observing the trend of loss changes, it can be found that in the early stage of training (the first 5 rounds), the loss changes rapidly and shows a faster decline rate because the model is easily affected by the local features in the training data. As the training progresses, the model gradually learns the global features and patterns of the data set. In the middle of the training (5-15 rounds), the rate of loss decline slows down and the parameter update step decreases. In the late stage of training (15-20 rounds), the model gradually approaches the optimal solution, the parameter update step further decreases, and the loss decreases slowly until it converges, indicating that there is no overfitting in the training process.
[0129] (3) Ablation experiment
[0130] In order to understand the role of the key steps in the method of the present invention, a targeted attack ablation experiment was conducted on the ImageNet dataset. In the experiment, local mapping is a method of generating adversarial samples only through local mapping, without performing adversarial transformation on the adversarial samples; local mapping + adversarial transformation is to add local mapping and adversarial transformation at the same time.
[0131] The ablation experiment results of this embodiment are shown in Table 2, where: represents white-box attack, and the rest are black-box attacks. It can be seen that although the perturbation limitation method and adversarial transformation module of the present invention reduce the attack success rate under the white-box to a certain extent, they effectively improve the attack success rate of the perturbation migration to the black-box. Specifically, the addition of thresholded heat maps only adds perturbations to the important areas of the classification network, which is more targeted. Compared with the mainstream method, the average attack success rate on different black-box models is increased by 2.96% and 3.12% respectively; after adding the adversarial transformation module, the diversity of adversarial samples is increased, and the aggressiveness of the perturbation is further improved on the basis of focusing on the important areas of the image. Therefore, the success rate of the black-box attack has increased again, increasing by 2.51% and 5.1%.
[0132] Table 2 Ablation experiment
[0133]
[0134] During the training process, the attack success rate of different black box models changes as follows: Figure 5 and Figure 6 As shown in the figure, the changes in attack success rate from the source model ResNet-152 to the target models Inception-v3 and GoogleNet are shown. The overall trend is Figure 3 , Figure 4The training process is similar to that shown in . The attack success rate of the black-box model also changes from fast to slow, and finally converges. Compared with other mainstream methods, the attack success rate of the method that only adds local mapping is 3% higher on average, which shows that adding perturbations to the key focus area of the classification network in a targeted manner is effective on various models. In the classification process, this area has a high weight for correct classification. When the target category information is added to the image features in this area, the classification network is more likely to interpret these perturbed image features as features related to the target category, thereby affecting its decision-making process during classification and leading to the final classification error. The method that adds local mapping and adversarial transformation achieves the best attack effect. This is because on the basis of adding perturbations to the key areas, the migration and robustness of the perturbations are further enhanced by adversarial transformation. In the training process, the adversarial transformation destroys the perturbation structure in the generated adversarial samples, resulting in a decrease in the cross entropy loss between the image and the target category. However, the purpose of training is to reduce the difference between the two. Through the game between destruction and optimization, the perturbation generation ability of the model is continuously enhanced, and finally the migration and robustness of the adversarial samples are improved.
[0135] Embodiment 2:
[0136] On the basis of Example 1, carry out:
[0137] (4) Transferability evaluation on different datasets
[0138] In this embodiment, the attack targets are set to attack 8 different categories simultaneously. Figure 7 The perturbation effects of four categories are listed, namely, Rock Snake, Sea Lion, Barometer and Swing. The attack methods MIM, DIM, ODI and CFM attack the eight categories respectively, and calculate their average attack success rates on different models. In contrast, C-GSP and the method of the present invention first train a model that can attack eight categories simultaneously on the training set, then use the trained model to generate adversarial samples on the test set, and attack on different models to obtain the attack success rate of each model.
[0139] The transferability comparison results of various methods on the ImageNet dataset are shown in Table 3. The first column on the left is different white-box proxy models, the second column is different attack methods, and columns 3-8 are the attack success rates on different models. represents white-box attack, and the rest are black-box attacks. As can be seen from Table 3, the baseline methods MIM and DIM show excellent attack effects in white-box attack, with attack success rates above 95%, but the adversarial sample transferability is poor.
[0140] Table 3 Comparison of transferability with targeted attacks on the ImageNet NeurIPS test set (1k images)
[0141]
[0142] By comparing the data in Table 3, it can be observed that for the ODI and CFM methods, the competition between multiple object angles and noise is considered on top of the baseline method, which ensures the success rate of white-box attacks while effectively improving the transferability to other models. ODI improved by an average of 17.2% over the baseline, and the CFM method improved by another 9% over the ODI method. However, it should be noted that these methods can only be trained to attack one target at the same time. In contrast, both the method of the present invention and C-GSP can attack multiple targets. Further comparison found that compared with the C-GSP method, the black-box attack success rate of the method of the present invention increased by an average of 7%, which indicates that adding local perturbations to salient areas has better attack effects than untargeted global perturbations. The comparison of specific perturbation effects is as follows: Figure 8 As shown in the figure, compared with the global perturbation, the perturbation in the adversarial sample of the present invention is only added to the salient area of the image, and the corresponding heat map of the image is as follows: Fig. 9 shown.
[0143] The comparison results on the MS-COCO dataset are shown in Table 4, where: Represents a white-box attack, and the rest are black-box attacks. Since the images in this dataset have higher resolution and more obvious image features, the attack success rates of various methods are higher than those of the ImageNet dataset. From the data comparison, it can be seen that the method of the present invention still has a higher attack success rate than other methods, and can still be improved by 4%~5% compared to the best CFM, which further verifies the effectiveness of the method of the present invention. It should be noted that the MS-COCO dataset focuses on target detection, and an image may contain more targets. In the experiments on the MS-COCO dataset, the experimental settings were not changed, only the training set was changed, which means that there is no label information corresponding to the image, and only the top-1 category obtained by the classification network is used to obtain the corresponding heat map, and perturbations are added to its salient areas. The adversarial samples generated under two different training sets are shown in the figure. Fig.10 As shown in the figure, it can be seen that even if the training set is changed, relatively similar perturbations can still be generated in the end. Therefore, the experimental results on the MS-COCO dataset indirectly prove the unsupervised learning of the method of the present invention. No original image labels or categories are required. Only the target attack category needs to be set, and the attack on the target category can be achieved by training the generated model.
[0144] Table 4 Comparison of transferability with targeted attack on the MS-COCO test set
[0145]
[0146] Embodiment 3:
[0147] On the basis of Example 2, carry out:
[0148] (5) Perturbation generalization evaluation
[0149] This embodiment also evaluates the generalization of the perturbation. Specifically, first, the generative model is trained on the ImageNet training set to generate perturbations for the ImageNet data distribution; second, the attack success rate of the model-generated perturbations is verified on the ImageNet test set to prove the effectiveness of the perturbation; finally, the test set is replaced, and adversarial samples are generated on the MS-COCO test set using the generative model for the ImageNet data distribution, and the generalization of the perturbation is proved by the attack success rate.
[0150] The experimental results are shown in Table 5. The first column on the left is training on different white-box models, the second column is different datasets, the third column is different methods, the fourth to tenth columns are the attack success rates on different models, and the eleventh column is the average attack success rate. Indicates that under the same data set, the results Represents a white-box attack.
[0151] Table 5. Generalization comparison of perturbations trained on the ImageNet dataset on the MS-COCO test set
[0152]
[0153] The experimental results in Table 5 are the attack effects of the perturbations learned on the general dataset ImageNet after generating adversarial samples on other test sets. The experimental results in Table 5 show that whether Inception-v3 is used as the source model or ResNet-152 is used as the source model, the perturbations generated from ImageNet training have an average attack success rate of 4% in the test on the MS-COCO dataset, which shows that the generated perturbations have certain applicability and effectiveness across datasets. It is worth noting that the CFM method can only generate adversarial samples for input data based on specific inputs, so the training and testing of CFM are the same dataset. Compared with the perturbations generated only for specific datasets in the CFM method, the adversarial samples generated by the method of the present invention on cross-datasets still show better attack performance, with an average improvement of 10.32%. Compared with the C-GSP method, the attack success rate increased by 6.56% on average. This improvement in generalization is due to the fact that the perturbations are added in the salient areas of the image, which has a certain impact on its classification performance. The difference in average attack success rate is shown in Figure 2. Fig.11As shown in the figure, we can see the comparison of the attack success rate of different methods when migrating from the ImageNet training set to the MS-COCO test set. CFM performs poorly on the Inception-v3 model, with an attack success rate of less than 30%, and only 31.8% on the MS-COCO test set, but it works better on the ResNet-152 model, with attack success rates of 45.1% and 56.2% on different test sets. The C-GSP method has similar attack effects on the two models, and the generalization attack success rate on the MS-COCO dataset can be improved by 5%~6%. The method of the present invention performs best on the two models. When the Inception-v3 model is used as the source model, the generalization attack success rate on the MS-COCO dataset can be improved by 11.1%, but it only improves by 4.1% on the ResNet-152 model. ResNet-152 has a deeper network structure and advanced feature extraction capabilities. Residual connections bypass one or more network layers, allowing information to be passed directly, thereby capturing more detailed image features, which makes it more defensive against adversarial examples, resulting in lower improvement in perturbation generalization.
[0154] (6) Disturbance area and disturbance size
[0155] The comparison of the perturbation area and the perturbation size is shown in Table 6. The average perturbation area of the proposed method is 30% smaller than that of the C-GSP method on the Inception-v3 model and 16% smaller on the ResNet-152 model, which proves that the proposed method can effectively convert label information into feature space and ensure that the perturbation is concentrated in the salient area of the image rather than scattered throughout the image. This strategy effectively reduces the perturbation area, but the effect on different source models varies greatly, which is related to the data set preprocessing method of different models.
[0156] Table 6 Disturbance area and l 2-norm comparison
[0157]
[0158] In order to match the input of different models, the dataset is preprocessed. When training with Inception-v3 as the source model, the image is first enlarged from 256×256 to 300×300, and then its center is cropped to 299×299. During this process, the proportion of the target object in the image has basically not changed; when training with ResNet-152 as the source model, only the center is cropped to 224×224, so the proportion of the target in the cropped image will increase, resulting in a larger area reduction when Inception-v3 is used as the source model, and a smaller reduction when ResNet-152 is used as the source model. In addition, due to the different sizes of objects in different images, the mean proportion is also unstable, so the final perturbation area only changes in a small range and does not decrease significantly.
[0159] In the comparison of disturbance size, we use l 2 distance to calculate adversarial samples x adv With the original sample x The perturbation distance between them is calculated, and the average value of the successfully attacked samples is used for comparison. The results are shown in Table 6 above. It can be seen that under the condition of the same perturbation size of 16 / 255, the adversarial samples generated by the method of the present invention are closer to the original samples.
Claims
1. A black-box targeted adversarial sample generation method for image salient areas, characterized by: Follow the steps below to implement it: Step 1: Threshold the heat map, retain the significant areas in the heat map that have a greater impact on the classification results, and obtain the thresholded heat map; then, based on the thresholded heat map, locally map the label information to the significant areas of the image, and use the generated local label feature map to construct an encoder to extract the mixed feature map; Step 2: Fuse the thresholded heatmap in the decoder so that the perturbation generated by the codec does not exceed the salient areas of the image; Step 3: Construct an adversarial transformation module and introduce it between the codec and the classification network to perform random affine transformations on the adversarial samples generated by the codec to destroy the perturbations in the image and promote the model to generate new adversarial samples that can resist such transformations. Step 1 is as follows: Step 1.1, obtain the heat map of the target image by using the gradient weighted class activation mapping method on the image in the data set, perform thresholding on the heat map, retain the significant area in the heat map, and obtain the thresholded heat map; Step 1.2: Use the mapping network to map the target label information into a label feature map, and then fuse the label feature map with the thresholded heat map obtained in step 1.1 to generate a local label feature map of the target category; Step 1.3: Use the local label feature map generated in step 1.2 to construct an encoder and extract a mixed feature map containing target label information and image features; The thresholding process in step 1.1 is defined as: in, For heatmap, Indicates the specific pixel position in the image. is the heat map after thresholding, T Set the threshold for the thermal threshold of the key area filter condition. T In the range of 0.5-0.9; The pixels in the heat map that exceed the threshold retain their original values, while those below the threshold are set to 0, thereby filtering the heat map; In step 1.2, the label feature map is converted into Heatmap with thresholding Fusion, get the local label feature map : in, , , Respectively represent the number of channels, height and width of the input image, is the heat map after thresholding; In step 1.3, the encoder first calculates the input image Feature map , and then the fused local label feature map With image feature map Concatenate them to get a mixed feature map , which contains the original image information and target label information; the above and Indicates the number of channels of the input image, , Respectively represent the height and width of the input image; The feature extraction process of the encoder is expressed as: Among them, Conv is the convolution operation, BN is batch normalization, is the activation function, K is the convolution kernel, k represents the convolution kernel size, s is the step length, p To fill in the parameters, i Indicates the coding layer number, when i =1, F 0 represents the feature map of the input image; when i =2,3, F i-1 Represents the output of the previous convolution layer; The encoder is represented as: After completing feature extraction, the encoder feeds its output into the subsequent network; Step 2 is implemented as follows: The feature map extracted by the encoder F It is fed into the residual block and expressed as: Among them, ResBlock represents the residual block; The residual features obtained by residual learning are decoded to generate perturbation features. At this time, the codec is updated as follows: in, UpSample (·) indicates the decoding process, represents the input image, is the local label feature map, The process of extracting features for the encoder; Fusion thresholded heatmap After that, the output of the decoder is expressed as: Use smooth projection Output feature map of the decoder Perform variable restrictions, recorded as: in, is the disturbance intensity, δ For disturbance, is the hyperbolic tangent function; The disturbance will be handled δ Add to Image In the output, we get the adversarial sample ; Thus, through the codec , completing the input image The latent vector with the target label Convert to output image ; The whole process is expressed as: ; Step 3 also includes the optimization of the parameters in the codec. After multiple rounds of optimization learning, the final adversarial sample is obtained. The specific optimization process is as follows: The transformed adversarial sample is input into the classifier to obtain the classification result of the adversarial sample, and the cross entropy loss between the classification result and the target category is calculated: in, i For multiple target categories T The serial number in represents the cross entropy loss, represents a classification network, For adversarial examples; Then, by continuously reducing the cross entropy loss, the adversarial samples are constantly moving closer to the target category, and the parameters in the codec are gradually optimized; the optimization during the training process is shown in the following formula: in, represents the input image, Represents the parameters in the generator, Represents the parameter for finding the minimum cross entropy loss value between the classification result and the target category , represents the adversarial transformation module, Represents smooth projection Restrictions on the perturbations generated by the encoder and decoder. The target category in the above formula is a set of multiple categories that can be set to attack at the same time. T .
2. The method for generating black-box targeted adversarial samples for image salient regions according to claim 1, characterized in that: In step 3, use the adversarial transformation module The adversarial samples generated by the encoder and decoder are processed as follows: in, For adversarial samples, To counter the transformation module, δ for disturbance; The random affine transformation includes random flipping (-15°, 15°), random translation (0.2, 0.2), random cropping (0.8, 1.2) and random scaling (-10°, 10°).
Citation Information
Patent Citations
X-ray image focus detection method and terminal equipment
CN110176002A
Adversarial sample generation method, neural network training method, device and equipment
CN113066002A
Physical confrontation sample generation method and system
CN113674140A
Black box confrontation sample generation method and device, equipment and storage medium
CN117853846A
End-to-end method for generating imperceptible adversarial patch
CN118229954A
