Adversarial Sample Generation Method Based on Regional Subspace Random Sampling

Through the regional subspace random sampling method, class activation map is used to divide the image area and optimize the perturbation, which solves the problems of low attack success rate and perturbation density caused by perturbation limitations in the prior art, and achieves weak perturbation and high attack success rate Adversarial sample generation.

CN118644750BActive Publication Date: 2024-11-15XIAN BUSTER INFORMATION TECH CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411111424.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-15
Estimated Expiration
2044-08-14

AI Technical Summary

Technical Problem

The existing adversarial sample generation method is limited to the local image, resulting in low attack success rate and too dense local perturbations, which cannot effectively reduce unnecessary perturbations.

Method used

The regional subspace random sampling method is adopted to generate heat maps through class activation mapping, divide high-focus and low-focus areas, random sampling generates perturbations in high-focus areas, and reduces the amount of perturbations through the iterative optimization process to generate weak and effective adversarial samples. .

Benefits of technology

It is achieved to significantly reduce the amount of perturbation of adversarial samples while maintaining a high attack success rate and generate adversarial samples with good image quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118644750B_ABST
    Figure CN118644750B_ABST
Patent Text Reader

Abstract

The present invention discloses an adversarial sample generation method of regional subspace random sampling. Firstly, a heat map is used to divide an image into different focus areas, and an initial perturbation set is randomly generated on the high focus area; then, a better perturbation is selected by assigning probability weights to calculate the sampling subspace; finally, the perturbation set generated on the sampling subspace is incorporated into the initial perturbation set and iterated continuously, and a perturbation re-evolution strategy is used to further reduce the perturbation, thereby improving the image quality of the adversarial sample; the method of the present invention solves the problem that the existing adversarial sample generation method has a low attack success rate and is too dense in local perturbations because the perturbation is limited to the local part of the image and is determined by the specific algorithm operation; sampling is performed by calculating the subspace of the important perturbation distribution, effectively changing the target class confidence while avoiding redundant perturbations, so as to generate adversarial samples with better image quality.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a method for generating adversarial samples by random sampling of regional subspaces. Background Art

[0002] With the development of artificial intelligence, deep learning technology has been widely used in many fields such as target detection, autonomous driving, and image classification. Related studies have shown that some deep neural networks are easily affected by carefully designed perturbations. For input data with specific perturbations added, the target model may give completely different results from the actual prediction results, thus causing serious security issues. In order to discover potential security vulnerabilities in the model, researchers will use various methods to generate adversarial samples to test the model. For the problems exposed during the test, targeted defense strategies are adopted to enhance the security of the model. Therefore, studying effective adversarial sample generation methods has important theoretical significance and practical value.

[0003] Among adversarial sample generation methods, generating adversarial samples through black-box attacks is more common in real-world scenarios. Many black-box attack methods only focus on how to reduce the amount of access to the target model to improve attack efficiency, while ignoring the problems caused by excessive perturbations, such as color distortion, excessive noise, and missing target features in adversarial samples. To address these issues, some black-box attack methods currently focus on reducing perturbations, and generate adversarial samples from three perspectives: modifying a small number of pixels, using gradient information, and perturbing key areas: (1) The method of modifying a small number of pixels only perturbs individual important pixels, but does not limit the value range of pixels. It uses the differential evolution algorithm to find high-quality solutions and generate adversarial samples with high image quality. Su et al. (Su JW, Vargas DV, Sakurai K. One pixel attack for fooling deep neural networks [J]. IEEE Transactions on Evolutionary Computation, 2019, 23 (5): 828-841.) used the differential evolution algorithm to propose a one pixel attack, which can generate adversarial samples with very little perturbation; Wang et al. (Wang J, Yin ZX, Tang J, et al. PICA: A Pixel Correlation-based Attentional Black-box Adversarial Attack [J]. arXiv preprint arXiv: 2101.07538, 2021.) used the image attention mechanism and pixel correlation, only taking one pixel out of every two adjacent pixels in the salient area as the target, and using a general multi-objective evolutionary algorithm to traverse the reduced pixels and generate perturbations that are imperceptible to human vision, which has a good attack effect. The adversarial samples generated by this type of method have a very low perturbation amount, but the attack success rate is low, and the adversarial samples may have obvious color difference noise. (2) Methods that utilize gradient information require estimating the gradient of the target model and then using a strategy similar to a white-box attack to generate adversarial samples.Huang Lifeng et al. (Huang Lifeng, Zhuang Wenzi, Liao Yongxian, et al. A black-box adversarial attack algorithm based on evolutionary strategy and attention mechanism [J]. Journal of Software, 2021, 32(11): 3512-3529.) limit the perturbation to the image feature area and propose a covariance matrix adaptive evolution strategy to generate weak perturbations; Chen et al. (Chen JY, Su MM, Shen SJ, et al. POBA-GA: Perturbation optimized black-box adversarial attacks via genetic algorithm [J]. Computers & Security, 2019, 85(3): 89-106.) use genetic algorithms to optimize perturbations and design a special fitness function to balance the attack capability and perturbation control. The generated adversarial sample has good attack performance. In terms of attack success rate and perturbation amount, this type of method performs better than the method that pursues efficiency, but because the estimated gradient information cannot completely replace the white-box condition and the perturbation often exists in the entire area of ​​the image, the perturbation can still be further reduced. (3) The method of perturbing the key area considers the degree of attention paid by the neural network to the background area and the target area, and only performs algorithm operations on the local area of ​​the image, thereby reducing redundant perturbations. Diego et al. (Diego G, Francesco M, Luisa V, et al. Perceptual quality-preserving black-box attack against deep learning image classifiers[J]. Pattern Recognition Letters, 2021, 147(8):142-149.) took classification confidence and good image quality as optimization objectives and imposed perturbations on image edges and key pixels; Chen et al. (Chen JY, Zheng HB, Xiong H, et al. Fine Fool: A novel DNN object contour attack on image recognition based on the attention perturbation adversarial technique[J]. Computers & Security, 2021, 104:102220.) proposed a new adversarial sample generation method based on the attention mechanism. By simultaneously considering channel space attention and pixel space attention, the perturbation attack is limited to the local area of ​​the model's attention, which can generate adversarial samples with good attack effects.Compared with methods based on global perturbations, this type of method can indeed generate adversarial samples with higher visual quality, but the perturbations are limited to the local area of ​​the image and are determined by the specific algorithm operations, which may cause problems such as low attack success rate and too dense local perturbations. Summary of the invention

[0004] The purpose of the present invention is to provide an adversarial sample generation method by random sampling of regional subspaces, which solves the problems in existing adversarial sample generation methods that the perturbation is limited to the local part of the image and is determined by the specific algorithm operation, resulting in a low attack success rate and too dense local perturbations.

[0005] The technical solution adopted by the present invention is a method for generating adversarial samples by random sampling of regional subspaces, and the specific implementation steps are as follows:

[0006] Step 1: Obtain a heat map from the images in the dataset through the class activation mapping method, segment the heat map according to the threshold and perform region mapping to generate a high-attention sampling space. and low-attention sampling space ;

[0007] Step 2: Sampling space based on high attention Initialize the perturbation set with inner elements E The disturbance value of the corresponding coordinate position in Random sampling is performed within the perturbation set, and the sampled perturbation set is input into the CNN model to calculate the fitness value, and the fitness values ​​are sorted to obtain the initial perturbation set. E ;

[0008] Step 3: Initial perturbation set E Iterate, generate subspace sampling results in each iteration, and merge the subspace sampling results into the perturbation set as new perturbations, add the perturbation corresponding to the maximum fitness value in the perturbation set to the original image, and generate adversarial samples that deceive the target model.

[0009] The present invention is also characterized in that:

[0010] In step 1, the dataset uses any one of MNIST, CIFAR-10, and ImageNet.

[0011] The specific process of step 1 is as follows: the images in the dataset are used to obtain heat maps through the Class Activation Map (CAM) method, the heat maps are segmented and region mapped, and a label array containing 0 and 1 and the same dimension as the image is generated. , 0 represents the value of the low-attention area, 1 represents the value of the high-attention area, and the label array The index of each element in corresponds to a sampling position, and the perturbation range at the sampling position is , label array The index set of all elements in constitutes the sampling space , where the set of indexes corresponding to elements with a value of 1 is the high-attention sampling space , the set of indexes corresponding to elements with a value of 0 is the low-attention sampling space ;

[0012] The sampling space is defined as:

[0013] (1)

[0014] The high-attention sampling space is defined as:

[0015] (2)

[0016] The low-attention sampling space is defined as:

[0017] (3)

[0018] in, w , h and ch are the width, height and number of channels of the image respectively, Represents the index position in the sampling space.

[0019] The specific process of step 2 is:

[0020] Step 2.1: Create a perturbation set , The dimension is The perturbation vector of represents a single perturbation; is the last sample value in the sampling space; the value of each perturbation vector coordinate is 0, The elements in represent the perturbation vector The coordinate position initialized in, the perturbation vector The values ​​at the coordinate positions initialized in will be The inside is obtained by random sampling;

[0021] Step 2.2: Change the perturbation vector Add images to the dataset , get the image in the dataset with added perturbation , calculate the fitness value of the disturbance :

[0022] (5)

[0023] in, Represents the images in the dataset Images in the dataset with added perturbations between distance, is much greater than A positive number, express The degree of change in the confidence of the target category is defined as follows:

[0024] (6)

[0025] in, , in untargeted attacks, express The correct category of express Target category;

[0026] Sort the fitness values ​​of the perturbation elements from large to small. For each perturbation vector , we can get the adversarial sample at this time :

[0027] (7)

[0028] in, lower and Uppper Represents the value range of image pixels in the digital space, It means that the adversarial samples are normalized within this value range.

[0029] Vector in step 2.1 The coordinate position expression formula initialized in is:

[0030] (4)

[0031] in, i Indicates the element number in the perturbation set. If the vector is in high dimension, it is taken, and if it is in low dimension, it is set to 0. Indicated in Random value in .

[0032] The specific process of step 3 is as follows:

[0033] Step 3.1: Sort the initial perturbation set from large to small according to the fitness value, and select the part with a value greater than the fitness value as the better perturbation set. , the part less than the fitness value is taken as the remaining perturbation set ; Include perturbation vector, Include perturbation vector; for For each disturbance in, the corresponding probability weight is calculated according to the fitness value , according to the probability weight Size selects a better perturbation ;

[0034] Step 3.2: According to the rate of change from Randomly select a number of coordinates and their corresponding values, which constitute the coordinate set , the coordinate set size is , the sampling subspace is ; The relevant definitions are as follows:

[0035] (8)

[0036] (9)

[0037] in, represents a hash function that transforms the coordinates in the coordinate set; Each element within is an array of length 2. Indicates the lower limit, the initial value is , Indicates the upper limit, the initial value is ; For any coordinate in the coordinate set , There will be a corresponding ,Will and Each disturbance within On coordinates Compare the values ​​at and count the values ​​greater than and less than The number of perturbations is denoted by and ,if , will Decrease, otherwise Increase; after compression calculation, the sampling subspace is obtained ;

[0038] Step 3.3: After calculating the sampling subspace After that, define , take the sampling results for , in the sampling subspace Random sampling is performed on The values ​​on some coordinates are realized The update formula is:

[0039] (10)

[0040] in, represents random sampling;

[0041] Step 3.4: Construct subspace perturbation set , get the updated subspace sampling result After that, calculate The fitness value of is incorporated into the subspace perturbation set as a new perturbation ; When the subspace perturbation set The number of perturbations contained is greater than When , the merged subspace perturbation set and the perturbation set , sort the fitness values ​​from large to small, and filter out the top perturbations to form a new perturbation set, and add 1 to the number of iterations. If the iteration termination condition is not met, return to step 2. If the iteration termination condition is met, the perturbation corresponding to the maximum fitness value in the current perturbation set is taken as the optimal perturbation and added to the original image to generate an adversarial sample.

[0042] The beneficial effects of the present invention are:

[0043] The adversarial sample generation method of regional subspace random sampling of the present invention uses the class activation mapping method to generate a heat map, and divides it into different regions to obtain the coordinates of the high-attention area in the digital space to form a sampling space; when generating the next generation of perturbations, the sampling subspace is calculated by optimizing the coordinates one by one, and the dimension of the sampling subspace is controlled by the rate of change; after generating a perturbation that can successfully deceive the target model, the perturbation optimization process is used as the second stage of evolution to effectively reduce unnecessary perturbations. The adversarial samples generated by the method of the present invention have weak perturbations and strong adversarial properties. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of the adversarial sample generation method of regional subspace random sampling of the present invention;

[0045] Figure 2 Generate a sampling space index map in the method of the present invention;

[0046] Figure 3 Generate a result graph for the subspace sampling results in the method of the present invention;

[0047] Figure 4 A diagram of the two-stage search process for adversarial samples in the method of the present invention;

[0048] Figure 5 is the average unit perturbation graph of each method of the MNIST dataset in the embodiment of the present invention;

[0049] Figure 6 is the average unit perturbation graph of each method of the CIFAR-10 dataset in the embodiment of the present invention;

[0050] Figure 7 is the average unit perturbation graph of each method of the ImageNet dataset in the embodiment of the present invention;

[0051] Figure 8 This is a disturbance distribution diagram corresponding to each method when the attack fails on the MNIST dataset in an embodiment of the present invention;

[0052] Fig. 9 This is a unit perturbation distribution diagram of adversarial samples that failed to attack the CIFAR-10 dataset in an embodiment of the present invention;

[0053] Fig.10 Adversarial samples and local area perturbation maps generated by the method of the present invention;

[0054] Fig.11 Adversarial samples and corresponding perturbation graphs generated by various methods in the embodiments of the present invention. DETAILED DESCRIPTION

[0055] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] The adversarial sample generation method of the regional subspace random sampling of the present invention is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0057] Step 1: Obtain a heat map from the images in the dataset through the class activation mapping method, segment the heat map according to a certain threshold and perform region mapping to generate a high-attention sampling space. and low-attention sampling space ; The dataset uses any one of MNIST, CIFAR-10, and ImageNet.

[0058] The method of the present invention uses a proxy model based on VGG16 to obtain the heat map of the input image, which is used to approximately estimate the degree of attention paid by the target model to each area of ​​the image. After obtaining the heat map of the target image, the heat map is segmented according to a certain threshold, and then region mapping is performed to generate a label array with the same dimension as the image and containing 0 and 1. , 0 represents the value of the low-attention area, and 1 represents the value of the high-attention area, such as Figure 2 The index of each element in the marker array corresponds to a sampling position, and the perturbation range at the sampling position is The set of indices of all elements in the label array constitutes the sampling space , where the set of indexes corresponding to elements with a value of 1 is the high-attention sampling space , the set of indexes corresponding to elements with a value of 0 is the low-attention sampling space , to divide the image into different areas. According to different sampling spaces, disturbances can be directly generated in the specified area of ​​the image. The method of the present invention selects As the disturbance area, to interfere with the key features of the image. Formula (1), Formula (2) and Formula (3) are the definitions of sampling space, high attention sampling space and low attention sampling space respectively. Among them, w , h and ch are the width, height and number of channels of the image respectively.

[0059] (1)

[0060] (2)

[0061] (3)

[0062] in, Represents the index position in the sampling space.

[0063] Step 2: Sampling space based on high attention Initialize the perturbation set with inner elements E The disturbance value of the corresponding coordinate position in Random sampling is performed within the perturbation set, and the sampled perturbation set is input into the CNN model to calculate the fitness value, and the fitness values ​​are sorted to obtain the initial perturbation set. E ; The specific process is:

[0064] Step 2.1: Create a perturbation set , The dimension is The perturbation vector of represents a single perturbation; is the last sample value in the sampling space. The value of each perturbation vector coordinate is 0. The elements in represent the perturbation vector The coordinate position initialized in, the perturbation vector The values ​​at the coordinate positions initialized in will be The inside is obtained by random sampling;

[0065] vector The coordinate position expression formula initialized in is:

[0066] (4)

[0067] in, i Indicates the element number in the perturbation set. If the vector is in high dimension, it is taken, and if it is in low dimension, it is set to 0. Indicated in Random value in .

[0068] Step 2.2: Change the perturbation vector Add images to the dataset , get the image in the dataset with added perturbation , calculate the fitness value of the disturbance :

[0069] (5)

[0070] in, Represents the images in the dataset Images in the dataset with added perturbations between distance, is much greater than A positive number, express The degree of change in the confidence of the target category is defined as follows:

[0071] (6)

[0072] in, , in untargeted attacks, express The correct category of express Target category;

[0073] Sort the fitness values ​​of the perturbation elements from large to small. For each perturbation vector , we can get the adversarial sample at this time :

[0074] (7)

[0075] in, lower and Uppper Represents the value range of image pixels in the digital space, It means that the adversarial samples are normalized within this value range.

[0076] Adversarial samples in the method of the present invention The calculation of has two stages. The first stage obtains deceptive adversarial samples, and the second stage reduces the perturbation amount of the adversarial samples. Figure 4 The fitness function has two stages. When the adversarial sample cannot deceive the model, The impact on the fitness value is almost negligible, that is, the adversarial sample moves from area 1 to area 2; when the adversarial sample successfully deceives the model, The value of is 0, It becomes the only factor that affects the fitness value, that is, the adversarial sample moves in a better direction in region 2.

[0077] Step 3: Initial perturbation set E Iterate, generate subspace sampling results in each iteration, and merge the subspace sampling results into the perturbation set as new perturbations, add the perturbation corresponding to the maximum fitness value in the perturbation set to the original image, and generate adversarial samples that deceive the target model. The specific process is:

[0078] Step 3.1: Figure 3 As shown, the initial perturbation set is sorted from large to small according to the fitness value, and the part with a value greater than the fitness value is selected as the better perturbation set. , the part less than the fitness value is taken as the remaining perturbation set ; Include perturbation vector, Include perturbation vector; for For each disturbance in, the corresponding probability weight is calculated according to the fitness value , according to the probability weight Size selects a better perturbation ;

[0079] Step 3.2: According to the rate of change from Randomly select a number of coordinates and their corresponding values, which constitute the coordinate set , the coordinate set size is , the sampling subspace is ; The relevant definitions are as follows:

[0080] (8)

[0081] (9)

[0082] in, Represents a hash function that transforms coordinates in a coordinate set. Each element within is an array of length 2. Indicates the lower limit, the initial value is , Indicates the upper limit, the initial value is ; For any coordinate in the coordinate set , There will be a corresponding ,Will and Each disturbance within On coordinates Compare the values ​​at and count the values ​​greater than and less than The number of perturbations is denoted by and ,if , then Decrease, otherwise Increase; after compression calculation, the sampling subspace is obtained ;

[0083] Step 3.3: After calculating the sampling subspace After that, define , take the sampling results for , in the sampling subspace Random sampling is performed on The values ​​on some coordinates are realized The update formula is:

[0084] (10)

[0085] in, Represents random sampling.

[0086] Step 3.4: Construct subspace perturbation set , get the updated subspace sampling result After that, calculate The fitness value of is incorporated into the subspace perturbation set as a new perturbation ; When the subspace perturbation set The number of perturbations contained is greater than When , the merged subspace perturbation set and the perturbation set , sort the fitness values ​​from large to small, and filter out the top perturbations to form a new perturbation set, and add 1 to the number of iterations. If the iteration termination condition is not met, return to step 2. If the iteration termination condition is met, the perturbation corresponding to the maximum fitness value in the current perturbation set is taken as the optimal perturbation and added to the original image to generate an adversarial sample.

[0087] Example 1

[0088] The code running environment is set as follows: Intel Core i9-10900K 3.7 GHz CPU, NVIDIA GeForceRTX 3090 GPU, 64 GB RAM, Ubuntu18.04, Python3.6, Tensorflow-gpu 1.15. For this embodiment, three indicators are used to evaluate the performance of the method of the present invention, and the evaluation of the method of the present invention focuses on the average unit perturbation and the attack success rate. The present invention uses MNIST, CIFAR-10 and ImageNet data sets and related recognition models to test the feasibility of the attack method and to evaluate some current work. The experimental results of the method of the present invention are compared with the experimental results of AutoZOOM, POBA-GA, ES-Attack and PQP.

[0089] The present invention sets the disturbance interval In order to prevent the difference between the adversarial sample and the original sample from being too large, a smaller initial value is used. At the same time, in order to ensure the success rate of the attack, when the initial value cannot meet the attack requirements, a step-by-step method is used to define Formula (11) is Defined on the MNIST dataset, formula (12) is Definition on CIFAR-10 and ImageNet datasets:

[0090] (11)

[0091] (12)

[0092] in, yes The initial value of is the number of visits to the target model at present. The parameters of the method of the present invention on each data set are shown in Table 1. is the size of the optimal sampling set, is the size of the remaining sample set, It is the rate of change when calculating the sampling subspace. On low-dimensional datasets such as MNIST and CIFAR-10, it is easier to generate adversarial samples, but the perturbation distribution is relatively simple, so a larger ; For the high-dimensional dataset ImageNet, it is more difficult to generate adversarial samples, and the perturbation distribution is more complicated. The value of should be appropriately reduced. Therefore, in this embodiment, and The values ​​of are set to 5 and 5 respectively, and on ImageNet and The values ​​of are 3 and 7 respectively. and rate of change It is an important factor affecting the performance of the attack method. Before the formal experiment, they were studied for parameter sensitivity and the more appropriate values ​​were determined. In the non-targeted attack experiments of MNIST and CIFAR-10, this embodiment uses the first 1000 correctly classified pictures, and for targeted attacks, the first 200 correctly classified pictures are used, and the maximum number of model accesses is 10,000. On the ImageNet dataset, this embodiment only conducts non-targeted attack experiments, and uses the first 200 correctly classified pictures, and the maximum number of model accesses is 20,000.

[0093] Table 1

[0094]

[0095] Parameter sensitivity analysis:

[0096] It can be seen from the adversarial sample generation method of the present invention that the performance of the method is affected by many factors, such as the sampling space , and , disturbance interval and rate of change Before conducting a quantitative study on the sensitivity of the parameters, this example conducts a brief qualitative analysis: obtained by the proxy model, but the characteristics of the heat map determine the The difference is small, so the different The impact on the results is small; and Determined by the data set, high-dimensional data sets often use smaller and larger ; and Positive correlation, It directly determines the range of the sampling subspace and the subspace sampling results, which has a significant impact on the attack success rate, attack efficiency and disturbance amount; the change rate It determines the similarity between the disturbance generated in a single iteration and the better disturbance, and has a more obvious impact on the disturbance amount and attack efficiency. and rate of change A sensitivity study was conducted, focusing on the performance of the method of the present invention in terms of average unit perturbation and attack success rate. In this example, 100 correctly classified images were randomly selected from three standard data sets for multiple experiments, and the maximum number of accesses to the model by the method of the present invention was limited to 10,000 times. Finally, relatively appropriate parameter values ​​were determined based on the sensitivity analysis results.

[0097] for The sensitivity study experimental results are shown in Table 2, Table 3 and Table 4, where Table 2 is the sensitivity study experimental results on the MNIST dataset. Table 3 shows the impact of attack methods on the performance of the CIFAR-10 dataset. Table 4 shows the impact of attack methods on the performance of ImageNet dataset. Impact on the performance of attack methods. On the one hand, with As the number of attacks increases, the attack success rate of the proposed method on the MNIST and ImageNet datasets also increases, especially When the value of is in a small range, the attack success rate increases. increased and rose sharply, and When the value of is in a larger range, the increase in attack success rate slows down. This is because there are a small number of data with relatively high attack difficulty. The impact on the output of the target model is not obvious. For the CIFAR-10 dataset, The impact on the attack success rate of the method of the present invention is small. On the other hand, on each data set, The larger the value, the more obvious the disturbance generated. Unlike the attack success rate, the average unit disturbance is For the attack method proposed in this embodiment, the disturbance amount should be controlled at the lowest possible level while maintaining a certain attack success rate. The lower limit should be taken within the optimal range. Considering the two important indicators of attack success rate and disturbance amount, it can be seen from the data in the table that The optimal value intervals on each data set are [0.30, 0.35], [0.04, 0.06], and [0.06, 0.08], respectively. The values ​​are 0.30, 0.04 and 0.06 respectively.

[0098] Table 2

[0099]

[0100] Table 3

[0101]

[0102] Table 4

[0103]

[0104] For the rate of change The sensitivity study experiments are shown in Table 5, Table 6 and Table 7, where Table 5 is the sensitivity study experiments on the MNIST dataset. Table 6 shows the impact of attack methods on the performance of CIFAR-10 dataset. Table 7 shows the impact of attack methods on the performance of ImageNet dataset. Impact on the performance of the attack method. Based on the conclusions in the previous article, this example uses MNIST, CIFAR-10, and ImageNet datasets to The values ​​of are set to 0.30, 0.04, and 0.06 respectively.

[0105] Table 5

[0106]

[0107] Table 6

[0108]

[0109] Table 7

[0110]

[0111] From Table 5, Table 6, and Table 7, we can see that the attack success rate is is not sensitive to changes in The average number of visits to the model only shows a trend of This is because: the better disturbance With each iteration generating a new perturbation The similarity between , and any two The minimum similarity is .when When smaller, and There is little difference, and each The difference between them is also small, then the redundant disturbance generated by a single iteration is relatively small; on the contrary, when When is larger, more redundant perturbations are generated in a single iteration. The value of should also be as small as possible while ensuring the success rate of the attack.

[0112] From the above experimental data, we can see that the rate of change on the MNIST dataset When it is between [0.15, 0.20], the attack method has better performance. In this embodiment, the lower limit is 0.15. Similarly, on CIFAR-10 and ImageNet The value of should be 0.10 and 0.10. The optimal value of is close, and only the average unit perturbation has an effect on This is because It only determines the pixel ratio used in a single iteration. For datasets of different dimensions, only the dimension is used to represent the image. That is to say, relative to the attack method itself, the rate of change The correlation with the dataset is weak. At the same time, whether it is Table 4 or Table 7, it can be seen that the overall attack success rate on ImageNet is low. This is because 10,000 times are used as the maximum number of model accesses. In order to improve this, the maximum number of model accesses is set to 20,000 times in the subsequent experiments.

[0113] Example 2

[0114] On the basis of Example 1, Figure 5 , Figure 6 and Figure 7 As shown, it is the comparison result of the average unit perturbation of adversarial samples between the method of the present invention and other methods on different data sets. For the MNIST and CIFAR-10 data sets, this embodiment respectively conducts untargeted attack and targeted attack experiments, while on ImageNet, this embodiment only conducts untargeted attack experiments.

[0115] On the MNIST dataset, the adversarial samples generated by the proposed method in the non-targeted attack mode have the lowest average unit perturbation, which is only 2.59×10 -3 . In this indicator, the method of the present invention is 42.6% lower than AutoZOOM with the highest perturbation amount and 9.4% lower than ES-Attack with the lowest perturbation amount compared with the other four attack methods. Although ES-Attack has the lowest average unit perturbation in the target attack mode, the method of the present invention is second only to ES-Attack and the values ​​of the two are very close. At the same time, according to the experimental results on the CIFAR-10 dataset, it can be seen that the average unit perturbation of the adversarial samples of the method of the present invention in both attack modes is the lowest. Under non-target attack, the average unit perturbation of the method of the present invention is reduced by 27.0% relative to AutoZOOM and 7.7% relative to PQP; under targeted attack, for this indicator, the method of the present invention is reduced by 31.7% relative to AutoZOOM and 7.0% relative to PQP. It can be observed that, whether it is MNIST or CIFAR-10, the perturbation values ​​of each attack method are relatively close. Even the method of the present invention, which performs best, has a perturbation amount that is more than half of the worst AutoZOOM under the same conditions. However, the situation on the ImageNet dataset is different. The average unit perturbation of the proposed method is only 9.80×10 -5 , compared with the other four methods, it is 29.3% of AutoZOOM with the highest perturbation amount and 61.6% of ES-Attack with the lowest perturbation amount.

[0116] From the above experimental results, it can be seen that the performance of the method of the present invention in terms of the amount of perturbation is related to the characteristics of the data set. First, for the targeted attack on the MNIST data set, the performance of the method of the present invention is inconsistent with the experimental conditions under other conditions, mainly because of the characteristics of handwritten digital images. For any handwritten digital image, its feature area corresponds to relatively uniform white pixels, and the non-feature area is a uniform black pixel. Under this distribution law, the method of the present invention only perturbs the area of ​​interest corresponding to the true category of the image, which can easily cause the target model to output an erroneous recognition result, thereby achieving a non-targeted attack. However, for targeted attacks, only a weak perturbation in a local area may not be enough to make it recognized as a specified category. In this case, increasing the perturbation of the feature area or adding obvious perturbations to the non-feature area can ensure the success rate of the attack. Therefore, in this case, the adversarial sample generated by the method of the present invention does not have the lowest amount of perturbation, and is closer to the performance of other methods.

[0117] Secondly, on the ImageNet dataset, the difference in perturbation values ​​between the method of the present invention and other methods is significantly increased, which is different from the results on MNIST and CIFAR-10. This is precisely because the strategy of the method of the present invention only perturbing the local image has greater advantages on high-dimensional datasets, which can avoid generating too many useless perturbations. AutoZOOM and POBA-GA are both based on perturbations in the global range of the image. Although the random gradient estimation strategy and the fitness function that suppresses perturbations they adopt can generate adversarial samples with good visual quality, it is difficult to avoid redundant perturbations as the data dimension increases. Although PQP and ES-Attack also apply perturbations to partial areas of the image, the two-stage search of the algorithm of the method of the present invention reduces the generated local perturbations again, and the space where perturbations can be compressed is larger on high-dimensional datasets, so the advantages of the method of the present invention are more obvious.

[0118] Example 3

[0119] Based on Example 2, Tables 8, 9 and 10 are comprehensive performance comparisons of the method of the present invention and other methods, wherein Table 8 is a comparison of 5 methods on the MNIST dataset, Table 9 is a comparison of 5 methods on the CIFAR-10 dataset, and Table 10 is a comparison of 5 methods on the ImageNet dataset (without targeted attack).

[0120] Table 8

[0121]

[0122] Table 9

[0123]

[0124] Table 10

[0125]

[0126] It can be seen from the comprehensive comparison table that the method of the present invention has lower disturbance and considerable attack success rate. The method of the present invention performs best in the average unit disturbance indicator. Among the five different comparison groups, the disturbance amount of the method of the present invention in four groups is the lowest. In terms of attack success rate, the method of the present invention performs well: it is second only to AutoZOOM and POBA-GA on the MNIST dataset; it is the best on the CIFAR-10 dataset; and it is second only to AutoZOOM on ImageNet. Among the comparison methods, ES-Attack is closest to the method of the present invention in terms of average unit disturbance, but its attack success rate is not as good as other methods; AutoZOOM and POBA-GA perform well in attack success rate, but the disturbance generated is large; PQP also has relatively few disturbances, but its performance in various aspects is not outstanding.

[0127] From the overall perspective of adversarial samples, the average unit perturbation reflects the average size of the perturbation amount, and embodies the performance of the attack method on a batch of samples. Specifically for a single adversarial sample, regardless of whether it can deceive the target model, the perturbation should be kept as small as possible to illustrate that the attack method does have stable and reliable performance in terms of the amount of perturbation. Therefore, this embodiment counts the number of adversarial samples generated by each method within different unit perturbation ranges to demonstrate the stability of the attack method in terms of the amount of perturbation. During the experiment, the present invention found that the majority of adversarial samples were successfully attacked, and their unit perturbation was close to the average unit perturbation of all adversarial samples; for a small number of adversarial samples that failed to attack, although the impact on the average unit perturbation indicator was not significant, their unit perturbation was significantly different from the average unit perturbation of all adversarial samples. Therefore, this embodiment only counts the adversarial samples that failed to attack, and analyzes the perturbation distribution corresponding to each method when the attack failed, such as Figure 8 , Fig. 9 shown.

[0128] exist Figure 8 , Fig. 9 In , each point on the horizontal axis represents a different unit perturbation interval, and the vertical axis represents the number of adversarial samples with unit perturbations in different intervals. Figure 8 It can be seen that when the proposed method and ES-Attack perform untargeted attacks on the MNIST dataset, the unit perturbations of most adversarial samples that fail to attack are in the range of [1.0×10 -3 , 2.0×10 -3 ], which is lower than their respective average unit perturbations, and the attack method has good stability. Under this condition, the unit perturbations of most adversarial samples corresponding to PQP are in the range of [2.5×10 -3 , 4.0×10-3 ], which is closer to the average unit perturbation, and the stability of the attack method is good. For adversarial samples that failed to attack, AutoZOOM and POBA-GA generated excessive perturbations and were not very stable. Although POBA-GA is second only to the method of the present invention in terms of average unit perturbation, it has the worst stability compared to other attack methods. Fig. 9 In the experiment, each method performs untargeted attack on the CIFAR-10 dataset. For most of the adversarial samples generated by the proposed method that fail the attack, the unit perturbation is between [5.0×10 -4 , 8.0×10 -4 ], the attack method has good stability. For ES-Attack, although there are some adversarial samples whose unit perturbations fail in the attack, the unit perturbations of more adversarial samples are in the range of [1.25×10 -3 , 1.70×10 -3 ], the attack method has poor stability. It can also be seen that the adversarial samples generated by PQP, POBA-GA and AutoZOOM that failed to attack are distributed in a lower unit perturbation range, but these methods have a higher probability of generating excessive perturbations when the attack fails. On this data set, the method of the present invention has the best stability compared with other methods, although the difference in average unit perturbation is not large.

[0129] From the indicator data, we can see that the method of the present invention performs well in terms of disturbance amount and attack success rate. In particular, the value of the average unit disturbance indicator is basically lower than that of other comparison methods. In order to reflect the performance of the method of the present invention from the perspective of visual effects, this example lists some adversarial samples generated by the method of the present invention and displays the corresponding local area disturbance map for comparison with the original image. Fig.10 shown.

[0130] exist Fig.10 In the perturbation graph, the difference between the original image and the adversarial sample is almost imperceptible. It can be seen from the perturbation graph that the perturbation is only distributed in the target feature area in the image. Moreover, perturbations of the same size have obvious visual differences when they exist in different areas of the image. The area of ​​high attention of the neural network to the image is exactly the area where the target features are located, which contains key points, edges and texture information. The perturbations distributed here are not visually obvious; in the background area, even weak perturbations may appear to be more prominent. Many black-box attack methods focus on efficiency and ignore the visual quality of adversarial samples. Even some black-box attack methods generate adversarial samples from the perspective of reducing perturbations, but most of them are based on perturbations in the global range. Compared with the method of the present invention, which perturbs in local feature areas, these attack methods are usually easier to implement, and the image quality of adversarial samples still has room for improvement.

[0131] Fig.11 It is a comparison diagram of adversarial samples generated by the method of the present invention and other methods. Although the difference between the adversarial samples seems small, the perturbation diagram can reflect the corresponding actual perturbation size. It can be seen that the method of the present invention can indeed generate the least perturbation among all the methods. Although ES-Attack and PQP only generate perturbations in local areas of the image, the perturbations appear to be more dense. For AutoZOOM and POBA-GA, the sparse perturbations are distributed within the overall range of the adversarial samples. Although they do not look obvious in the target area of ​​the image, the image quality of the background area is reduced in terms of visual effect.

[0132] In the above manner, the adversarial sample generation method of regional subspace random sampling of the present invention, the proposed regional subspace random sampling, is a black box attack method that can generate weak perturbations and has a high attack success rate. This attack method takes into account the sensitivity of the model recognition results to perturbations in different regions of the image, introduces a class activation mapping method, generates perturbations only in some key areas, and performs sampling by calculating the subspace of the distribution of important perturbations, effectively changing the target class confidence while avoiding redundant perturbations to generate adversarial samples with better image quality. The experimental results on MNIST, CIFAR-10 and ImageNet show that the adversarial samples generated by the method proposed in the present invention have significantly lower perturbations and considerable attack success rates compared to other black box attack methods.

Claims

1. The adversarial sample generation method based on regional subspace random sampling is characterized by: Follow the steps below to implement it: Step 1: Obtain a heat map from the images in the dataset through the class activation mapping method, segment the heat map according to the threshold and perform region mapping to generate a high-attention sampling space. and low-attention sampling space ; Step 2: Sampling space based on high attention Initialize the perturbation set with inner elements E The disturbance value of the corresponding coordinate position in Random sampling is performed within the perturbation set, and the sampled perturbation set is input into the CNN model to calculate the fitness value, and the fitness values ​​are sorted to obtain the initial perturbation set. E ; Step 3: Initial perturbation set E Iterate, generate subspace sampling results in each iteration, and merge the subspace sampling results into the perturbation set as new perturbations, add the perturbation corresponding to the maximum fitness value in the perturbation set to the original image, and generate adversarial samples that deceive the target model; The specific process of step 2 is as follows: Step 2.1: Create a perturbation set , The dimension is The perturbation vector of represents a single perturbation; is the last sample value in the sampling space; the value of each perturbation vector coordinate is 0, The elements in represent the perturbation vector The coordinate position initialized in, the perturbation vector The values ​​at the coordinate positions initialized in will be The inside is obtained by random sampling; Step 2.2: Change the perturbation vector Add images to the dataset , get the image in the dataset with added perturbations , calculate the fitness value of the disturbance : (5) in, Represents the images in the dataset Images in the dataset with added perturbations between distance, is the confidence weight, the purpose is to reduce Impact on fitness value; express The degree of change in the confidence of the target category is defined as follows: (6) in, , in untargeted attacks, express The correct category of express Target category; Sort the fitness values ​​of the perturbation elements from large to small. For each perturbation vector , we can get the adversarial sample at this time : (7) in, lower and Uppper Represents the value range of image pixels in the digital space, Indicates that the adversarial sample is normalized within this value range; The specific process of step 3 is as follows: Step 3.1: Sort the initial perturbation set from large to small according to the fitness value, and select the part with a value greater than the fitness value as the better perturbation set. , the part less than the fitness value is taken as the remaining perturbation set ; Include perturbation vector, Include perturbation vector; for For each disturbance in, the corresponding probability weight is calculated according to the fitness value , according to the probability weight Size selects a better perturbation ; Step 3.2: According to the rate of change from Randomly select a number of coordinates and their corresponding values, which constitute the coordinate set , the coordinate set size is , the sampling subspace is ; The relevant definitions are as follows: (8) (9) in, represents a hash function that transforms the coordinates in the coordinate set; Each element within is an array of length 2. Indicates the lower limit, the initial value is , Indicates the upper limit, the initial value is ; For any coordinate in the coordinate set , There will be a corresponding ,Will and Each disturbance within On coordinates Compare the values ​​at and count the values ​​greater than and less than The number of perturbations is denoted by and ,if , will Decrease, otherwise Increase; after compression calculation, the sampling subspace is obtained ; Step 3.3: After calculating the sampling subspace After that, define , take the sampling results for , in the sampling subspace Random sampling is performed on The values ​​on some coordinates are realized The update formula is: (10) in, represents random sampling; Step 3.4: Construct subspace perturbation set , get the updated subspace sampling result After that, calculate The fitness value of is incorporated into the subspace perturbation set as a new perturbation ; When the subspace perturbation set The number of perturbations contained is greater than When , the merged subspace perturbation set and the perturbation set , sort the fitness values ​​from large to small, and filter out the top perturbations to form a new perturbation set, and add 1 to the number of iterations. If the iteration termination condition is not met, return to step 2. If the iteration termination condition is met, the perturbation corresponding to the maximum fitness value in the current perturbation set is taken as the optimal perturbation and added to the original image to generate an adversarial sample.

2. The method for generating adversarial samples by random sampling of regional subspaces according to claim 1, characterized in that: The dataset described in step 1 uses any one of MNIST, CIFAR-10, and ImageNet.

3. The method for generating adversarial samples by random sampling of regional subspaces according to claim 1, characterized in that: The specific process of step 1 is: obtain a heat map from the images in the dataset through the class activation mapping method, segment the heat map and perform region mapping, and generate a labeled array with the same dimension as the image and containing 0 and 1 , 0 represents the value of the low-attention area, 1 represents the value of the high-attention area, the label array The index of each element in corresponds to a sampling position, and the perturbation range at the sampling position is , label array The index set of all elements in constitutes the sampling space , where the set of indexes corresponding to elements with a value of 1 is the high-attention sampling space , the set of indexes corresponding to elements with a value of 0 is the low-attention sampling space ; The sampling space is defined as: (1) The high-interest sampling space is defined as: (2) The low-attention sampling space is defined as: (3) in, w , h and ch are the width, height and number of channels of the image respectively, Represents the index position in the sampling space.

4. The method for generating adversarial samples by random sampling of regional subspaces according to claim 1, characterized in that: The vector described in step 2.1 The coordinate position expression formula initialized in is: (4) in, i Indicates the element number in the perturbation set. If the vector is in high dimension, it is taken, and if it is in low dimension, it is set to 0. Indicated in Random value in .

Citation Information

Patent Citations

  • Adversarial defense method based on class activation mapping

    CN111754519A

  • Confrontation sample generation method based on belief attack and salient region disturbance limitation

    CN114399630A

  • General adversarial disturbance generation method based on correlation class activation mapping

    CN114676852A

  • Confrontation sample generation method and system based on sparse disturbance group

    CN116503692A

  • Adaptive meta-attack system and method for target tracker under automatic driving condition

    CN118196742A