Modeling method and system for generating transferable adversarial samples based on middle layer
By adding perturbations to the intermediate layers of deep neural networks and calculating feature importance, high-quality transferable adversarial examples are generated, which solves the problems of insufficient transferability and low model generalization ability in existing methods and improves the testing effect of adversarial examples.
Patent Information
- Application Number
- CN202310514286.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2025-12-19
- Estimated Expiration
- 2043-05-09
AI Technical Summary
Existing methods for generating transferable image adversarial examples fail to fully consider the internal features of neural network models in black-box scenarios, resulting in insufficient transferability and perceptuality of adversarial examples, low model generalization ability, and high error rate of generated adversarial examples on the test set.
By adding perturbations to the intermediate layers of the model and calculating the feature importance of each layer, the layers with the highest feature importance are selected for perturbation to generate initial adversarial examples. The images are then smoothed using a Gaussian filter to improve the transferability and quality of the adversarial examples.
It improves the transferability and generation efficiency of adversarial examples, and the generated adversarial examples can more effectively cause errors in the test model, thereby improving the robustness of the model and image quality.
Smart Images
Figure CN116543260B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a model intermediate layer-based transferable adversarial sample generation method, belonging to the field of artificial intelligence testing. BACKGROUND
[0002] With the rapid development of deep learning technology, the emergence of big data and the significant improvement of computing power, deep neural networks (DNN) are widely used in image classification, speech recognition and other fields. Due to the high recognition accuracy of DNN itself, it is also widely deployed in security-critical and mission-critical fields such as autonomous driving, medical diagnosis, and malware detection. In the field of image classification, the classification ability of DNN models can even rival human recognition level.
[0003] While DNNs are playing their roles in various fields, they are also facing many threats. Due to the complex structure of deep neural networks, a small perturbation added to an image may cause the deep neural network to make a wrong judgment, even if it cannot be discovered by humans, and thus output a wrong result. Such an input sample that can induce a classification model to make a wrong judgment by adding a perturbation intentionally is called an adversarial example. Taking image adversarial examples as an example, human eyes are difficult to perceive the addition of adversarial perturbations to the original input image, but they can fool the neural network model and make it make a wrong classification judgment. The existence of adversarial examples is a great threat to fields with high security requirements. For example, in the autonomous driving scenario, an attacker changes the road signs into corresponding adversarial samples, causing the autonomous driving system to make a wrong judgment about the road signs, which may lead to traffic accidents. In the image classification task, many DNN models show poor robustness when facing some maliciously modified images. In order to ensure the safety of DNN models in related applications, it is urgent to find an effective testing method to discover potential defects in DNN models and thus improve their robustness. Adversarial examples can be used as effective test cases to make the model make a wrong judgment and help discover some potential defects in the model. In addition, adversarial examples can also be used to retrain the model with a dataset containing adversarial samples to improve the robustness of the model.
[0004] Some methods for generating adversarial samples have been proposed, which can be divided into two categories according to whether the internal state of the DNN is needed: white-box adversarial sample generation method and black-box adversarial sample generation method. The white-box method needs to obtain the internal structure of the DNN network during the generation of adversarial samples, while the black-box method only needs to obtain the output of the DNN network under various inputs without considering the internal state. Since it is very difficult to obtain the internal parameter information of the DNN system, the black-box method is more practical than the white-box method in practice. In black-box adversarial sample generation, the transferability of adversarial samples is an important indicator, which enables adversarial samples to work across models and even across tasks. Transferable adversarial samples not only cause incorrect classification of specific models, but also have a certain impact on such models, and can be used not only for testing specific models, but also for testing such models. Because of the high practicability of such transfer-based adversarial sample generation methods, more and more researchers have begun to study the generation method of transferable adversarial samples.
[0005] Currently, some methods for generating transferable image adversarial samples have been proposed, among which the better ones are: (1) The MI-FGSM method based on momentum accumulates the gradient of the last iteration in the iterative process, making the optimization direction more stable; (2) The DMI based on input transformation randomly scales the input image sample, increasing the diversity of the input image sample; (3) The TMI based on gradient smoothing uses a Gaussian convolution kernel to perform Gaussian smoothing on the generated gradient, increasing the discriminant region of the model on the input image sample; (4) The VMI based on neighbor gradient variance adjustment generates multiple neighbor image samples of the input image sample, and calculates the difference between the gradients of the multiple neighbor image samples and the current input image sample as the correction direction of the next gradient, further stabilizing the optimization direction. Although the above methods have good results, these methods generally have the following problems:
[0006] (1) In the black-box scenario, whether the query-based adversarial sample generation method or the transfer-based adversarial sample generation method, the adversarial sample transferability indeed has certain improvement. However, some methods still ignore the internal features of the neural network model, and the generated adversarial samples only pass through the last layer output of the neural network model, without considering the output of some feature space layers of the neural network model. From the aspect of image perception, compared with the gradient-based transfer attack algorithm, the adversarial samples output through the intermediate layers of the neural network have higher perceptibility and transferability.
[0007] (2) Since the deep learning network model has strong learning ability, the model may often extract the general features of the image data as the characteristics of predicting a certain type of result, which often leads to the model predicting good results on the training set, but having a high error rate on the test set and the validation set, and the generalization ability of the model is low. In the process of generating transferable adversarial samples, the adversarial samples generated by the substitute model may overfit the substitute model, limiting the transferability of the adversarial samples. SUMMARY
[0008] The purpose of the application is to consider that the transferable image adversarial samples pose a greater security threat to the deployment of deep neural network models in real environments, and the current transferable image adversarial sample generation method encounters problems in the testing process. The application provides a transferable adversarial sample generation method based on the intermediate layer of the model, which mainly improves the generation method of transferable adversarial samples and improves the transferability of adversarial samples by adding perturbations in the intermediate layer of the model.
[0009] Technical scheme: In order to achieve the above-mentioned purpose of the application, the transferable adversarial sample generation method based on the intermediate layer of the model comprises the following steps:
[0010] Step 1: training a target image classification model and a substitute model using an image sample set;
[0011] Step 2: adding perturbations to the input image sample in the substitute model using a white-box adversarial sample generation method to generate an initial adversarial sample, and calculating the feature importance of each intermediate layer of the substitute model during the generation of the initial adversarial sample;
[0012] Step 3: obtaining the intermediate features of the input image sample and the initial adversarial sample in the intermediate layer with the highest feature importance, and taking the difference as the attack direction of the initial adversarial sample, in order to find a better adversarial sample on the initial adversarial sample as the target, calculate the intermediate layer loss function of the substitute model, and the better adversarial sample maintains the attack direction of the initial adversarial sample as much as possible and maximizes the perturbation strength; calculate the gradient by back propagation, and obtain the perturbation of the current iteration according to the gradient direction and size;
[0013] Step 4: adding the generated perturbation to the input image sample, and fine-tuning the image sample with the perturbation according to the perturbation constraint to meet the constraint condition, and obtaining the intermediate image sample of the current iteration;
[0014] Step 5: reducing the high-frequency noise of the intermediate image sample using a Gaussian filter to make the image sample smoother;
[0015] Step 6: Determine whether the number of iterations can reach the maximum number of iterations, if yes, the smoothed intermediate image sample is taken as the generated transferable image adversarial sample, otherwise the smoothed intermediate image sample is taken as the input of the substitute model again, and steps 3-5 are repeated.
[0016] Further, the method for generating the initial adversarial sample in step 2 adopts FGSM (Fast Gradient Sign Attack), I-FGSM (Iterative FGSM) or MI-FGSM (Momentum Iterative FGSM) method. The method for calculating the feature importance of the intermediate layer is as follows: in the process of generating the initial adversarial sample, first, forward propagation is performed through the substitute model, then the loss function is calculated according to the true label, second, back propagation is performed to calculate the gradient of each intermediate layer; then the gradient values of each layer in the iteration process are averaged, and the absolute value of the average value is calculated as the importance score of the feature map of the layer; finally, the importance score of each layer is divided by the sum of the importance scores of all layers, thereby obtaining the normalized importance score of each layer.
[0017] Further, the step 3 includes the following steps:
[0018] Step 31: Select the layer l with the highest feature importance, obtain the intermediate features f l (x) and f l (x') of the original clean image sample x and the initial adversarial sample x' in layer l through forward propagation, and take Δx' l = f l (x')-f l (x) as the attack direction of the initial adversarial sample x', then the goal is to find a better adversarial sample x" by optimizing x' in the intermediate layer l of the model, so that x" maintains the original attack direction as much as possible and maximizes the perturbation strength, and the loss function is as follows:
[0019]
[0020] Where f l (x") is the intermediate feature of x" in layer l, Δx" l = f l (x")-f l (x), g represents dot product operation, and || ||2 represents 2-norm for normalizing perturbation size, represents calculating mean square error;
[0021] Step 32: Back propagation to obtain the gradient of the loss function of the model relative to the image sample x" The gradient of the model relative to the image sample x" is normalized to obtain:
[0022]
[0023] The perturbation for the current iteration is obtained as: Δx”=grad' x” .
[0024] Step 4: Add the generated perturbation to the input image samples, and fine-tune the perturbation-injected image samples according to the perturbation constraints to satisfy the constraints, thereby obtaining the intermediate image samples for the current iteration:
[0025] x” N+1 =Clip ε {x” N +lrsign(Δx”)}
[0026] Where x” N and x” N+1 The adversarial examples generated in the Nth and N+1th iterations are respectively, where lr represents the learning rate, sign represents the sign function, and Clip represents the symbol. ε This indicates that the image is cropped by pixels, and the perturbation is limited to the range ε.
[0027] Furthermore, step 5 uses a Gaussian filter to reduce high-frequency noise in the intermediate image samples, making the image samples smoother, and then convolves the image using a convolution kernel:
[0028]
[0029] Where m and n are the coordinates of the center point of the convolution kernel, and k and l are the values of the convolution kernel; f(k,l) is the vector composed of the pixels surrounding the currently smoothed points m and n, and the Gaussian function. σ is the Gaussian filter parameter.
[0030] Furthermore, the method also includes testing the target image classification model using transferable image adversarial examples and calculating the accuracy of the model under test in classifying adversarial examples.
[0031] Based on the same inventive concept, this invention provides a transferable adversarial example generation system based on a model intermediate layer, comprising:
[0032] The initial training module is used to train the target image classification model and the alternative model using the image sample set;
[0033] The intermediate layer importance calculation module is used to add perturbations to the input image samples using a white-box adversarial sample generation method on the alternative model to generate initial adversarial samples, and calculate the feature importance of each intermediate layer of the alternative model during the generation of initial adversarial samples.
[0034] The disturbance determination module is used for obtaining the intermediate features of the input image sample and the initial adversarial sample in the intermediate layer with the highest feature importance, and taking the difference as an attack direction of the initial adversarial sample, so as to find a better adversarial sample on the initial adversarial sample as a target, calculate the intermediate layer loss function of the substitute model, and the better adversarial sample can keep the attack direction of the initial adversarial sample as much as possible and maximize the disturbance intensity; the gradient is calculated by back propagation, and the disturbance is obtained according to the gradient direction and size;
[0035] The intermediate image generation module is used for adding the generated disturbance to the input image sample, fine-tuning the image sample with the added disturbance according to the disturbance constraint, so that the constraint condition is met, and an intermediate image sample is obtained; and a Gaussian filter is used to reduce the high-frequency noise of the intermediate image sample, so that the image sample is smoother;
[0036] The sample generation control module is used for iteratively generating intermediate image samples of each round through the disturbance determination module and the intermediate image generation module, and judging whether the iteration number can reach the maximum iteration number, if yes, taking the smoothed intermediate image sample as the generated transferable image adversarial sample, otherwise, taking the smoothed intermediate image sample as the input of the substitute model again, and repeating the intermediate image sample generation process.
[0037] Based on the same inventive concept, the present application provides a computer system, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program realizes the model intermediate layer based transferable adversarial sample generation method when loaded into the processor.
[0038] Based on the same inventive concept, the present application provides a computer readable storage medium, which stores a computer program, and the computer program realizes the model intermediate layer based transferable adversarial sample generation method when executed by a processor.
[0039] Beneficial effects: the present application provides a model intermediate layer based migratable adversarial sample generation method and system. In order to improve the migratability of the adversarial sample, the method generates disturbance in the intermediate layer of the neural network model and adds it to the image sample, mainly focuses on the internal features of the neural network model, so that the generated adversarial sample has higher migration. In the past research on model intermediate layer, by adding disturbance to each intermediate layer, the effect of adding which layer is better is obtained from the final result, which will lead to low experimental efficiency. The present application calculates the importance score of each layer of the model while generating the initial adversarial sample, so as to obtain the best intermediate layer preferentially, without the need to select the optimal layer from the experimental results, thereby improving the efficiency of generating adversarial samples. In addition, the present application first generates an initial adversarial sample, and then finds a better adversarial sample based on it, so that the better adversarial sample increases the disturbance while maintaining the attack direction of the initial adversarial sample, thereby generating an adversarial sample with higher migration rate. In order to improve the quality of the image sample, the present application uses a Gaussian filter to remove high-frequency disturbance and smooth the image sample, thereby generating an adversarial sample with higher quality. Compared with the existing technology, the present application can find more errors caused by migratable image adversarial samples in the to-be-tested model, that is, the success rate of generating migratable image adversarial samples is higher, while ensuring the generation quality of the image. BRIEF DESCRIPTION OF DRAWINGS
[0040] Figure 1 The overall step diagram of the embodiment of the present application is shown in the figure;
[0041] Figure 2 The flowchart of the intermediate layer based migratable adversarial sample generation method of the present application is shown in the figure. DETAILED DESCRIPTION
[0042] The present application will be further illustrated in combination with specific embodiments, and it should be understood that these embodiments are only used to illustrate the present application and not to limit the scope of the present application. After reading the present application, those skilled in the art can make various equivalent modifications of the present application, which all fall within the scope defined by the appended claims of the present application.
[0043] As shown in the figure, Figure 1 The intermediate layer based migratable adversarial sample generation method disclosed by the embodiment of the present application mainly includes the following steps:
[0044] Step 1: training a target image classification model and a substitute model using an image sample set;
[0045] Step 2: adding disturbance to the input image sample on the substitute model using a white-box adversarial sample generation method to generate an initial adversarial sample, and calculating the feature importance of each intermediate layer of the substitute model in the process of generating the initial adversarial sample;
[0046] Step 3: Obtain intermediate features of the input image sample and the initial adversarial sample in the intermediate layer with the highest feature importance, and use their difference as the attack direction of the initial adversarial sample. With the goal of finding a better adversarial sample on the initial adversarial sample, calculate the loss function of the intermediate layer of the alternative model. The better adversarial sample should maintain the attack direction of the initial adversarial sample as much as possible and maximize the perturbation strength. Backpropagation is used to calculate the gradient, and the perturbation of the current iteration is obtained according to the gradient direction and magnitude.
[0047] Step 4: Add the generated perturbation to the input image sample, and fine-tune the perturbation-added image sample according to the perturbation constraints to make it meet the constraints, and obtain the intermediate image sample of the current iteration.
[0048] Step 5: Use a Gaussian filter to reduce high-frequency noise in intermediate image samples, making the image samples smoother;
[0049] Step 6: Determine whether the number of iterations can reach the maximum number of iterations T. If so, use the smoothed intermediate image sample as the generated transferable adversarial image sample. Otherwise, use the intermediate image sample as the input of the alternative model again and repeat steps 3 to 5.
[0050] like Figure 2 As shown, taking a deep neural network for image classification as an example, the detailed steps of a transferable adversarial example generation method based on the intermediate layer of the model disclosed in this embodiment of the invention are illustrated. Assuming there is a target image classification model DenseNet121 trained using ImageNet that needs to be tested, the specific steps are as follows:
[0051] Step 1: Train the deep neural network model DenseNet121 and the alternative model ResNet18 for image classification using the preprocessed ImageNet training set. In this step, download the ImageNet training set, validation set, test set, and their corresponding labels from the ImageNet dataset website https: / / image-net.org / challenges / LSVRC / 2012 / index.php; decompress the necessary data from the downloaded compressed file to a local folder. Read the ImageNet training and test sets, including the corresponding sample labels, and store them in a matrix tensor; crop the read image samples to 299×299 pixels and normalize the pixel values to the range [-1.0, 1.0]. Place the original test set into the clean sample set; the training set is used to train the deep neural network model for image classification.
[0052] In this example, the parameter configuration information of the migratable image adversarial sample generation method is set, specifically including: setting the maximum disturbance change value ε = 16.0, the maximum iteration number N = 10, the learning rate lr = 1.0, the weight α = 0.005, and the Gaussian filter parameter σ = 1.5. max
[0053] Step 2: Add disturbance to the input image sample using the white-box adversarial sample generation method on the substitute model Resnet18 to generate the initial adversarial sample, and calculate the feature importance of each intermediate layer of the substitute model during the generation of the initial adversarial sample. The specific steps are as follows:
[0054] Step 21: Obtain the current input image sample x, whose corresponding original class is t, and N' represents the current iteration round. Use the white-box adversarial sample generation method to generate the initial adversarial sample, which can use methods such as FGSM, I-FGSM, MI-FGSM, etc. In this example, I-FGSM is taken as an example: initialize the adversarial sample (where N' = 0), input the image sample x into the model, calculate the cross-entropy loss function and calculate the gradient Iterative generation of adversarial samples:
[0055]
[0056] where Clip ε is the pixel clipping of the image, and the initial adversarial sample x' is finally obtained. The sign function is a sign function, if then sign(x) = 1; if then sign(x) = 0; if then sign(x) = -1.
[0057] Step 22: During the generation of the initial adversarial sample, first perform forward propagation through the model, then calculate the loss function according to the true label, and then perform backpropagation to calculate the gradient of each intermediate layer. In the network model, fl(x) represents the feature map of the model f on the intermediate layer l, and f(x)[t] represents the probability that the image x belongs to t, i.e. the logical value output, then the gradient of the intermediate layer l is Secondly, using the calculated intermediate layer gradient information, the importance score of each layer feature map can be calculated. Specifically, the gradient values of each layer in the iteration process can be averaged, and then the absolute value of the average value is calculated as the importance score of the layer feature map. The importance score of the l-th layer can be represented as (where ω l,i The gradient of the ith iteration of the lth layer is T, and the total number of iterations is T. Since the gradient of different layers may differ in size, the importance score of each layer needs to be normalized for comparison. The importance score of each layer can be divided by the sum of the importance scores of all layers to obtain the normalized importance score of each layer, and the final importance score of the lth layer is Score l : (L is the total number of layers of the model).
[0058] Step 3: Obtain the intermediate features of the clean image and the initial adversarial sample in the intermediate layer with high feature importance, calculate the intermediate layer loss function of the substitute model, backpropagate to calculate the gradient, and obtain the perturbation of the current round of iteration according to the gradient direction and size. The specific steps are as follows:
[0059] Step 31: Select a layer l with high importance score, and obtain the intermediate features of the original image x and the initial adversarial sample x' in layer l by forward propagation:f l (x) and f l (x'), and Δx' l = f l (x')-f l (x) is regarded as the attack direction of the initial adversarial sample x', then our goal is to find a better adversarial sample x" by optimizing x' in the intermediate layer l of the model, so that x" maintains the original attack direction as much as possible and maximizes the perturbation strength, and the loss function is as follows:
[0060]
[0061] The loss function includes two parts, the first part represents maintaining the original attack direction, and the second part represents increasing the perturbation strength, where f l (x") is the intermediate feature of x" in layer l, Δx" l = f l (x")-f l (x), g represents the dot product operation, and || ||2 represents the 2-norm used to normalize the perturbation size, represents the calculation of the mean square error;
[0062] Step 32: Backpropagation to obtain the gradient of the loss function of the model relative to the image sample x" In order to further stabilize the direction of the perturbation, we normalize the gradient of the model relative to the image sample x" to obtain:
[0063]
[0064] The perturbation of the current round of iteration is Δx" = grad' x” .
[0065] Step 4: the generated perturbation is added to the input image sample, and the image sample with the added perturbation is fine-tuned according to the perturbation constraint to meet the constraint condition, to obtain an intermediate image sample of the current round of iteration:
[0066] x” N+1 = Clip ε {x” N + lrsign(Δx”)}
[0067] where x” N and x” N+1 are the adversarial samples generated in the Nth iteration and the N+1th iteration, respectively.
[0068] Step 5: use a Gaussian filter to reduce the high-frequency noise of the intermediate image sample, so that the image sample is smoother, and use a convolution kernel to convolve the image:
[0069]
[0070] where m and n are the coordinates of the center point of the convolution kernel, and k and l are the values of the convolution kernel; f(k, l) is a vector composed of the pixels around the current smoothed point m, n, and the Gaussian function
[0071] Step 6: determine whether the number of iterations can reach the maximum number of iterations T, if yes, proceed to step 11, otherwise, the intermediate image sample is re-used as the input of the model, and steps 3-5 are repeated.
[0072] Step 7: restore the generated image sample to the data form of the original image sample and save it to the transferable image adversarial sample data set as a test case.
[0073] Step 8: test the DenseNet121 model using the transferable image adversarial sample, and calculate the classification accuracy ACRC adv of the DenseNet121 model on the transferable adversarial sample to test the robustness of the model.
[0074] Based on the same inventive concept, the embodiment of the present application discloses a model intermediate layer-based migratable adversarial sample generation system, which comprises an initial training module configured to train a target image classification model and a substitute model using an image sample set; an intermediate layer importance calculation module configured to add perturbation to input image samples on the substitute model using a white-box adversarial sample generation method to generate initial adversarial samples, and calculate the feature importance of each intermediate layer of the substitute model in the process of generating the initial adversarial samples; a perturbation determination module configured to obtain the intermediate features of the input image samples and the initial adversarial samples in the intermediate layer with the highest feature importance, and take the difference as the attack direction of the initial adversarial samples, so as to find a better adversarial sample on the initial adversarial sample as the target, calculate the intermediate layer loss function of the substitute model, and the better adversarial sample can keep the attack direction of the initial adversarial sample as much as possible and maximize the perturbation strength; calculate the gradient through back propagation, and obtain the perturbation according to the gradient direction and size; an intermediate image generation module configured to add the generated perturbation to the input image samples, fine-tune the image samples with the added perturbation according to the perturbation constraint, so that the image samples meet the constraint condition, and obtain intermediate image samples; and use a Gaussian filter to reduce the high-frequency noise of the intermediate image samples, so that the image samples are smoother; and a sample generation control module configured to generate intermediate image samples of each round through the perturbation determination module and the intermediate image generation module, and judge whether the iteration number can reach the maximum iteration number, if yes, take the smoothed intermediate image samples as the generated migratable image adversarial samples, and if not, take the smoothed intermediate image samples as the input of the substitute model again, and repeat the intermediate image sample generation process.
[0075] Based on the same inventive concept, the embodiment of the present application discloses a computer system, which comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program realizes the model intermediate layer-based migratable adversarial sample generation method when loaded into the processor.
[0076] Based on the same inventive concept, the embodiment of the present application discloses a computer readable storage medium, which stores a computer program, and the computer program realizes the model intermediate layer-based migratable adversarial sample generation method when executed by a processor.
Claims
1. A model-based intermediate layer transferable adversarial sample generation method, characterized in that, The method comprises the following steps: Step 1: training a target image classification model and a substitute model using an image sample set; Step 2: adding perturbations to input image samples on the substitute model using a white-box adversarial sample generation method to generate initial adversarial samples, and calculating the feature importance of each intermediate layer of the substitute model in the process of generating the initial adversarial samples; Step 3: obtaining the intermediate features of the input image samples and the initial adversarial samples at the intermediate layer with the highest feature importance, and taking the difference as the attack direction of the initial adversarial samples, so as to find a better adversarial sample on the initial adversarial sample as a target, calculate the intermediate layer loss function of the substitute model, and the better adversarial sample can keep the attack direction of the initial adversarial sample as much as possible and maximize the perturbation strength; the gradient is calculated by back propagation, and the perturbation of the current iteration is obtained according to the gradient direction and size; Step 4: adding the generated perturbation to the input image sample, and fine-tuning the image sample with the added perturbation according to the perturbation constraint to make it meet the constraint condition, and obtaining the intermediate image sample of the current iteration; Step 5: reducing the high-frequency noise of the intermediate image sample using a Gaussian filter to make the image sample smoother; Step 6: judging whether the iteration number can reach the maximum iteration number, if yes, taking the smoothed intermediate image sample as the generated transferable image adversarial sample, otherwise, taking the smoothed intermediate image sample as the input of the substitute model again, and repeating steps 3-5; The method for calculating the feature importance of the intermediate layer in step 2 is: in the process of generating the initial adversarial sample, firstly, forward propagation is performed through the substitute model, then the loss function is calculated according to the true label, secondly, back propagation is performed, and the gradient of each intermediate layer is calculated; then, the gradient values of each layer in the iteration process are averaged, and then the absolute value of the average value is calculated as the importance score of the feature map of the layer; finally, the importance score of each layer is divided by the sum of the importance scores of all layers, so as to obtain the normalized importance score of each layer; The step 3 comprises the following steps: Step 31: Select layer l with the highest feature importance, and obtain the intermediate features of the original clean image sample x and the initial adversarial sample x' in layer l through forward propagation: f l (x) and f l (x'), Δx l '=f l (x')-f l If x is considered as the attack direction of the initial adversarial sample x', then our goal is to find a better adversarial sample x” by optimizing x' in the intermediate layer l of the model, so that x” maintains the original attack direction as much as possible and maximizes the perturbation strength. The loss function is as follows: where f l (x") is the intermediate feature of x" in layer l, Δx l " = f l (x") - f l (x), · denotes the dot product operation, and || ||2denotes the 2-norm to normalize the perturbation size, denotes the computation of the mean squared error; Step 32: Backpropagation to obtain the gradient of the loss function of the model with respect to the image sample x Standardizing the gradient of the model with respect to the image sample x gives: Obtain the perturbation for the current iteration of the loop: Δx" = grad' x” .
2. The model intermediate layer based transferable adversarial sample generation method according to claim 1, characterized in that, The method for generating the initial adversarial sample in step 2 adopts the FGSM, I-FGSM or MI-FGSM method.
3. The method of claim 1, wherein, In step 4, the generated perturbation is added to the input image sample, and the image sample with the added perturbation is fine-tuned according to the perturbation constraint to make it meet the constraint condition, and the intermediate image sample of the current iteration is obtained: x” N+1 = Clip ε {x” N + lrsign(Δx”)} where x N and x N+1 are adversarial samples generated in the Nth iteration and the N+1th iteration, respectively, lr denotes a learning rate, Δx" denotes a perturbation, sign denotes a sign function, and Clip ε denotes pixel clipping of an image, limiting the perturbation within the range of ε.
4. The model intermediate layer based transferable adversarial sample generation method of claim 1, wherein, In step 5, a Gaussian filter is used to reduce the high-frequency noise of the intermediate image sample to make the image sample smoother, and the image is convolved with a convolution kernel: Wherein, m, n is the center point coordinates of the convolution kernel, k, l is the value of the convolution kernel; f(k, l) is the vector composed of the pixels around the current smoothing point m, n, Gaussian function σ is the Gaussian filter parameter.
5. The method of claim 1, wherein, Further comprising: using the transferable image adversarial sample to test the target image classification model, and calculating the accuracy rate of the tested model in classifying the adversarial sample.
6. A model-agnostic intermediate layer based transferable adversarial sample generation system, comprising: Comprise: An initial training module for training a target image classification model and a substitute model using an image sample set; An intermediate layer importance calculation module for adding perturbations to input image samples on the substitute model using a white-box adversarial sample generation method to generate initial adversarial samples, and calculating the feature importance of each intermediate layer of the substitute model in the process of generating the initial adversarial samples; An intermediate layer importance calculation module for adding perturbations to input image samples on the substitute model using a white-box adversarial sample generation method to generate initial adversarial samples, and calculating the feature importance of each intermediate layer of the substitute model in the process of generating the initial adversarial samples; The method for calculating the importance of the features of the intermediate layer is: in the process of generating the initial adversarial sample, first, forward propagation is performed through the substitute model, and then the loss function is calculated according to the true label, and second, the gradient of each intermediate layer is calculated by back propagation; then the gradient values of each layer in the iteration process are averaged, and the absolute value of the average value is calculated as the importance score of the feature map of the layer; Finally, the importance score of each layer is divided by the sum of the importance scores of all layers, thereby obtaining the normalized importance score of each layer; The perturbation determination module is configured to obtain the intermediate features of the input image sample and the initial adversarial sample at the intermediate layer with the highest feature importance, and take the difference as the attack direction of the initial adversarial sample, so as to find a more optimal adversarial sample on the initial adversarial sample as the target, calculate the loss function of the intermediate layer of the substitute model, and the more optimal adversarial sample maintains the attack direction of the initial adversarial sample as much as possible and maximizes the perturbation strength; The reverse propagation calculates the gradient, and obtains the perturbation according to the gradient direction and size; including: selecting the layer l with the highest feature importance, obtaining the intermediate features:f l (x) and f l (x') of the original clean image sample x and the initial adversarial sample x' in the layer l through forward propagation l ' = f l (x') - f l (x) is regarded as the attack direction of the initial adversarial sample x', then our goal is to find a better adversarial sample x" by optimizing x' on the intermediate layer l of the model, so that x" can maintain the original attack direction as much as possible and maximize the perturbation strength, and the loss function is as follows: where f l (x") is the intermediate feature of x" in layer l, Δx l " = f l (x") - f l (x), · denotes the dot product operation, and || ||2 denotes the 2-norm to normalize the perturbation size, denotes the computation of the mean square error; the loss function of the model is backpropagated to obtain the gradient of the model with respect to the image sample x" The gradient of the model with respect to the image sample x" is normalized to obtain: Obtain the perturbation for the current iteration of the loop: Δx" = grad' x” ; The intermediate image generation module is configured to add the generated perturbation to the input image sample, and fine-tune the image sample with the added perturbation according to the perturbation constraint to make it satisfy the constraint condition, thereby obtaining an intermediate image sample; and use a Gaussian filter to reduce the high-frequency noise of the intermediate image sample, so that the image sample is smoother; The sample generation control module is configured to generate the intermediate image sample of each round through the perturbation determination module and the intermediate image generation module, and determine whether the iteration number can reach the maximum iteration number, if yes, the smoothed intermediate image sample is taken as the generated transferable image adversarial sample, otherwise the smoothed intermediate image sample is taken as the input of the substitute model again, and the intermediate image sample generation process is repeated.
7. A computer system comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The computer program is loaded into the processor to implement the model intermediate layer based transferable adversarial sample generation method according to any one of claims 1-5.
8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to implement the model intermediate layer based transferable adversarial sample generation method according to any one of claims 1-5.