A black-box attack method based on gradient alignment adversarial distillation

Abstract

The present application relates to the field of deep neural network security, and more particularly to a black-box attack method based on gradient alignment adversarial distillation. The method comprises the following steps: step 1: designing a sample perturbation generator maximizing gradient difference; step 2: designing a gradient alignment adversarial distillation modeling framework for the substitute model; step 3: alternately training the substitute model and the perturbation generator; step 4: completing the attack on the black-box model based on the white-box attack method and the substitute model. The experimental results on different data sets and models show that, compared with the traditional static distillation, the dynamic distillation based on Gaussian distribution random perturbation, and the dynamic distillation based on adversarial perturbation, the present application method has achieved significant improvement in the success rate of confrontation, and has certain innovation and universality.

Description

Technical Field

[0001] This invention relates to the field of deep neural network security, and in particular to a black-box attack method based on gradient alignment to counter distillation. Background Technology

[0002] Deep neural network models suffer from vulnerability to attacks, proving exceptionally susceptible to subtle, carefully crafted perturbations. These perturbations, known as adversarial examples, can lead to incorrect decisions by the model in ways imperceptible to the human eye. This phenomenon not only reveals the shortcomings of deep learning models in terms of robustness but also poses a serious potential threat to their application in security-sensitive fields, such as medical diagnostics and autonomous driving systems.

[0003] Methods for generating adversarial examples are called adversarial attack methods. Although the term "attack" can be misleading, the essence of adversarial attack research is to improve the security and robustness of models. The current mainstream model defense method is adversarial training, whose basic idea is to enhance the robustness of the model using adversarial examples. Most existing adversarial training methods rely on white-box attack methods to generate adversarial examples. Since white-box attacks typically assume that the attacker can access the model's internal information, the generated adversarial examples often directly utilize gradient information, resulting in relatively simple forms and distributions. Consequently, while the trained model may have some resistance to specific white-box attacks, its defense effectiveness is limited when facing other white-box attack methods or even more realistic black-box attacks.

[0004] Existing research has shown that adversarial examples generated by black-box attacks are significantly more diverse than those generated by white-box attacks, and therefore have the potential to be used to build more robust defense models. However, introducing black-box attacks into model training still faces two major challenges: (1) Black-box attacks usually rely on a large number of model query operations, and the time to generate an adversarial example is much longer than that of white-box attacks. If used for training, it will greatly increase the model training time and computational cost; (2) The adversarial examples generated by traditional black-box attacks are of limited quality and have a low success rate, making it difficult to substantially improve the model's defense capabilities.

[0005] The following section will classify and analyze existing adversarial attack methods, point out their key challenges in practical applications, and outline the improvement scheme and technical advantages proposed by this invention.

[0006] In the field of adversarial attacks, early research focused on white-box attacks, assuming that detailed information about the target model, such as its architecture, network gradient calculations, parameters, and even the data used during training, is known to the attacker. While white-box attacks have yielded significant research results, their application is limited by their reliance on information such as model structure, parameters, and training samples. In practical applications, most attackers cannot obtain this information; therefore, black-box attacks are more relevant and valuable in practice. Black-box attacks refer to attack methods that generate adversarial examples through the model's output without direct access to the target model's internal structure and parameters. These attacks simulate the most common and covert attack scenarios in the real world, where attackers can only understand the model's behavior through interaction with it.

[0007] The challenge of black-box attacks lies in the lack of understanding of the internal workings of the target model, requiring attackers to use more creative and computationally intensive methods to infer the model's behavior. With the increasing application of deep learning models in security-critical fields, research on black-box attacks is crucial for understanding and improving model robustness. Currently, based on the attacker's information gathering methods, black-box attacks can be broadly categorized into three types: transfer attacks based on substitute models, gradient approximation attacks, and decisions boundary search attacks. Among these, transfer attacks have a relatively low success rate, and their success depends heavily on the quality of the constructed substitute model. However, compared to the other two types, transfer attacks offer a significant time advantage in generating adversarial examples, making them feasible for adversarial training to improve model robustness. Therefore, researching how to further improve their success rate is of great importance.

[0008] Alternative model-based transfer attacks generate adversarial examples that can deceive the target black-box model by constructing an alternative model to mimic its behavior. In alternative model attacks, the alternative model is based on the white-box assumption, meaning its network parameters and architecture are fully known. While the internal details of the target black-box model are unknown, attackers can typically query its output from input samples, which can then be used to train and fine-tune the alternative model. The core challenge for this type of attack is constructing an alternative model that closely resembles the target black-box model.

[0009] Currently, knowledge distillation is the primary technique used to train alternative models. The core idea is to construct an alternative model (student model) with functional similarity to the target black-box model (teacher model) through knowledge distillation, thus transforming the difficult-to-access black-box attack problem into a relatively easy-to-handle white-box attack problem. In this process, the attacker continuously queries the target black-box model, collecting training datasets with similar decision-making behavior to train and optimize the alternative model. However, current knowledge distillation methods are mostly static distillation, fitting the outputs of the teacher and student models to fixed samples. Since fixed samples inevitably suffer from sparsity in a high-dimensional sample space, static distillation can only approximate the teacher model at a limited number of sparse points, making it difficult to ensure consistency between the student and teacher models across the overall decision boundaries. Because adversarial attack methods are largely dependent on the model's decision boundaries, the quality of adversarial examples generated for the target black-box model based on the alternative model is often low. Furthermore, in actual neural network training, the structural complexity of the model is extremely high, and the sample space is also high-dimensional. Traversing dynamically perturbed samples across all dimensions of the model is difficult in terms of both complexity and time efficiency. Therefore, it is necessary to design an efficient perturbation generation mechanism to support the dynamic distillation process, which is a key issue facing dynamic distillation.

[0010] Furthermore, since most attack methods are based on model gradients, they rely on the gradient information of the target model to generate adversarial examples that can lead to incorrect predictions. Adversarial examples generated by these methods will only be effective on the target black-box model if the gradients of the surrogate model and the target black-box model are consistent. If the gradients of the surrogate model and the target black-box model are inconsistent, then even if their outputs are similar, the adversarial examples generated by the surrogate model may differ significantly from the effective adversarial examples of the target black-box model. Therefore, the consistency of gradients between the surrogate model and the target black-box model is crucial for the success of transfer attacks. Summary of the Invention

[0011] To address the problems of existing technologies, this invention proposes to enhance the gradient consistency between the surrogate model and the target black-box model during the distillation process through gradient matching loss. Furthermore, it trains a perturbation generator to generate small perturbations that maximize the gradient difference between the surrogate and target black-box models. By introducing the generator and utilizing the idea of ​​adversarial learning, the perturbation generator and the surrogate model are trained alternately in an adversarial manner. This transforms the training process of the surrogate model into a minimization-maximization problem.

[0012]

[0013] in, For the target black-box model, As an alternative model, For the first One input sample, For generator G pairs The resulting tiny perturbations For a fixed number of samples 。

[0014] This invention helps to promote the development of real-time and efficient black-box attack techniques and provides important technical support for building more robust deep neural network models.

[0015] Technical solution

[0016] A black-box attack method based on gradient alignment to counter distillation includes the following steps:

[0017] Step 1: Design a sample perturbation generator that maximizes the gradient difference ;

[0018] The sample perturbation generator Used for input samples Generate small perturbations Fixed sample Plus disturbance A sample neighborhood is formed for training the alternative model; by maximizing the gradient difference between the alternative model and the target black-box model in the sample neighborhood, the alternative model is guided to continuously improve its fitting ability to the target black-box model in the process of repairing these differences.

[0019] Step 2: Design a gradient-aligned adversarial distillation modeling framework for the alternative model;

[0020] The gradient-aligned adversarial distillation method, abbreviated as GRAD, is introduced to generate an alternative model for the target black-box model; it not only focuses on mimicking the output of the target black-box model. It also focuses on mimicking its gradient. Train an alternative model through distillation. This makes its output and target black box model The output should be as close as possible to the sample neighborhood, while also minimizing the gradient of the alternative model. gradient with the target black-box model Matching.

[0021] Step 3: Alternately train alternative models and perturbation generator ;

[0022] Disturbance generator The training objective is to generate perturbations that maximize the gradient difference between the surrogate model and the target black-box model; while the surrogate model The training objective is to minimize the gradient difference between the surrogate model and the target black-box model, achieving gradient consistency between the two models; therefore, the surrogate model... and perturbation generator The training process is a minimization-maximization problem, which is suitable for adversarial training; the overall training objectives are as follows:

[0023]

[0024] in, For the target black-box model, As an alternative model, For the first One input sample, For generator G pairs The resulting tiny perturbations For a fixed number of samples 。

[0025] Step 4: Based on white-box attack methods and alternative models Complete the black box model The attack;

[0026] Reading alternative models The model weight file is used to select a white-box attack method to perform an attack on the original dataset using an alternative model. Perform white-box attacks to obtain an adversarial example set; test the black-box model on the adversarial example dataset. The output is used to calculate the attack success rate. .

[0027] In summary, this invention innovatively proposes a gradient-aligned adversarial distillation transfer attack method. It ensures consistency of both output and gradient during the training of the substitute model. By introducing dynamic sample loss, it guides the substitute model and the target black-box model to achieve output consistency in the sample neighborhood, reducing the difference between the two models at the decision boundary. Furthermore, by introducing gradient matching loss, it ensures consistency in gradient information between the two models, enhancing the effectiveness of adversarial examples generated based on the gradient information of the substitute model against the target black-box model and improving the attack success rate.

[0028] Beneficial effects

[0029] This invention proposes a novel black-box attack method. The proposed gradient alignment adversarial distillation method effectively alleviates the problem of large differences in decision boundaries between the surrogate model and the target black-box model. By increasing the gradient similarity between the two, the success rate of the transfer attack is improved, and the generated adversarial examples are of higher quality. Simultaneously, while maintaining the advantages of black-box attacks (such as no need to access internal model information and high sample diversity), the generation time is significantly shortened, making it computationally efficient comparable to white-box attacks. This provides a technical foundation and feasible path for introducing black-box attacks into model training, thereby systematically improving the model's defense capabilities in real-world deployment environments. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the processing flow of the method of the present invention;

[0031] Figure 2 This is the modeling framework for gradient alignment adversarial distillation in this embodiment of the invention;

[0032] Figure 3 This is a two-dimensional probability density comparison diagram of black-box and white-box adversarial samples in an embodiment of the present invention. Detailed Implementation

[0033] The technical solution provided in this application will be further described below with reference to specific embodiments and accompanying drawings. The advantages and features of this application will become clearer from the following description.

[0034] The embodiments of the present invention are illustrated using image classification as an example, but the method of the present invention is also applicable to other tasks.

[0035] A black-box attack method based on gradient alignment to counter distillation includes the following steps:

[0036] Step 1: Design a sample perturbation generator that maximizes the gradient difference

[0037] The sample perturbation generator Used for input samples Generate small perturbations Fixed sample Plus disturbance A sample neighborhood is formed for training the surrogate model. The design strategy is to maximize the gradient difference between the surrogate model and the target black-box model in the sample neighborhood, thereby guiding the surrogate model to continuously improve its fitting ability to the target black-box model as it repairs these differences.

[0038] To improve the training efficiency of alternative models, perturbation The perturbation generator should be designed to be the most challenging input to the surrogate model, similar to the "hardest sample." Given that the training objective is to make the surrogate model approximate the target black-box model in terms of gradients, the generation of the perturbation should consciously expose its weaknesses in gradient consistency. To this end, the most effective strategy is to maximize the gradient difference between the two models in the sample neighborhood, thereby guiding the surrogate model to continuously improve its fitting ability to the target black-box model while addressing these weaknesses. The perturbation generator is discussed below. The network structure design and loss function design are explained.

[0039] Step 1.1 Design the disturbance generator Network Structure

[0040] The perturbation generator network perturbs the input samples Generate small perturbations The overall network is represented as: ,in, It is a generator network, and the network parameters are: The input is a fixed sample. The output is a tiny perturbation with the same size as the input.

[0041] In this embodiment, the perturbation generator network adopts the UNet architecture. The UNet network is recommended because its output size matches the input, making it suitable for pixel-level generation tasks. UNet employs a symmetrical downsampling-upsampling path and skip connections. The downsampling layer expands the receptive field to capture global contextual information, while the upsampling layer restores spatial details. This combination ensures that the perturbation is reasonably distributed both globally and locally.

[0042] The UNet network includes: an initial convolutional module, a downsampling layer, a ResNet block, an upsampling layer, and a final convolutional module.

[0043] The initial convolutional module adopts a 5×5 convolutional layer plus a normalization layer design to convert the number of input sample channels into the number of feature channels;

[0044] The downsampling layer consists of m layers, each a 4×4 convolution with a stride of 2, and is superimposed with activation functions and normalization layers. By using multiple convolutions and activation functions, the feature map size is gradually reduced, thereby increasing the receptive field.

[0045] The ResNet block consists of n residual blocks, with the number of channels remaining constant. Feature representation capability is enhanced through skip connections between multiple residual blocks.

[0046] The upsampling layer consists of m layers, each of which is a 4×4 transposed convolution with a stride of 2, and is superimposed with activation functions and normalization layers. The feature map size is gradually restored through deconvolution.

[0047] The final convolutional module uses a 5×5 convolution and selects Tanh as the activation function to limit the perturbation range, ultimately outputting perturbed samples.

[0048] Furthermore, for sample data of varying complexity, the network design maintains the core framework of UNet and is flexibly adjusted according to requirements:

[0049] For simple samples (such as low-resolution, simple images), reduce the number of downsampling layers m and the number of residual blocks n to reduce model complexity and computational cost;

[0050] For complex samples (such as high-resolution, richly textured images), it is necessary to increase the network depth and width to improve the receptive field and expressive power, and ensure the precision and effectiveness of perturbation generation.

[0051] Taking image samples as an example, for simple MNIST sample images, you can choose to have m=2 sampling layers, n=6 residual blocks, and ngf=32 initial channels. For relatively complex CIFAR-10 sample images, you can choose to have m=3 sampling layers, n=12 residual blocks, and ngf=64 initial channels.

[0052] Step 1.2 Design the disturbance generator loss function

[0053] In a fixed sample Approximate calculation of the target black box model in the neighborhood first gradient

[0054] ;

[0055] Similarly, alternative models The approximate first-order gradient on is

[0056] ,

[0057] The design strategy is to generate perturbations. To make the gradient difference between the two models Maximum. Therefore, the generator loss function The design is as follows:

[0058]

[0059] In addition to the gradient difference loss term (the first half of the formula), a perturbation amplitude constraint is added (the second half of the formula) to ensure that the perturbation is small. Let be the mean square error function. For the loss term coefficient, A perturbation threshold, specified by the user, is used as a hyperparameter for model training. For example, in this first embodiment, The value is 30. The value is 0.01. Values .

[0060] Step 2: Design a gradient-aligned adversarial distillation modeling framework for the alternative model

[0061] To improve the gradient consistency between the surrogate model and the target black-box model, this invention proposes a gradient-aligned adversarial distillation method, abbreviated as GRAD (GRadient-aligned Adversarial Distillation), which generates a surrogate model for the target black-box model, focusing not only on mimicking the output of the target black-box model. It also focuses on mimicking its gradient. The core idea of ​​this method is to train an alternative model through distillation. This makes its output and target black-box model The output should be as close as possible to the sample neighborhood, while also minimizing the gradient of the alternative model. gradient with the target black-box model Matching.

[0062] This goal is achieved by adding a gradient matching term to the loss function, thereby ensuring that the alternative model can simulate the behavior of the target black-box model in terms of gradient information, which is of utmost concern to adversarial attacks.

[0063] Step 2.1: Design a gradient alignment adversarial distillation modeling framework

[0064] Gradient alignment anti-distillation modeling framework, such as Figure 2 As shown, it includes: the target model, the alternative model, and the perturbation generator.

[0065] Among them, the disturbance generator Input Sample Generate small perturbations Thus, dynamic samples are synthesized: ;

[0066] Target Model As a black-box model, for input samples and perturbation samples Its output is and ;

[0067] Alternative Model For a trainable white-box model, for input samples and perturbation samples The output is and Used to fit the target black-box model The gradient;

[0068] The target black box model is approximated using the finite difference method. and alternative models First gradient:

[0069] ,

[0070] It should be noted that the choice of alternative models is generally task-related, and each task typically has commonly used models. For example, for image tasks, convolutional neural networks might be chosen, while for NLP tasks, Transformer or RNN-like networks might be selected. This invention does not limit this choice.

[0071] Step 2.2: Design an alternative model loss function

[0072] Alternative Model loss function Including: losses from the original mission Static sample distillation loss Dynamic sample distillation loss Matching loss with first-order gradient .

[0073] The specific explanation is as follows:

[0074] (1) The original mission losses are as follows:

[0075]

[0076] in This represents the loss function corresponding to the black-box model task itself. Let y be the input sample and y be the sample label. For classification tasks, the loss is usually cross-entropy; for regression tasks, the loss is usually mean squared error.

[0077] (2) Static sample distillation loss is as follows:

[0078]

[0079] in, The distance metric function for the difference is typically the KL divergence of the model's final predicted probability or the L2 loss of intermediate layer features.

[0080] (3) The dynamic sample distillation loss is as follows:

[0081]

[0082] in, Indicates a fixed sample Tiny perturbations on the surface The distance metric function for the difference is the same as above.

[0083] (4) The first-order gradient matching loss is as follows:

[0084]

[0085] The parameters have the same meaning as above.

[0086] In summary, the alternative model loss function The definition is as follows:

[0087]

[0088] in, These are weights used to control the original task loss, static samples, dynamic samples, and the degree of first-order gradient matching, respectively, and are used to balance the loss term. For example, in Implementation Example 1... The values ​​are 0.2, 0.8, 3, and 3.

[0089] Step 3: Alternately train alternative models and perturbation generator ;

[0090] Disturbance generator The training objective is to generate perturbations that maximize the gradient difference between the surrogate model and the target black-box model; while the surrogate model The training objective is to minimize the gradient difference between the surrogate model and the target black-box model, achieving gradient consistency between the two models. Therefore, the surrogate model... and perturbation generator The training process is a minimization-maximization problem, suitable for adversarial training. The overall training objectives are as follows:

[0091]

[0092] The training process is as follows:

[0093] Step 3.1 Initialize the alternative model (parameter Initialize the disturbance generator Training dataset Target black box model Disturbance generator Loss weights in loss Alternative Model Loss weight of the Central Plains mission Static sample distillation weights Dynamic sample distillation weights Gradient weights Learning rate of the alternative model Learning rate of the perturbation generator .

[0094] For example, in Example 1 The values ​​are 30 and 0.01 respectively. The values ​​are 0.2, 0.8, 3, and 3 respectively. , The value of is 1e-4.

[0095] Step 3.2 Training the perturbation generator

[0096] Fixed substitution model Training the perturbation generator Generate disturbance The perturbation is generated by maximizing the gradient difference between the surrogate model and the target black-box model through the loss function. It points in the direction where the gradient consistency between the two is weakest.

[0097] Step 3.3 Training the Alternative Model

[0098] Fixed disturbance generator In a fixed sample neighborhood, an alternative model is trained using gradient alignment adversarial distillation. By minimizing the gradient difference through the loss function, the alternative model can be optimized. Maximize the gradient information of the fitted target black-box model.

[0099] Step 3.4 Training data for all batches Perform steps 3.2 and 3.3 alternately to complete the adversarial training. Save the network weight file of the alternative model.

[0100] In summary, the alternative model and perturbation generator The adversarial training process is as follows:

[0101]

[0102] Step 4: Based on white-box attack methods and alternative models Complete the black box model attack

[0103] Reading alternative models For the model weight file, choose a white-box attack method to attack the original dataset. In alternative models Perform white-box attacks to obtain adversarial sample sets. In adversarial sample datasets Test black-box model The output is used to calculate the attack success rate. :

[0104]

[0105] in, Indicates the number of samples; indicator function [ The symbol ] indicates a truth value; it is 1 if true and 0 otherwise. Represents the black box model For input Predicted labels.

[0106] Example 1: Advantages of Gradient Alignment-Based Adversarial Distillation Black-Box Attack Method in Adversarial Attacks

[0107] Step 1, Determine the baseline method

[0108] The following three baseline methods are designed and compared to demonstrate the advantages of the method of the present invention in terms of attack success rate.

[0109] Baseline Method 1: The substitution model is trained using a static distillation method, and the loss function of the substitution model is as follows:

[0110]

[0111] Baseline Method 2: The substitution model is trained using a dynamic distillation method (utilizing random perturbations). The loss function of the substitution model is as follows:

[0112] This consists of a fixed sample plus a small perturbation. Forming dynamic samples, where tiny perturbations occur. Generated by random perturbations following a Gaussian distribution, assuming the dimension of the static sample image is... The perturbation consists of an image of the same size, with each pixel drawn from an independent and identically distributed Gaussian distribution, with the following distribution parameters:

[0113]

[0114] Baseline Method 3: The substitution model is trained using a dynamic distillation method (utilizing adversarial perturbations), and the loss function of the substitution model is as follows:

[0115] Unlike baseline method 2, here the small perturbation Adversarial attack methods (FGSM, PGD, etc.) in alternative models The above is calculated based on the gradient.

[0116] Step 2: Setting up the experimental environment

[0117] (1) Benchmark dataset

[0118] Different datasets and black-box models need to be selected for comparative experiments to verify the innovativeness and universality of the invention method.

[0119] The dataset chosen is the commonly used public dataset SVHN (Street View House Numbers), which is frequently used as a benchmark dataset in adversarial attack research, helping to evaluate and compare the performance of different models and methods. SVHN is a large-scale image dataset containing approximately 1 million images, which are house numbers cropped from Google Street View. These images are taken in natural environments with a wide variety of backgrounds and lighting conditions. The dataset is typically divided into training and testing sets, with the training set containing approximately 73,000 images and the testing set containing approximately 26,000 images.

[0120] (2) The architecture of the attacked deep neural network model

[0121] The typical neural network architecture chosen is ResNet-18, which has wide applications in image classification tasks. Evaluating its performance under adversarial attacks is of great significance for practical applications.

[0122] (3) Attack methods and attack parameters

[0123] Transfer attacks involve first performing a white-box attack on the surrogate model, and then using white-box adversarial examples from the surrogate model to transfer attacks on the black-box model. The white-box attack methods selected here include FGSM, PGD, BIM, MI-FGSM, NI-FGSM, VNI-FGSM, and PI-FGSM++, covering a wide range of attack methods (from single-step attacks to multi-step enhancement attacks, considering transfer enhancement, and incorporating temperature regulation strategies, etc.), enabling a comprehensive evaluation of the model's vulnerability under different attack methods. For all attack methods, a maximum perturbation limit was set. This means that the maximum possible change in perturbation an attacker can add during an attack is 8 color channel units, measured at the RGB value of each pixel in the image. This limitation is to simulate real-world attack scenarios where attackers might attempt to deceive the model with tiny, imperceptible perturbations. For multi-step iterative attack methods, such as PGD and BIM, a single-step perturbation limit is set. This parameter defines the perturbation size allowed in each iteration, ensuring the attack is gradual rather than applying a large perturbation all at once. Furthermore, the number of iterations is uniformly set to 7, representing the number of iterations the attacker performs while attempting to deceive the model. By fixing the single-step perturbation limit and the number of iterations, fair comparisons can be made between different attack methods and models.

[0124] Step 3: Conduct comparative experiments and analyze the results.

[0125] A ResNet-18 model was trained on the SVHN dataset as the target black-box model, and VGG-16 was used as a substitute model during distillation.

[0126] In the dynamic distillation method with random perturbation, the noise mean was set to 0 and the variance to 8 / 255. In the dynamic distillation method with adversarial perturbation, the parameters for PGD and FGSM were the attack parameters from the "Experimental Environment Configuration". Table 1 details the attack success rates under various attack modes on the ResNet-18 model / SVHN dataset. Here, the attack model refers to the adversarially trained defense model.

[0127] Experimental results show that white-box attack methods achieved high success rates on the models, with the PGD attack method achieving a 46.31% success rate on the ResNet-18 target model. However, when transferred to black-box attack scenarios, the success rate of traditional static distillation decreased significantly. For example, on the ResNet-18 model, the best attack success rate of static distillation was 23.79%, far lower than the best attack success rate of 52.01% for white-box attacks. The gradient alignment adversarial distillation proposed in this invention considers fixed sample output, dynamic sample output, and first-order gradient output, ensuring consistency between the replacement model and the target black-box model in these aspects. This significantly improves the performance of the transfer attack, achieving the best attack effect compared to baseline methods. On the ResNet-18 model, the best attack success rate of gradient alignment adversarial distillation reached 40.29%, far exceeding the best attack success rate of 23.79% for traditional static distillation. Overall, the average attack success rate of the gradient alignment adversarial distillation proposed in this invention is improved by 6.7%, 9.0%, 37%, and 74% respectively compared with the dynamic distillation methods of PGD adversarial perturbation, FGSM adversarial perturbation, dynamic distillation of random perturbation, and static distillation, demonstrating the significant advantages of the method proposed in this invention.

[0128] Table 1. Comparison of black-box attack success rates (%) under different distillation methods (ResNet-18 model / SVHN). The optimal index under different attack modes is marked in bold.

[0129]

[0130] Example 2: Benefits of a Gradient Alignment-Based Adversarial Distillation-Based Black-Box Attack Method for Model Adversarial Defense

[0131] The experimental setup in this embodiment is basically the same as that in Embodiment 1, including the selection of datasets, the selection of models, and the setting of adversarial attack parameters.

[0132] Adversarial training is a mainstream method for improving the adversarial robustness of models. Its core idea is to introduce adversarial examples during training, enabling the model to learn attack patterns and thus enhance its ability to identify and defend against adversarial attacks. Existing adversarial training methods mostly generate adversarial examples based on PGD white-box attack methods, which presents a trade-off between improved robustness accuracy (accuracy on adversarial examples) and decreased natural accuracy (accuracy on clean examples). The lack of diversity in the generated adversarial examples leads to overfitting on these examples, a major reason for the decline in natural accuracy. Furthermore, models trained adversarially typically do not perform well enough against black-box attacks and require further improvement.

[0133] White-box attacks, which directly utilize the model's structure and gradient information, typically find vulnerable regions quickly along the steepest loss direction. While efficient, they are prone to getting trapped in local optima, resulting in a concentrated and undiversified distribution of generated adversarial examples. Black-box attacks, on the other hand, cannot access the model's internal information and often require searching for perturbations in a broader input space, leading to a more diverse range of generated adversarial examples.

[0134] The following shows the distribution of white-box and black-box adversarial examples. First, the high-level features of the adversarial examples are extracted by the model. Then, these high-dimensional features are compressed and mapped to a two-dimensional space using dimensionality reduction techniques (such as t-SNE). Finally, their two-dimensional probability density map is plotted. Figure 3 This paper presents the distribution of white-box and black-box adversarial examples for the ResNet-18 model (SVHN data). The white-box attack methods used include FGSM, PGD, BIM, NI-FGSM, VNI-FGSM, and PI-FGSM++. White-box adversarial examples are obtained by directly calculating gradients on the corresponding models using these methods, while black-box adversarial examples are calculated on alternative models (obtained through distillation training) using these methods. Specifically, the ResNet-18 model uses VGG16 as an alternative model. The figures show that white-box adversarial examples often exhibit multimodal and skewed distributions, tending to concentrate in specific perturbation directions, thus falling into local patterns. In contrast, black-box adversarial examples have a more uniform and symmetrical distribution, demonstrating higher diversity and wider coverage.

[0135] Step 1, Traditional Model Training

[0136] The SVNH samples were divided into training and testing datasets, with 73,257 images in the training dataset and 26,032 images in the testing dataset. The model was trained using ResNet18 as the architecture, employing cross-entropy loss as the loss function. Model parameters were adjusted upon convergence of the loss value. save.

[0137] Step 2, Adversarial Training of the Model Based on White-Box Attacks

[0138] Step 2.1, Generation of white-box adversarial examples

[0139] Adversarial examples are generated using the PGD method, specifically, given the original input sample. and tags and model Adversarial examples are generated for model iteration in the following manner. :

[0140]

[0141] Step 2.2, Adversarial Training Process

[0142] Adversarial training is an iterative process; during iterative training, the model... The above utilizes white-box adversarial attack methods (such as PGD) to generate white-box adversarial examples. The training process is completed, and the final update is based on the combined loss function applied to clean samples and white-box adversarial examples. ,get The specific adversarial training process is as follows: The trained model parameters... Save it.

[0143]

[0144] Step 3: Fine-tuning the model based on black-box attacks

[0145] Step 3.1, Generation of black-box adversarial examples

[0146] First, a model is given. The gradient alignment adversarial distillation method of this invention is used to generate alternative models. Then, the PGD method is used to randomly select 80% of the training dataset, given the original input samples. and tags , for the model Generate adversarial examples (as follows), at this time It is a model The black-box adversarial examples were used. Finally, a fine-tuning dataset was composed of 80% black-box adversarial examples and 20% original clean examples. .

[0147]

[0148] Step 3.2, Model Fine-tuning

[0149] The model obtained from adversarial training Based on this, fine-tuning the dataset As training data, for the model Fine-tuning training is performed, with 2-3 iterations, to obtain the fine-tuned model. Save model parameters .

[0150] Step 4: Model performance testing and comparative analysis

[0151] Step 4.1 Model Robustness Evaluation Method

[0152] In the experiments, FGSM and PGD, representative white-box attack methods, were used to evaluate the model's robustness under known gradient conditions; Square was used for black-box attack to simulate real-world attack scenarios where the model's internal information is not visible. This combination covers typical white-box and black-box threats, ensuring the comprehensiveness of the model's robustness assessment.

[0153] Step 4.2 Model Comprehensive Evaluation Results

[0154] On the same test dataset, the standard training model obtained in step 1 was tested. Step 2: Adversarial training model obtained based on white-box adversarial examples And the model obtained by fine-tuning training based on black-box adversarial examples in step 3. A comprehensive performance evaluation was conducted, with test samples including clean samples and adversarial samples. The accuracy on the clean test samples is called natural accuracy, and the accuracy on adversarial samples generated by different attack methods is called robust accuracy. The test results are shown in Table 2. It can be seen that after fine-tuning training with black-box adversarial samples, both the natural accuracy and robust accuracy of the model are improved compared to adversarial training. In particular, when facing black-box attacks, the model accuracy is improved by 6%, verifying the effectiveness and feasibility of black-box attack methods in further improving the model's defense capabilities.

[0155] Table 2 Comparison of Model Performance Indicators

[0156]

[0157] The above description is merely a description of preferred embodiments of this application and is not intended to limit the scope of this application in any way. Any changes or modifications made by those skilled in the art based on the above-disclosed technical content should be considered as equivalent and valid embodiments and fall within the scope of protection of the technical solution of this application.

Claims

1. A black-box attack method based on gradient alignment to counter distillation, characterized in that, Includes the following steps: Step 1: Design a sample perturbation generator that maximizes the gradient difference ; The sample perturbation generator Used for input image samples Generate small perturbations Fixed sample Plus disturbance Forming sample neighborhoods for alternative model training; By maximizing the gradient difference between the alternative model and the target black-box model in the sample neighborhood, the alternative model is guided to continuously improve its fitting ability to the target black-box model in the process of repairing these differences. Step 2: Design a gradient-aligned adversarial distillation modeling framework for the alternative model; A gradient alignment adversarial distillation method is introduced to generate an alternative model for the target black-box model; an alternative model is trained through distillation. This makes its output and target black box model The output is close to the sample neighborhood, while also making the gradient of the alternative model... gradient with the target black-box model Matching; Step 3: Alternately train alternative models and perturbation generator ; Disturbance generator The training objective is to generate perturbations that maximize the gradient difference between the surrogate model and the target black-box model; while the surrogate model The training objective is to minimize the gradient difference between the surrogate model and the target black-box model, achieving gradient consistency between the two models; therefore, the surrogate model... and perturbation generator The training process is a minimization-maximization problem, which is accomplished using adversarial training; the overall training objective is as follows: in, For the target black-box model, As an alternative model, For the first One input sample, For generator G pairs The resulting tiny perturbations For a fixed sample size; Step 4: Based on white-box attack methods and alternative models Complete the black box model The attack; Reading alternative models The model weight file is used to select a white-box attack method to perform an attack on the original dataset using an alternative model. Perform white-box attacks to obtain an adversarial example set; test the black-box model on the adversarial example dataset. The output is used to calculate the attack success rate. ; Step 1 is as follows: Step 1.1 Design the disturbance generator Network structure; The perturbation generator network perturbs the input samples Generate small perturbations The overall network is represented as: ,in, It is a generator network, and the network parameters are: The input is a fixed sample. The output is a tiny perturbation with the same size as the input. Step 1.2 Design the disturbance generator Loss function; In a fixed sample Approximate calculation of the target black box model in the neighborhood first gradient ； Similarly, alternative models The approximate first-order gradient on is ， The design strategy is to generate perturbations. To make the gradient difference between the two models Maximum; therefore, generator loss function The design is as follows: In addition to the gradient difference loss term in the first half of the formula, a perturbation amplitude constraint is added in the second half of the formula to ensure that the perturbation is small; where Let be the mean square error function. For the loss term coefficient, The perturbation threshold is specified by the user and used as a hyperparameter for model training.

2. The black-box attack method based on gradient alignment against distillation according to claim 1, characterized in that, The perturbation generator network adopts the UNet architecture design; The UNet network includes: an initial convolutional module, a downsampling layer, a ResNet block, an upsampling layer, and a final convolutional module; The initial convolutional module adopts a 5×5 convolutional layer plus a normalization layer design to convert the number of input sample channels into the number of feature channels; The downsampling layer consists of m layers, each being a 4×4 convolution with a stride of 2, and is superimposed with activation functions and normalization layers; by using multiple convolutions and activation functions, the feature map size is gradually reduced, thereby increasing the receptive field; The ResNet block consists of n residual blocks, with the number of channels remaining constant. The feature representation capability is enhanced through skip connections of multiple residual blocks. The upsampling layer consists of m layers, each of which is a 4×4 transposed convolution with a stride of 2, and is superimposed with activation functions and normalization layers. The feature map size is gradually restored through deconvolution. The final convolutional module uses a 5×5 convolution and selects Tanh as the activation function to limit the perturbation range, ultimately outputting perturbed samples.

3. The black-box attack method based on gradient alignment against distillation according to claim 1, characterized in that, Step 2 is as follows: Step 2.1: Design a gradient alignment adversarial distillation modeling framework; The modeling framework for gradient alignment adversarial distillation includes: a target model, an alternative model, and a perturbation generator; Among them, the disturbance generator Input Sample Generate small perturbations Thus, dynamic samples are synthesized: ; Target Model As a black-box model, for input samples and perturbation samples Its output is and ; Alternative Model For a trainable white-box model, for input samples and perturbation samples The output is and Used to fit the target black-box model The gradient; The target black box model is approximated using the finite difference method. and alternative models First gradient: ， Step 2.2: Design an alternative model Loss function; Alternative Model loss function Including: losses from the original mission Static sample distillation loss Dynamic sample distillation loss Matching loss with first-order gradient ; The specific explanation is as follows: (1) The original mission losses are as follows: in This represents the loss function corresponding to the black-box model task itself. Let y be the input sample and y be the sample label; (2) Static sample distillation loss is as follows: in, For the distance metric function of the difference, use the KL divergence of the model's final predicted probability or the L2 loss of the intermediate layer features; (3) The dynamic sample distillation loss is as follows: in, Indicates a fixed sample Tiny perturbations on the surface For the distance metric function of the difference, see (2); (4) The first-order gradient matching loss is as follows: The parameters have the same meaning as above; In summary, the alternative model loss function The definition is as follows: in, These are weights that control the original task loss, static samples, dynamic samples, and the degree of first-order gradient matching, respectively, and are used to balance the loss term.

4. The black-box attack method based on gradient alignment against distillation according to claim 1, characterized in that, Step 3 specifically involves: Step 3.1 Initialize the alternative model Initialize the perturbation generator Training dataset Target black box model Disturbance generator Loss weights in loss Alternative Model Loss weight of the Central Plains mission Static sample distillation weights Dynamic sample distillation weights Gradient weights Learning rate of the alternative model Learning rate of the perturbation generator ; Step 3.2 Train the perturbation generator; Fixed substitution model Training the perturbation generator Generate disturbance The perturbation is generated by maximizing the gradient difference between the surrogate model and the target black-box model through the loss function. Pointing in the direction where the gradient consistency between the two is weakest; Step 3.3 Train the alternative model; Fixed disturbance generator In a fixed sample neighborhood, an alternative model is trained using gradient alignment adversarial distillation. ; By minimizing the gradient difference through the loss function, the alternative model is made possible. Maximize the gradient information of the fitted target black-box model; Step 3.4 Training data for all batches Alternately execute steps 3.2 and 3.3 to complete the adversarial training; save the network weight file of the alternative model.

5. The black-box attack method based on gradient alignment against distillation according to claim 1, characterized in that, Step 4 specifically involves: Reading alternative models For the model weight file, choose a white-box attack method to attack the original dataset. In alternative models Perform white-box attacks to obtain adversarial sample sets. In adversarial sample datasets Test black-box model The output is used to calculate the attack success rate. : in, Indicates the number of samples; indicator function [ The symbol ] indicates a truth value; it is 1 if true and 0 otherwise. Represents the black box model For input Predicted labels.

Images