A combined black-box attack method for detecting robustness of a deep learning model

Abstract

The application provides a combined black box attack method for detecting robustness of a deep learning model, comprising the following steps: S1: parameterized modeling by a generative model: a generative model is trained to directly generate an adversarial perturbation, and historical query information of a target model is fused by parameterized modeling to improve the efficiency of subsequent attacks on the target model; S2: improving the efficiency of query attack by using the output of the generative model; S3: improving the attack efficiency by fine-tuning the substitute model and the generative model: the similarity between the substitute model and the target model is improved by fine-tuning the substitute model to imitate the output result of the target model. The application considers that the query attack is a continuous process, and the efficiency of subsequent query attacks is enhanced by using historical query information, that is, the attack efficiency on the target model is improved by using previous attack sample information, the attack success rate is improved, and the query cost is reduced, so that the robustness of the current deep learning model can be better detected.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a combined black-box attack method for detecting the robustness of deep learning models. Background Technology

[0002] The goal of adversarial example generation tasks is to add covert and malicious perturbations to clean samples, deceiving deep learning models without being detected by the human eye. Based on the accessible information of the attacked model, existing adversarial attacks can generally be divided into white-box and black-box attacks. This aims to test the robustness of current deep learning models and can be used for model validation and testing. It helps improve the security and versatility of models in real-world deployments.

[0003] In white-box attack tasks, it is assumed that the attacker has access to the parameters and structural information of the target model, and therefore can construct adversarial perturbations based on the gradient information of the target model. In black-box attack tasks, the target model is unknown to the attacker; the attacker can only obtain the output feedback of the target model. Compared to white-box scenarios, black-box scenarios are more similar to real-world scenarios and are more challenging.

[0004] Existing methods for generating black-box adversarial examples can generally be divided into three categories: migration attacks, query attacks, and combination attacks.

[0005] 1) Transfer attacks exploit the structural similarity between deep learning models to construct adversarial examples based on alternative models and then attack the target model. They are typically based on the transferability of the models. Model transferability refers to the fact that due to the structural similarity of deep learning models, adversarial examples generated based on one model also have a certain degree of obfuscation for other deep learning models. This means that adversarial examples can be generated based on local alternative models and then transferred to the target model for attack. Strategies such as model ensembles and momentum gradient updates can further improve the transferability of the samples. However, when the structural differences between models are significant, the success rate of transfer attacks drops significantly, resulting in a low attack success rate.

[0006] 2) Query-based attacks iteratively query the target model and update adversarial samples based on feedback information, thereby obfuscating the target model. By utilizing multiple feedback results from the target model, adversarial samples are iteratively adjusted to ultimately obfuscate the target model. Compared to transfer attacks, query-based attacks significantly improve the success rate, but require additional query costs and computational overhead. Query-based attacks can be further classified into gradient estimation-based methods and stochastic search-based methods based on their update strategies. The former estimates the zero-order gradient of the sample in the model space based on query information to update the adversarial perturbation; the latter searches around the model's decision boundary, retaining the search path if it optimizes the objective function, otherwise abandoning it. These query-based attacks suffer from low query efficiency and high computational costs.

[0007] 3) Combination attacks utilize the advantages of both migration attacks and query attacks, combining different strategies to improve the efficiency of adversarial sample generation.

[0008] Therefore, existing methods for detecting robustness suffer from low success rates or high query costs. Summary of the Invention

[0009] The purpose of this invention is to solve the problem of improving the success rate of robustness detection while reducing query costs, and to provide a combined black-box attack method for detecting the robustness of deep learning models.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] A combined black-box attack method for detecting the robustness of deep learning models includes the following steps:

[0012] S1: Parametric modeling through generative models: Generative models are trained to directly generate adversarial perturbations. Parametric modeling is used to fuse historical query information of the target model to improve the efficiency of subsequent attacks on the target model.

[0013] S2: Improve the efficiency of query attacks by leveraging the output of generative models;

[0014] S3: Fine-tuning the substitute and generative models to improve attack efficiency: By fine-tuning the substitute model to make its output mimic the output of the target model, the similarity between the substitute model and the target model is improved.

[0015] In some embodiments, step S1 includes:

[0016] S11: Input clean image samples into the generation model as conditional parameters of the model;

[0017] S12: Generate the model based on the adversarial perturbation corresponding to the current input and output. This perturbation is within a given norm range, making it consistent with the actual scene and imperceptible to the human eye.

[0018] S13: Add the generated perturbation to the original clean image sample to obtain the adversarial sample;

[0019] S14: Input the adversarial examples into the alternative model, which is a local model that tests the current task.

[0020] S15: Calculate the adversarial loss function based on the generated results, and optimize the parameters of the generated model using gradient descent based on the adversarial loss function.

[0021] In some embodiments, the norm in step S12 is the L2 norm or the Linf norm.

[0022] In some embodiments, the model described in step S14 is pre-trained, and this step also obtains the parameters and gradient information of the model, which outputs a predicted result based on the input adversarial examples.

[0023] In some embodiments, step S15 is further included: calculating an adversarial loss function based on the generation result, and optimizing the parameters of the generation model using gradient descent based on the adversarial loss function.

[0024] In some embodiments, in step S2, during the attack phase, the generative model obtained in the first phase is used with the current sample as a condition to generate the initial value or the mean of the initial distribution of the adversarial action as the downstream query attack model, and the query attacker queries the model to generate the final adversarial sample in an iterative update manner.

[0025] In some embodiments, step S2 includes:

[0026] T1: Use the obtained adversarial samples as the initial values ​​for the downstream query-based black-box attacker, which significantly improves the attack efficiency of the attacker.

[0027] T2: The query attack repeatedly accesses the black-box target model and adjusts the current adversarial sample based on the target model's output until the target model gives an incorrect result.

[0028] In some embodiments, in step T2, each access process is for the attacker to generate or sample an adversarial sample as input to the target model, and the target model gives the corresponding output.

[0029] In some embodiments, step S3 includes:

[0030] E1: Use clean image samples as input to the surrogate model and the target model to obtain the output of the surrogate model and the output of the target model, respectively;

[0031] E2: Calculate the difference between the two outputs as the loss and update the parameters of the alternative model using gradient descent to obtain the updated alternative model.

[0032] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the above-described combined black-box attack method for detecting the robustness of deep learning models.

[0033] The present invention has the following beneficial effects:

[0034] This invention takes into account that query attacks are a continuous process and uses historical query information to enhance the efficiency of subsequent query attacks. That is, by using information from previous attack samples to improve the attack efficiency on the target model, the success rate of the attack can be increased and the query cost can be reduced, thereby enabling better detection of the robustness of current deep learning models. Attached Figure Description

[0035] Figure 1 This is a flowchart of a combined black-box attack method for detecting the robustness of deep learning models in an embodiment of the present invention;

[0036] Figure 2 This is a schematic diagram of the first stage in an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of the second stage in an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of the expected substitution model adjustment parameters in the third stage of this embodiment of the invention;

[0039] Figure 5 This is a schematic diagram illustrating the injection of information from the target model into the generated model in the third stage of an embodiment of the present invention. Detailed Implementation

[0040] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0041] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0042] Example

[0043] The combined black-box attack method for detecting the robustness of deep learning models in this invention is implemented through an efficient combined black-box attack method, such as... Figure 1 As shown, it includes the following steps:

[0044] Phase 1 (parametric modeling via generative models): such as Figure 2 As shown, this stage directly generates adversarial perturbations by training a generative model. Parameterized modeling can fuse historical query information from the target model to improve the efficiency of subsequent attacks on the target model. During training, to save on query costs for the target model, alternative deep models can be used for training, leveraging the structural similarity of deep models. Specifically, the training method can utilize the model's adversarial loss function to optimize the generative model based on gradient descent, making the output of the generative model approximate the distribution of adversarial examples. Figure 2 The alternative model in the text is filled with blanks, indicating an unadjusted alternative model. Specifically, it includes the following steps:

[0045] S11: Input clean image samples into the generative model as conditional parameters of the model; where clean refers to the original image, as opposed to the adversarial sample after adding adversarial noise, that is, the original classification model can correctly classify and identify the correct sample.

[0046] S12: The generative model generates adversarial perturbations based on the current input and output. These perturbations are within a given norm range, with the aim of making the perturbations consistent with the real-world scenario and imperceptible to the human eye. In some embodiments, the norm can be the L2 norm or the Linf norm; in this embodiment, the Linf norm is selected, and the original value range of the image is 0 to 255. The perturbation is between -0.05*255 and 0.05*255, which is between -12 and 13.

[0047] S13: Add the generated perturbation to the original clean image sample to obtain the adversarial sample;

[0048] S14: Input the adversarial examples into the alternative model, which is a local model that tests the current task. This model is pre-trained, and we can obtain the parameters and gradient information of the model. The model outputs the predicted results based on the input adversarial examples.

[0049] S15: Calculate the adversarial loss function based on the generated results, and optimize the parameters of the generated model using gradient descent based on the adversarial loss function;

[0050] S16: Iterate from S11 to S15 to complete the training of the generative model.

[0051] The second stage (using the output of generative models to improve the efficiency of query-based attacks): such as Figure 3As shown, in the attack phase, the generative model obtained in the first phase is used with the current sample as a condition to generate adversarial perturbations as the initial values ​​or the mean of the initial distribution of the downstream query-based attack model. The query-based attacker then queries the model to generate the final adversarial sample through iterative updates. Since the output perturbation of the generative model is close to the final adversarial sample, it can improve the optimization efficiency of the attacker and reduce the query cost. In the actual attack phase, the target model is the model we want to test the attack in this scenario. We expect the adversarial sample generated by the generative model to confuse the model and cause it to give incorrect prediction results.

[0052] The specific steps include:

[0053] T0: Input a clean image sample into the generator model as a conditional parameter; the generator model outputs an adversarial perturbation corresponding to the current input, which is within a given norm; the generated perturbation is added to the original clean image sample to obtain the adversarial sample.

[0054] T1: The obtained adversarial samples are used as the initial values ​​for the downstream query-based black-box attacker, which helps the attacker significantly improve its attack efficiency.

[0055] T2: During the query, the attacker accesses the black-box target model multiple times. (Each access process generates or samples an adversarial sample, which is used as input to the target model, and the target model provides the corresponding output.) The attacker adjusts the current adversarial sample based on the output of the target model until the target model gives an incorrect result.

[0056] The third stage (fine-tuning the substitute model and the generative model to improve attack efficiency): Since the historical attack samples in this embodiment obtained information about the target model, this information can be used to further improve the efficiency of our attack process. By fine-tuning the substitute model to make its output mimic the output of the target model, the similarity between the substitute model and the target model is improved. Then, the generative model is fine-tuned using the same method as training, allowing the generative model to acquire more information about the target model. At this point, the adversarial perturbations generated by the generative model are more helpful for downstream attacks, further improving the attack efficiency of subsequent query-based attacks. This stage means that the substitute model is expected to adjust its parameters based on the input and output of the target model (wherein the input and output are cached from the input and output of the second-stage attack process), so that the substitute model gradually approaches the target model. That is, for the same input, the substitute model can give an output close to that of the target model, but the substitute model is a white-box model that can provide gradient update information, while the target model is a black-box model that cannot obtain gradient information. (Reference) Figure 4 , Figure 5 : Figure 4This means that the local surrogate model is expected to simulate the output of the black-box target model, and the parameters of the surrogate model are adjusted according to the input and output of the target model. Figure 5 The alternative model in the code is filled with dots, representing the updated alternative model. The specific steps include:

[0057] E1: Using clean image samples as input to the surrogate model and the target model, we obtain the outputs of the surrogate model and the target model, respectively.

[0058] E2: Calculate the difference between the two outputs as the loss and update the parameters of the alternative model using gradient descent to obtain the updated alternative model.

[0059] Figure 4 This can be expressed as follows: We have obtained an updated alternative model, which means we have obtained some information from the target model. We hope to inject this information back into the generative model, so we perform the first stage of training. The difference is that the alternative model here is the updated alternative model.

[0060] Experimental Example

[0061] This experiment uses image classification as the task, employing a ResNet-50 model for the target classification and a VGG-16 model for the surrogate classification. The task involves predicting the category of an image given a given image, such as grass or sky. Both the target and surrogate models return a vector representing the predicted probabilities of different categories during the prediction process. The image can be any type of image. A Generative Adversarial Network (GAN) can be used as the generative model.

[0062] Phase 1: Training Phase

[0063] A training set needs to be constructed using image samples, and then a generative adversarial model (GAN) is trained. An image x is input into the generative model to obtain noise zx, which is then added to the image to obtain adversarial examples x'. These adversarial examples are then input into the surrogate model VGG-16, which predicts the probabilities of a set of vectors. The probability value of the true label t of image x is selected as the loss, and the parameters of the generative model can then be optimized using the Adam optimizer. This process is repeated multiple times to complete the training of the surrogate model.

[0064] Phase Two: Attack Phase

[0065] The given image sample y is input into the trained generative model GAN ​​to obtain noise zy. The noise is added to the image to obtain the adversarial sample y' of the current image. This adversarial sample is then passed to the downstream query-based black-box attacker. Here, the attacker can be the Square Attack attacker. The attacker iteratively queries the target model. The query process includes sampling new attack samples, inputting them into the target model ResNet-50, and updating the attack samples in a gradient-free manner based on the output of the target model, i.e., the probability of the predicted class, until the attack is successful.

[0066] Phase Three: Adjustment Phase

[0067] At this point, it is expected that the updated generative model GAN ​​will generate more aggressive adversarial examples against the current target model ResNet-50:

[0068] 1. First, we use the input y and output of the target model ResNet in the second stage to adjust the parameters of the alternative model VGG. We can select the difference between the predicted outputs of the two models as the loss and use the Adam optimizer for optimization.

[0069] 2. Repeat the first-stage process using the adjusted alternative model to update the parameters of the generated model.

[0070] Table 1 shows the attack results on the ImageNet dataset for image classification tasks. The target models are VGG-16, WRN-50, and Inception-V3, and the alternative model is ResNet-50. The first row shows the attack results for Square, and the second row shows the attack results for Square Attack after providing initial values ​​for the model. ASR represents the attack success rate, Mean represents the query mean, and Median represents the query median.

[0071] Table 1

[0072]

[0073] This invention proposes a combined black-box attack method for detecting the robustness of deep learning models, representing cutting-edge academic research. This method integrates commonly used query-based attack methods, significantly improving their attack efficiency and success rate, with an average increase of approximately 15%. It also reduces the number of accesses to the target model, decreasing query costs by an average of approximately 60%, which is significant for guiding the design of robust and secure machine learning systems. Furthermore, it can serve as a plugin to enhance the efficiency of other query-based attacks.

[0074] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0076] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0077] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0078] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A combined black-box attack method for detecting the robustness of deep learning models, characterized in that, Includes the following steps: S1: Parametric modeling through generative models: Generative models are trained to directly generate adversarial perturbations. Parametric modeling is used to fuse historical query information of the target model to improve the efficiency of subsequent attacks on the target model. S2: Improve the efficiency of query attacks by leveraging the output of generative models; S3: Fine-tuning the substitution and generation models to improve attack efficiency: By fine-tuning the substitution model to make its output mimic the output of the target model, the similarity between the substitution model and the target model is improved; Step S1 includes: S11: Input clean image samples into the generative model as conditional parameters for the generative model; S12: Generate the model based on the adversarial perturbation corresponding to the current input and output. This perturbation is within a given norm range, making it consistent with the actual scene and imperceptible to the human eye. S13: Add the generated perturbation to the original clean image sample to obtain the adversarial sample; S14: Input the adversarial examples into the alternative model, which is a local model that tests the current task; S15: Calculate the adversarial loss function based on the generated results, and optimize the parameters of the generated model using gradient descent based on the adversarial loss function; Step S2 includes: T1: Use the obtained adversarial samples as the initial value or the mean of the initial distribution of the downstream query-based black-box attacker, so that the downstream query-based black-box attacker can significantly improve the attack efficiency. T2: Downstream query-based black-box attackers access the black-box target model multiple times and adjust the current adversarial sample based on the target model's output until the target model gives an incorrect result. Step S3 includes: E1: Use clean image samples as input to the surrogate model and the target model to obtain the output of the surrogate model and the output of the target model, respectively; E2: Calculate the difference between the two outputs as the loss and update the parameters of the alternative model using gradient descent to obtain the updated alternative model; E3: Using the replacement model updated in step E2, repeat steps S11 to S15 to fine-tune the parameters of the generated model.

2. The combined black-box attack method for detecting the robustness of deep learning models as described in claim 1, characterized in that, The norm mentioned in step S12 is either the L2 norm or the Linf norm.

3. The combined black-box attack method for detecting the robustness of deep learning models as described in claim 1, characterized in that, The alternative model described in step S14 is pre-trained, and this step also obtains the parameters and gradient information of the model. The alternative model outputs the predicted result based on the input adversarial example.

4. The combined black-box attack method for detecting the robustness of deep learning models as described in claim 3, characterized in that, It also includes step S16: iterating from S11 to S15 to complete the training of the generative model.

5. The combined black-box attack method for detecting the robustness of deep learning models as described in claim 1, characterized in that, In step T2, each access process generates or samples an adversarial sample for the attacker, which serves as the input to the target model, and the target model outputs the corresponding sample.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program can implement the combined black-box attack method for detecting the robustness of deep learning models as described in any one of claims 1-5.

Images