Generative transferable anti-attack method based on additional prompt image

Abstract

The invention provides a generative transferable anti-attack method based on an additional prompt image, and belongs to the field of artificial intelligence security. According to the method, different guide images are flexibly utilized to improve a directional or non-directional transferable attack method, and for data input of an adversarial sample generator, besides a natural image, an additional prompt image is introduced to improve the performance of a simplified generative attack method. The additional prompt image-based generative transferable adversarial attack method disclosed by the invention has high mobility, and the generated adversarial sample has high adversarial aggressiveness in both directional attack tasks and non-directional attack tasks. The method is high in efficiency, the generated adversarial sample can be directly used for cross-dataset and cross-model attacks, and computing resource consumption needed by wide robustness evaluation is greatly reduced.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence security, and in particular relates to a generative transferable counterattack method based on additional prompt images. Background Art

[0002] In recent years, deep neural networks (DNNs) have achieved remarkable success in various fields, including key areas such as computer vision and natural language processing. In computer vision in particular, deep neural network models are widely used in practical tasks such as image classification, object detection, and image generation. However, the inherent fragility of deep neural network models makes them vulnerable to adversarial samples, which are samples that make deep neural network models make wrong predictions by adding tiny and imperceptible adversarial perturbations to clean samples.

[0003] In addition, adversarial examples have been shown to be able to perturb different models in different architectures. [1] and data distribution [2] Transfer, that is, adversarial transferability. The core idea of ​​transferable adversarial attacks is to use a substitute model to replace the black-box model to be attacked. The attacker has full access to the substitute model, so the attacker uses a white-box attack on the substitute model to generate adversarial samples, and then uses this adversarial sample to attack the black-box model. Transferable adversarial attacks can be roughly divided into two main types: iterative methods and generative methods. Iterative attacks directly optimize the input space to generate adversarial samples, while generative attacks focus on first training a generative model to produce these samples. Compared with generative methods, iterative methods are usually more time-consuming and may result in poor adversarial transferability. Representative methods related to generative methods include BIA attacks [1] and CDA attacks [2] , which are representative methods of intermediate layer feature attack and output layer feature attack respectively. Taking the intermediate layer feature method as an example, the optimization goal of the training phase can be expressed as follows:

[0004]

[0005] Among them, f l (x) and f l (x′) are the features of the intermediate layer (l) of the surrogate model obtained by taking the clean sample and the adversarial sample as the input of the surrogate model, and θ is the model parameter of the adversarial generator. The optimization goal is to maximize the cosine similarity of the intermediate layer features.

[0006] The output layer attack method aims to establish a similarity loss on the output distribution of the alternative model. For example, the CDA method optimizes the adversarial sample generator by establishing a cross entropy loss on the model output layer. The optimization objective can be expressed as follows:

[0007]

[0008] In addition, when the adversarial noise generator is trained on a specific target dataset or alternative model architecture, the perturbations it generates may overfit to that specific context. Existing methods have enhanced the transferability of adversarial samples from different perspectives, such as improving the transferability of attack methods by improving the overfitting problem of the generated model, but have not truly achieved the improvement of the transferability of adversarial samples that is unrelated to the model structure and data distribution.

[0009] [1] Zhang Q, Li X, Chen Y, et al.Beyond imagenet attack: Towards crafting adversarial examples for black-box domains[C] / / International Conference on Learning Representations.2022.

[0010] [2]Naseer MM, Khan SH, Khan MH, et al. Cross-domain transferability of adversarial perturbations[C] / / Advances in Neural Information ProcessingSystems.2019. Summary of the invention

[0011] The present invention is made to solve the above-mentioned problems, and its purpose is to provide a generative and transferable adversarial attack method based on additional prompt images, breaking through the single input limitation of the existing adversarial sample generator, and introducing additional prompt images to realize the semantic correlation transferability of adversarial samples.

[0012] The present invention provides a generative transferable adversarial attack method based on additional prompt images, which has the following characteristics, including the following steps: S10, initializing the adversarial generator G(·, ·), θ g ) and the alternative model f(·,θ c ) and load the model parameters, where (·, ·) is the adversarial generator input, θ g is the model parameter of the adversarial generator, · is the clean sample image or guided sample image input to the adversarial generator, θ c is the replacement model parameter; S20, randomly sample natural images x and guide images x from the training dataset g ; S30, natural image x and guide image x g Input into the adversarial generator to generate adversarial samples x adv, where the adversarial generator performs a pixel-level feature affine change in each BatchNorm layer in the middle of its model to complete the semantic injection process. The affine transformation parameters of the feature affine change are determined by the guide image x g The input is convolutionally transformed; S40, the adversarial sample x adv Perform standardization to make the size uniform; S50, calculate the optimization loss in step S40 and perform gradient backpropagation to optimize the model parameters of the adversarial generator; S60, repeat steps S20 to S50 until the model of the adversarial generator converges.

[0013] The generative transferable adversarial attack method based on additional prompt images provided by the present invention may also have the following features: wherein, in step S30, the adversarial generator performs a data feedforward to generate an adversarial sample x adv .

[0014] The generative transferable adversarial attack method based on additional prompt images provided by the present invention may also have the following features: wherein, in step S30, the affine transformation parameters are learnable, and the i-th layer in the semantic injection process is:

[0015] x i g =Interp(x g ,w i ,h i )

[0016] α i =Conv(x i g )

[0017] β i =Conv(x i g )

[0018] f i '=(1+α i )f i +β i ,

[0019] In the above formula, Conv and Interp are convolution and interpolation calculations respectively, x i g is the guidance image input of the affine transformation of the i-th layer, w i is the width of the affine transformation input guidance image of the i-th layer, h i is the height of the input guidance image of the i-th layer affine transformation, α i is the learnable parameter of the affine transformation of the i-th layer, β i is the learnable parameter of the affine transformation of the i-th layer, f i' is the affine transformation output of the i-th layer, f i It is the feature layer input of the i-th layer affine transformation.

[0020] The generative transferable adversarial attack method based on additional prompt images provided by the present invention may also have the following features: wherein, in step S40, the method of normalization processing is: adv Random cropping and scaling are performed to make the size uniform.

[0021] The generative transferable adversarial attack method based on additional prompt images provided by the present invention may also have the following features: wherein, in step S50, for the targeted attack method, a similarity loss and a contrast loss are constructed on the intermediate layer features and the output layer features, respectively.

[0022] Functions and Effects of the Invention

[0023] According to the generative transferable adversarial attack method based on additional prompt images involved in the present invention, on the one hand, a new adversarial sample generator is designed to break through the single input limitation of the existing adversarial sample generator; on the other hand, additional prompt images are introduced to realize the semantic relevance transferability of adversarial samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flowchart of a generative transferable anti-attack method based on additional prompt images in an embodiment of the present invention;

[0025] Figure 2 It is a flowchart of a generative transferable anti-attack method based on additional prompt images in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following embodiments and the accompanying drawings specifically illustrate a generative and transferable anti-attack method based on additional prompt images of the present invention.

[0027] <Example>

[0028] Figure 1 is a flowchart of a generative transferable anti-attack method based on additional prompt images in an embodiment of the present invention; Figure 2 It is a flowchart of a generative transferable anti-attack method based on additional prompt images in an embodiment of the present invention.

[0029] like Figure 1 and 2 As shown, this embodiment provides a generative transferable anti-attack method based on additional prompt images, comprising the following steps:

[0030] S10, initialize the adversarial generator G((·,·),θ g ) and the alternative model f(·,θ c ) and load the adversarial generator model framework, load the alternative model and its parameters, and then load the data into the computing device.

[0031] Among them, (·, ·) is the adversarial generator input, θ g is the model parameter of the adversarial generator, · is the clean sample image or guided sample image input to the adversarial generator, θ c are alternative model parameters.

[0032] S20, randomly sample natural images x and guidance images x from the training dataset g and loaded into the computing device.

[0033] S30, computing the natural image x and the guide image x in the device g Input into the adversarial generator to generate adversarial samples x adv In this step, the adversarial generator performs a data feed forward to obtain the generated adversarial sample x adv .

[0034] The adversarial generator performs a pixel-level feature affine change in each BatchNorm layer in the middle of its model to complete the semantic injection process. The affine transformation parameters of the feature affine change are determined by the guidance image x g The input is convolutionally transformed. The affine transformation parameters are learnable, and the semantic guidance injection process of the i-th layer can be expressed as:

[0035] x i g =Interp(x g ,w i ,h i )

[0036] α i =Conv(x i g )

[0037] β i =Conv(x i g )

[0038] f i '=(1+α i )f i +β i ,

[0039] In the above formula, Conv and Interp are convolution and interpolation calculations respectively, x ig is the guidance image input of the affine transformation of the i-th layer, w i is the width of the affine transformation input guidance image of the i-th layer, h i is the height of the input guidance image of the i-th layer affine transformation, α i is the learnable parameter of the affine transformation of the i-th layer, β i is the learnable parameter of the affine transformation of the i-th layer, f i ' is the affine transformation output of the i-th layer, f i It is the feature layer input of the i-th layer affine transformation.

[0040] S40, standardize the input data and convert the adversarial sample x adv Random cropping and scaling are performed to make the size uniform.

[0041] S50, calculating the optimization loss in step S40. For the targeted attack method, this embodiment constructs similarity loss and contrast loss on the intermediate layer features and output layer features respectively. After the loss calculation is completed, the gradient is returned and the adversarial generator model parameters are optimized.

[0042] S60, repeat steps S20 to S50 until the model of the adversarial generator converges. The whole process is an iterative process. By continuously optimizing the adversarial generator parameters until the model converges, the generated adversarial samples can maximize the impact on the output of the model without significantly changing the image content, so as to train and test the robustness of the deep learning model. The adversarial samples generated in this way can be applied to the robustness evaluation of various practical scenarios such as cross-model structure and data distribution.

[0043] Functions and Effects of the Embodiments

[0044] This embodiment provides a generative transferable adversarial attack method based on additional prompt images, which introduces additional prompt images to improve the performance of the simplified generative attack method.

[0045] The adversarial samples generated by the generative transferable adversarial attack method based on additional prompt images in this embodiment have high adversarial aggressiveness in both targeted attack and non-targeted attack tasks.

[0046] The adversarial samples generated by the generative transferable adversarial attack method based on additional prompt images in this embodiment can be directly used for cross-dataset and cross-model attacks, greatly reducing the computing resource consumption required for extensive robustness evaluation.

[0047] This embodiment can provide additional semantic information guidance for the adversarial generator by introducing additional prompt images, thereby avoiding the situation of overfitting of the generator, and further improving the adversarial transferability of adversarial samples across model structures and data distributions.

[0048] Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A generative transferable adversarial attack method based on additional prompt images, characterized in that: The following steps are involved: S10, initialize the adversarial generator G((·,·),θ g ) and the alternative model f(·,θ c ) and load the model parameters, where (·, ·) is the adversarial generator input, θ g is the model parameter of the adversarial generator, · is the clean sample image or guided sample image input to the adversarial generator, θ c are alternative model parameters; S20, randomly sample natural images x and guidance images x from the training dataset g ; S30, combining the natural image x and the guide image x g Input into the adversarial generator to generate adversarial samples x adv , wherein the adversarial generator performs a pixel-level feature affine change in each BatchNorm layer in the middle of its model to complete the semantic injection process, and the affine transformation parameters of the feature affine change are determined by the guide image x g The input is transformed by convolution; S40, the adversarial sample x adv Standardization is performed to make the size uniform; S50, calculating the optimization loss in step S40 and performing gradient backpropagation, thereby optimizing the model parameters of the adversarial generator; S60, repeat steps S20 to S50 until the model of the adversarial generator converges.

2. The generative transferable adversarial attack method based on additional prompt images according to claim 1, characterized in that: in, In step S30, the adversarial generator performs a data feedforward to generate the adversarial sample x adv .

3. The generative transferable adversarial attack method based on additional prompt images according to claim 1, characterized in that: in, In step S30, the affine transformation parameters are learnable. The i-th layer in the semantic injection process is: x i g =Interp(x g ,w i ,h i ) α i =Conv(x i g ) β i =Conv(x i g ) f i '=(1+a i )f i +b i , In the above formula, Conv and Interp are convolution and interpolation calculations respectively, x i g is the guiding image input of the affine transformation of the i-th layer, w i is the width of the affine transformation input guidance image of the i-th layer, h i is the height of the input guidance image of the i-th layer affine transformation, α i is the learnable parameter of the affine transformation of the i-th layer, β i is the learnable parameter of the affine transformation of the i-th layer, f i ' is the affine transformation output of the i-th layer, f i It is the feature layer input of the i-th layer affine transformation.

4. The generative transferable adversarial attack method based on additional prompt images according to claim 1, characterized in that: in, In step S40, the method of the standardization process is: adv Random cropping and scaling are performed to make the size uniform.

5. The generative transferable adversarial attack method based on additional prompt images according to claim 1, characterized in that: in, In step S50, for the targeted attack method, a similarity loss and a contrast loss are constructed on the intermediate layer features and the output layer features respectively.

Images