A generative transferable adversarial attack method based on additional prompt image

Abstract

The application provides a generative transferable adversarial attack method based on an additional prompt image, and belongs to the field of artificial intelligence security. Different guide images are flexibly used to improve the directed or non-directed transferable attack method. In addition to natural images, the application also introduces additional prompt images for improving the performance of the simplified generative attack method. The generative transferable adversarial attack method based on the additional prompt image has high transferability, and the generated adversarial samples have high adversarial attackability in directed attack and non-directed attack tasks. The method is efficient, and the generated adversarial samples can be directly used for cross-dataset and cross-model attacks, greatly reducing the computational resource consumption required for extensive robustness evaluation.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence security, and specifically relates to a generative transferable counter-attack method based on additional prompt images. Background Art

[0002] In recent years, deep neural networks (DNNs) have achieved remarkable success in various fields, including key areas such as computer vision and natural language processing. In computer vision, in particular, DNN models have been widely used in practical tasks such as image classification, object detection, and image generation. However, the inherent fragility of DNN models makes them vulnerable to adversarial examples, which are attacks that cause DNN models to make incorrect predictions by adding small and imperceptible adversarial perturbations to clean examples.

[0003] In addition, adversarial examples have been shown to be able to perturb the model in different architectures. [1] and data distribution [2] Transfer, that is, adversarial transferability. The core idea of ​​transferable adversarial attacks is to use a substitute model instead of the black-box model to be attacked. The attacker has full access to the substitute model, so the attacker uses a white-box attack on the substitute model to generate adversarial samples, and then uses this adversarial sample to attack the black-box model. Transferable adversarial attacks can be roughly divided into two main types: iterative methods and generative methods. Iterative attacks directly optimize the input space to generate adversarial samples, while generative attacks focus on first training a generative model to produce these samples. Compared with generative methods, iterative methods are usually more time-consuming and may result in poor adversarial transferability. Representative methods related to generative methods include BIA attacks [1] and CDA attacks [2] , which are representative methods of intermediate layer feature attack and output layer feature attack, respectively. Taking the intermediate layer feature method as an example, the optimization goal of the training phase can be expressed as follows:

[0004]

[0005] Among them, f l (x) and f l (x′) are the features of the intermediate layer (l) of the surrogate model obtained by taking the clean sample and the adversarial sample as the input of the surrogate model, and θ is the model parameter of the adversarial generator. The optimization goal is to maximize the cosine similarity of the intermediate layer features.

[0006] The output layer attack method aims to establish a similarity loss on the output distribution of the alternative model. For example, the CDA method optimizes the adversarial sample generator by establishing a cross entropy loss on the model output layer. The optimization objective can be expressed as follows:

[0007]

[0008] Furthermore, when an adversarial noise generator is trained on a specific target dataset or alternative model architecture, the perturbations it generates may overfit to that specific context. Existing methods have enhanced the transferability of adversarial examples from different perspectives. For example, some methods improve the transferability of attack methods by addressing the overfitting problem of generative models, but none of them truly improve the transferability of adversarial examples that are independent of model structure and data distribution.

[0009] [1] Zhang Q, Li X, Chen Y, et al.Beyond imagenet attack: Towards crafting adversarial examples for black-box domains[C] / / International Conference on Learning Representations.2022.

[0010] [2]Naseer MM, Khan SH, Khan MH, et al. Cross-domain transferability of adversarial perturbations[C] / / Advances in Neural Information ProcessingSystems.2019. Summary of the Invention

[0011] The present invention is made to solve the above problems, and its purpose is to provide a generative and transferable adversarial attack method based on additional prompt images, breaking through the single input limitation of the existing adversarial sample generator, and introducing additional prompt images to achieve semantically relevant transferability of adversarial samples.

[0012] The present invention provides a generative transferable adversarial attack method based on additional prompt images, which has the following characteristics: S10, initializing the adversarial generator G(·,·), θ g ) and the alternative model f(·,θ c ) and load the model parameters, where (·, ·) is the adversarial generator input, θ g is the adversarial generator model parameter, · is the clean sample image or guidance sample image input to the adversarial generator, θ c is the replacement model parameter; S20, randomly sample natural images x and guide images x from the training dataset g ; S30, natural image x and guide image x g Input into the adversarial generator to generate adversarial samples x advwhere the semantic injection process is completed by the adversarial generator performing a pixel-level feature affine transformation on the BatchNorm layer of each intermediate layer of its model, and the affine transformation parameters of the feature affine transformation are derived from the guide image x g S40, the input is subjected to a convolutional transformation to obtain an adversarial sample x adv S50, the optimization loss in step S40 is calculated and gradient backpropagation is performed to optimize the model parameters of the adversarial generator; and S60, steps S20-S50 are repeated until the model of the adversarial generator converges.

[0013] In the generative transferable adversarial attack method based on an additional prompt image provided by the present application, the adversarial generator can be further characterized in that in step S30, the adversarial generator is subjected to one data feedforward to generate an adversarial sample x adv .

[0014] In the generative transferable adversarial attack method based on an additional prompt image provided by the present application, the adversarial generator can be further characterized in that in step S30, the adversarial generator is subjected to one data feedforward to generate an adversarial sample x

[0015] x i g = Interp(x g ,w i ,h i )

[0016] α i = Conv(x i g )

[0017] β i = Conv(x i g )

[0018] f i ' = (1 + α i )f i + β i ,

[0019] In the above formula, Conv and Interp are convolution and interpolation calculations, respectively, x i g is the guide image input of the i-th layer affine transformation, w i is the width of the guide image input of the i-th layer affine transformation, h i is the height of the guide image input of the i-th layer affine transformation, α i is the i-th layer affine transformation learnable parameter, β i is the i-th layer affine transformation learnable parameter, and f iis the output of the i-th layer affine transformation, f i is the feature layer input of the i-th layer affine transformation.

[0020] In the method for generating a generative transferable adversarial attack based on an additional prompt image provided by the application, the method for standardization in step S40 can be as follows: the adversarial sample x adv Randomly crop and scale to unify the size.

[0021] In the method for generating a generative transferable adversarial attack based on an additional prompt image provided by the application, the method for generating a generative transferable adversarial attack based on an additional prompt image provided by the application can be as follows: in step S50, for the targeted attack method, similarity loss and contrast loss are respectively constructed on the intermediate layer features and the output layer features.

[0022] Effects of the application

[0023] According to the method for generating a generative transferable adversarial attack based on an additional prompt image provided by the application, on the one hand, the design of a new type of adversarial sample generator breaks through the single input limitation of the existing adversarial sample generator; on the other hand, an additional prompt image is introduced to realize the semantic correlation transferability of the adversarial sample. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a flowchart of the method for generating a generative transferable adversarial attack based on an additional prompt image in the embodiment of the application;

[0025] Figure 2 is a flowchart of the method for generating a generative transferable adversarial attack based on an additional prompt image in the embodiment of the application. DETAILED DESCRIPTION

[0026] In order to make the technical means, creative features, purposes and effects achieved by the application easy to understand, the following embodiments will be described in detail in combination with the drawings.

[0027] <EMBODIMENT>

[0028] Figure 1 is a flowchart of the method for generating a generative transferable adversarial attack based on an additional prompt image in the embodiment of the application; Figure 2 is a flowchart of the method for generating a generative transferable adversarial attack based on an additional prompt image in the embodiment of the application.

[0029] As shown in Figure 1 and 2 The embodiment provides a method for generating a generative transferable adversarial attack based on an additional prompt image, which comprises the following steps:

[0030] S10, initialize the adversarial generator G((·,·),θ g ) and the alternative model f(·,θ c ) and load the adversarial generator model framework, load the replacement model and its parameters, and then load the data into the computing device.

[0031] Among them, (·, ·) is the adversarial generator input, θ g is the adversarial generator model parameter, · is the clean sample image or guidance sample image input to the adversarial generator, θ c are replacement model parameters.

[0032] S20, randomly sample natural images x and guidance images x from the training dataset g and loaded into the computing device.

[0033] S30, computing the natural image x and the guide image x in the device g Input into the adversarial generator to generate adversarial samples x adv In this step, the adversarial generator performs a data feed forward to obtain the generated adversarial sample x adv .

[0034] Among them, the adversarial generator performs a pixel-level feature affine change in each BatchNorm layer in the middle of its model to complete the semantic injection process. The affine transformation parameters of the feature affine change are determined by the guide image x g The input is convolutionally transformed. The affine transformation parameters are learnable, and the semantic guidance injection process of the i-th layer can be expressed as:

[0035] x i g =Interp(x g ,w i ,h i )

[0036] α i =Conv(x i g )

[0037] β i =Conv(x i g )

[0038] f i '=(1+α i )f i +β i ,

[0039] In the above formula, Conv and Interp are convolution and interpolation calculations respectively, x ig is the guidance image input of the affine transformation of the i-th layer, w i is the width of the affine transformation input guidance image of the i-th layer, h i is the height of the affine transformation input guidance image of the i-th layer, α i is the learnable parameter of the affine transformation of the i-th layer, β i is the learnable parameter of the affine transformation of the i-th layer, f i ' is the affine transformation output of the i-th layer, f i It is the feature layer input of the i-th layer affine transformation.

[0040] S40, normalize the input data and convert the adversarial sample x adv Random cropping and scaling are performed to make the size uniform.

[0041] S50: Calculate the optimization loss from step S40. For targeted attack methods, this embodiment constructs a similarity loss and a contrast loss based on the intermediate and output layer features, respectively. After the loss calculation is complete, gradient backpropagation is performed to optimize the adversarial generator model parameters.

[0042] S60: Repeat steps S20 to S50 until the adversarial generator model converges. This entire process is an iterative process. By continuously optimizing the adversarial generator parameters until the model converges, the generated adversarial examples can maximize the impact on the model output without significantly changing the image content, thereby training and testing the robustness of deep learning models. Adversarial examples generated in this way can be applied to robustness assessments across various practical scenarios, including model structures and data distributions.

[0043] Functions and Effects of the Embodiments

[0044] This embodiment provides a generative transferable adversarial attack method based on additional prompt images, which introduces additional prompt images to improve the performance of the simplified generative attack method.

[0045] The adversarial samples generated by the generative transferable adversarial attack method based on additional prompt images in this embodiment have high adversarial aggressiveness in both targeted attack and non-targeted attack tasks.

[0046] The adversarial samples generated by the generative transferable adversarial attack method based on additional prompt images in this embodiment can be directly used for cross-dataset and cross-model attacks, greatly reducing the computing resource consumption required for extensive robustness evaluation.

[0047] This embodiment introduces additional prompt images to provide additional semantic information guidance for the adversarial generator, thereby avoiding the generator from overfitting and improving the adversarial transferability of adversarial samples across model structures and data distributions.

[0048] Those skilled in the art will appreciate that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A generative transferable adversarial attack method based on additional prompt images, characterized by: The following steps are involved: S10, initialize the adversarial generator G((·,·),θ g ) and the alternative model f(·,θ c ) and load the model parameters, where (·, ·) is the adversarial generator input, θ g is the adversarial generator model parameter, · is the clean sample image or guidance sample image input to the adversarial generator, θ c are the replacement model parameters; S20, randomly sample natural images x and guidance images x from the training dataset g ; S30, the natural image x and the guide image x g Input the adversarial generator to generate adversarial samples x adv , wherein the adversarial generator performs a pixel-level feature affine change in each BatchNorm layer in the middle of its model to complete the semantic injection process, and the affine transformation parameters of the feature affine change are determined by the guide image x g The input is convolution transformed; S40, the adversarial sample x adv Standardization is performed to make the size uniform; S50, calculating the optimization loss in step S40 and performing gradient backpropagation to optimize the model parameters of the adversarial generator; S60, repeat steps S20 to S50 until the model of the adversarial generator converges.

2. The generative transferable adversarial attack method based on additional prompt images according to claim 1, characterized in that: in, In step S30, the adversarial generator performs a data feed forward to generate the adversarial sample x adv .

3. The generative transferable adversarial attack method based on additional prompt images according to claim 1, characterized in that: in, In step S30, the affine transformation parameters are learnable. The i-th layer in the semantic injection process is: x i g =Interp(x g ,w i ,h i ) α i =Conv(x i g ) β i =Conv(x i g ) f i '=(1+a i )f i +b i , In the above formula, Conv and Interp are convolution and interpolation calculations respectively, x i g is the guidance image input of the affine transformation of the i-th layer, w i is the width of the affine transformation input guidance image of the i-th layer, h i is the height of the affine transformation input guidance image of the i-th layer, α i is the learnable parameter of the affine transformation of the i-th layer, β i is the learnable parameter of the affine transformation of the i-th layer, f i ' is the affine transformation output of the i-th layer, f i It is the feature layer input of the i-th layer affine transformation.

4. The generative transferable adversarial attack method based on additional prompt images according to claim 1, characterized in that: in, In step S40, the method of normalization is: adv Random cropping and scaling are performed to make the size uniform.

5. The generative transferable adversarial attack method based on additional prompt images according to claim 1, characterized in that: in, In step S50, for the targeted attack method, similarity loss and contrast loss are constructed on the intermediate layer features and output layer features respectively.

Images