A method for generating an adversarial sample with high mobility and weak perception

Abstract

This invention discloses a method for generating adversarial examples with high transferability and weak perceptibility. The method includes: first, inputting the original image into a diffusion model and a surrogate model respectively, obtaining reconstructed samples reconstructed by the diffusion model and guided samples generated by attacking the surrogate model using the PGD algorithm; then, performing discrete wavelet transform on the original samples, reconstructed samples, and guided samples, using only the decomposed low-frequency components for image restoration, obtaining three samples containing only low-frequency components; next, obtaining the L-th layer features of these three samples containing only low-frequency components in the surrogate model, constructing a triplet loss to achieve feature guidance; finally, generating adversarial examples with high transferability and weak perceptibility after a preset number of iterations. This invention generates adversarial examples with higher transferability and are difficult to detect.

Description

Technical Field

[0001] This invention relates to the field of deep learning image adversarial attacks, specifically to a method for generating adversarial examples with high transferability and weak perceptualization. Background Technology

[0002] In recent years, deep neural networks (DNNs) have achieved tremendous success in the field of computer vision. However, the security and reliability of neural networks still face significant challenges. Research shows that neural networks are extremely vulnerable to adversarial examples. Attackers can fool state-of-the-art deep neural networks by adding only small perturbations to clean samples. Moreover, these adversarial examples are transferable, meaning that adversarial examples generated on a proxy model can be used to attack other models. Therefore, it is necessary to discover as many vulnerabilities as possible in DNNs to improve their robustness and avoid their potential risks.

[0003] Since Szegedy et al. discovered the existence of adversarial examples in 2014, various attack algorithms have been proposed to study the vulnerabilities of DNNs. In a black-box environment, attackers only have query privileges over the target model, unable to access its structure, parameters, or other information. Transfer-based adversarial attacks can easily overcome these limitations because they only need to generate adversarial examples on a local proxy model to successfully mislead the target model. Based on L... p Norm-constrained attacks are currently the mainstream approach. In 2021, Wang et al. proposed disrupting key features of model decision-making, guiding adversarial example searches towards disrupting these key features to achieve stronger transferability. In 2023, Li et al. introduced the idea of ​​model ensemble, incorporating probabilistic measures into the weights and biases of the proxy model to obtain an infinite set of DNNs from a single training iteration. Adversarial examples were then created on this model distribution to obtain highly transferable adversarial examples. On the other hand, unconstrained adversarial attacks have also gained widespread attention. These attacks typically modify images but generate adversarial examples that are imperceptible to humans. In 2022, Yuan et al. sampled image colors from public datasets to construct a color distribution library. Guided by this library, they optimized adversarial examples through neighborhood search and initialization reset. However, research shows that L... p Norm-constrained adversarial examples exhibit good transferability, but their perturbations are very obvious, and their noise is extremely similar to high-frequency noise. This not only hinders the transfer of adversarial examples to other black-box models but also makes them easily sanitized and defended against. Conversely, unconstrained attacks generate perturbations that are more concentrated in regions with high-level semantics, enhancing their stealth, but their transferability lags far behind constrained attack methods. Therefore, balancing the high transferability and imperceptibility of adversarial examples has become a challenge. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by proposing a highly transferable and weakly perceptual adversarial example generation method. It aims to enhance the transferability of adversarial examples by applying triplet loss constraints to feature guidance, ensuring that the features of the adversarial examples are far removed from the original classification region. Furthermore, this invention restricts the attack perturbation to the low-frequency component space, concentrating the perturbation on the semantic features of the image, enhancing the stealth of the attack, while maintaining the visual consistency of the image, making the adversarial examples more difficult to detect.

[0005] The technical solution adopted in this invention includes the following steps:

[0006] S1. Transfer the original image x clean The input is fed into a stable diffusion model, and the encoder converts x... clean z0 is obtained by mapping from the image space to the latent space, and z0 is obtained by forward diffusion of the latent variable z0. t In the back diffusion stage, the Unet model is used to analyze z. t Denoising is performed to obtain z′, and z′ is decoded to obtain x′;

[0007] S2. Let x′, x clean x guide Discrete wavelet transform is performed on three samples, where x guide This is an adversarial sample generated by attacking a proxy model using the PGD algorithm; each sample is decomposed into one low-frequency component and three high-frequency components, namely x′. ll 、x′ h 、x′ hl 、x′ hh , and The high-frequency components of all three samples were discarded, and only the low-frequency component x′ was used. ll , Image restoration was performed, and the following results were obtained:

[0008] S3. Utilize and right Feature-guided learning involves obtaining... In the Lth layer features of the surrogate model, respectively The surrogate model is an image classifier based on a deep neural network; The triplet loss L tri , for z t Optimize the triplet loss and implement feature guidance;

[0009] S4. Perform N attack iterations, updating x′ by optimizing the loss function during the iterations; the optimization process includes four sub-loss functions, namely the loss L used for the attack. attack Loss L used to enhance mobility transfer Loss L used to preserve content structure structure and the loss L used for feature guidance tri After updating x′ N times, the final adversarial sample x is obtained. adv .

[0010] Compared with the prior art, the present invention has the following main advantages:

[0011] 1. This invention restricts the attack perturbation to the low-frequency component space, ensuring that the generated perturbation is mainly concentrated in the semantic feature part of the image, making adversarial examples more difficult for the human eye to detect.

[0012] 2. This invention utilizes feature guidance to perturb more features, ensuring that during the optimization process, the generated adversarial example features are as far away as possible from the category regions of the original clean sample, thereby generating imperceptible adversarial examples with higher transferability. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the steps of a method for generating adversarial samples with high transferability and weak perceptuality according to the present invention.

[0014] Figure 2 This is a schematic diagram illustrating the ternary loss of the present invention.

[0015] Figure 3 This is a schematic diagram of the discrete wavelet transform of the present invention. Detailed Implementation

[0016] The technical solution of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The accompanying drawings are for illustrative purposes only and should not be construed as limiting the present invention. The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] The steps in the following embodiments are provided for ease of explanation only, and there is no limitation on the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0018] Combined with appendix Figure 1 As shown, this invention provides a method for generating adversarial examples with high transferability and weak perceptuality, the method comprising the following steps:

[0019] First, obtain the original image x with category label y from the target dataset.clean The original image is used as input to both the diffusion model and the surrogate model, where the surrogate model is an image classifier based on a deep neural network. The result is a reconstructed sample obtained through the diffusion model and a guided sample generated by attacking the surrogate model using the PGD algorithm. The specific process is as follows:

[0020] The original image is input into stable diffusion and then processed by an encoder to convert x clean From image space to latent space, x clean The sample at forward diffusion step size t = 0 is z0; z0 is inverted to z after t = 20 time steps. t Then, after the reverse denoising stage, the Unet model is used to predict the noise at each step. After denoising for t=20 time steps, z′ is obtained. Then, z′ is decoded to obtain the reconstructed sample x′.

[0021] Original image x clean The surrogate model is used as the initial sample, and the Projected Gradient Descent (PGD) algorithm is employed to attack it. The process involves first evaluating the loss function of the current sample using the surrogate model to calculate the gradient of the input sample. Then, the calculated gradient is multiplied by 2 / 255 and applied to the current sample. Finally, the perturbation magnitude is limited to ensure that the perturbation of each channel of the adversarial sample is between 0 and 16. After 10 attack iterations, a guiding sample x is obtained. guide ;

[0022] For x′ and x respectively clean x guide The decomposition is achieved using a two-dimensional Haar wavelet transform. First, a one-dimensional Haar wavelet transform is applied to the image along the horizontal direction to obtain the low-frequency and high-frequency components in the horizontal direction. Then, the low-frequency and high-frequency components obtained in the previous step are applied again along the vertical direction to obtain the low-frequency and high-frequency components in the vertical direction. After the two-dimensional Haar wavelet transform, each sample is decomposed into one low-frequency component and three high-frequency components, namely x′. ll 、x′ lh 、x′ hl 、x′ hh , and

[0023] x′ ll =Lx′L T ,x′ lH =Hx′L T ,x′ hl =Lx′H T ,x′ hh =Hx′H T

[0024]

[0025]

[0026] Where L and H are the low-pass and high-pass filters of the orthogonal wavelet, respectively;

[0027] Combination Figure 3 Analysis shows that low-frequency components contain basic image information, which often contains label features. Discarding high-frequency components and using only low-frequency components not only allows for focused perturbation of the label's corresponding feature regions but also enhances its imperceptibility. Therefore, the high-frequency components of all three samples are discarded, and only the low-frequency component x′ is used. ll , Image restoration was performed, and the following results were obtained:

[0028]

[0029]

[0030]

[0031] Where IDWT(·) represents the discrete wavelet inverse transform, and L is the low-pass of the orthogonal wavelet;

[0032] right Perform feature guidance, Optimization is performed using triplet loss; during the optimization process, triplet pairs are used. The triplet loss L tri ,in The anchor term representing the loss of the triple. The positive term representing the loss of the triple is... The negative term of the triplet loss is represented by L, which is used for feature-guided triplet loss. tri Defined as:

[0033]

[0034] Where, α∈R + It is the distance between positive and negative pairs, F L (·) represents the features of the Lth layer of the model, and D(·) represents the metric function;

[0035] according to Figure 2 It can be seen that the loss L of the triplet tri Optimization can push adversarial examples away from the original label's category region and bring them closer to other classes;

[0036] The process involves N iterations, where N = 30. During these iterations, x′ is updated by optimizing the loss function. The optimization process includes four sub-loss functions, namely the loss L used for the attack. attack Loss L used to enhance mobility transfer Loss L used to preserve content structure structure and the triplet loss L used for feature guidance tri ,

[0037] By directly interfering with z in the potential space t , for z t Optimize to achieve attack L attack Defined as:

[0038]

[0039] in, This represents the proxy model. For the surrogate model parameters, y represents the true label, and x represents the true label. adv Let J(·) represent a carefully designed adversarial example, and J(·) represent the cross-entropy loss.

[0040] In the latent space, by accumulating the cross-attention mapping between images and labels, the strong recognition ability between images and labels is destroyed, thus enhancing the adversarial sample transferability. transfer Defined as:

[0041]

[0042] Where Var(·) represents the variance of the input, Cross(·) represents the sum of all cross-attention mappings between image pixels and label C during the denoising process, Average(·) represents the average value of the input, and z t This represents the potential variable obtained after t steps of forward diffusion, where t represents the steady-state diffusion step size, and SDM stands for steady-state diffusion.

[0043] Incorporating self-attention control to ensure the adversarial example content structure remains unchanged L structure Defined as:

[0044]

[0045] z t(fix) This represents a copy of the inverted latent variable, which is fixed in the absence of perturbations; calculate z separately. t(fix) and z t The self-attention mappings are denoted as S. t(fix) and S t Forced S t Approaching S t(fix) ;

[0046] The final objective function is:

[0047] arg min L=ρL attack +βL transfer +γL structure +μL tri

[0048] Where ρ, β, γ, and μ are weighting factors set to 10, 10000, 100, and 10 respectively; until the maximum number of attack iterations of 30 is reached, the updated image sample x′ is output. N As the final adversarial example x adv And save it.

[0049] To implement this invention, the performance of the attack was evaluated using the publicly available ImageNet-Compatible Dataset development set, which contains 1000 images of size 299×299×3. Considering that stable diffusion cannot handle the original input size of the ImageNet-Compatible Dataset, all experiments focused on a resized version of 224×224×3. This will be used as an example to illustrate the specific implementation process.

[0050] Top-1 accuracy was used to evaluate the performance of the attack on this invention. Furthermore, Frechet Inception Distance (FID) was used as an indicator to judge the imperceptibility of adversarial examples to humans. Generally, FID aligns with human judgment and reflects the degree of interference well.

[0051] In the table, "S" represents the proxy model and "T" represents the target model. For white-box attacks, where the proxy model is the same as the target model, "*" is used. "AVG(w / o self)" represents the average accuracy of all target models except those identical to the proxy model. Table 1 shows a performance comparison of the transferability and weak perceptualization of adversarial examples on normally trained CNNs, transformers, and MLPs. The results show that the present invention significantly improves performance when adversarial examples generated on the Res-50 model are transferred to other normally trained models, reducing the average accuracy of these models to 32.8% and the FID of the adversarial examples to 62.7. To verify the robustness of the adversarial examples generated by the present invention, Table 2 shows a performance comparison of the transferability and weak perceptualization of adversarial examples generated on Inc-v3 on defensive models. The results show that the present invention significantly improves performance when adversarial examples generated on the Inc-v3 model are transferred to other defensive models, reducing the average accuracy of these models to 44.8% and the FID of the adversarial examples to 62.3. Based on the results of the two tables, this invention can effectively balance the two indicators of high transferability and weak perception.

[0052] Table 1: Performance comparison of the transferability and weak perceptualization of the present invention on a normal training model.

[0053]

[0054] Table 2: Performance comparison of the present invention in terms of transferability and weak perception in defense models

[0055]

[0056] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, any technical solution that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for generating adversarial examples with high transferability and weak perceptuality, characterized in that, Includes the following steps: S1. Transfer the original image x clean The input is fed into the stable diffusion model, and after forward diffusion and backward denoising, the reconstructed sample x' is obtained; S2. Let x', x clean x guide Discrete wavelet transform is performed on three samples, where x guide To attack adversarial examples generated by a proxy model using the PGD algorithm, the high-frequency components obtained from the decomposition are discarded, and only the low-frequency components are used for image reconstruction to obtain the following results: S3. Utilize and right Feature-guided learning involves obtaining... In the Lth layer features of the surrogate model, respectively The surrogate model is an image classifier based on a deep neural network; The triplet loss L tri , for z t Triple loss optimization and feature-guided processing are employed; among these... The anchor term representing the loss of the triple. The positive term representing the loss of the triple is... The negative term of the triplet loss is represented by L, which is used for feature-guided triplet loss. tri Defined as: Where, α∈R + It is the distance between positive and negative pairs, F L (·) represents the features of the Lth layer of the model, and D(·) represents the metric function; S4. Perform N attack iterations, updating x' by optimizing the loss function during the iterations; the loss function contains four sub-loss functions, namely the loss L used for the attack. attack Loss L used to enhance mobility transfer Loss L used to preserve content structure structure and the triplet loss L used for feature guidance tri After updating x' N times, the final adversarial sample x is obtained. adv .

2. The adversarial example generation method with high transferability and weak perceptualization according to claim 1, characterized in that, The original image x clean The reconstructed sample x' is obtained after forward diffusion and backward denoising. First, x' is processed by an encoder. clean Mapping from the image space to the latent space, after a forward diffusion stage, the latent variable z0 diffuses for t steps to obtain z. t In the reverse denoising stage, the Unet model is used to denoise z. t Denoising is performed to obtain z', and z' is then decoded to obtain x'.

3. The adversarial example generation method with high transferability and weak perceptualization according to claim 1, characterized in that, The aforementioned x', x clean x guide Discrete wavelet transform is performed on three samples, where x guide This is an adversarial sample generated by attacking a proxy model using the PGD algorithm; each sample is decomposed into one low-frequency component and three high-frequency components, namely x'. ll 、x' lh 、x' hl 、x' hh , and The high-frequency components of all three samples were discarded, and only the low-frequency component x' was used. ll , Image restoration was performed, and the following results were obtained:

4. The adversarial example generation method with high transferability and weak perceptualization according to claim 1, characterized in that, The process of updating x' during iteration includes four sub-loss functions, namely the loss L used for the attack. attack Loss L used to enhance mobility transfer Loss L used to preserve content structure structure and the triplet loss L used for feature guidance tri , By directly interfering with z in the potential space t , for z t Optimize to achieve attack L attack Defined as: in, This represents the proxy model. For the surrogate model parameters, y represents the true label, and x represents the true label. adv Let J(·) represent a carefully designed adversarial example, and J(·) represent the cross-entropy loss. In the latent space, by accumulating the cross-attention mapping between images and labels, the strong recognition ability between images and labels is destroyed, thus enhancing the adversarial sample transferability. transfer Defined as: Where Var(·) represents the variance of the input, Cross(·) represents the sum of all cross-attention mappings between image pixels and label C during the denoising process, Average(·) represents the average value of the input, and z t This represents the potential variable obtained after t steps of forward diffusion, where t represents the steady-state diffusion step size, and SDM stands for steady-state diffusion. Adding self-attention control ensures that the content structure of adversarial examples remains unchanged. structure Defined as: z t(fix) This represents a copy of the inverted latent variable, which is fixed in the absence of perturbations; calculate z separately. t(fix) and z t The self-attention mappings are denoted as S. t(fix) and S t Forced S t Approaching S t(fix) ; The final objective function is: arg min L=ρL attack +βL transfer +γL structure +μL tri Where ρ, β, γ, and μ are weighting factors, and N attack iterations are performed. During the iterations, x' is updated by optimizing the loss function argmin L; until the maximum number of attack iterations N is reached, the updated image sample x' is output. N As the final adversarial example x adv .

Images