Migration stealthy adversarial attack method based on controllable diffusion guidance

Abstract

This invention relates to a transferable covert adversarial attack method based on tunable diffusion guidance. It includes: mapping the original image to an initial state of backdiffusion through a forward diffusion process of a denoising diffusion implicit model; in each backdiffusion time step, performing semantic preprocessing on the original image using a gradient-weighted class activation mapping method to construct a multi-granularity semantic mask combining block-level and pixel-level methods; estimating the predicted clean image corresponding to the current state; calculating the ensemble gradient of multiple proxy models in each backdiffusion time step, and constructing a rescaling variance reduction gradient by combining the variance reduction gradient of a randomly sampled proxy model; updating the diffusion noise based on the gradient; filtering the updated terms element-wise using the multi-granularity semantic mask; and limiting the perturbation amplitude through budget-constrained projection. After several backsampling processes, imperceptible adversarial examples are generated. This method exhibits better imperceptibility and transferability in attacks against the target model.

Description

Technical Field

[0001] This invention relates to the field of deep learning security and adversarial attack technology, and in particular to a mobile covert adversarial attack method based on tunable diffusion guidance. Background Technology

[0002] In recent years, while deep neural networks have achieved remarkable results in the field of computer vision and have been widely applied in various practical scenarios, they are highly sensitive to minute perturbations and vulnerable to adversarial example attacks, leading to erroneous output results and threatening system security and reliability. Compared to white-box attacks, black-box attacks are closer to the real environment. Their core is to generate adversarial examples with strong transferability and as little visual notice as possible using a proxy model, under the condition that the internal information of the target model cannot be obtained, in order to improve the practicality and stealth of the attack.

[0003] Most existing transferability-based attack methods directly optimize gradients in the pixel space, such as FGSM, I-FGSM, PGD, and their various ensembles and momentum enhancement methods. While these methods can improve the success rate of cross-model attacks to some extent, their perturbations are usually quite dense, easily introducing visible noise into the image, thus compromising the perceptual quality of the sample and resulting in poor, imperceptible effects. With the development of diffusion models, AdvAD describes adversarial attacks as a non-parametric diffusion process from a modeling perspective. It injects adversarial guidance based on the attacked model during the back-diffusion process and uses pixel-level constraint modules to control the noise offset. However, these constraints are more based on overall pixel-level limitations and are insufficient in distinguishing the importance of different semantic regions in the image, making it difficult to achieve precise attacks in critical regions and suppress perturbations in non-critical regions. Furthermore, it does not consider the overfitting problem of a single model, and the potential for transferability improvement remains limited in complex cross-model black-box scenarios. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, the purpose of this invention is to solve the issues of insufficient mobility, limited visual concealment, and difficulty in flexibly controlling the adversarial samples generated by existing attack methods, and to provide a mobile concealment adversarial attack method based on adjustable diffusion guidance.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a mobile covert countermeasure attack method based on tunable diffusion guidance, comprising the following steps:

[0006] S1: Obtain the original image The real label corresponding to the original image The diffusion coefficients of multiple surrogate models and diffusion models will affect the original image. The forward diffusion process of the denoising diffusion implicit model is mapped to the initial state of the reverse diffusion. And initialize the diffused noise, starting from time step Start executing the reverse diffusion iteration.

[0007] S2: Use gradient-weighted class activation mapping to perform semantic preprocessing on the original image to obtain a class activation map corresponding to the spatial size of the original image. ; activation graph of the class As a pixel-level semantic graph ,Will The image is divided into several non-overlapping blocks. The semantic importance of each block is calculated, and the blocks with semantic importance above a preset percentile are retained to construct a block-level semantic mask. Furthermore, within the preserved image blocks, pixels with activation values ​​above a preset percentile are retained to construct a multi-granularity semantic mask. .

[0008] S3: At each backdiffusion time step, estimate the predicted clean image corresponding to the current time step based on the current diffusion state and the diffusion noise of the previous time step. Calculate the adversarial gradients of multiple surrogate models on the current predicted clean image, and integrate these multiple adversarial gradients to obtain the integrated gradient. Randomly sample a surrogate model, calculate the variance reduction gradient corresponding to that surrogate model, and apply the rescaling factor... Adjust the relative weights of the integrated gradient and the variance reduction gradient to construct a rescaled variance reduction gradient.

[0009] S4: Update the diffuse noise according to the rescaling variance reduction gradient, filter the multi-granularity semantic mask and the diffuse noise update term element by element to obtain the semantically constrained intermediate noise, constrain the semantically constrained intermediate noise to obtain the diffuse noise that meets the perturbation budget constraint;

[0010] S5: Perform a denoising diffusion implicit model backsampling based on the updated diffusion noise to obtain the diffusion state at the next time step. ,Will Continue iterating as the input for the next time step, until... Back diffusion ends. Output.

[0011] Furthermore, in step S2, the original image is semantically preprocessed using a gradient-weighted class activation mapping method to obtain a class activation map corresponding to the spatial size of the original image. ;Activate the class graph Directly used as a pixel-level semantic graph Among them, pixel-level semantic graph Used to characterize the semantic relevance and semantic importance of each pixel location in the original image to the target category discrimination result; for pixel-level semantic maps any pixel position ,have

[0012]

[0013] in, and These represent the height and width of the original image, respectively. express The semantic response value of a location is determined by its magnitude; a larger value indicates a greater semantic contribution and higher semantic importance of the corresponding pixel to the target category. By constructing the pixel-level semantic map, key semantic regions in the original image can be preliminarily characterized at the pixel level, providing a basis for subsequent block-level semantic filtering and fine-grained perturbation constraints.

[0014] Furthermore, in S2, a block-level semantic mask is constructed. The method is as follows:

[0015] pixel-level semantic graph Divide into several sizes Non-overlapping image patches ,in , , , The semantic importance of each image patch is calculated using the Frobenius norm.

[0016]

[0017] set up Represents the set of semantic importance of all image patches. Percentiles, then block-level semantic mask Defined as:

[0018]

[0019] in Represents image blocks It includes all pixel locations. At a coarser granular level, regions with strong semantic responses can be prioritized and invalid updates in regions with weak semantic contributions can be suppressed, thereby reducing the diffusion of perturbations into the background.

[0020] Furthermore, in step S2, a multi-granularity semantic mask is constructed. The method is as follows:

[0021] In block-level semantic mask Each image patch retained Internally, the semantic importance of each pixel in the image patch is further evaluated, and the activation values ​​of the corresponding pixels are sorted, retaining only the activation values ​​located at the top of the image patch. Percentile pixels are used to achieve fine-grained semantic filtering within the block; wherein, for pixels located in the image block any pixel position within If it is in the pixel-level semantic graph Activation value in The first activation value greater than or equal to the activation value within the image patch Percentile threshold Then the pixel will be in the multi-granularity semantic mask. The corresponding position is recorded as 1, otherwise as 0; the multi-granularity semantic mask Represented as:

[0022]

[0023] in, Represents image blocks The first of all pixel activation values The percentile. The pixel activation value is equal to the corresponding pixel position in the pixel-level semantic map. The semantic response value is used. By combining block-level filtering with intra-block pixel filtering, both region-level semantic integrity and pixel-level fine constraints can be taken into account, making the generated adversarial examples have a better imperceptible effect.

[0024] Furthermore, in S3, the original image After the forward diffusion process of the denoising diffusion implicit model, the initial state of the reverse diffusion is obtained. Each reverse diffusion time step, based on the current diffusion state And the diffusion noise of the previous time step Estimate the predicted clean image at the current time step. Its expression is:

[0025]

[0026] in, For the first The diffusion coefficients corresponding to each diffusion time step. The predicted clean image is used to approximately represent the noise-free image state at the current back-diffusion time step, providing a more stable input representation for the gradient calculation of the surrogate model.

[0027] Furthermore, in S3, the ensemble gradient of multiple proxy models is obtained by averaging the adversarial gradients of all proxy models at the current time step, and is expressed as:

[0028]

[0029] in, For the number of proxy models, Indicates the first Each proxy model corresponds to an adversarial objective function. By integrating the gradients of multiple proxy models, the influence of the bias of a single proxy model on the attack direction can be reduced, making the obtained gradients more reflective of the shared vulnerability features of different models, thereby enhancing the ability to perform transfer attacks. This represents the adversarial gradient of a proxy model at the current time step;

[0030] This indicates that the adversarial objective function is relative to the current diffusion state. The gradient, i.e., the gradient of the first... The adversarial gradient of each proxy model at the current time step; the adversarial objective function is expressed as:

[0031]

[0032] in, Indicates the first A proxy model, This indicates that the surrogate model will predict clean images. Determined as a true category The probability of.

[0033] The rescaling variance reduction gradient is constructed as follows: a surrogate model is randomly sampled at each backdiffusion time step. The surrogate model was calculated to predict clean images. and the current diffusion state The gradient on the time step is obtained, and the rescaling variance reduction gradient at the current time step is obtained according to the following formula:

[0034]

[0035] in, It is the rescaling factor, and The rescaling factor is used to rescale the variance reduction term and serves as a control parameter for the trade-off between mobility and imperceptibility. Indicates the first The adversarial objective function of the proxy model relative to the predicted clean image gradient,

[0036] Rescaling variance reduction gradient Equivalent to integrated gradient Reduce gradient with variance The weighted combination is expressed as:

[0037]

[0038] in, This is used to adjust the relative weights of the integrated gradient and the variance reduction gradient in the rescaling variance reduction gradient, in order to achieve an adjustable balance between adversarial sample migration and imperceptibility.

[0039] Furthermore, in step S4, the current diffuse noise is updated based on the rescaling variance reduction gradient, and the updated intermediate noise is represented as:

[0040]

[0041] in, To initialize the diffused noise, For the first The diffusion coefficient corresponding to each diffusion time step.

[0042] Utilizing multi-granular semantic masks Element-by-element filtering of the noise update term yields the semantically constrained intermediate noise, expressed as follows:

[0043]

[0044] in, This represents element-wise multiplication. By filtering element-wise, noisy updates are retained only in semantically important regions, thereby suppressing redundant perturbations in non-critical regions and reducing the impact on the overall visual quality of the image.

[0045] Updated intermediate noise by budget constraint projection By applying constraints, diffuse noise that satisfies the disturbance budget limit is obtained. Its expression is:

[0046]

[0047] in, Indicates based on Projection operations on the norm sphere, Preset disturbance budget.

[0048] Furthermore, in S5, after completing the budget constraint projection, the updated diffuse noise is used. Perform a backsampling operation on the denoising diffusion implicit model to obtain the diffusion state at the next time step. Its expression is:

[0049]

[0050] And Continue iterating as the input for the next time step, until... Output the final adversarial sample at time By iteratively updating over time steps, adversarial perturbations can be smoothly embedded into the diffusion backsampling process, enabling the final generated adversarial examples to maintain strong attack capabilities while possessing better visual naturalness and imperceptibility.

[0051] Compared with the prior art, the present invention has at least the following advantages:

[0052] 1. Most current gradient-based transfer attack methods, such as FGSM, I-FGSM, PGD, and their momentum or ensemble improvements, typically perform perturbation optimization directly along the gradient direction in pixel space. While these methods can improve the success rate of cross-model attacks to some extent, the lack of explicit constraints on the perceived quality of the perturbation often results in dense perturbations that easily introduce visible noise into the image, making it difficult to balance transferability and imperceptibility. In contrast, this invention embeds adversarial optimization into the backdiffusion process, utilizing the progressive denoising characteristics of diffusion to gradually control the perturbation, and combining multi-model gradient information for joint optimization. This not only reduces the directional bias caused by a single proxy model but also effectively reduces the visual distortion caused by direct pixel space updates while maintaining attack capability, thereby significantly improving the visual naturalness and actual concealment of adversarial examples.

[0053] 2. Compared to existing traditional gradient attack methods that rely solely on single-step or iterative gradient accumulation to enhance transferability, this invention proposes a rescaled variance reduction gradient and its adjustable trade-off mechanism. This mechanism combines multi-model ensemble gradients with variance reduction gradients to suppress gradient inconsistencies between different surrogate models during the diffusion inversion process, resulting in a smoother optimization trajectory and preventing excessive oscillations or ineffective shifts in the backdiffusion process. Furthermore, this invention introduces a trade-off control factor into the rescaled variance reduction gradient. This is used to adjust the relative contributions of the integrated gradient and the variance-reduced gradient to the final update direction. When the gradient is smaller, the final gradient is more biased towards the multi-model ensemble gradient, which is beneficial for making full use of the shared attack direction between different proxy models, thereby improving the black-box transfer attack capability; when When the variance is large, the final gradient is more biased towards the variance-reduced gradient, which helps to weaken gradient oscillations and the accumulation of invalid perturbations, thereby reducing the perturbation amplitude and improving visual imperceptibility. Therefore, this invention can not only improve the stability of gradient updates through the rescaling variance reduction mechanism, but also... Achieve flexible adjustment between portability and imperceptibility.

[0054] 3. Compared to AdvAD, which primarily relies on a single proxy model to construct adversarial guidance during the backdiffusion process, this invention fully considers the problem of overfitting caused by single-model guidance, thus limiting black-box transfer performance. Since the gradient information of a single proxy model can only reflect the local decision-making characteristics of a specific model, the generated perturbations tend to be biased towards the model's own vulnerabilities, resulting in insufficient generalization attack capability against unknown target models. This invention effectively alleviates overfitting to a single proxy model through multi-model collaborative optimization, enabling perturbations to learn more common attack features among multiple models to provide a more consistent and robust attack direction. Therefore, while maintaining the stealth advantage of the diffusion attack, it improves the transferability and attack stability of adversarial examples in black-box scenarios.

[0055] 4. This invention further addresses the problem of insufficient utilization of semantically important regions in image space by existing methods. It designs a multi-granularity semantic guidance mechanism that filters and constrains semantically significant regions at both the block and pixel levels. This ensures that perturbations are preferentially distributed in regions more relevant to the target semantics, while suppressing invalid modifications in semantically unimportant regions. Compared to traditional gradient attacks that rely on global uniform updates or simple pixel constraints, this mechanism significantly improves the targeting and spatial sparsity of perturbation allocation, reducing visual noise accumulation in non-critical regions. Unlike AdvAD, which primarily uses pixel-level projection constraints, this invention further enhances the finesse of perturbation control, enabling the generated adversarial examples to maintain strong attack performance while exhibiting better structural consistency, perceptual quality, and imperceptibility. Attached Figure Description

[0056] Figure 1 is a simplified flowchart of the method of the present invention.

[0057] Figure 2 is a visual comparison of adversarial perturbation under single-granularity and multi-granularity semantic importance constraints.

[0058] Figure 3 shows a visual comparison of various ensemble methods under different gradient variance indices.

[0059] Figure 4 shows a comparison of the attack effects. Detailed Implementation

[0060] The present invention will now be described in further detail.

[0061] The proxy models used in this invention include convolutional neural networks, visual Transformer models, and visual sequence models. The adversarial examples generated by this invention can be used to attack black-box target classification models and to analyze the vulnerability of different models in transferable attack scenarios. Target models that can be attacked using the adversarial examples generated by this invention include CNN models, ViT models, and their robust variants. Addressing the problem that existing diffusion-based adversarial attack methods, while generating relatively smooth and imperceptible perturbations, are easily affected by the bias of a single proxy model in black-box scenarios, leading to insufficient transferability, this invention proposes a transferable covert adversarial attack method based on adjustable diffusion guidance. This method integrates multi-model ensemble, rescaling variance reduction optimization, and block-level and pixel-level joint semantic constraints into the backdiffusion process, thereby improving attack transferability while maintaining the visual imperceptibility of the perturbations.

[0062] Compared to AdvAD, which primarily relies on a single proxy model to construct adversarial guidance during backdiffusion, this invention fully considers the problem of overfitting in a single model, thus limiting black-box transfer performance. To address this, this invention employs multi-model collaborative optimization, enabling perturbations to learn more common attack features across multiple models. This maintains the stealth advantage of the diffusion attack while improving the transferability and attack stability of adversarial examples. Furthermore, this invention designs a multi-granularity semantic guidance mechanism, jointly constraining semantically significant regions at both the block and pixel levels. This ensures that perturbations are preferentially distributed in key semantic regions, reducing invalid modifications and visual noise in non-key regions. The purpose of this invention is to organically embed multi-model integration, rescaling variance reduction optimization, and block-level and pixel-level joint semantic constraints into the backdiffusion process, thereby enhancing the adversarial example transfer attack capability while ensuring the perturbations maintain good visual imperceptibility.

[0063] Given the original image Its true label , A collection of proxy models Total diffusion time step Initialize diffused noise and preset disturbance budget First, map the original image to... The interval is determined, and the initial state for reverse diffusion is obtained through the forward diffusion process of the denoising diffusion implicit model. Let the first... The diffusion coefficient at each time step is The initial diffusion state can then be expressed as:

[0064]

[0065] in, This is the initial state for reverse diffusion. To initialize with Gaussian noise. Then from... Begin executing the backdiffusion iteration. To obtain the predicted clean image corresponding to the current diffusion state at each time step, based on the non-Markov property of the denoising diffusion implicit model, the 1st... The predicted clean image at each time step is estimated as follows:

[0066]

[0067] in, This represents the diffusion state at the current time step. This is the diffused noise used for backsampling in the previous time step. Used to approximate the current reverse trajectory at time step The corresponding noise-free results are shown below.

[0068] Furthermore, to ensure that perturbations are preferentially concentrated in semantically critical regions, this invention performs multi-granular semantic preprocessing on the original image before backdiffusion begins. First, the original image is generated using the Grad-CAM method. Class activation graph And it is used as a pixel-level semantic response map, that is:

[0069]

[0070] in, Indicates the first in the image Line 1 The semantic relevance of each column pixel to the target category is calculated, with higher values ​​indicating greater importance. Then, the class activation map is divided into several non-overlapping image patches. ,in , ,and , For each image patch, its block-level semantic importance is calculated using the Frobenius norm:

[0071]

[0072] in, Representing an image The set of all pixel locations contained therein. For all block-level importance sets middle The percentile, then the block-level semantic mask Defined as:

[0073]

[0074] In block-level semantic mask Each image patch retained Internally, the semantic importance of each pixel in the image patch is further evaluated, and the activation values ​​of the corresponding pixels are sorted, retaining only the activation values ​​located at the top of the image patch. Percentile pixels are used to achieve fine-grained semantic filtering within the block; wherein, for pixels located in the image block any pixel position within If it is in the pixel-level semantic graph Activation value in The first activation value greater than or equal to the activation value within the image patch Percentile threshold Then the pixel will be in the multi-granularity semantic mask. The corresponding position is recorded as 1, otherwise as 0; the multi-granularity semantic mask Represented as:

[0075]

[0076] in, Represents image blocks The first of all pixel activation values The percentiles were constructed from this. At the same time, it retains the block-level backbone semantic region and high-response pixels within the block, which can further compress the perturbation distribution range in non-critical regions while ensuring the semantic integrity of the target, resulting in better imperceptible effects.

[0077] At each reverse diffusion time step To enhance the black-box migration attack capability, this invention no longer relies on a single proxy model, but integrates the adversarial gradients of all proxy models. To this end, the first definition is... The adversarial objective function of the proxy model is:

[0078]

[0079] in, Representative proxy model In the input Predict the true label The probability of predicting a clean image at the current time step. For all surrogate models at the current time step. Averaging the gradients yields the integrated gradient:

[0080]

[0081] While this can reduce the bias effect of a single surrogate model, using only the average gradient can still be affected by gradient inconsistencies between models. To further reduce this inconsistency, a surrogate model is randomly sampled at each backdiffusion time step. Calculate its effect on predicting clean images respectively. and the current diffusion state Find the gradient on the variance and construct the variance-reducing gradient:

[0082]

[0083] in, The rescaling factor can balance control over portability and imperceptibility, and .when When the variance is smaller, the proportion of multi-model ensemble gradients in the rescaling variance reduction gradient is higher, and the update direction more fully reflects the vulnerable directions shared among multiple proxy models, thus improving the black-box transfer attack capability of adversarial examples; when When the variance is large, the proportion of variance reduction in gradient is higher during rescaling variance reduction, which can weaken gradient oscillations between different surrogate models and different time steps, making the perturbation injection in the diffusion reverse process smoother, thereby reducing the perturbation amplitude and improving the visual imperceptibility of adversarial examples. Therefore, this invention can achieve this without changing the diffusion sampling process and the surrogate model structure, simply by adjusting... This allows for different attack modes, including those prioritizing mobility, those prioritizing undetectability, and those balancing the two.

[0084] After obtaining the rescaling variance reduction gradient, the current diffuse noise is updated based on this gradient. First, the noise update term for the current time step is defined as:

[0085]

[0086] Without semantic constraints, the intermediate noise can be written as:

[0087]

[0088] To concentrate the perturbation on semantically critical regions, multi-granular semantic masks are used. Applying this to the noise update term and filtering it element by element, we obtain the semantically constrained intermediate noise:

[0089]

[0090] Gradient updates are retained only at key locations corresponding to multi-granularity semantic masks, while noise offset is suppressed or weakened in non-key regions, thus preventing disordered diffusion of perturbations into visually unimportant areas. Subsequently, to ensure the perturbation satisfies budget constraints, a norm-sphere-based projection operation is performed on the updated intermediate noise to obtain diffused noise that satisfies the perturbation budget limit.

[0091]

[0092] in, Indicates based on Projection operations on the norm sphere, Preset disturbance budget.

[0093] After completing the budget constraint projection, the updated diffuse noise is used. Perform a backsampling operation on the denoising diffusion implicit model to obtain the diffusion state at the next time step:

[0094]

[0095] And Continue iterating as the input for the next time step, until... Output the final adversarial sample at time After the diffusion process is complete, the output result is transformed back from the data range corresponding to the diffusion model to the original image value range. After one reverse diffusion iteration, this method can generate adversarial examples with high cross-model transfer attack capabilities while maintaining the naturalness of image structure and visual concealment.

[0096] Experimental Results and Analysis

[0097] Dataset and Model Selection: The dataset selection is consistent with the baseline method experiments, using the ImageNet-compatible Dataset. This dataset contains 1000 images from various categories on ImageNet, with original image sizes of [size missing]. In all experiments, the input size was uniformly adjusted to the standard input size. The purpose is to evaluate the attack effectiveness and generalization performance of the method proposed in this invention.

[0098] To verify the effectiveness of the method under different model architectures, three types of models—CNN, ViT, and VisionMamba—were selected as proxy models for adversarial example generation. For the CNN proxy model, ResNet-50 and MobileNet-V2 were selected as single-model proxies, and two ensemble proxy models were further set up: {ResNet-50, MobileNet-V3, DenseNet121} and {MobileNet-V2, ResNet-18, Inception-V4}. For the ViT proxy model, ViT-B and PiT-B were selected as single-model proxies, and two ensemble proxy models were set up: {ViT-B, ViT-T, ViT-S} and {PiT-B, PiT-T, PiT-S}. Furthermore, to further verify the applicability of the method of the present invention to the novel state-space visual model, VisionMamba was selected as the proxy model, and two sets of integrated proxy models were set up, namely {VisionMamba,MobileNet-V3,DenseNet-121} and {VisionMamba, ViT-B, PiT-B}.

[0099] To test the transfer attack capability of the generated adversarial examples in a black-box scenario, the experiment further selected various target models with different structures for evaluation. These included CNN target models such as Inception-v3, Vgg11, Dense201, ResNet-v2, Inception-ResNet-v2, and GoogleNet; and ViT target models such as CaiT, Visformer, LeViT, DeiT, ConVit, and Swin. Through testing on multiple heterogeneous target models, the cross-model transfer capability and actual attack effect of the adversarial examples generated by the method of this invention can be more comprehensively evaluated.

[0100] Benchmark Methods: To verify the effectiveness of the method of this invention, several representative undetectable attack methods were selected as benchmark methods for comparison, including PGD, AdvDrop, PerC-AL, SSAH, and AdvAD. Comparison with the above benchmark methods and variant methods fully demonstrates the comprehensive advantages of the method of this invention in terms of improved portability and preservation of undetectability.

[0101] Evaluation criteria and parameter settings: To systematically evaluate the effectiveness of the attack method proposed in this invention, the experiment evaluates it from two aspects: transferability and imperceptibility.

[0102] Regarding transferability, the Average Attack Success Rate (ASR) is used as the evaluation metric to measure the proportion of times an adversarial example causes a target model to misclassify in a black-box attack scenario, where Avg represents the average attack success rate across all target models. This metric can intuitively reflect the cross-model transferability attack capability of the adversarial examples generated by the method of this invention.

[0103] Regarding imperceptibility, considering that adversarial perturbations are usually strictly constrained and have small visual differences, subjective human evaluation is difficult to reliably and objectively reflect the quality of the perturbation. Therefore, multiple objective quantitative indicators are used to evaluate the differences between the original image and the adversarial example. These include: the perturbation's... Norm, Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Perceptual Patch Similarity (LPIPS), and Fréchet Inception Distance (FID). The norm is used to measure the overall magnitude of the perturbation, PSNR and SSIM reflect the degree of image distortion at the pixel level, while LPIPS and FID measure the visual similarity between the adversarial example and the original image at the perceptual level. By jointly evaluating these metrics, the ability of the method of this invention to maintain image visual quality while ensuring attack performance can be more comprehensively reflected.

[0104] Parameter settings: To verify the effectiveness of the attack method proposed in this invention, the method was compared with various benchmark methods under different black-box attack settings. To ensure a fair comparison, the perturbation budget for all methods was uniformly set to... Image patch size set to For different diffusion steps Adjust the hyperparameters separately.

[0105] Specifically, when When, set the rescaling factor to At this point, a smaller weight is assigned to the integrated gradient. To mitigate its impact and enhance the imperceptibility of adversarial examples; when and At that time, Setting it to 0.1 enhances the effect of the integrated gradient, thereby improving attack transferability. This applies to multi-granularity semantic guidance parameters. It is used to control the sparsity of block-level and pixel-level masks; when At that time, the retention ratio for both block-level and pixel-level pixels was set to 10%; when At that time, the block-level retention ratio was adjusted to 90%, and the pixel-level retention ratio was kept at 10% in order to provide stronger semantic guidance capabilities with fewer diffusion steps.

[0106] Gradient variance analysis: To further verify the effect of the rescaling variance reduction gradient on optimization stability in this invention, the gradient variance changes of different ensemble strategies during the backdiffusion process were visualized and compared, as shown in Figure 3. It can be seen that compared to the comparison methods such as Ens, SVRG, and Cosine, the method of this invention maintains a lower and more stable trend in both gradient numerical variance and significant principal variance, indicating that the rescaling variance reduction ensemble gradient constructed in this invention can effectively reduce gradient fluctuations between diffusion time steps and alleviate the gradient inconsistency problem between different proxy models. Therefore, this invention can provide a more stable optimization direction for subsequent diffusion noise updates, thereby improving the transferability and stability of the adversarial examples generation process.

[0107] First, the transferability and imperceptibility of the proposed method were verified on a CNN model. The experimental results are shown in Table 1, under different diffusion step settings ( , , The method of this invention demonstrates superior overall performance compared to benchmark methods on various CNN proxy models. Overall, the method maintains high visual quality while consistently achieving a higher attack success rate, indicating a better balance between transferability and imperceptibility. Especially when the number of diffusion steps is small, this method still possesses strong attack capabilities. For example, on ResNet-50, when the number of diffusion steps is reduced to... At that time, this method improved the average attack success rate from 36.3% to 40.4%, demonstrating its good migration attack performance even under limited diffusion iteration conditions. Similarly, on MobileNet-V2, this method also achieved the best results in terms of visual quality metrics, exhibiting the smallest [value missing]. The perturbation (0.80), the highest PSNR (53.82), and the SSIM close to 1 (0.9990) indicate that the method of this invention can not only effectively improve the migration attack capability of adversarial examples in black-box scenarios, but also better maintain the visual naturalness and concealment of the generated examples.

[0108] Table 1 compares the transferability and imperceptibility of the proposed method and the comparative method on multiple CNN target models with different diffusion steps when the surrogate model is a CNN. The bolded result indicates the optimal result.

[0109]

[0110] Furthermore, the transferability and imperceptibility of the proposed method were further verified on the ViT model. Consistent with the testing method on the CNN model, the generated adversarial examples were input into each target ViT model, and the classification results were calculated. The experimental results are shown in Table 2. Under different diffusion step settings, the proposed method significantly outperformed various benchmark methods, demonstrating stronger black-box transfer attack capabilities. When using ViT-B as the proxy model, the proposed method increased the average attack success rate from 30.9% to 33.3%, while generating smaller perturbations, indicating that it can better control the perturbation intensity while improving attack performance. Further analysis of PSNR, SSIM, and LPIPS metrics shows that the adversarial examples generated by the proposed method still possess good visual naturalness and high concealment. This demonstrates that the proposed method not only effectively enhances the transfer attack effect on the ViT model but also maintains superior performance in terms of imperceptibility.

[0111] Table 2 compares the transferability and imperceptibility of the proposed method and the comparative method on multiple ViT target models with different diffusion steps when the surrogate model is ViT. The bolded result indicates the optimal result.

[0112]

[0113] Furthermore, the transferability and imperceptibility of the proposed method were further verified on the VisionMamba model. Experimental results are shown in Tables 3 and 4. Using VisionMamba-Small as the surrogate model and combining it with various CNN models (MobileNet-V3, DenseNet-121) and ViT models (ViT-B, PiT-B), the proposed method demonstrated superior overall performance compared to AdvAD under different diffusion step counts. Overall, the proposed method exhibits significant advantages in both transfer attack success rate and visual quality metrics. For example, the method of this invention achieves an average attack success rate of 34.5% and 32.9% on CNN target models and ViT target models, respectively. It also achieves better results in perturbation amplitude control and perceptual quality metrics, demonstrating its ability to effectively maintain the visual naturalness of adversarial examples while enhancing attack effectiveness. Furthermore, the method maintains relatively stable attack performance under different combinations of proxy and target models, showcasing good adaptability to differences in model structure. The comprehensive evaluation results of multiple attack and perceptual metrics show that the method of this invention is applicable not only to convolutional neural network models but also to Transformer-type visual models and VisionMamba-type models, exhibiting good versatility and achieving high cross-model transfer attack effectiveness without significantly sacrificing visual fidelity.

[0114] Table 3 compares the transferability and imperceptibility of AdvAD and our method for various CNN models at different diffusion steps when using VisionMamba as the surrogate model. The best results are marked in bold.

[0115]

[0116] Table 4 compares the transferability and imperceptibility of AdvAD and our method for various ViT models at different diffusion steps when using VisionMamba as the surrogate model. The best results are marked in bold.

[0117]

[0118] Ablation experiments: This invention designed three sets of ablation experiments to verify the effectiveness of the proposed method, namely (1) rescaling factor (2) Component combination method. (3) Image block size. Experimental results show that each module in the proposed method plays an important role in improving attack mobility and maintaining imperceptibility, and there is a clear performance trade-off between different parameter settings.

[0119] (1) Rescaling factor First, the rescaling factor was examined in the rescaling variance reduction gradient. The effect of [the experiment]. The experimental results are shown in Table 5. [With the development of] [the experiment], As the value increases, the proportion of the integrated gradient gradually decreases, while the dominant role of the variance-reducing gradient gradually strengthens. While this effectively suppresses perturbation amplitude and distribution shift, resulting in a continuous improvement in the image quality of generated samples, it also weakens the attack strength, leading to a gradual decrease in attack success rate. This explains the rescaling factor. It has a significant impact on the balance between mobility and imperceptibility: smaller It is more conducive to improving attack performance, and larger This is more conducive to maintaining visual quality. Furthermore, by introducing a rescaling mechanism, this invention enables flexible adjustment between attack effectiveness and image fidelity.

[0120] Table 5 shows the rescaling factors. Impact on attack success rate (ASR) and image quality.

[0121]

[0122] (2) Component Combination Method: Further, the combined effects of different components in the method of this invention were analyzed. The experimental results are shown in Table 6. Different components have obvious complementary effects in the method of this invention. When only the rescaling variance reduction gradient is introduced, the attack success rate is significantly improved, indicating that this module can effectively enhance the ability to transfer attacks against perturbations. However, due to the more aggressive perturbation update, the image quality is reduced. When only multi-granularity semantic guidance is introduced, the perceptual quality of the samples is significantly improved, but the attack success rate is significantly reduced. In contrast, only when both the rescaling variance reduction gradient and multi-granularity semantic guidance are introduced can a better balance be achieved between the attack success rate and visual quality. This result shows that the two key modules in the method of this invention are not independent of each other, but work together to improve transferability and maintain imperceptibility, thus demonstrating good synergistic gains.

[0123] Table 6 compares the attack performance and image quality of this method under different component configurations. Here, w / r-SVRG indicates that rescaling variance reduction gradient (r-SVRG) is enabled, w / MGS indicates that multi-granularity semantic guidance (MGS) is enabled, and w / rSVRG andMGS represent this method.

[0124]

[0125] (3) Image patch size Furthermore, this invention also analyzes image patch size. Impact on attack performance. Experimental results are shown in Table 7. When using a medium block size... When the block size is reduced to a certain value, the highest attack success rate can be achieved, indicating that this setting can more effectively capture key semantic regions in the image, thereby enhancing the ability to resist perturbations. However, at the same time, its perceptual quality index is relatively reduced. In contrast, when the block size is reduced to a certain value, the attack success rate is higher. While the attack success rate decreased slightly, metrics such as PSNR, SSIM, FID, and LPIPS all showed significant improvement, indicating that finer-grained image segmentation helps to localize perturbations more effectively, thereby enhancing the visual naturalness and fidelity of adversarial examples. This result demonstrates that image patch size directly affects the semantic mask's ability to control the perturbation distribution range, and appropriate patch size selection is crucial for balancing transferability and imperceptibility. Based on the comprehensive experimental results, this invention ultimately selects... This is considered a better setting.

[0126] Table 7 shows the image patch sizes. Impact on attack success rate (ASR) and image quality.

[0127]

[0128] Attack Performance Analysis: The method of this invention exhibits stable and superior attack performance under different proxy models, target models, and diffusion step settings. In terms of transferability, the method achieves higher average attack success rates on both CNN and ViT target models. Under the ViT-B proxy model, the average attack success rate can be increased from 30.9% to 33.3%; under the VisionMamba proxy model, when… At that time, the average attack success rate on CNN and ViT target models reached 34.5% and 32.9% respectively, demonstrating strong cross-architecture transfer attack capabilities. Regarding imperceptibility, the method of this invention can effectively control the perturbation intensity while improving the attack success rate. Overall, the method outperforms or significantly outperforms benchmark methods in metrics such as norm, PSNR, SSIM, FID, and LPIPS, indicating that the generated adversarial examples are closer to the original images at both the pixel and perceptual levels, exhibiting better visual naturalness and stealth. Furthermore, multi-granular semantic guidance can generate smaller perturbations and better perceptual quality while ensuring attack performance, and the joint introduction of block-level and pixel-level semantic constraints further verifies that the method of this invention has a better comprehensive balance between improving transferability and maintaining imperceptibility.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for transferable stealthy adversarial attack based on controllable diffusion guidance, characterized in that, Includes the following steps: S1: Obtain the original image The real label corresponding to the original image The diffusion coefficients of multiple surrogate models and diffusion models will affect the original image. The forward diffusion process of the denoising diffusion implicit model is mapped to the initial state of the reverse diffusion. And initialize the diffused noise, starting from time step Begin executing the back diffusion iteration; S2: Use gradient-weighted class activation mapping to perform semantic preprocessing on the original image to obtain a class activation map corresponding to the spatial size of the original image. ; activation graph of the class As a pixel-level semantic graph ,Will The image is divided into several non-overlapping blocks. The semantic importance of each block is calculated, and the blocks with semantic importance above a preset percentile are retained to construct a block-level semantic mask. Furthermore, within the preserved image blocks, pixels with activation values ​​above a preset percentile are retained to construct a multi-granularity semantic mask. ; S3: At each backdiffusion time step, estimate the predicted clean image corresponding to the current time step based on the current diffusion state and the diffusion noise of the previous time step. Calculate the adversarial gradients of multiple surrogate models on the current predicted clean image, and integrate these multiple adversarial gradients to obtain the integrated gradient. Randomly sample a surrogate model, calculate the variance reduction gradient corresponding to that surrogate model, and apply the rescaling factor... Adjust the relative weights of the integrated gradient and the variance reduction gradient to construct a rescaled variance reduction gradient; S4: Update the diffuse noise according to the rescaling variance reduction gradient, filter the multi-granularity semantic mask and the diffuse noise update term element by element to obtain the semantically constrained intermediate noise, constrain the semantically constrained intermediate noise to obtain the diffuse noise that meets the perturbation budget constraint. S5: Perform a denoising diffusion implicit model backsampling based on the updated diffusion noise to obtain the diffusion state at the next time step. ,Will Continue iterating as the input for the next time step, until... After backdiffusion ends, the denoised and reconstructed image obtained at this point is used as the final adversarial example. Output.

2. The mobile covert counterattack method based on adjustable diffusion guidance as described in claim 1, characterized in that: In step S2, the class activation graph is... Directly used as a pixel-level semantic graph Among them, pixel-level semantic graph Used to characterize the semantic relevance and semantic importance of each pixel location in the original image to the target category discrimination result; for pixel-level semantic maps any pixel position ,have in, and These represent the height and width of the original image, respectively. for The semantic response value of a location is such that the larger the value, the greater the semantic contribution of the corresponding pixel to the target category and the higher its semantic importance.

3. The mobile covert counterattack method based on adjustable diffusion guidance as described in claim 2, characterized in that: In S2, a block-level semantic mask is constructed. The method is as follows: pixel-level semantic graph Divide into several sizes Non-overlapping image patches ,in , , , The semantic importance of each image patch is calculated using the Frobenius norm. in, Represents image blocks The semantic importance; set up Represents the set of semantic importance of all image patches. Percentiles, then block-level semantic mask Defined as: in Represents image blocks All pixel positions contained therein Indicates the pixel position.

4. The mobile covert counterattack method based on adjustable diffusion guidance as described in claim 3, characterized in that: In S2, a multi-granularity semantic mask is constructed. The method is as follows: In block-level semantic mask Each image patch retained Internally, the semantic importance of each pixel in the image patch is further evaluated, and the corresponding pixel activation values ​​are sorted, retaining only the pixels with activation values ​​located at the top of the image patch. Percentile pixels are used to achieve fine-grained semantic filtering within blocks; Among them, for the image block any pixel position within If it is in the pixel-level semantic graph Pixel activation values The first activation value greater than or equal to the activation value within the image patch Percentile threshold Then the pixel will be in the multi-granularity semantic mask. The corresponding position is recorded as 1, otherwise as 0; the multi-granularity semantic mask Represented as: in, Represents image blocks The first of all pixel activation values The percentile, the pixel activation value is equal to the corresponding pixel position in the pixel-level semantic map. The semantic response value in the text.

5. The mobile covert counterattack method based on adjustable diffusion guidance as described in claim 4, characterized in that: In S3, the original image After the forward diffusion process of the denoising diffusion implicit model, the initial state of the reverse diffusion is obtained. Each reverse diffusion time step, based on the current diffusion state And the diffusion noise of the previous time step Estimate the predicted clean image at the current time step. Its expression is: in, For the first The diffusion coefficient corresponding to each diffusion time step.

6. The mobile covert counterattack method based on adjustable diffusion guidance as described in claim 5, characterized in that: In S3, the integrated gradient of multiple proxy models It is obtained by averaging the adversarial gradients of all proxy models at the current time step, and is expressed as: in, For the number of proxy models, Indicates the first The adversarial objective function corresponding to each proxy model This represents the adversarial gradient of a proxy model at the current time step; This indicates that the adversarial objective function is relative to the current diffusion state. The gradient, i.e., the gradient of the first... The adversarial gradient of each proxy model at the current time step; the adversarial objective function is expressed as: in, Indicates the first A proxy model, This indicates that the surrogate model will predict clean images. Determined as a true category The probability of; The rescaling variance reduction gradient is constructed as follows: a surrogate model is randomly sampled at each backdiffusion time step. Calculate the performance of the surrogate model in predicting clean images. and the current diffusion state The gradient on the current time step is obtained by subtracting the gradient from the rescaling variance according to the following formula. : in, It is the rescaling factor, and , Indicates the first The adversarial objective function of the proxy model relative to the predicted clean image The gradient, wherein the rescaling factor is used to rescale the variance reduction term and serves as a control parameter for the trade-off between mobility and imperceptibility. Rescaling variance reduction gradient Equivalent to integrated gradient Reduce gradient with variance The weighted combination is expressed as: in, This is used to adjust the relative weights of the integrated gradient and the variance reduction gradient in the rescaling variance reduction gradient, in order to achieve an adjustable balance between adversarial sample migration and imperceptibility.

7. The mobile covert counterattack method based on adjustable diffusion guidance as described in claim 6, characterized in that: In step S4, the current diffuse noise is updated based on the rescaling variance reduction gradient, and the updated intermediate noise is represented as follows: in, To initialize the diffused noise, For the first The diffusion coefficient corresponding to each diffusion time step; Utilizing multi-granular semantic masks Element-by-element filtering of the noise update term yields the semantically constrained intermediate noise, expressed as follows: in, This represents element-wise multiplication. intermediate noise after semantic constraints is projected using budget constraints. By applying constraints, diffuse noise that satisfies the disturbance budget limit is obtained. Its expression is: in, Indicates based on Projection operations on the norm sphere, Preset disturbance budget.

8. The mobile covert counterattack method based on adjustable diffusion guidance as described in claim 7, characterized in that: In step S5, after completing the budget constraint projection, the following steps are taken: Perform a backsampling operation on the denoising diffusion implicit model to obtain the diffusion state at the next time step. Its expression is: in, For the first +1 diffusion coefficient corresponding to diffusion time step.

Images