A Visual Complexity-Guided Diffusion Model Adversarial Defense Method

Abstract

This invention proposes a visual complexity-guided diffusion model adversarial defense method, which includes the following steps: Step 1) Using the attacked video adversarial sample as input, calculate the visual complexity of the adversarial video sample frame by frame; Step 2) Select an appropriate diffusion purification scale based on the mapping relationship between the sample visual complexity and the diffusion purification scale; Step 3) Perform forward diffusion on the adversarial video sample, and based on the purification scale selected in Step 2), use Gaussian noise of appropriate intensity to submerge the adversarial perturbation while preserving the principal components of the video frame as much as possible to obtain the diffused video sample; Step 4) Perform reverse diffusion on the diffused video sample, and use a one-time denoising strategy to efficiently restore the diffused sample to the purified sample; Step 5) Input the purified benign video into an intelligent video system to obtain the correct decision and complete the defense against adversarial attacks.

Description

Technical Field

[0001] This invention belongs to the field of visual model adversarial defense, and particularly relates to a visual complexity-guided diffusion model adversarial defense method. Background Technology

[0002] Intelligent video systems are designed to automatically process and analyze video data, integrating multiple functions including target tracking and recognition in videos. Therefore, they are widely used in video surveillance, human-computer interaction, and other fields. However, research shows that existing deep neural network-based intelligent video systems are highly vulnerable to adversarial attacks. Once adversarial perturbations, imperceptible to the human eye, are added to naturally collected benign samples, these otherwise high-performance deep neural networks can easily be misled, leading to incorrect predictions. Accordingly, adversarial defense aims to eliminate the negative impact of adversarial attacks, thereby maximizing the reliability and security of intelligent video systems.

[0003] Previous adversarial defense methods have the following design shortcomings: 1) Insufficient robustness: Existing methods are unable to balance the performance of the target model on benign and adversarial samples. In other words, adversarial defense methods cannot enhance the robustness against adversarial samples without affecting the original performance of the model; 2) Insufficient versatility: Existing defense methods are usually designed for a specific adversarial attack or are highly coupled with the target model, while general-purpose defense methods can be applied to different types of adversarial attacks and target models; 3) Low computational efficiency: Since most existing defense methods are designed for image-level visual models, they do not adequately consider computational efficiency. Large-scale video data places high demands on the computational consumption and inference time of adversarial defense. Therefore, given the above-mentioned shortcomings of existing technologies, a robust, highly versatile, and efficient adversarial defense method is needed to address the security threats faced by intelligent video systems. Summary of the Invention

[0004] Objective: This invention designs an adversarial defense method based on a diffusion model. Before adversarial examples are input into the system, the diffusion model is used to cleanse them into benign examples. To address the high time consumption of the back-diffusion process, this invention employs a one-time denoising strategy, accelerating computation while ensuring effective cleansing. Furthermore, recognizing that cleansing complex scenes is often more challenging than simple ones, this invention designs a visual complexity-guided diffusion cleansing scale selection mechanism. This enables adaptive adjustment of the cleansing process, achieving a balance between adversarial perturbation elimination and high-quality image reconstruction. Therefore, this invention meets the requirements of robustness, versatility, and efficiency in adversarial defense method design, providing a practical guarantee for the secure use of intelligent video systems.

[0005] Technical Solution: To solve the above-mentioned technical problems and achieve the above-mentioned objectives, this invention proposes a visual complexity-guided diffusion model adversarial defense method, which includes the following steps:

[0006] Step 1) Take the video adversarial sample to be attacked as input and calculate the visual complexity of the adversarial video sample frame by frame;

[0007] Step 2) Select the diffusion cleanup scale based on the mapping relationship between sample visual complexity and diffusion cleanup scale;

[0008] Step 3) Forward diffusion is performed on the adversarial video samples. Based on the sanitization scale selected in Step 2), Gaussian noise is used to submerge the adversarial perturbation while retaining the principal components of the video frames to obtain the diffused video samples.

[0009] Step 4) Perform reverse diffusion on the diffused video samples and use a one-time denoising strategy to restore the diffused samples to high-quality clean samples;

[0010] Step 5) Input the purified, benign video into the intelligent video system to obtain the correct decision and complete the defense against adversarial attacks.

[0011] Furthermore, the method for step 1) is as follows:

[0012] Step 1.1) Given a naturally acquired benign video sequence X = [x 1 ,x 2 ,x 3 ,…,x N ], where N is the number of frames in the video segment, adversarial video sequence Defined as:

[0013]

[0014] Wherein, IVS(·) represents Intelligent Video System (IVS) based on deep neural networks. For the parameters of the system; r and These represent the output decisions of the intelligent video system for benign and adversarial video sequences, respectively. Since the distinction between benign and adversarial video sequences must be imperceptible to the human eye, we use l here. p Norm constraints are used to counteract the dimensions of disturbances. Generally, p = 0, 1, 2, ∞, while τ is the upper limit of the disturbance.

[0015] Step 1.2) Calculate the adversarial video sequence frame by frame. The edge energy is used as a measure of the visual complexity of the video frame, given the adversarial video frame of the nth frame. The frame is convolved using filters with a Sobel kernel along both the horizontal and vertical directions. Specifically, the horizontal Sobel operation can be represented as:

[0016]

[0017] Correspondingly, the vertical Sobel operation is denoted as:

[0018]

[0019] Based on G h With G v It can be calculated Edge density map G:

[0020]

[0021] Countering video frames Visual complexity can be calculated by taking the mean of G:

[0022]

[0023] Where, vc represents The visual complexity is given by H and W, which represent the height and width of the edge density map G, respectively, and i and j are the indices of the pixels in G.

[0024] Furthermore, in step 2), based on the mapping relationship between the visual complexity and the diffusion model purification scale, the optimal diffusion purification scale is selected. The operation process is as follows:

[0025]

[0026] Where Proj(vc) is the mapping function between visual complexity and diffusion sanitization scale, with the visual complexity vc of the adversarial video frame as the independent variable, controlling the parameters of the diffusion model sanitization scale. The optimal purification scale is obtained by taking (and monotonically decreasing as the diffusion time step T increases) as the dependent variable. Specifically, Proj(vc) is designed as a piecewise function: vc min With VC max It is a manually set visual complexity threshold, used to counteract video frames. Visual complexity VC is less than minimum visual complexity VC min or greater than the maximum visual complexity VC max At that time, the maximum purification scale is used directly. or minimum cleanliness level As the optimal diffusion purification scale; and when VC min ≤vc≤vcmax In this invention, a linear function is used as the relationship between VC and VC. Mapping relationship:

[0027] J(vc; w, b) = w·vc + b

[0028] in:

[0029]

[0030] Furthermore, based on the optimal purification scale obtained in step 2),

[0031] Step 3) Countering video frames Forward diffusion can be formally expressed as follows:

[0032]

[0033] Where ∈ represents Gaussian noise. For a diffused sample, as shown in the above formula, the granularity of forward diffusion is... Decision. Considering the need to counter video frames. It is composed of benign video frames x n It is obtained by combining with the counter-perturbation δ, that is, Therefore, the above equation can be rewritten as:

[0034]

[0035] Due to the imperceptibility of the human eye to counteract disturbances, and The value of δ decreases monotonically as the diffusion time step T increases, and its dimension is much smaller than that of the added Gaussian noise. Therefore, if the time step T... * With appropriate selection, adversarial information in the diffusion sample Gaussian noise While being submerged, Principal component information was preserved in benign image samples.

[0036] Furthermore, step 4) involves processing the diffusion sample obtained in step 3). Reverse diffusion is performed to generate high-quality cleaned samples. Considering the high computational demands of video tasks, this invention employs a one-time denoising strategy, directly denoising from the final diffusion time step T. * Directly predict the fully denoised image

[0037]

[0038] Where, ∈ θ(·) is a deep generative network model parameterized by θ. This invention uses a pre-trained DDIM diffusion model to predict the Gaussian noise ∈ added in step 3) through only one forward propagation. The T value is different from that set by the standard diffusion model. * Compared to 1000, this invention will use T * The value range of T is limited to [25, 60], that is, T min =25,T max =60, in decreasing ∈ θ (·) While reducing the prediction difficulty of ∈, it greatly improves the computational efficiency of the diffusion model. The one-time denoising strategy can achieve high-quality and high-efficiency disturbance elimination.

[0039] In step 5), the purified video sequence Input into the intelligent video system:

[0040]

[0041] Make That is, enabling intelligent video systems to make correct decisions, thereby completing the defense against adversarial attacks.

[0042] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0043] (1) The visual complexity-guided diffusion purification scale selection mechanism can enhance the quality of purified video sequences, thereby enhancing the robustness of intelligent video systems.

[0044] (2) The proposed defense method is independent of the intelligent video system being protected, and therefore has good versatility.

[0045] (3) Using a one-time denoising strategy can significantly reduce the computational and time consumption required for defense. Attached Figure Description

[0046] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0047] like Figure 1 As shown, this invention proposes a visual complexity-guided diffusion model adversarial defense method, which includes the following steps:

[0048] Step 1) Take the video adversarial sample to be attacked as input and calculate the visual complexity of the adversarial video sample frame by frame;

[0049] Step 2) Select the diffusion cleanup scale based on the mapping relationship between sample visual complexity and diffusion cleanup scale;

[0050] Step 3) Forward diffusion is performed on the adversarial video samples. Based on the sanitization scale selected in Step 2), Gaussian noise is used to submerge the adversarial perturbation while retaining the principal components of the video frames to obtain the diffused video samples.

[0051] Step 4) Perform reverse diffusion on the diffused video samples and use a one-time denoising strategy to restore the diffused samples to high-quality clean samples;

[0052] Step 5) Input the purified, benign video into the intelligent video system to obtain the correct decision and complete the defense against adversarial attacks.

[0053] Furthermore, the method for step 1) is as follows:

[0054] Step 1.1) Given a naturally acquired benign video sequence X = [x 1 ,x 2 ,x 3 ,…,x N ], where N is the number of frames in the video segment, adversarial video sequence Defined as:

[0055]

[0056] Wherein, IVS(·) represents Intelligent Video System (IVS) based on deep neural networks. For the parameters of the system; r and These represent the output decisions of the intelligent video system for benign and adversarial video sequences, respectively. Since the distinction between benign and adversarial video sequences must be imperceptible to the human eye, we use l here. p Norm constraints are used to counteract the dimensions of disturbances. Generally, p = 0, 1, 2, ∞, while τ is the upper limit of the disturbance.

[0057] Step 1.2) Calculate the adversarial video sequence frame by frame. The edge energy is used as a measure of the visual complexity of the video frame, given the adversarial video frame of the nth frame. The frame is convolved using filters with a Sobel kernel along both the horizontal and vertical directions. Specifically, the horizontal Sobel operation can be represented as:

[0058]

[0059] Correspondingly, the vertical Sobel operation is denoted as:

[0060]

[0061] Based on G h With Gv It can be calculated Edge density map G:

[0062]

[0063] Countering video frames Visual complexity can be calculated by taking the mean of G:

[0064]

[0065] Where, vc represents The visual complexity is given by H and W, which represent the height and width of the edge density map G, respectively, and i and j are the indices of the pixels in G.

[0066] Furthermore, in step 2), based on the mapping relationship between the visual complexity and the diffusion model purification scale, the optimal diffusion purification scale is selected. The operation process is as follows:

[0067]

[0068] Where Proj(vc) is the mapping function between visual complexity and diffusion sanitization scale, with the visual complexity vc of the adversarial video frame as the independent variable, controlling the parameters of the diffusion model sanitization scale. The optimal purification scale is obtained by taking (and monotonically decreasing as the diffusion time step T increases) as the dependent variable. Specifically, Proj(vc) is designed as a piecewise function: vc min With VC max It is a manually set visual complexity threshold, used to counteract video frames. Visual complexity VC is less than minimum visual complexity VC min or greater than the maximum visual complexity VC max At that time, the maximum purification scale is used directly. or minimum cleanliness level As the optimal diffusion purification scale; and when VC min ≤vc≤vc max In this invention, a linear function is used as the relationship between VC and VC. Mapping relationship:

[0069] J(vc; w, b) = w·vc + b

[0070] in:

[0071]

[0072] Furthermore, based on the optimal purification scale obtained in step 2),

[0073] Step 3) Countering video frames Forward diffusion can be formally expressed as follows:

[0074]

[0075] Where ∈ represents Gaussian noise. For a diffused sample, as shown in the above formula, the granularity of forward diffusion is... Decision. Considering the need to counter video frames. It is composed of benign video frames x n It is obtained by combining with the counter-perturbation δ, that is, Therefore, the above equation can be rewritten as:

[0076]

[0077] Due to the imperceptibility of the human eye to counteract disturbances, and The value of δ decreases monotonically as the diffusion time step T increases, and its dimension is much smaller than that of the added Gaussian noise. Therefore, if the time step T... * With appropriate selection, adversarial information in the diffusion sample Gaussian noise While being submerged, Principal component information was preserved in benign image samples.

[0078] Furthermore, step 4) involves processing the diffusion sample obtained in step 3). Reverse diffusion is performed to generate high-quality cleaned samples. Considering the high computational demands of video tasks, this invention employs a one-time denoising strategy, directly denoising from the final diffusion time step T. * Directly predict the fully denoised image

[0079]

[0080] Where, ∈ θ (·) is a deep generative network model parameterized by θ. This invention uses a pre-trained DDIM diffusion model to predict the Gaussian noise ∈ added in step 3) through only one forward propagation. The T value is different from that set by the standard diffusion model. * Compared to 1000, this invention will use T * The value range of T is limited to [25, 60] (i.e., T). min =25,T max =60), in reducing ∈ θ (·) While reducing the prediction difficulty of ∈, it significantly improves the computational efficiency of the diffusion model. Therefore, the one-time denoising strategy can achieve high-quality and high-efficiency perturbation elimination.

[0081] In step 5), the purified video sequence Input into the intelligent video system:

[0082]

[0083] Make That is, enabling intelligent video systems to make correct decisions, thereby completing the defense against adversarial attacks.

[0084] This invention defends against attacks on the target tracking model SiamRPN++ and the behavior recognition model Non-Local on the publicly available visual target tracking dataset OTB100 and behavior recognition dataset UCF101. Tables 1 and 2 show the defense effectiveness of this invention against the aforementioned intelligent video systems. Furthermore, this invention also has advantages in computational efficiency and time consumption, achieving 2.25 × 10⁻⁶ floating-point operations per second (FLOPs). 3 G, with a frame rate of 14.03 frames per second (FPS). Experimental results in real-world scenarios demonstrate that this invention can provide effective defense for intelligent video systems.

[0085] Table 1. Defense performance of this invention against the target tracking model SiamRPN++ on the OTB100 dataset.

[0086] Original performance Post-attack performance Post-defense performance Tracking success rate 69.6％ 32.4％ 61.1％ Tracking accuracy 91.4％ 47.1％ 83.3％

[0087] Table 2 shows the effectiveness of this invention in defending against non-local behavior recognition models on the UCF101 dataset.

[0088] Original performance Post-attack performance Post-defense performance Recognition accuracy 94.6％ 27.3％ 80.5％

Claims

1. A visual complexity-guided diffusion model adversarial defense method, characterized in that, The method includes the following steps: Step 1) Take the video adversarial sample to be attacked as input and calculate the visual complexity of the adversarial video sample frame by frame; Step 2) Select the diffusion cleanup scale based on the mapping relationship between sample visual complexity and diffusion cleanup scale; Step 3) Forward diffusion is performed on the adversarial video samples. Based on the sanitization scale selected in Step 2), preset Gaussian noise is used to submerge the adversarial perturbation while retaining the principal components of the video frames to obtain the diffused video samples. Step 4) Perform reverse diffusion on the diffused video samples and use a one-time denoising strategy to restore the diffused samples to clean samples; Step 5) Input the purified, benign video into the intelligent video system to obtain the correct decision and complete the defense against adversarial attacks.

2. The visual complexity-guided diffusion model adversarial defense method according to claim 1, characterized in that, The method for step 1) is as follows: Step 1.1) Set the naturally acquired benign video sequence X = [x 1 x 2 x 3 , ..., x N ], where N is the number of frames in the video segment, adversarial video sequence Defined as: Where IVS(·) represents an intelligent video system based on deep neural networks. Let r be the parameter of the system. These represent the output decisions of the intelligent video system for benign and adversarial video sequences, respectively, with τ being the upper limit of the perturbation. Step 1.2) Calculate the adversarial video sequence frame by frame. The edge energy is used as a measure of the visual complexity of the video frame, given the adversarial video frame of the nth frame. The frame is convolved using a filter with a Sobel kernel along both the horizontal and vertical directions. The Sobel operation in the horizontal direction is represented as: The vertical Sobel operation is denoted as: Based on G h With G v Calculate Edge density map G: Countering video frames Visual complexity is calculated by taking the mean of G: Where, vc represents The visual complexity is represented by H and W, which represent the height and width of the edge density map G, respectively, and i and j are the indices of the pixels in G.

3. The visual complexity-guided diffusion model adversarial defense method according to claim 2, characterized in that, In step 2), based on the mapping relationship between the visual complexity and the diffusion model purification scale, the optimal diffusion purification scale is selected. The operation process is as follows: Where Proj(vc) is the mapping function between visual complexity and diffusion sanitization scale, with the visual complexity vc of the adversarial video frame as the independent variable, controlling the parameters of the diffusion model sanitization scale. As the dependent variable, it decreases monotonically as the diffusion time step T increases. Among them, VC min With VC max It is a set visual complexity threshold, which is used to counteract video frames. Visual complexity VC is less than minimum visual complexity VC min or greater than the maximum visual complexity VC max At that time, the maximum purification scale is used directly. or minimum cleanliness level As the optimal diffusion purification scale; when vc min ≤vc≤vc max When using linear functions as VC and Mapping relationship: J(vc; w, b) = w·vc + b in:

4. The visual complexity-guided diffusion model adversarial defense method according to claim 3, characterized in that, Step 3) Countermeasure video frames Forward diffusion can be formally expressed as follows: Where ∈ represents Gaussian noise. For the diffused samples, I is the variance of the standard Gaussian noise, taking into account adversarial video frames. It is composed of benign video frames x n It is obtained by combining with the counter-perturbation δ, that is The above equation can be rewritten as: Based on the obtained optimal purification scale Adversarial information in diffused samples Gaussian noise While being submerged, By preserving the principal components of the video frames, diffuse video samples are obtained.

5. The visual complexity-guided diffusion model adversarial defense method according to claim 4, characterized in that, In step 4), the diffused video samples are back-diffused, and a one-time denoising strategy is used to restore the diffused samples to clean samples, as follows: Where, ∈ θ (·) is a deep generative network model parameterized by θ, which uses a pre-trained DDIM diffusion model to simultaneously remove adversarial perturbations δ and Gaussian noise ∈ through only one forward propagation.

6. The visual complexity-guided diffusion model adversarial defense method according to claim 5, characterized in that, In step 5), the purified video sequence Input into the intelligent video system: Make That is, enabling intelligent video systems to make correct decisions and complete the defense against adversarial attacks.

Images