Confrontation sample generation method and system based on space-time double-domain momentum fusion

Abstract

The invention discloses an adversarial sample generation method and system based on space-time double-domain momentum fusion, and aims to solve the problems of poor sample mobility, insufficient disturbance stability, low attack efficiency and easy detection of generated samples in an existing gradient-based adversarial attack algorithm. The invention provides a space-time enhanced momentum iteration fast gradient symbol method (TEMI-FGSM), and global convergence and disturbance stability generation of the gradient direction are realized by fusing multiple randomly transformed gradient directions of the same image in a space momentum domain and introducing a cross-iteration momentum accumulation mechanism in a time momentum domain. The confrontation sample generated by the method shows stronger aggressiveness and mobility in both the white box scene and the black box scene, and keeps higher concealment visually. When the method is used in combination with adversarial training, the robustness and generalization ability of the target model can be effectively improved. The system provided by the invention is simple in structure and stable in training, and is suitable for a robustness test, security defense and algorithm evaluation platform of a deep neural network model.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, artificial intelligence security, and deep learning, specifically to a method and system for generating adversarial examples based on spatiotemporal momentum fusion. Background Technology

[0002] Deep neural networks (DNNs) are widely used in finance, healthcare, and autonomous driving, but their adversarial robustness is poor. Attackers can construct adversarial examples by adding small perturbations to the input data, causing the model to output incorrect results. Existing attack methods based on algorithms such as FGSM and MI-FGSM show good performance in terms of generation efficiency and attack success rate, but the following problems still exist: (1) The attack transferability is insufficient, and it is difficult to maintain consistent performance of the generated samples across different models; (2) Momentum updates only consider the time dimension and ignore the spatial transformation characteristics of the input samples; (3) The direction of the disturbance is greatly affected by local gradient fluctuations and is prone to getting trapped in local optima; (4) The samples generated during adversarial training are singular, making it difficult to improve the model's generalization robustness.

[0003] Therefore, there is an urgent need for an adversarial example generation method that can simultaneously introduce momentum mechanisms in both the temporal and spatial dimensions. On the one hand, by accumulating gradient information over multiple iterations, the perturbation direction can be stabilized and the adverse effects of single-step gradient fluctuations can be reduced. On the other hand, by applying various random spatial transformations (such as translation, rotation, scaling, shearing, and random cropping) to the input samples, the spatial invariance of the input can be explicitly modeled, so that gradient updates no longer depend on a single image structure, thereby avoiding the local optima problem caused by spatial transformation sensitivity in traditional methods.

[0004] Furthermore, existing methods such as FGSM, I-FGSM, and MI-FGSM are all based solely on "single-path gradients," meaning the gradient information of the same image at a single scale and with a fixed spatial layout. The direction of the perturbation is entirely constrained by the spatial structure of the original sample. When faced with different model structures (ResNet / VGG / Inception, etc.), this type of perturbation lacks cross-spatial consistency and generalization, resulting in weak transferability. Moreover, due to the lack of spatial diversity information, traditional momentum iteration methods struggle to generate diverse perturbations, often leading to "pattern collapse" during adversarial training. This results in overfitting of the model to a particular type of attack, hindering the improvement of generalization robustness.

[0005] Therefore, a joint momentum mechanism is needed that can simultaneously integrate "cross-transformation space gradient" and "cross-time iterative gradient," enabling the perturbation direction to achieve stable convergence in the time dimension while also achieving diversity and global consistency in the spatial dimension. By constructing a temporal-spatial dual-domain momentum fusion framework, the limitations of traditional FGSM series algorithms can be overcome, enabling the generated adversarial examples to have stronger cross-model transfer capabilities, a more stable perturbation generation process, and to provide high-quality samples with variability and controllability for adversarial training. Summary of the Invention

[0006] Objective of the Invention: To address the shortcomings of the existing technologies, this invention provides a method and system for generating adversarial examples based on spatiotemporal dual-domain momentum fusion. High-quality adversarial examples are generated using the Spatiotemporal Enhanced Momentum Iterative Fast Gradient Signed Method (TEMI-FGSM) and applied to adversarial training of image classification models, thereby improving the effectiveness of adversarial attacks and enhancing model robustness.

[0007] Technical Solution: An adversarial example generation method based on spatiotemporal dual-domain momentum fusion introduces spatial momentum domain fusion and temporal momentum domain accumulation mechanisms into the FGSM framework, and generates high-quality adversarial examples based on the spatiotemporal enhanced momentum iterative fast gradient sign method, which are then used for adversarial training of image classification models. The method includes the following steps: (1) Perform several random geometric transformations on the input sample, including rotation, clipping or translation, calculate the gradient of each transformed sample, and perform weighted fusion to obtain the spatial momentum; Obtain the target model F and the original samples x Let the upper limit of the disturbance amplitude be Initialization time momentum spatial momentum =0; Spatial momentum fusion calculation: for input samples x Perform various random transformations (such as rotation, cropping, translation, etc.), calculate the gradients of these transformed samples, and then weight and fuse them. Weights for different transformation operations:

[0008] (2) Accumulate the gradient information of the previous moment in each iteration to form time momentum, and weight and fuse the spatial momentum and time momentum to generate the total momentum vector; The fusion representation of spatial momentum is:

[0009] The computational representation of time momentum update fusion is as follows:

[0010] In the formula, This indicates random transformations performed on the image, such as rotation, translation, scaling, and cropping. This represents the true class label corresponding to the input sample. This represents the set of trainable parameters for the target model. This indicates the calculation of the gradient. For the current gradient, The momentum decay coefficient, and Representing the first i Spatial momentum and temporal momentum of the next iteration This represents the expectation of all stochastic geometric transformations under the transformation distribution T, used to fuse gradient information from different spatial transformations. Represents the weights of different transformation operations. For the first i Adversarial examples in the next iteration; (3) Update the adversarial perturbation and generate adversarial samples based on the total momentum direction. The total momentum fusion process is defined as follows:

[0011] in Spatial momentum, For time momentum, These correspond to the weighting coefficients for time momentum and spatial momentum, respectively.

[0012] Furthermore, spatial momentum is obtained by calculating a weighted average of the gradients after performing stochastic transformations, including rotation, translation, scaling, and cropping, on the input samples, including: Random padding: Dynamically add a padding region of 0-12 pixels around the input image. The padding values ​​are uniformly sampled from the range [0,1]. The padding operation is represented as:

[0013] Based on the above operation to eliminate the special characteristics of image edge pixels, in the formula, This indicates a fill operation. Indicates the sample Perform a four-sided fill operation and output the filled image. U is a uniform distribution that describes the random value selection rule. Each value within the interval has an equal probability, ensuring the randomness of the operation.

[0014] After filling, the image is cropped proportionally to 0.8-1.0 times its original size. The top-left corner coordinates are randomly generated. The output after the clipping operation is represented as follows: The formula is:

[0015] The operation represented by the above formula breaks the original image boundaries, expanding the region of interest of the target model F from local pixels to the global scope. This introduces spatial diversity to enhance the robustness of gradient calculation, providing a basic context for subsequent enhancement transformations. It is the clipping operation function. It is the vertical coordinate of the top left corner of the cropped area (offset from the top of the filled image). t (pixels) This represents the horizontal coordinate of the top-left corner of the cropped area (offset from the left side of the filled image). (pixels) This represents the scale factor, which corresponds to the size of the cropped area. and These represent height and width, respectively. Translation: Lateral offset Pixels, Vertical Offset Pixels simulate the tiny displacements of a target within an image; Rotation: Angle It covers common image deflection scenarios; Scaling: Scale Factor s ∈[0.9,1.1], simulating changes in near and far perspective; Shear: Shear angle This introduces nonlinear geometric deformation; The comprehensive transformation formula is: This represents a transformation function, and by setting the parameter range, the transformation can include common natural disturbances without destroying the semantic information of the image.

[0016] The spatial momentum domain fusion and temporal momentum domain accumulation mechanism described in this method uses the principle of ensemble learning to ensure that the gradient direction is not affected by the local extrema of a single transformation.

[0017] Furthermore, during the total momentum fusion process, the adaptive weights are iterated in the following manner: First 1 / 3 iterations: The focus is on the rapid convergence of time momentum to the initial perturbation direction; Middle 1 / 3 iteration: Balancing spacetime momentum to explore the global optimum; Last 1 / 3 iterations: Emphasis is placed on enhancing spatial momentum to improve resistance to transformation; This division is based on the phased characteristics of attack iteration. In the early stage, time momentum is needed to guide the rapid approximation of the misclassified region, and in the later stage, spatial momentum is needed to ensure the stability of the perturbation under transformation.

[0018] In the method described above, adversarial examples are generated using the following formula: in For disturbance intensity parameters, Represents the fusion momentum gradient A symbolic operation is performed to extract the direction of the perturbation update. It is the fusion gradient of spatiotemporal momentum.

[0019] Furthermore, momentum perturbation Simultaneously constrained by both L∞ and L1 norms to balance perturbation amplitude and concealment, the mathematical expression is:

[0020] in, This is an adjustment coefficient used to balance the constraint weights of the two norms.

[0021] In training the target model F, clean samples and adversarial samples are mixed and input into the model for iterative training to improve the model's robustness; a mixed loss function is defined as follows:

[0022] in These are balancing parameters used to adjust the weights of clean sample loss and adversarial sample loss during training. This indicates that the target model is such as VGG, ResNet, etc. This indicates that the model is based on the input samples. x and real labels y The loss function value (e.g., cross-entropy loss).

[0023] Furthermore, the method supports both white-box and black-box attack modes, and the generated adversarial examples have cross-model transfer capabilities, including image classification models.

[0024] On the other hand, the present invention also provides an image classification model robustness enhancement system, which uses the adversarial example generation method based on spatiotemporal dual-domain momentum fusion to obtain adversarial examples, and performs training of the target model F based on the generated adversarial examples; The system includes: Input module: Used to load the original samples and the target model to be trained; Momentum fusion module: Performs spatial momentum calculation and temporal momentum accumulation and fusion; Adversarial generation module: Generates and outputs adversarial examples based on fused momentum; Training module: Trains the model using a mixture of clean and adversarial samples and outputs a robust model.

[0025] Furthermore, the system includes an evaluation module that calculates the attack success rate, mobility, and stealth index of adversarial examples.

[0026] Beneficial effects: The substantial features and significant advancements of this invention include the introduction of spatial momentum domain fusion and temporal momentum domain accumulation mechanisms on the traditional FGSM framework to achieve global convergence of gradient directions, thereby generating adversarial samples with greater attack transferability and stealth, which can be used for adversarial training to enhance model robustness. It also includes: (1) Dual-domain momentum fusion: Combining spatial and temporal momentum information, the generated perturbation direction is smoother, improving the global consistency of adversarial samples; (2) Strong attack transferability: The success rate of transfer between different model architectures (ResNet, VGG, LeNet) is improved by about 10%; (3) Significant improvement in defense performance: After adversarial training based on generated samples, the robustness accuracy is improved by an average of about 15%; (4) Strong universality: It can be applied to various tasks such as image classification and object detection. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the overall process of the method described in this invention. Figure 2 A schematic diagram of a spatiotemporal enhanced momentum fusion structure; Figure 3 This is a schematic diagram of the overall structure of the model robustness enhancement system; Figure 4 This is a comparison chart of experimental results (comparison of TEMI-FGSM with FGSM, MI-FGSM, C&W, and DeepFool). Detailed Implementation

[0028] To more clearly illustrate the technical solution of the present invention, a method and system for generating adversarial samples based on a spatiotemporal enhanced momentum mechanism will be described in detail with reference to the accompanying drawings and embodiments.

[0029] Combination Figure 1-3 As shown, based on the technical solution provided by this invention, it is possible to construct such a... Figure 3 The image classification model robustness enhancement system shown is used for training computer vision, artificial intelligence security, and deep learning models. This invention generates high-quality adversarial examples based on the Spatiotemporal Enhanced Momentum Iterative Fast Gradient Symbolic Method (TEMI-FGSM) and uses them for adversarial training of image classification models, thereby improving the effectiveness of adversarial attacks and enhancing model robustness.

[0030] The image classification model robustness enhancement system mainly includes the following functional modules: input data preprocessing module, spatiotemporal gradient enhancement module, momentum perturbation generation module, adversarial example generation module, model robustness training module, and evaluation and optimization module. The system uses the TEMI-FGSM algorithm as its core to achieve multi-level gradient perception, spatiotemporal feature aggregation, and adaptive perturbation generation of input samples, thereby improving the quality of generated adversarial examples and the robustness of the model.

[0031] Step S1: Construction of the input data preprocessing module.

[0032] This module is used to standardize and normalize the input sample set, laying the foundation for subsequent perturbation generation. The system receives the original input samples. x Perform the following operations: The input image is scaled and channelized to ensure that each pixel value falls within the range [0,1][0,1][0,1]. Calculate the initial prediction result y=F(x) of the target model F, and record the gradient information corresponding to the prediction. The target model includes deep learning architectures such as ResNet-18, ResNet-50, or VGG-19.

[0033] In batch mode, preprocessed samples and corresponding gradients are cached to support subsequent spatiotemporal feature calculations.

[0034] Step S2: Construction of the spatiotemporal gradient enhancement module

[0035] This module corresponds to the Temporal-Spatial Enhancement (TSE) mechanism in the TEMI-FGSM algorithm, used to jointly model gradient information in both temporal and spatial dimensions, thereby improving directionality and stability against perturbations. The module consists of two sub-units: S201 Temporal Gradient Aggregator This unit defines a momentum accumulation term based on historical gradient information accumulated during multiple iterations:

[0036] in, This is the momentum decay coefficient. Dynamic consistency of the perturbation direction is achieved through cumulative smoothing of historical gradients.

[0037] S202 Spatial Feature Attention Unit

[0038] Based on various spatial transformations of the input samples, such as rotation, translation, scaling, and flipping, specifically: Random padding involves dynamically adding a 0-12 pixel padding region around the input image, with the padding values ​​uniformly sampled from the [0,1] interval. The formula is as follows:

[0039] The core function of this operation is to eliminate the special characteristics of edge pixels in images: traditional image edge pixels lack context, which can easily lead to the model learning local bias features. After filling, the edge pixels are integrated into a new virtual context, forcing the model to focus on the content itself rather than location information.

[0040] After filling, crop proportionally to 0.8-1.0 times the original image size. The coordinates of the top-left corner are randomly generated using the following formula:

[0041] This operation breaks the original image boundaries, expanding the model's region of interest from local pixels to the global area. By introducing spatial diversity, it enhances the robustness of gradient calculation and provides a basic context for subsequent enhancement transformations.

[0042] Translation: Lateral offset Pixels, Vertical Offset Pixels simulate the minute displacement of a target within an image; rotation: angle. Covers common image deflection scenarios; Scaling: Scale factor s ∈[0.9,1.1], simulating changes in near and far perspective; shearing: shearing angle This introduces nonlinear geometric deformation. The comprehensive transformation formula is:

[0043] Calculate the transformed spatial momentum:

[0044] in The weights, varying in different ways, are used in this weight matrix to amplify the perturbation contribution in salient regions within the spatial domain, thereby concentrating the perturbation more in the model's decision-sensitive regions. This indicates random transformations performed on the image, such as rotation, translation, scaling, and cropping.

[0045] Step S3: Construction of the momentum perturbation and adversarial sample generation module.

[0046] This module implements the core perturbation generation process of TEMI-FGSM and is a key part of the system. Perturbation generation follows the calculation logic below: In each iteration t, the perturbation direction is calculated based on the accumulated gradient: ; The perturbation is superimposed on the input sample to obtain a temporary adversarial sample: ; And limit the disturbance amplitude to satisfy: .

[0047] The above expression is used to calculate and control the stealth of adversarial examples.

[0048] Dynamically update momentum term The process is repeated iteratively until the preset number of steps T is reached.

[0049] This process integrates both temporal and spatial enhancement mechanisms, which, compared to traditional FGSM / MI-FGSM, can significantly improve the directional stability of perturbations and the transferability of samples while ensuring the success rate of attacks.

[0050] Step S4: Model Robustness Training, Evaluation, and Optimization Module

[0051] This module uses high-quality adversarial examples generated by TEMI-FGSM for training to enhance model robustness. The training process is as follows: Mix clean samples in each training batch x Corresponding adversarial examples This forms the input set; Define the mixed loss function:

[0052] in For balancing parameters.

[0053] Use an adaptive learning rate optimizer (such as Adam) to update parameters so that the model gradually learns robust features to perturbations; The perturbation budget and number of iterations are dynamically increased as the training progresses, ensuring that the model adapts stepwise from low-intensity attacks to high-intensity attacks.

[0054] After the system completes training, the model is evaluated using various standard attack algorithms (FGSM, I-FGSM, DeepFool, C&W, etc.). This includes calculating the attack success rate and model robustness.

[0055] The results show that the TEMI-FGSM samples generated by the system of this invention improve the average attack success rate by about 12% in black-box migration attacks, while the robust model improves by more than 30% under various attacks, verifying the effectiveness and generalization ability of the system.

Claims

1. A method for generating adversarial examples based on spatiotemporal dual-domain momentum fusion, characterized in that, This method introduces a spatial momentum domain fusion and temporal momentum domain accumulation mechanism into the FGSM framework, and generates high-quality adversarial examples based on the spatiotemporal enhanced momentum iterative fast gradient sign method, which are then used for adversarial training of an image classification model. The method includes the following steps: (1) Perform several random geometric transformations on the input sample, including rotation, cropping or translation, calculate the gradient of the transformed sample after the random geometric transformation, and perform weighted fusion to obtain the spatial momentum; (2) Accumulate the gradient information of the previous moment in each iteration to form time momentum, and weight and fuse the spatial momentum and time momentum to generate the total momentum vector; The fusion representation of spatial momentum is: The computational representation of time momentum update fusion is as follows: In the formula, This indicates random transformations performed on the image, such as rotation, translation, scaling, and cropping. This represents the true class label corresponding to the input sample. This represents the set of trainable parameters for the target model. Let the gradient of the loss function with respect to the adversarial examples be given in the current iteration step. The momentum decay coefficient, and Representing the first i Spatial momentum and temporal momentum of the next iteration This represents the expectation of all stochastic geometric transformations under the transformation distribution T, used to fuse gradient information from different spatial transformations. Represents the weights of different transformation operations. For the first i Adversarial examples in the next iteration; (3) Update the adversarial perturbation and generate adversarial samples based on the total momentum direction. The total momentum fusion process is defined as follows: in Spatial momentum, For time momentum, These are the weighting coefficients.

2. The adversarial example generation method based on spatiotemporal dual-domain momentum fusion according to claim 1, characterized in that, Spatial momentum is obtained by calculating a weighted average of the gradients after performing stochastic transformations, including rotation, translation, scaling, and cropping, on the input sample. Random padding: Dynamically add a padding region of 0-12 pixels around the input image, with the padding value uniformly sampled from the range [0,1]. The formula is as follows: This method is based on the above operation to eliminate the specificity of pixels at the image edges; After filling, crop proportionally to 0.8-1.0 times the original image size. The coordinates of the top-left corner are randomly generated using the following formula: The operation represented by the above formula breaks the original image boundary, expands the region of interest of the target model F from local pixels to the global, enhances the robustness of gradient calculation by introducing spatial diversity, and provides a basic context for subsequent enhancement transformation. Translation: Lateral offset Pixels, Vertical Offset Pixels simulate the tiny displacements of a target within an image; Rotation: Angle It covers common image deflection scenarios; Scaling: Scale Factor s ∈[0.9,1.1], simulating changes in near and far perspective; Shear: Shear angle This introduces nonlinear geometric deformation; The comprehensive transformation formula is: By setting the parameter range, the transformation can include common natural disturbances without destroying the semantic information of the image.

3. The adversarial example generation method based on spatiotemporal dual-domain momentum fusion according to claim 2, characterized in that, The spatial momentum domain fusion and temporal momentum domain accumulation mechanism described in this method uses the principle of ensemble learning to ensure that the gradient direction is not affected by the local extrema of a single transformation.

4. The adversarial example generation method based on spatiotemporal dual-domain momentum fusion according to claim 1, characterized in that, During the total momentum fusion process, the adaptive weights are iterated in the following manner: First 1 / 3 iterations: The focus is on the rapid convergence of time momentum to the initial perturbation direction; Middle 1 / 3 iteration: Balancing spacetime momentum to explore the global optimum; Last 1 / 3 iterations: Emphasis is placed on enhancing spatial momentum to improve resistance to transformation; This division is based on the phased characteristics of attack iteration. In the early stage, time momentum is needed to guide the rapid approximation of the misclassified region, and in the later stage, spatial momentum is needed to ensure the stability of the perturbation under transformation.

5. The adversarial example generation method based on spatiotemporal dual-domain momentum fusion according to claim 1, characterized in that, Adversarial examples are generated using the following formula: in For disturbance intensity parameters, Represents the fusion momentum gradient A symbolic operation is performed to extract the direction of the perturbation update. It is the fusion gradient of spatiotemporal momentum.

6. The adversarial example generation method based on spatiotemporal dual-domain momentum fusion according to claim 1 or 5, characterized in that, momentum disturbance Simultaneously constrained by both L∞ and L1 norms to balance perturbation amplitude and concealment, the mathematical expression is: in, This is an adjustment coefficient used to balance the constraint weights of the two norms.

7. The adversarial example generation method based on spatiotemporal dual-domain momentum fusion according to claim 1, characterized in that, To improve model robustness, a mixture of clean and adversarial samples is used as input to the model for iterative training; a mixed loss function is defined as follows: in These are balancing parameters used to adjust the weights of clean sample loss and adversarial sample loss during training. Representing the target model, This indicates that the model is based on the input samples. x and real labels y The loss function value under the given conditions.

8. The adversarial example generation method based on spatiotemporal dual-domain momentum fusion according to claim 1, characterized in that, The method supports both white-box and black-box attack modes, and the generated adversarial examples have cross-model transfer capabilities, including image classification models.

9. A robustness enhancement system for an image classification model, characterized in that, The system uses the adversarial example generation method based on spatiotemporal dual-domain momentum fusion as described in any one of claims 1-8 to obtain adversarial examples, and performs training of the target model F based on the generated adversarial examples; The system includes: Input module: Used to load the original samples and the target model to be trained; Momentum fusion module: Performs spatial momentum calculation and temporal momentum accumulation and fusion; Adversarial generation module: Generates and outputs adversarial examples based on fused momentum; Training module: Trains the model using a mixture of clean and adversarial samples and outputs a robust model.

10. The image classification model robustness enhancement system according to claim 9, characterized in that, The system is equipped with an evaluation module, which is used to calculate the attack success rate, mobility, and stealth index of adversarial examples.

Images