An integrated attack method using unsystematic preprocessing and dynamic weights

Abstract

The present disclosure provides an integrated attack method using non-systematic preprocessing and dynamic weight. It comprises the following steps: step 1: selecting an experimental data set and determining a model; step 2: performing input sample data enhancement; step 3: multi-model dynamic optimization integrated attack, generating an integrated model of the adversarial sample; step 4: model weight rebalancing strategy, generating an optimized adversarial sample; step 5: cyclically executing steps 3 to 4 until exceeding the set iteration threshold, updating the adversarial sample; step 6: cycle end, output the final integrated model generated adversarial sample; step 7: using the obtained adversarial attack sample to attack the target black box model, calculating the success rate of the adversarial sample attack model. The present application can monitor and optimize the contribution of each agent model to the attack effect, significantly improve the migration ability of the adversarial sample among different models, and show obvious technical progress and advantages, especially for models with large structural differences.

Description

Technical Field

[0001] This disclosure relates to the fields of computer vision and deep learning security, and more specifically, to an integrated attack method that utilizes unsystematic preprocessing and dynamic weights. Background Technology

[0002] With the significant advancements in deep learning technology across various fields such as image recognition, natural language processing, and recommender systems, its application in computer vision has been particularly prominent, achieving or even surpassing human-level image recognition capabilities. However, with the widespread application of these technologies, especially in critical areas such as aerospace, autonomous vehicles, and security monitoring, security issues have gradually emerged, highlighting the importance of the security of deep neural network models.

[0003] Adversarial attacks, which mislead model predictions by adding subtle perturbations to input samples, not only pose a threat to deep learning models but also serve as a tool for evaluating and enhancing model robustness. Research on adversarial attacks helps reveal potential vulnerabilities in models and, through training with adversarial examples, enriches the model's learning sample space, thereby improving its robustness. Although adversarial attacks are divided into white-box and black-box attacks, in real-world applications, black-box attacks are more prevalent and practical due to limitations in prior knowledge.

[0004] Against this backdrop, improving the transferability of adversarial examples, even those capable of successfully attacking unknown black-box models, has become a key research focus. The success rate of transferability depends on the similarity between the proxy model and the target model. Existing research primarily aims to improve transferability by enhancing the commonalities between adversarial examples and multiple models, such as by maximizing perturbations of key neurons or introducing momentum. However, these methods have limited effectiveness when facing models with significant structural differences.

[0005] Model ensemble methods have emerged based on the assumption that if an adversarial example can deceive multiple models with different structures, its success rate in attacking unknown models is also higher. However, most ensemble methods simply combine the outputs of different models, failing to fully consider the differences in the contribution of each model to the attack, thus limiting the improvement of the transferability of adversarial examples.

[0006] Previous attempts to improve the transferability of adversarial examples have primarily focused on enhancing their generalization ability through standardized input preprocessing techniques (such as scaling, cropping, and rotation). However, these methods often employ a fixed preprocessing workflow applied to all models, neglecting the fact that different models may have varying sensitivities to different types of input perturbations. While simple and easy to implement, this "one-size-fits-all" preprocessing approach often limits the transferability of adversarial examples in practical applications due to a lack of sufficient input diversity. In other fields, such as image enhancement and data augmentation, non-uniform, targeted transformation strategies have proven to significantly improve the generalization ability and robustness of models, providing new insights into the field of adversarial example generation.

[0007] The relevant patents at home and abroad mainly include a method for generating adversarial examples of images based on black-box scenarios [CN113537494B], an adversarial example generation method based on attention heatmap and ensemble learning [CN115311708A], a highly transferable adversarial example generation method and system [CN116011558B], a transferable image adversarial example generation and deep neural network testing method and system [CN114741310A], and a transferable image adversarial example generation device and method [CN112215292B].

[0008] Analysis of the aforementioned patents revealed the following shortcomings:

[0009] (1) Black-box attacks are time-consuming due to their high query costs. Therefore, when implementing black-box adversarial attacks, white-box proxy models are often relied upon for migration attacks. However, adversarial examples generated by relying on a single proxy model often face the problem of low attack success rate;

[0010] (2) In the process of integrating multiple proxy models to combat attacks, the different contributions of each proxy model structure to the attack are not taken into account. The logic, probability and loss of integration do not really consider the fundamental reasons for improving portability, which makes it impossible to maximize the potential of the integrated model.

[0011] (3) Traditional adversarial example generation methods often introduce a unified input preprocessing technique for data augmentation. Although this method is simple and easy to implement, it often leads to a lack of sufficient input diversity in the generated adversarial examples because it ignores the differences in sensitivity of different models to different types of input perturbations, thus limiting their transferability between different models. In order to solve the above technical problems,

[0012] To address the aforementioned technical problems, this invention provides an ensemble attack method utilizing unsystematic preprocessing and dynamic weights. This method aims to evaluate and enhance the robustness of the model by combining it with other basic adversarial example generation techniques. First, the method employs unsystematic preprocessing techniques to transform the input data into different models using various input transformation methods, enhancing the input diversity of the ensemble model. Then, it monitors and adjusts the contribution weight of each model to the generated adversarial examples through multi-model dynamic tuning and weight rebalancing strategies. This strategy not only improves the transferability of adversarial examples but also adapts to various proxy models, greatly enhancing the generalization ability of generated adversarial examples. Summary of the Invention

[0013] The purpose of this disclosure is to provide an integrated attack method utilizing unsystematic preprocessing and dynamic weights, aiming to evaluate and enhance the robustness of models by combining it with other basic adversarial example generation techniques. Traditional adversarial example generation techniques mainly focus on averaging the attack effects of multiple models, without fully considering the contribution of differences between models to the attack. Existing techniques cannot effectively adapt to the characteristics of different models, nor can they provide adversarial examples with strong adaptability and high generalization ability.

[0014] In general, an integrated attack method utilizing unsystematic preprocessing and dynamic weights is provided, comprising the following steps:

[0015] Step 1: Select CIFAR_10 as the experimental dataset and determine four white-box proxy models and multiple black-box models;

[0016] Step 2: Apply different random input transformation strategies to the four white-box models to augment the input sample data, and save the transformed data as a copy of the benign sample.

[0017] Step 3: Ensemble attack with dynamic tuning of multiple models. For a given image, a white-box substitution model is used for the attack. The next updated adversarial attack sample is predicted, the contribution of each model to the attack effect is identified, the weights of each model are determined, and adversarial samples of the ensemble model are generated.

[0018]

[0019] Step 4: Model Weight Rebalancing Strategy. Based on the prediction performance of the adversarial examples on the sub-models after the initial ensemble, the model weights are readjusted:

[0020]

[0021] ω new_1 ,ω new_2 ...ω new_k =softmax(ρ new_1,ρ new_2 ...ρ new_3 )

[0022] Predictive performance is reflected by the loss value of the adversarial example on the model. A higher loss value indicates a lower probability of accurate prediction for that adversarial example, and a greater contribution of that model to the adversarial objective when using an ensemble model. Finally, optimized adversarial examples are generated based on normalized weights.

[0023]

[0024] Step 5: Repeat steps 3 to 4 until the set iteration threshold is exceeded. Adjust the model fusion weights based on the fusion attack of multi-model dynamic optimization integration and weight rebalancing, and update the adversarial examples.

[0025] Step 6: The loop ends, and the final adversarial examples generated by the ensemble model are output.

[0026] Step 7: Use the obtained adversarial attack sample to attack the target black-box model.

[0027] The white-box proxy models are ResNet-18, Inception v3, ViT-Tiny, and DeiT-Tiny.

[0028] The black-box models are ResNet-50, WideResNet-50, BiT-M-R50, BiT-M-R101, ViT-Base, DeiT-Base, Swin-Base, and Swin-Small.

[0029] The input sample data augmentation includes random noise or random resizing.

[0030] Step 3 includes the following sub-steps:

[0031] Step 3.1: For the adversarial examples generated by the ensemble model, for an image x of arbitrary size that serves as the input to the deep neural network in the image recognition model, set the starting point for the adversarial attack example update iteration process.

[0032] This represents a current adversarial attack sample. The adversarial example generation process is as follows:

[0033]

[0034] Among them, w k It is the kth proxy model f k The integrated weights satisfy all w k ≥0 and

[0035] Step 3.2: Obtain each proxy model f from the input image. i Output (f) i (x) and gradient information g i Right now

[0036] Step 3.3: Based on the loss of adversarial examples generated by different models on other models, determine the contribution of each model to generating effective adversarial examples, and thus determine the weight of each model:

[0037]

[0038] Where, p k (·) indicates that it comes from f k The output of logits, S k,i This can be viewed as the loss of the k-th model on adversarial examples generated using the gradient of the i-th model. β represents the hyperparameter controlling the ensemble weighting effect, and ρ... i Higher values ​​indicate a range from g i The generated adversarial examples have better migration attack capabilities, meaning g i It contains more transferable adversarial information.

[0039] Step 3.4: Based on the adversarial ratio of each model, normalize each model using the softmax function: ω1, ω2...ω k =sofymax(ρ1,ρ2...ρ k Wherein, ω i Weights for each model.

[0040] Step 3.5: Using the weights of each model, calculate the initial adversarial examples of the ensemble pattern, and calculate the ensemble adversarial examples based on the weights of each model.

[0041] The innovation of this invention lies in:

[0042] (1) This technique does not require information about a specific white-box proxy model or target model, and has compatibility and wide applicability with other generative adversarial example methods.

[0043] (2) By dynamically adjusting the weights of multiple models and rebalancing these weights, this technique can monitor and optimize the contribution of each proxy model to the attack effect, significantly improving the ability of adversarial samples to migrate between different models. In particular, it shows obvious technical progress and advantages for models with large structural differences.

[0044] (3) Data preprocessing is performed using non-systematic data augmentation techniques to enhance the diversity of model inputs, thereby effectively improving the transferability of generated adversarial examples. Attached Figure Description

[0045] The above and other objects and features of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings.

[0046] Figure 1 This is a flowchart illustrating an integrated attack method utilizing unsystematic preprocessing and dynamic weights according to an embodiment of the present disclosure;

[0047] Figure 2 This is a schematic diagram illustrating an integrated attack method utilizing unsystematic preprocessing and dynamic weights according to an embodiment of the present disclosure.

[0048] Figure 3 This is a graph showing test results of an integrated attack method utilizing unsystematic preprocessing and dynamic weights according to an embodiment of the present disclosure; Detailed Implementation

[0049] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0050] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided only to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein, which will become clear upon understanding the disclosure of this application.

[0051] As used herein, the term “and / or” includes any one of the associated listed items and any combination of any two or more.

[0052] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts should not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0053] In the specification, when an element (such as a layer, region, or substrate) is described as being "on" another element, "connected to," or "bonded to" another element, the element may be directly "on" another element, directly "connected to," or "bonded to" the other element, or one or more other elements may be present in between. Conversely, when an element is described as being "directly on" another element, "directly connected to," or "directly bonded to" another element, no other elements may be present in between.

[0054] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the described features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0055] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains upon understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalistic manner.

[0056] Furthermore, in the description of the examples, detailed descriptions of well-known related structures or functions will be omitted when it is believed that such detailed descriptions would lead to a vague interpretation of this disclosure.

[0057] Figure 1 This is a schematic diagram illustrating an integrated attack method utilizing unsystematic preprocessing and dynamic weights according to an embodiment of the present disclosure.

[0058] In the problem of adversarial example generation optimization, existing methods mainly fall into three categories: gradient optimization, input transformation, and ensemble models. While ensemble models can improve the transferability of adversarial examples, existing methods, which integrate models by averaging the logits, probabilities, and losses of multiple surrogate models, often neglect the individual characteristics between models and the effectiveness of individual models in the transferability of adversarial examples. Therefore, dynamically adjusting the weights of models throughout the ensemble process, considering the contribution and transferability impact of each surrogate model to the attack target, is an effective way to fully leverage the individual characteristics of the models. Furthermore, simply summarizing their contributions may offset the overall attack target. Guiding adversarial example generation by rebalancing the importance of sub-model gradients can optimize the weights of models that contribute more to the attack within the ensemble model, thereby improving the transferability of the generated adversarial examples from the entire ensemble model. Moreover, input diversity strategies (e.g., resizing and padding, translation, scaling) have been shown to enhance transferability, but applying the same transformation to the inputs of multiple models may limit input diversity and potentially hinder overall transferability. Therefore, applying non-systematic data augmentation techniques to the inputs of all models to increase input diversity within the ensemble can improve the transferability of the generated adversarial examples. Based on the above principles, this invention innovatively proposes an integrated attack method utilizing non-systematic preprocessing and dynamic weights. The specific implementation process of this invention is as follows: Figure 1 As shown, Figure 2 This is a schematic diagram of the principle of this embodiment.

[0059] Specific embodiments of the present invention are as follows:

[0060] Step 1: First, CIFAR_10 was selected as the experimental dataset for image classification. The chosen black-box models included ResNet-50, WideResNet-50 (WRN101-2), BiT-M-R50, BiT-M-R101, ViT-Base, DeiT-Base, Swin-Base, and Swin-Small. For the white-box surrogate models, ResNet-18, Inception v3, ViT-Tiny, and DeiT-Tiny were selected.

[0061] Step 2: Apply different random input transformation strategies to the four white-box models to augment the input sample data, such as random noise, random resizing, etc. Save the transformed data as a copy of the benign sample.

[0062] Step 3: Input the data transformed by the corresponding model in Step 2 into the corresponding model to obtain the model's output (f).i (x) and the gradient information g after backpropagation i Right now

[0063] An adversarial sample clip(x+γ·sign(g) is obtained based on the gradient information of each model. i Based on the loss of adversarial examples generated by different models on other models and the contribution of each model to generating effective adversarial examples, the weight of each model is determined: S k,i =-1 y

[0064]

[0065] Where, p k (·) indicates that it comes from f k The output of logits, S k,i This can be viewed as the loss of the k-th model on adversarial examples generated using the gradient of the i-th model. β represents the hyperparameter controlling the ensemble weighting effect, and ρ... i Higher values ​​indicate a range from g i The generated adversarial examples have better migration attack capabilities, meaning g i It contains more transferable adversarial information; based on the adversarial ratio of each model, the softmax function is used to normalize each model: ω1, ω2... ω k =softmax(ρ1,ρ2...ρ k ), where ω i Weights for each model.

[0066] Calculate ensemble adversarial examples based on the weights of each model.

[0067] Step 4: Based on the prediction performance of the adversarial examples on the sub-models after initial ensemble, readjust the model weights: Predictive performance is reflected in the loss value of the adversarial example on the model. A higher loss value indicates a lower probability of accurate prediction for that adversarial example, and a larger contribution of that model to the adversarial objective when using an ensemble model. ω new_1 ,ω new_2 ...ω new_k =softmax(ρ new_1 ,ρ new_2 ...ρ new_3 Finally, optimized adversarial examples are generated based on the normalized weights.

[0068] Step 5: Repeat steps 3 to 4 until the set iteration threshold is exceeded. Adjust the model fusion weights based on the fusion attack of multi-model dynamic optimization integration and weight rebalancing, and update the adversarial examples.

[0069] Step 6: The loop ends, and the final adversarial examples generated by the ensemble model are output.

[0070] Step 7: Use the obtained adversarial attack samples to attack the target black-box model and calculate the success rate of the adversarial attack model.

[0071] This invention compares the experimental method Ours with the experimental results of two existing ensemble models, Ens and SVRE, as follows: Figure 3 As shown in Table 1, under the same parameter configuration, the success rate of the adversarial example attack black-box model generated by this method is higher than that of the other two, demonstrating the effectiveness of the method proposed in this invention in improving the transferability of adversarial examples.

[0072] Table 1 Experimental results of the present invention

[0073] Ens 21.32 16.22 12.58 10.69 7.17 10.68 8.3 15.23 12.77 SVRE 26.05 20.61 21.26 18.87 17.84 22.23 17.66 25.99 21.31 Ours 52.53 34.7 32.61 29.35 38.83 53.36 43.66 56.34 42.6725

[0074] Beneficial effects of this invention

[0075] Universality and Flexibility: This invention is not limited to any specific white-box proxy model type, nor does it require any prior knowledge of the target black-box model, allowing it to be flexibly combined with other adversarial example generation methods. This characteristic of not relying on specific model types and black-box information endows the method with broad applicability and flexibility, enabling the generation of effective adversarial examples in a wider range of application scenarios.

[0076] Enhancing the transferability of adversarial examples: Through multi-model dynamic tuning and model weight rebalancing mechanisms, this invention can monitor and optimize the contribution of different proxy models to the attack target, maintaining efficient weight adjustment even when model structures differ significantly. Furthermore, the use of non-systematic data augmentation strategies enhances the diversity of input data. These methods work together to improve the transferability of adversarial examples across different models, maintaining attack effectiveness even against models with different structures. This indicates that this invention represents a significant improvement over existing technologies in enhancing the applicability and effectiveness of adversarial examples in multi-model environments.

[0077] While some embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.

Claims

1. An integrated attack method utilizing unsystematic preprocessing and dynamic weights, characterized in that, Includes the following steps: Step 1: Select CIFAR_10 as the experimental dataset for image classification tasks, and determine four white-box proxy models and multiple black-box models; Step 2: Apply different random input transformation strategies to the four white-box models to augment the input sample data, and save the transformed data as a copy of the benign sample; Step 3: Ensemble attack with dynamic multi-model tuning. For a given image, a white-box substitution model is used for the attack. The next updated adversarial attack sample is predicted, the contribution of each model to the attack effect is identified, the weight of each model is determined, and adversarial samples of the ensemble model are generated. Step 4: Model weight rebalancing strategy. Based on the prediction performance of the adversarial examples on the sub-models after the initial ensemble, the model weights are adjusted again. Predictive performance is reflected by the loss value of the adversarial examples on the model. A higher loss value indicates a lower probability of accurate prediction for that adversarial example, and a greater contribution of that model to the adversarial objective when using an ensemble model. Finally, optimized adversarial examples are generated based on normalized weights. Step 5: Repeat steps 3 to 4 until the set iteration threshold is exceeded. Adjust the model fusion weights and update the adversarial examples based on the fusion attack of multi-model dynamic optimization integration and weight rebalancing. Step 6: The loop ends, and the final adversarial examples generated by the ensemble model are output; Step 7: Use the obtained adversarial attack samples to attack the target black-box model and calculate the success rate of the adversarial attack model.

2. The integrated attack method utilizing unsystematic preprocessing and dynamic weights as described in claim 1, characterized in that, The white-box proxy models are ResNet-18, Inception v3, ViT-Tiny, and DeiT-Tiny.

3. The integrated attack method utilizing unsystematic preprocessing and dynamic weights as described in claim 1, characterized in that, The black-box models are ResNet-50, WideResNet-50, BiT-M-R50, BiT-M-R101, ViT-Base, DeiT-Base, Swin-Base, and Swin-Small.

4. The integrated attack method utilizing non-systematic preprocessing and dynamic weights as described in claim 1, characterized in that, The input sample data augmentation includes random noise or random resizing.

5. The integrated attack method utilizing unsystematic preprocessing and dynamic weights as described in claim 1, characterized in that, Step 3 includes the following sub-steps: Step 3.1: For the adversarial examples generated by the ensemble model, for an image x of arbitrary size that serves as the input to the deep neural network in the image recognition model, set the starting point for the adversarial attack example update iteration process. This represents a current adversarial attack sample. The adversarial example generation process is as follows: Among them, w k It is the kth proxy model f k The integrated weights satisfy all w k ≥0 and Step 3.2: Obtain each proxy model f from the input image. i Output (f) i (x) and gradient information g i Right now Step 3.3: Based on the loss of adversarial examples generated by different models on other models, determine the contribution of each model to generating effective adversarial examples, and thus determine the weight of each model: Where, p k (·) indicates that it comes from f k The output of logits, S k,i It can be viewed as the loss of the k-th model on adversarial examples generated using the gradient of the i-th model, where β represents the hyperparameter controlling the ensemble weighting effect, and ρ i Higher values ​​indicate values ​​from g i The generated adversarial examples have better migration attack capabilities, meaning g i It contains more transferable adversarial information; Step 3.4: Based on the adversarial ratio of each model, normalize each model using the softmax function: ω1, ω2...ω k =softmax(ρ1,ρ2...ρ k Wherein, ω i Weights for each model; Step 3.5: Using the weights of each model, calculate the initial adversarial examples of the ensemble pattern, and calculate the ensemble adversarial examples based on the weights of each model.

Images