Adversarial sample generation method based on deep neural network
By generating balanced adversarial examples through a category-aware perturbation balancing and adversarial round weighting strategy, the problems of model adversarial robustness and class imbalance under long-tailed datasets are solved, and the performance and robustness of the model under long-tailed distribution are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SICHUAN UNIV
- Filing Date
- 2026-01-19
- Publication Date
- 2026-05-01
AI Technical Summary
Existing adversarial training methods have failed to effectively address the issues of model adversarial robustness and class imbalance on long-tail datasets, resulting in a decrease in the model's generalization ability to tail classes. Furthermore, existing methods have failed to effectively consider the impact of class imbalance on the generation of adversarial examples.
By using a category-aware perturbation balance (CPB) strategy and an adversarial round weighting (AIW) strategy, balanced adversarial examples are generated and trained. The objective function is used to optimize the model parameters to achieve a balance between adversarial robustness and inter-class performance. The specific steps include generating perturbation strength, generating adversarial perturbations, and generating and training adversarial examples.
It significantly improves adversarial robustness and inter-class performance balance under long-tailed data, enhances model performance in tail classes, while maintaining good transferability and computational efficiency, and improves standard accuracy and accuracy under adversarial attacks.
Smart Images
Figure CN121960604A_ABST
Abstract
Description
A method for generating adversarial examples based on deep neural networks Technical Field
[0001] This specification relates to the fields of computer vision and natural language processing, and in particular to a method for generating adversarial examples based on deep neural networks. Background Technology
[0002] In recent years, deep neural networks (DNNs) have achieved remarkable success in fields such as computer vision and natural language processing, demonstrating groundbreaking performance. However, recent research has revealed the vulnerability of deep learning models to adversarial example attacks. Adversarial examples are generated by attackers adding non-random and imperceptible perturbations (i.e., adversarial perturbations) to natural samples, which can induce the target model's classifier to produce incorrect outputs. The existence of adversarial examples poses a serious security threat to many applications that rely on deep learning technologies.
[0003] Currently, improving the adversarial robustness of deep learning models to defend against adversarial attacks has become a cutting-edge research topic in the field of deep learning. Among the various defense methods proposed by researchers, adversarial training (AT) is widely considered the most effective. Its core idea is to add adversarial examples to the training set for training, alternately minimizing and maximizing the target model's loss on these adversarial examples. Specifically, adversarial training first requires adversarial examples, which are typically carefully generated against the target model using adversarial attack methods such as the Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD). The optimization objective is to maximize the adversarial loss so that the model outputs incorrect predictions. After obtaining the adversarial examples, adversarial training adds them to the training set to train the classifier, adjusting the classifier's parameters to minimize the adversarial loss so that the classifier can correctly classify the adversarial examples. In recent years, numerous studies, inspired by other fields, have improved and optimized traditional adversarial training methods, including metric learning, self-supervised learning, ensemble learning, and fair learning.
[0004] However, existing adversarial training methods are primarily designed based on class-balanced training data, neglecting class imbalance. Specifically, they still use balanced datasets like CIFAR10 and CIFAR100 to evaluate adversarial training performance, which fails to reflect the prevalent class imbalance problem in the real world. Real-world data often exhibits a long-tail distribution, meaning a few head classes dominate while a large number of tail class samples are scarce. Models trained on long-tail datasets tend to show higher confidence in head class samples, thus weakening their generalization ability to tail class samples and leading to a significant performance drop on these classes. This overconfidence problem is extremely challenging. In real adversarial scenarios, attackers are not limited by the distribution of class frequencies and can intentionally target tail classes. This distributional difference prompts a re-examination and improvement of existing adversarial training methods, making their performance on long-tail data a more practical and challenging evaluation criterion. Although some research has attempted to address the fairness issue in adversarial robustness, these methods mainly focus on the inherent learning difficulty differences between classes in balanced datasets. Adversarial robustness in long-tail distribution scenarios remains largely unexplored.
[0005] Existing long-tail adversarial training methods often overlook the coupling relationship between the model and adversarial examples: on the one hand, an overconfident model may introduce bias when generating adversarial examples, causing the generated adversarial examples to be indirectly affected by the imbalance of data classes; on the other hand, the model learns from biased adversarial examples, which will further exacerbate the performance imbalance. Summary of the Invention
[0006] In view of the above-mentioned shortcomings in the prior art, the present invention provides an adversarial example generation method based on deep neural networks, which solves the problem that deep neural network models are easily affected by sample imbalance when learning on adversarial examples, resulting in biased results.
[0007] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: a method for generating adversarial examples based on deep neural networks, comprising: S1: inputting the original long-tail dataset into a deep neural network to obtain the perturbation intensity of the current round; S2: based on the perturbation intensity of the current round, the model parameters of the current round, and the original long-tail dataset, analyzing the data using a deep neural network to obtain adversarial perturbations; S3: superimposing the adversarial perturbations with the original long-tail dataset to obtain adversarial examples, thus completing the generation of adversarial examples; wherein, the adversarial examples are used to train the deep neural network to obtain a trained deep neural network.
[0008] Furthermore, the expression for the disturbance intensity of the current round is: ;in, This indicates the intensity of the disturbance in the current round. This represents the proportion used to control the differences in perturbation intensity between categories. This represents the upper limit of the overall disturbance intensity. Indicates category , Represents the set consisting of all categories. express A certain category The probability of occurrence This indicates the degree of imbalance of category y relative to the class with the largest number of samples. Indicates category The degree of imbalance relative to the class with the largest number of samples.
[0009] Furthermore, the expression for the anti-disturbance is: ;in, This indicates the perturbation strength applicable to category y in the current round. This represents the function that takes the minimum value. This represents the iteration ratio at which the disturbance intensity gradually increases. This represents the total number of rounds of model training iterations. Indicates the current round. This indicates the intensity of the disturbance in the current round.
[0010] Furthermore, the adversarial examples are used to train the deep neural network to obtain a trained deep neural network. This includes: the deep neural network is trained to minimize the balance of the training objective, maintaining a bulldozer distance between the distribution corresponding to the adversarial examples generated in the current round and the distribution generated in the previous iteration, and using the objective function to train the deep neural network to obtain a trained deep neural network.
[0011] Furthermore, the expression for the objective function is: ;in, Let h be the parameter to be optimized and the optimization objective be to minimize the expected value of the adversarial loss on the distribution p. This represents the expectation of the adversarial loss over the overall data distribution p. This indicates that the optimization objective against disturbances is to maximize the loss. This indicates that the rebalancing counter-perturbation strength mechanism does not apply to the counter-perturbation loss term. This indicates that the model has been iterated and updated t-1 times. Indicates input data, This indicates resistance to disturbances. This indicates the label corresponding to the data.
[0012] The beneficial effects of this invention are as follows: This invention provides an adversarial example generation method based on deep neural networks, which generates balanced adversarial examples for training by rebalancing the adversarial perturbation intensity, while ensuring the adversarial robustness of the model and the balance of performance between categories. (1) Significantly improves adversarial robustness under long-tail data. At the same time, it improves the balance, significantly improves the performance of the model on the tail class, and thus improves the balance of model performance between categories; (2) It does not increase significant computational overhead and has good transferability, and can be seamlessly integrated into mainstream adversarial training algorithms; (3) After theoretically verifying that "adversarial examples can help balance long-tail distribution and achieve a balance between adversarial robustness and performance", this invention is designed based on theoretical guidance. This invention can significantly improve the performance of the model under long-tail distribution, specifically in terms of improving key performance indicators such as standard accuracy (accuracy on original samples) and accuracy under adversarial attacks (including PGD, Auto Attack, etc.). Attached Figure Description
[0013] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same reference numerals denote the same structures, wherein: Figure 1 is an exemplary flowchart of an adversarial example generation method based on a deep neural network according to some embodiments of this specification. Detailed Implementation
[0014] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.
[0015] Figure 1 is an exemplary flowchart illustrating an adversarial example generation method based on a deep neural network according to some embodiments of this specification. As shown in Figure 1, the process includes the following steps. In some embodiments, the process can be executed by a processor.
[0016] S1: Input the original long-tail dataset into the deep neural network to obtain the perturbation intensity of the current round.
[0017] The long-tailed distribution is a class imbalanced data distribution in which the minority class (tail class) samples are significantly fewer than the majority class (head class).
[0018] In some embodiments, the processor may be configured For the measurable sample space, where Representing the feature space, Represents the tag space. (Note: The original text contains some formatting errors and inconsistencies. A more accurate translation would require the full context.) For definition in The sample distribution on the given surface, and this distribution is class imbalanced, satisfying the following condition: and At the same time, define For the corresponding balanced distribution, for all categories All satisfied And balanced sample distribution Imbalanced sample distribution The only difference lies in the category frequency, satisfying . This represents the current iteration round.
[0019] Deep neural networks are an extension of traditional artificial neural networks, with multiple hidden layers. Compared to traditional neural networks, they have deeper layers and higher complexity.
[0020] Perturbation strength is a general adversarial training framework that achieves both robustness and fairness under long-tailed distribution.
[0021] In some embodiments, the category-aware perturbation balance (CPB) policy strength aims to improve the generation of adversarial examples by assigning class-related perturbation strengths to different categories, thereby mitigating the skew problem in the training objective. Let... For commonly used disturbance strengths (e.g.) ), and define Representing the Each category relative to the category with the largest number of samples The imbalance rate in terms of any... The corresponding disturbance intensity can be obtained through calculation.
[0022] Category-aware perturbation equilibrium is a mechanism that adaptively adjusts the intensity of perturbations for different categories.
[0023] In some embodiments, the expression for the disturbance intensity of the current round is: ;in, This indicates the intensity of the disturbance in the current round. This represents the proportion used to control the differences in perturbation intensity between categories. This represents the upper limit of the overall disturbance intensity. Indicates category , Represents the set consisting of all categories. express A certain category The probability of occurrence This indicates the degree of imbalance of category y relative to the class with the largest number of samples. Indicates category The degree of imbalance relative to the class with the largest number of samples.
[0024] Used to control the proportion of perturbation intensity differences assigned between categories. The larger the value, the more pronounced the imbalance in perturbation intensity, and the more significant the reinforcement effect on the minority class. Simultaneously, the class-adaptive perturbation intensity can ensure that the Wasserstein distance between the original sample distribution and the perturbed adversarial sample distribution does not exceed [a certain value]. ,Right now This allows for control over the magnitude of the perturbation at the distribution level, ensuring that the semantic information contained in the original distribution is not compromised.
[0025] S2: Based on the perturbation intensity of the current round, the model parameters of the current round, and the original long-tail dataset, deep neural networks are used for analysis to obtain adversarial perturbations.
[0026] To achieve high-level class-balanced adversarial robustness, adversarial training needs to simultaneously satisfy the following two key characteristics: (1) Balanced training objective: The model should be updated to minimize the balanced training objective, i.e., the robustness risk of the previous model on the balanced training data distribution. Instead of a direct but unbalanced training objective This objective can be minimized under carefully designed category-level perturbation strengths. To achieve this. (2) Stable adversarial distribution: targeting Distribution of generated adversarial examples It should be the same as the distribution generated in the previous iteration. Maintaining a close Wasserstein distance. Therefore, the rebalancing adversarial perturbation strength mechanism proposed in this invention consists of two parts: a class-wise perturbation balancing (CPB) strategy and an adversarial iteration weighting (AIW) strategy, corresponding to the two key characteristics mentioned above. That is, the situation where the number of samples of different classes in the training set is unbalanced and there are differences.
[0027] The goal of the adversarial round-weighted (AIW) strategy is to stabilize the evolution of the adversarial distribution, i.e., to control all training rounds. In the example, the Wasserstein distance between two adjacent adversarial sample distributions. As training progressed, The model gradually converges as the value increases, which means... and As they get closer, the distribution distance gradually decreases. Conversely, in the early stages of training, and The significant differences resulted in a greater distance. Warming up against the intensity of the disturbance occurred before training. In each iteration, the disturbance strength is gradually increased from 0 to... This constrains the strength of the resistance to perturbations in the early stages. Since this distribution distance can be bounded by the perturbation strength used in the corresponding iteration, it is possible to obtain the strength of any category through calculation. In the The perturbation intensity in each iteration round.
[0028] Adversarial round weighting is a mechanism in adversarial training that applies different weights to the perturbation intensity in different rounds.
[0029] In some embodiments, the expression for resisting disturbances is: ;in, This indicates the perturbation strength applicable to category y in the current round. This represents the function that takes the minimum value. This represents the iteration ratio at which the disturbance intensity gradually increases. This represents the total number of rounds of model training iterations. Indicates the current round. This indicates the intensity of the disturbance in the current round.
[0030] S3: Overlay the adversarial perturbation with the original long-tail dataset to obtain adversarial examples, thus completing the generation of adversarial examples; among them, the adversarial examples are used to train the deep neural network to obtain a trained deep neural network.
[0031] Adversarial examples are samples that are incorrectly predicted by the model by adding small perturbations to the original samples.
[0032] In some embodiments, adversarial examples are used to train a deep neural network to obtain a trained deep neural network. This includes: training the deep neural network with a training objective that minimizes the balance, maintaining a bulldozer distance between the distribution corresponding to the adversarial examples generated in the current round and the distribution generated in the previous iteration, and using an objective function to obtain a trained deep neural network.
[0033] In some embodiments, the processor can combine a category-aware perturbation balance (CPB) strategy with an adversarial round-weighted strategy (AIW) to implement an adversarial perturbation strength (RAIL) method designed for long-tailed data in the first round. In the next iteration, the objective function of the optimized model is trained, but the rebalancing adversarial perturbation strength mechanism does not apply to the adversarial loss term. It makes no specific assumptions, thus it is compatible with a variety of existing adversarial training algorithms, and can produce well-trained deep neural networks.
[0034] Adversarial training is a general method for defending against adversarial attacks by generating adversarial examples and training a model to correctly predict adversarial examples.
[0035] In some embodiments, the expression for the objective function is: ;in, Let h be the parameter to be optimized and the optimization objective be to minimize the expected value of the adversarial loss on the distribution p. This represents the expectation of the adversarial loss over the overall data distribution p. This indicates that the optimization objective against disturbances is to maximize the loss. This indicates that the rebalancing counter-perturbation strength mechanism does not apply to the counter-perturbation loss term. This indicates that the model has been iterated and updated t-1 times. Indicates input data, This indicates resistance to disturbances. This indicates the label corresponding to the data.
[0036] In some embodiments of this specification, a method for generating adversarial examples based on deep neural networks is provided. By rebalancing the adversarial perturbation intensity, balanced adversarial examples are generated for training, while ensuring the adversarial robustness of the model and the balance of performance between categories. (1) Significantly improves adversarial robustness under long-tailed data. At the same time, it improves the balance, significantly improves the performance of the model on the tail class, and thus improves the balance of model performance between categories; (2) It does not increase significant computational overhead and has good transferability, and can be seamlessly integrated into mainstream adversarial training algorithms; (3) After theoretically verifying that "adversarial examples can help balance long-tailed distribution and achieve a balance between adversarial robustness and performance", this invention is designed based on theoretical guidance. This invention can significantly improve the performance of the model under long-tailed distribution, specifically in terms of improving key performance indicators such as standard accuracy (accuracy on original samples) and accuracy under adversarial attacks (including PGD, Auto Attack, etc.).
Claims
1. A method for generating adversarial examples based on deep neural networks, characterized in that, include: S1: Input the original long-tail dataset into the deep neural network to obtain the perturbation intensity of the current round; S2: Based on the perturbation intensity of the current round, the model parameters of the current round, and the original long-tail dataset, analyze using a deep neural network to obtain adversarial perturbations; S3: Superimpose the adversarial perturbations with the original long-tail dataset to obtain adversarial examples, thus completing the generation of adversarial examples; among them, adversarial examples are used to train the deep neural network to obtain a trained deep neural network.
2. The adversarial example generation method based on deep neural networks according to claim 1, characterized in that, The expression for the disturbance intensity of the current round is: ;in, This indicates the intensity of the disturbance in the current round. This indicates the proportion used to control the differences in perturbation intensity between categories. This represents the upper limit of the overall disturbance intensity. Indicates category ， Represents the set consisting of all categories. express A certain category The probability of occurrence This indicates the degree of imbalance of category y relative to the class with the largest number of samples. Indicates category The degree of imbalance relative to the class with the largest number of samples.
3. The adversarial example generation method based on deep neural networks according to claim 1, characterized in that, The expression for the counter-disturbance is: ;in, This indicates the perturbation strength applicable to category y in the current round. This represents the function that takes the minimum value. This represents the iteration ratio at which the disturbance intensity gradually increases. This represents the total number of rounds of model training iterations. Indicates the current round. This indicates the intensity of the disturbance in the current round.
4. The adversarial example generation method based on deep neural networks according to claim 1, characterized in that, The adversarial examples are used to train the deep neural network. The trained deep neural network is obtained by: minimizing the balance of the training objective, keeping the distribution corresponding to the adversarial examples generated in the current round at a bulldozer distance from the distribution generated in the previous iteration, and using the objective function to train the deep neural network to obtain the trained deep neural network.
5. The adversarial example generation method based on deep neural networks according to claim 4, characterized in that, The process of training a deep neural network using an objective function to obtain a trained deep neural network includes: combining a category-aware perturbation balancing strategy with an adversarial round-weighted strategy, and an adversarial perturbation strength method designed for long-tail data in the [number] round. In each iteration, the objective function of the optimized model is trained to obtain a trained deep neural network.
6. The adversarial example generation method based on deep neural networks according to claim 4, characterized in that, The expression for the objective function is: ;in, Let h be the parameter to be optimized and the optimization objective be to minimize the expected value of the adversarial loss on the distribution p. This represents the expectation of the adversarial loss over the overall data distribution p. This indicates that the optimization objective against disturbances is to maximize the loss. This indicates that the rebalancing counter-perturbation strength mechanism does not apply to the counter-perturbation loss term. This indicates that the model has been iterated and updated t-1 times. Indicates input data, This indicates resistance to disturbances. This indicates the label corresponding to the data.