Neural Network Adversarial Training via Selective Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
When training neural networks with adversarial examples, existing methods fail to effectively account for the accuracy of these examples, leading to inefficient training and potentially low accuracy in the neural network's predictions.
Innovation Solution
A learning device and method that acquires both base data and adversarial examples, using normalization layers to determine error induction in neural network estimation, and updates parameter values using both types of data to improve training efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If adversarial examples are used to train neural networks, then the neural network's robustness against attacks is improved, but the training efficiency deteriorates due to indiscriminate use of all adversarial examples
Solution Approach 1:
The patent extracts and selects only the effective adversarial examples that actually induce errors in the neural network, discarding ineffective ones. This is achieved by evaluating each adversarial example's ability to cause misclassification and selectively using only those that succeed, thereby improving training efficiency while maintaining robustness.
Solution Approach 2:
Instead of using all generated adversarial examples, the patent applies partial action by selectively using only a subset of adversarial examples that are proven to be effective. This avoids the excessive computation of processing all adversarial examples while still achieving the desired robustness improvement.
2Adaptability or versatility
If all adversarial examples are used for training, then comprehensive coverage of attack scenarios is improved, but the training time increases due to processing ineffective examples
Solution Approach 1:
The patent extracts only the effective adversarial examples that successfully induce errors, separating them from ineffective examples. This extraction process filters out time-wasting examples while preserving the diversity of effective attack scenarios, thus maintaining adaptability while reducing training time.
Solution Approach 2:
The patent changes the parameter of adversarial example selection from 'all examples' to 'effective examples only'. By evaluating and selecting based on the error-induction capability parameter, the system maintains comprehensive coverage of effective attack scenarios while eliminating time-consuming processing of ineffective examples.
3Quantity of substance
If adversarial examples with low accuracy are used for training, then the training data volume is increased, but the learning effectiveness deteriorates due to inclusion of ineffective examples
Solution Approach 1:
The patent extracts and identifies effective adversarial examples by evaluating their error-induction capability. This extraction ensures that only high-quality, effective examples are used for training, maintaining learning effectiveness while optimizing the training data volume by excluding ineffective examples.
Solution Approach 2:
The patent changes the quality parameter of training data from 'all adversarial examples' to 'effective adversarial examples'. By introducing an effectiveness evaluation parameter, the system ensures that training data volume is optimized with only those examples that contribute to learning effectiveness.
Data Source
AI summary
A learning device for a neural network uses a base data group, which is a group including a plurality of data, to update a parameter value of the partial network and a parameter value of the second normalization layer, and uses an adversarial example determined to induce an error in estimation using the neural network, among adversarial examples included in an adversarial data group, which is a group including a plurality of adversarial examples with respect to the data included in the base data group, to update the parameter value of the partial network and a parameter value of the first normalization layer. The neural network includes a partial network, a first normalization layer normalizing data input to the first normalization layer itself, and a second normalization layer normalizing data input to the second normalization layer itself.