Neural Network Normalization Layer for Adversarial Training Diversity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
When training neural networks using adversarial examples, existing methods fail to effectively reflect the diversity of these examples, leading to inadequate training and potential weaknesses in the network's ability to handle varied adversarial conditions.
Innovation Solution
A learning device and method that acquire a base data group and generate adversarial data groups under different conditions, using normalization layers to update parameter values for both the partial network and the entire network, ensuring each normalization layer normalizes data with specific average and variance values, thereby reflecting the diversity of adversarial examples in the training process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If adversarial examples are used to train neural networks, then the network's ability to handle adversarial conditions is improved, but the diversity of adversarial examples is not effectively reflected in the training
Solution Approach 1:
The patent divides the training process into multiple segments by creating separate normalization layers for each adversarial example generation condition. Instead of using a single normalization layer for all adversarial examples, the system segments the training data by condition (e.g., different attack methods, noise levels, or perturbation types) and applies condition-specific normalization. This segmentation allows each normalization layer to capture the statistical characteristics of its specific condition, thereby effectively reflecting the diversity of adversarial examples in the training process while improving the network's reliability against various adversarial conditions.
2Adaptability or versatility
If multiple normalization layers are used for different adversarial example conditions, then the diversity of training is improved, but the device complexity increases
Solution Approach 1:
The patent applies local quality by making each normalization layer specialized for a specific adversarial example generation condition rather than using a uniform normalization approach. Each normalization layer has its own learned parameters (mean and variance) that are adapted to the local characteristics of its specific condition. This allows the system to maintain high adaptability to diverse adversarial conditions while managing complexity through targeted specialization rather than blanket complexity across all layers.
Solution Approach 2:
The patent introduces dynamics by making the normalization process adaptive to different conditions. During training, the system dynamically selects which normalization layer to apply based on the adversarial example generation condition. This dynamic approach allows the network to flexibly adapt to various adversarial conditions without requiring a static, overly complex architecture. The conditional normalization layers are activated selectively, providing the necessary diversity handling while keeping the overall system complexity manageable through conditional activation rather than permanent structural complexity.
Data Source
AI summary
A learning device for a neural network uses the base data group to update a parameter value of the partial network and a parameter value of the normalization layer associated with the entire base data group, and uses each group of adversarial examples of each adversarial example generation condition to update the parameter value of the partial network and the parameter value of the normalization layer associated with the condition. The neural network includes a partial network, a normalization layer associated with the entirety of a base data group including a plurality of data, and a normalization layer associated with each condition of adversarial example generation.