Variational Randomized Smoothing for Adversarially Robust Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing adversarial training techniques for neural networks are inefficient and fail to provide robust defense against adversarial attacks due to the lack of effective methods for generating high-quality adversarial examples and selecting optimal noise levels for randomized smoothing.
Innovation Solution
A variational framework is introduced to determine input sample-wise noise levels for randomized smoothing using a noise level selector neural network, combined with stochastic regularization and dual smoothing techniques to enhance robustness against adversarial attacks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional randomized smoothing adds fixed noise level for all input samples, then the implementation is simple, but the robustness against adversarial attacks is insufficient and accuracy on clean data deteriorates
Solution Approach 1:
The patent applies dynamics by transitioning from a fixed noise level to a dynamic, adaptive noise level that varies based on input sample characteristics. The noise level selector neural network dynamically adjusts the noise level for each input sample, allowing the system to adapt to different data conditions and optimize the balance between robustness and accuracy for each case.
Solution Approach 2:
The patent changes the parameter of noise level from a static value to a variable parameter that is selected based on input data characteristics. By introducing a noise level selector that determines the noise level as a function of the input sample, the system optimizes the noise level parameter for each input, improving robustness while maintaining accuracy on clean data.
2Productivity
If adversarial training uses gradient-based optimization algorithms to generate adversarial examples, then the generation speed is fast, but the quality and diversity of adversarial examples is insufficient
Solution Approach 1:
The patent merges multiple approaches to adversarial example generation by combining gradient-based optimization algorithms with generative models and evolutionary algorithms. This hybrid approach leverages the speed of gradient-based methods while incorporating the quality and diversity generation capabilities of generative models and evolutionary algorithms, achieving both fast generation and high quality.
Solution Approach 2:
The patent uses a composite approach to adversarial example generation, combining different generation methods (gradient-based, generative models, evolutionary algorithms) into a unified system. This composite methodology integrates the strengths of each approach to produce high-quality, diverse adversarial examples efficiently.
3Reliability
If adversarial training augments training data with adversarial examples, then the robustness is improved, but the computational cost and training time increases
Solution Approach 1:
The patent applies partial action by selectively applying adversarial training only to specific portions of the training data or using a subset of adversarial examples rather than processing all data exhaustively. This allows the system to improve robustness effectively while reducing the computational time and resources required for training.
Data Source
AI summary
Embodiments disclose a method and a system for robust transformation of input with a neural network. The method comprises processing the input data with a variational neural network (VNN) trained with ML to produce static parameters including noise level for the input data, injecting a set of random noises sampled on a probabilistic distribution according to the statistic parameters defined by the VNN to produce a set of perturbed input samples. The method comprises processing each of the set of perturbed input samples with a transformation neural network to produce a set of transformations and outputting a combination of the set of transformations as the robust transformation of the input data. Some embodiments consider training the variational neural network and transformation neural network by using adversarial examples from an attack model via alternating, explicit, and implicit gradient frameworks.