Neural Network Discretization for Gradient Obfuscation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks (NNs) are vulnerable to adversarial attacks, particularly when attackers have access to gradients or can approximate them, leading to misclassification and potential security breaches.
Innovation Solution
Discretize and scramble the neuron inputs and outputs of NNs, converting them into indices and storing operations in lookup tables (LUTs) to obscure the math operations, making auto-differentiation processes difficult and gradients inaccessible.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If neural network operations are kept in continuous domain with standard math operations, then auto-differentiation works efficiently for training and attack analysis, but the network becomes vulnerable to gradient-based adversarial attacks
Solution Approach 1:
The patent transforms the continuous parameters of neural network operations into discrete parameters by quantizing input values to a finite set of discrete levels. This parameter transformation fundamentally changes the mathematical properties of the operations, making gradient-based attacks inapplicable while preserving the essential functionality of the neural network through carefully designed discretization schemes and lookup table representations.
Solution Approach 2:
The patent replaces the continuous mathematical operations (addition, multiplication, activation functions) with discrete lookup table operations. Instead of computing results through mathematical formulas that require differentiation, the system uses pre-computed lookup tables that map discrete input combinations to discrete output values, substituting the mechanical/mathematical system with a discrete computational system that is resistant to gradient-based analysis.
2Reliability
If discretization is applied to neuron inputs and outputs, then gradients become inaccessible and auto-differentiation fails, but the computational efficiency and simplicity of standard neural network operations is lost
Solution Approach 1:
The patent performs preliminary discretization of the input space and pre-computes all possible operation outcomes for discrete inputs, storing them in lookup tables before runtime. This preliminary action eliminates the need for complex runtime computations and gradient calculations, as the discrete operations can be executed through simple table lookups that are both efficient and gradient-obfuscating.
Solution Approach 2:
The patent creates discrete copies of the neural network operations in the form of lookup tables that replicate the input-output behavior of the original continuous operations. These copied discrete operations maintain the functional equivalence needed for security while being computationally efficient through pre-computation and avoiding the need for expensive gradient calculations during inference or attack resistance.
Data Source
AI summary
Embodiments regard neural network (NN) obfuscation. A method can include receiving data representing the NN architecture. The NN architecture includes neurons, each neuron comprising corresponding neuron input weights, an activation function, and one or more interconnections between one or more other neurons of the neurons. The method can include mapping inputs to the neurons from a continuous domain to a discrete domain resulting in discretized neuron inputs. The method can include applying the activation function to the discretized neuron inputs resulting in discretized neuron outputs.