Stochastic Noise Layers for Neural Network Exfiltration Resistance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models are vulnerable to model exfiltration attacks, where threat actors probe and infer model architecture and parameters, posing risks to trade secrets and intellectual property, and existing defenses are computationally inefficient or impair model performance.
Innovation Solution
Integrate stochastic noise layers into neural networks, applying noise to inputs, outputs, or weights, sampled from learned probability distributions to obscure model operations while preserving performance, using differentiable training methods to optimize stochastic layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If stochastic noise layers are introduced to impede model exfiltration attacks, then model security is improved, but model performance may deteriorate
Solution Approach 1:
The patent applies parameter changes by introducing stochastic noise layers that modify the weight parameters of the neural network dynamically. During training, noise is added to weights to prevent attackers from inferring exact parameter values, while the noise level is controlled to maintain model performance. This resolves the contradiction by changing the parameter representation from fixed to stochastic, improving security without sacrificing performance.
Solution Approach 2:
The patent implements dynamics by making the network weights dynamic and time-varying through stochastic noise injection. Instead of static weights that can be easily probed, the weights now change over time according to a stochastic process, making it difficult for attackers to clone the model while preserving the model's functional performance through proper noise management.
2Reliability
If stochastic noise is applied to weights or outputs, then model exfiltration resistance is improved, but output consistency deteriorates
Solution Approach 1:
The patent applies dynamics by introducing time-varying stochastic noise to the network weights and outputs. This makes the model's behavior dynamic rather than static, so repeated inputs produce different outputs due to the evolving noise patterns. This dynamic behavior prevents attackers from probing the model to extract architecture and parameters, while the underlying model structure remains stable and performant.
Data Source
AI summary
Provided is a process including: obtaining, with a computer system, with a stochastic layer of a multi-layer neural network, inputs to the stochastic layer from, wherein the multi-layer neural network comprises both deterministic layers and the stochastic layer, and the stochastic layer comprises a plurality of parameters that vary stochastically according to respective probability distributions; determining values of the plurality of parameters by randomly sampling from the statistical distributions; determining an output of the stochastic layer based on both the determined values of the plurality of parameters and the inputs to the stochastic layer; and providing the output of the stochastic layer to a downstream layer of the multi-layer neural network or as an output of the multi-layer neural network.