Neural Network Inference Privacy Using Adaptive Laplace Perturbation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The prevalent use of Deep Neural Networks (DNNs) in cloud-based inference poses significant privacy risks as raw data sent to the cloud can be misused or leaked, compromising user privacy without significant sacrifice of neural network performance.
Innovation Solution
The method involves optimizing stochastic perturbations using Laplace distributions to obfuscate private data features before inference, ensuring ε-differential privacy and minimizing mutual information loss while maintaining inference accuracy, without altering the pre-trained neural network architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If stochastic perturbations are added to obfuscate private data features, then data privacy is improved, but inference accuracy deteriorates
Solution Approach 1:
The patent applies parameter changes by optimizing the scale parameters of Laplace distributions for stochastic perturbations. The loss function includes terms that control the trade-off between privacy (mutual information reduction) and utility (inference accuracy), allowing dynamic adjustment of perturbation强度 to achieve optimal balance
Solution Approach 2:
The patent implements dynamics by making the perturbation parameters adaptive rather than static. The scale parameters of the Laplace distributions are optimized through gradient descent based on the loss function, allowing the system to dynamically adjust the amount of perturbation applied to different features based on their importance to the inference task
2Object-affected harmful factors
If stochastic perturbations are added to protect data privacy, then privacy security is improved, but data utility deteriorates
Solution Approach 1:
The patent implements feedback by using the inference performance as a signal to adjust the perturbation parameters. The loss function incorporates the inference accuracy metric, and gradient descent uses the performance feedback to iteratively optimize the scale parameters, ensuring that privacy protection does not excessively degrade utility
Solution Approach 2:
The system changes parameters of the perturbation distributions based on the observed trade-off between privacy and utility. The scale parameters are continuously adjusted to find the optimal point where privacy protection is maximized while maintaining acceptable inference performance
3Loss of information
If optimal stochastic perturbations are found using optimization, then privacy protection is improved, but computational complexity increases
Solution Approach 1:
The patent replaces complex combinatorial optimization methods with gradient-based optimization. By formulating the problem as differentiable optimization with a smooth loss function, the system can use efficient gradient descent algorithms rather than exhaustive search or complex discrete optimization methods
Solution Approach 2:
The optimization focuses on finding optimal parameters for a parametric family of distributions (Laplace distributions) rather than searching through all possible perturbation functions. This parameterization reduces the search space and makes the optimization computationally tractable
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods and systems that provide data privacy for implementing a neural network-based inference are described. A method includes injecting stochasticity into the data to produce perturbed data, wherein the injected stochasticity satisfies an ε-differential privacy criterion and transmitting the perturbed data to a neural network or to a partition of the neural network for inference.