Private Deep Neural Network Training via Secure Multi-Party Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for training deep neural networks across multiple machines face challenges in maintaining data privacy, as they often require sharing sensitive data among machines, which can be impractical due to privacy and legal concerns.
Innovation Solution
The implementation of a system where multiple machines store private data and share secure, information-theoretically computed values to enable secure multi-party computation, allowing for the training of deep neural networks without revealing the original data, using cryptographic techniques like secret sharing and information-theoretic protocols.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If private data is shared among multiple machines for training deep neural networks, then training accuracy is improved, but data privacy is compromised
Solution Approach 1:
The patent segments private data into multiple secret shares distributed across different machines. Each machine holds only a portion of the data (a share), and no single machine can reconstruct the original private data alone. This segmentation enables collaborative training while preserving privacy, as the data is divided into independent parts that only become meaningful when combined through secure computation protocols.
2Object-affected harmful factors
If secret sharing is implemented across multiple machines, then data privacy is protected, but system complexity increases
Solution Approach 1:
The patent introduces secure multi-party computation protocols as intermediaries that facilitate collaboration between machines holding secret shares. These computational intermediaries enable joint training operations without requiring direct data sharing, acting as a mediator layer that manages the complexity of secure computation while preserving privacy. The intermediaries handle the intricate coordination needed for distributed secret sharing operations.
3Reliability
If secure multi-party computation is used for private DNN training, then information-theoretic security is achieved, but computational overhead increases
Solution Approach 1:
The patent performs preliminary secret sharing of data and model parameters before the actual training process. By pre-distributing secret shares and setting up the secure computation framework in advance, the system reduces the computational overhead during the main training iterations. This preliminary preparation enables more efficient secure computations during training, as the infrastructure is already in place rather than being established during each training step.
Data Source
AI summary
Systems and methods for private deep neural network training are disclosed. Method includes storing first private values at first machine and second private values at second machine; providing, to third machine, first share of first private values and first share of second private values; providing, to fourth machine, second share of first private values and second share of second private values; computing, at third machine, third machine-value based on first share of first private values and first share of second private values; computing, at fourth machine, fourth machine-value based on second share of first private values and second share of second private values; providing, to first machine and second machine, third machine-value and fourth machine-value; and computing, at first machine, a mathematical function of first private values and second private values, mathematical function being computed based on first private values stored at first machine, third machine-value, and fourth machine-value.