Privacy-Preserving Machine Learning via Secret Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current privacy-preserving machine learning techniques face inefficiencies and accuracy degradation when training models on large datasets, particularly in neural networks, due to high computational overhead and leakage of sensitive information.
Innovation Solution
The implementation of secret-sharing methods among training computers, using multiplication triplets and garbled circuits, allows for efficient weight updates and activation function computations, ensuring privacy preservation and scalability to large datasets without exposing private data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If secret-sharing methods with multiplication triplets and garbled circuits are used for privacy-preserving machine learning, then privacy preservation is improved, but training time and computational overhead increase
Solution Approach 1:
The patent pre-generates multiplication triplets (a, b, c) where c = a × b before the actual training process. These triplets are stored and reused during training to eliminate the need for real-time secure multiplication computations. The garbled circuits are also prepared in advance for activation functions, allowing the main training loop to proceed with simpler operations that maintain privacy while reducing computational overhead.
Solution Approach 2:
The patent creates copies of data in encrypted or secret-shared form that can be processed without exposing the original plaintext. Multiple copies of multiplication triplets are generated and distributed to different computing nodes. These copies enable parallel processing and reduce the computational burden on any single node while maintaining the security guarantees of secret-sharing.
2Reliability
If secret-sharing methods are used for training on large datasets, then privacy preservation is improved, but scalability and efficiency deteriorate
Solution Approach 1:
The patent divides the large dataset into multiple partitions and distributes them across different computing nodes. Each node holds only a portion of the data in secret-shared form, processing its local partition independently. This segmentation enables parallel training across multiple machines, improving scalability while maintaining privacy guarantees through the distributed secret-sharing architecture.
Solution Approach 2:
The patent performs data preprocessing and partitioning before the distributed training process. Multiplication triplets are pre-computed and distributed to relevant nodes in advance. This preliminary setup enables the distributed system to proceed with efficient local computations without needing to coordinate complex secure operations during the actual training of large datasets.
3Reliability
If garbled circuits are used for activation functions in neural networks, then privacy preservation is improved, but computational complexity and training time increase
Solution Approach 1:
The patent converts activation functions into garbled circuits before training begins. These garbled circuits are compiled and optimized in advance, transforming complex non-linear operations into pre-processing tables and lookup structures. During training, the system simply performs table lookups rather than evaluating complex mathematical functions, dramatically reducing computational complexity while maintaining the privacy-preserving properties of garbled circuits.
Data Source
AI summary
New and efficient protocols are provided for privacy-preserving machine learning training (e.g., for linear regression, logistic regression and neural network using the stochastic gradient descent method). A protocols can use the two-server model, where data owners distribute their private data among two non-colluding servers, which train various models on the joint data using secure two-party computation (2PC). New techniques support secure arithmetic operations on shared decimal numbers, and propose MPC-friendly alternatives to non-linear functions, such as sigmoid and softmax.