Shuffled Neural Network Parts for Secure Outsourced Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Secure Multi-Party Computation (SMPC) techniques using Homomorphic Encryption are not compatible with Deep Learning Accelerators (DLAs) lacking high precision circuits, as they require longer encryption keys that exceed the precision capabilities of these accelerators, limiting their ability to perform operations like multiplication and accumulation.
Innovation Solution
The proposed solution involves shuffling and splitting data samples and artificial neural network models into randomized parts, allowing external parties to perform computations without reconstructing the original data or model, using a shuffling map and offset keys to enhance privacy protection and reduce the need for high precision circuits, enabling DLAs without high precision circuits to participate in outsourced deep learning computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Homomorphic Encryption is used for Secure Multi-Party Computation, then data privacy is protected, but the encryption key length exceeds the precision capability of Deep Learning Accelerators lacking high precision circuits
Solution Approach 1:
The patent segments the artificial neural network model into multiple parts, where each part contains a subset of neurons and associated weights. This segmentation allows the encrypted computation to be distributed across multiple less-precise circuits, with each circuit handling only a portion of the overall computation. The results from all parts are then aggregated to produce the final output, effectively dividing the precision requirement across multiple components rather than requiring one high-precision circuit.
Solution Approach 2:
The patent changes the parameter of encryption key length by using shorter encryption keys in conjunction with model segmentation. Instead of relying on a single long encryption key that requires high-precision circuits for manipulation, the system uses multiple shorter keys corresponding to different model parts. This parameter change makes the computation compatible with DLAs that have limited precision capabilities while maintaining security through the combined effect of multiple encrypted operations.
2Reliability
If data samples are split into parts for secure computation, then data privacy is enhanced, but the complexity of managing and reconstructing results increases
Solution Approach 1:
The patent applies preliminary action by pre-processing the artificial neural network model into structured parts with clearly defined responsibilities before the secure computation begins. Each part is prepared with its corresponding encryption key and data subset in advance. This preliminary organization simplifies the management during computation, as the system only needs to coordinate predefined parts rather than dynamically managing arbitrary data splits. The reconstruction process is also simplified because the pre-structured parts can be directly aggregated in a predetermined manner.
3Adaptability or versatility
If Deep Learning Accelerators without high precision circuits are used, then hardware usability is improved, but the ability to perform secure multiparty computation with long encryption keys is limited
Solution Approach 1:
The patent segments both the computational model and the encryption scheme to match the capabilities of DLAs without high precision circuits. By dividing the model into parts that can be processed with shorter encryption keys, the system enables secure computation on hardware that would otherwise be incompatible. Each DLA processes a segment with a key length it can handle, and the collective result achieves the security level that would require longer keys if computed on a single high-precision system.
Solution Approach 2:
The patent enhances universality by creating a secure computation framework that works across different hardware capabilities. The model segmentation approach allows the same system to operate on DLAs with varying precision levels - whether high-precision circuits are available or not. The system adapts its encryption key length and partitioning strategy based on the target hardware, making secure multiparty computation universally applicable across diverse deep learning accelerator platforms.
Data Source
AI summary
Protection of access to artificial neural network (ANN) models in outsourcing deep learning computations via shuffling parts. For example, an ANN model can be configured as the sum of a plurality of randomized model parts. Some of the randomized parts can be applied an offset operation and/or encrypted to generate modified parts for outsourcing. Such model parts from different ANN models can be shuffled and outsourced to one or more external entities to obtain the responses of the model parts to inputs. Data samples as inputs to the ANN models can also be split into sample parts as inputs to model parts to protect the data samples. The result of a data sample as an input applied to an ANN model can be obtained from a sum of responses of model parts with the sample parts applied as inputs.