Tensor Shuffling for Privacy in Outsourced Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Secure Multi-Party Computation (SMPC) techniques using Homomorphic Encryption are not compatible with Deep Learning Accelerators (DLAs) lacking high precision circuits, as they require longer encryption keys that exceed the precision capabilities of these accelerators, limiting their ability to perform operations like multiplication and accumulation.
Innovation Solution
The approach involves shuffling and splitting data samples and artificial neural network models into randomized parts, allowing external entities to perform computations without reconstructing the original data or model, using a shuffling map and offset keys to enhance privacy protection and prevent brute-force attacks, thereby enabling DLAs without high precision circuits to participate in outsourced deep learning computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Homomorphic Encryption is used for Secure Multi-Party Computation, then data privacy is protected, but the encryption key length exceeds the precision capabilities of Deep Learning Accelerators lacking high precision circuits
Solution Approach 1:
The patent divides the data sample into multiple parts (first part, second part, third part) that can be independently processed. Each part is processed separately by different entities or at different times, with the final result reconstructed by combining the processed parts. This segmentation allows DLAs without high precision circuits to handle smaller, manageable data portions that fit within their precision constraints while maintaining overall computational accuracy.
2Reliability
If data samples are split into multiple parts for processing, then privacy protection is enhanced, but the complexity of coordinating and reconstructing results increases
Solution Approach 1:
The patent applies offset keys to the data parts before processing to mask the actual data values. This preliminary action ensures that even if intermediate results are exposed, the original data remains protected. The offset keys are applied in advance to transform the data into a form that preserves privacy while allowing meaningful computation to proceed on the transformed values.
Solution Approach 2:
The patent introduces intermediate transformed values as mediators between the original data and the final results. These intermediate representations (offset and split data parts) serve as placeholders that preserve privacy during processing while containing sufficient information to reconstruct accurate results. The intermediaries decouple the privacy protection mechanism from the computational logic.
3Productivity
If external entities process data parts independently, then computational parallelism is achieved, but the risk of brute-force attacks on individual parts increases
Solution Approach 1:
The patent transforms the data by applying offset keys and splitting operations that change the parameter space in which the data exists. The offset keys shift the data values into different numerical ranges, and the splitting operation distributes data across multiple dimensions. These parameter changes ensure that brute-force attacks on individual parts become computationally infeasible, as the attack would need to succeed across multiple transformed dimensions simultaneously.
Data Source
AI summary
Protection of access to a tensor in outsourcing deep learning computations via shuffling. For example, the tensor in the computation of an artificial neural network can have elements arranged in a first dimension of rows and a second dimension of columns. The tensor can be partitioned along the first dimension and the second dimension to generate computing tasks that are shuffled and/or mixed with other tasks for outsourcing to external entities. Computing results returned from the external entities can be used to generate a computing result of the tensor in the computation of the artificial neural network. The partitioning and shuffling can prevent the external entities from accessing and/or reconstructing the tensor.