Private ANN Training with TEE and Quadratic Homomorphic Encryption
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for training artificial neural networks (ANNs) on remote hosts face challenges due to resource constraints in Trusted Execution Environments (TEEs) and the inefficiencies of Homomorphic Encryption (HE) in handling complex activation functions, leading to sub-optimal performance and privacy concerns.
Innovation Solution
A method that combines Trusted Execution Environments (TEEs) with quadratic homomorphic encryption to split ANN training between a trusted and an untrusted process, where the trusted process computes outputs of neurons based on encrypted inputs, while the untrusted process computes encrypted inputs, using a key-pair generated by the trusted process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Homomorphic Encryption is used to compute over encrypted data, then data privacy is preserved, but performance overhead becomes intolerable
Solution Approach 1:
The patent segments the ANN training computation into two distinct parts: linear operations (matrix multiplications, additions) performed using homomorphic encryption on encrypted data, and non-linear operations (activation functions) performed using polynomial approximations on decrypted data. This segmentation allows the system to leverage the privacy-preserving properties of HE for suitable operations while avoiding its performance penalties for operations where it is less efficient.
Solution Approach 2:
The patent transforms the activation function computation by changing the mathematical representation from direct neural network activation functions to polynomial approximations. This parameter change enables the use of homomorphic encryption for the transformed computation, as polynomials can be efficiently evaluated using homomorphic operations, thereby improving the overall performance while maintaining privacy.
2Reliability
If TEEs are used for arbitrary computation on sensitive data, then data privacy is protected, but resource constraints prevent resource-demanding computations
Solution Approach 1:
The patent extracts the computationally intensive non-linear activation function computations from the TEE environment and performs them outside using polynomial approximations. Only the essential linear operations remain inside the TEE, reducing the resource demands on the TEE while maintaining data privacy protection for the sensitive inputs and weights.
Solution Approach 2:
The patent introduces polynomial approximations as an intermediary mechanism that bridges the gap between the need for private computation in TEEs and the requirement for efficient non-linear operations. The polynomial approximation serves as a mediator that can be computed both inside the TEE with acceptable performance and outside with enhanced efficiency.
3Reliability
If TEEs run on main processors, then data privacy is maintained, but dedicated hardware architectures cannot be leveraged for speedup
Solution Approach 1:
The patent segments the computation to place only the privacy-critical linear operations inside the TEE on the main processor, while moving the computationally intensive activation function evaluations to dedicated hardware architectures (GPUs or FPGAs) running outside the TEE. This segmentation enables the system to simultaneously maintain data privacy in the TEE and leverage the high-speed parallel processing capabilities of dedicated hardware for the non-linear operations.
Data Source
AI summary
A method training an artificial neural network (ANN) on a remote host computes, using a trusted process deployed in a trusted execution environment (TEE) on the remote host, a key-pair for a homomorphic encryption scheme and shares, by the trusted process, the public key (PK) of the key-pair with an untrusted process deployed on the remote host. The method splits the training procedure of the ANN between the untrusted process and the trusted process, wherein the untrusted process computes encrypted inputs to neurons of the ANN by means of the homomorphic encryption scheme, while the trusted process computes outputs of the neurons based on the respective encrypted inputs to the neurons as provided by the untrusted process.