Homomorphic Encrypted Neural Network Approximation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current homomorphic encryption methods for neural networks require high-degree polynomials as activation functions, leading to increased computational time and resource usage, especially when bootstrapping is necessary, and retraining is required, which limits performance and efficiency.
Innovation Solution
The use of weighted least squares to generate an approximate polynomial for neural network operations, based on input data mean, standard deviation, and correction constants, allowing for homomorphic encrypted data operations without the need for high-degree polynomials, thereby reducing computational overhead and resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If high-degree polynomials are used as activation functions in homomorphic encryption neural networks, then the neural network can perform operations on encrypted data, but the computational time and resource usage increase significantly
Solution Approach 1:
The patent changes the parameter of polynomial degree from high-degree to low-degree approximations. By using low-degree polynomials that approximate activation functions like ReLU, the system maintains the ability to perform neural network operations on encrypted data while significantly reducing computational complexity and resource requirements.
Solution Approach 2:
The patent employs temporary approximation polynomials that are computationally inexpensive to evaluate. These low-degree polynomial approximations serve as disposable computational building blocks that enable encrypted neural network operations without requiring expensive high-degree polynomial calculations, thus improving processing efficiency.
2Reliability
If high-degree polynomials are used for neural network operations, then encryption can be maintained, but bootstrapping and retraining are required which limits performance
Solution Approach 1:
The patent performs preliminary approximation of activation functions using low-degree polynomials before actual neural network inference. By pre-computing these approximations and avoiding the need for runtime bootstrapping, the system eliminates time-consuming operations while maintaining encryption integrity throughout the computation process.
Solution Approach 2:
The system uses temporary low-degree polynomial approximations that replace the need for expensive bootstrapping operations. These approximation polynomials are computationally lightweight and do not require the time-consuming bootstrapping process, thereby reducing overall computation time while maintaining security.
3Reliability
If conventional homomorphic encryption methods are used, then encrypted data operations are possible, but computational overhead and resource requirements are high
Solution Approach 1:
The patent fundamentally changes the computational parameters by replacing high-degree polynomial operations with low-degree polynomial approximations. This parameter change reduces the computational overhead and resource requirements for encrypted data operations while maintaining the essential capability to perform neural network inference on encrypted data.
Solution Approach 2:
The patent substitutes complex high-degree polynomial mechanical computations with simpler low-degree polynomial approximations. This substitution replaces the heavy computational machinery of conventional homomorphic encryption with lighter, more efficient polynomial evaluations that require fewer computational resources and less energy.
Data Source
AI summary
An apparatus includes one or more processors configured to execute instructions; and one or more memories storing the instructions; wherein the execution of the instructions by the one or more processors configures the one or more processors to generate an approximate polynomial, approximating a neural network operation, of a portion of a deep neural network model that is configured to receive input data, by using weighted least squares based on parameters corresponding to the generation of the approximate polynomial, a mean of the input data, and a standard deviation of the input data; and generate a homomorphic encrypted data operation result based on the input data and the approximate polynomial that approximates the neural network operation.