Fully Homomorphic Encryption Neural Network Inference Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing fully homomorphic encryption (FHE) schemes, such as CKKS and FHEW/TFHE, when combined for neural network inference, result in a significant decrease in inference accuracy due to accumulated numerical errors from scale-down processes and limited precision in functional bootstrapping.
Innovation Solution
The proposed solution involves an operating system and method for a fully homomorphic encryption neural network model that uses a combination of CKKS for linear operations and FHEW/TFHE for functional bootstrapping, along with the PEGASUS framework for converting between CKKS and LWE ciphertexts. Additionally, a lookup table-aware fine-tuning method is applied to align the input value range with the message domain of the ciphertext, updating convolution weights and activation functions to mitigate numerical errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If CKKS and FHEW/TFHE encryption schemes are combined for neural network inference, then data privacy is protected through full homomorphic encryption, but inference accuracy significantly decreases due to accumulated numerical errors
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the scale factor and precision parameters during the homomorphic encryption inference process. Specifically, it modifies the CKKS scheme's scaling parameters and the FHEW/TFHE functional bootstrapping parameters to minimize numerical error accumulation while maintaining encryption security, thereby resolving the contradiction between privacy protection and inference accuracy
Solution Approach 2:
The patent employs a composite encryption approach by integrating CKKS (which supports floating-point operations) and FHEW/TFHE (which supports functional bootstrapping) into a hybrid FHE system. This composite structure leverages the strengths of both schemes: CKKS for linear computations and FHEW/TFHE for non-polynomial operations, thereby maintaining both data privacy and inference accuracy
2Adaptability or versatility
If functional bootstrapping is used to support non-polynomial operations in encrypted neural networks, then computational capability is enhanced, but numerical precision is lost due to limited precision in the bootstrapping process
Solution Approach 1:
The patent introduces an intermediary scaling mechanism that acts as a buffer between the functional bootstrapping operation and the final decryption process. This intermediary layer performs precision-preserving scaling operations that compensate for the precision loss inherent in functional bootstrapping, thereby maintaining numerical precision while enabling non-polynomial computations
Solution Approach 2:
The patent applies beforehand cushioning by pre-calculating and applying compensation factors before the functional bootstrapping operation. These compensation factors are designed to counteract the expected precision loss from bootstrapping, effectively cushioning the numerical precision against degradation while maintaining the enhanced computational capability
Data Source
AI summary
An operating method of a fully homomorphic encrypted neural network model is provided, wherein the fully homomorphic encrypted neural network model includes a plurality of layers, and the method performed by a processor includes: for one of the plurality of layers, encrypting a plaintext input with a first encryption algorithm to generate a ciphertext vector, performing a convolution operation according to the ciphertext vector to generate a result vector, transforming the result vector into a plurality of result ciphertexts adopting a second encryption algorithm, inputting the plurality of result ciphertexts into an activation function to generate a plurality of encrypted activation values, and repacking the plurality of encrypted activation values to generate an output vector adopting the first encryption algorithm.