Homomorphic Encryption Neural Network Extension via Packing Frameworks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training and running neural networks with homomorphic encryption (HE) require HE-friendly architectures that often result in lower accuracy due to limitations such as using polynomial functions instead of ReLU activations and MAX-Pooling, and HE processing is computationally intensive, being three to five orders of magnitude slower than plaintext operations.
Innovation Solution
A system that receives a machine learning network and a selected HE packing framework, generates a list of HE packings, extends the network with additional neurons based on these packings, and trains the extended network, optimizing HE-friendly architectures to improve accuracy without degrading latency or increasing processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If HE-friendly architectures are used for homomorphic encryption processing, then security and privacy are improved, but accuracy deteriorates due to limitations in activation functions and pooling operations
Solution Approach 1:
The patent segments the neural network architecture into distinct components that can be independently optimized for HE compatibility. By dividing the network into layers with specific activation functions and pooling operations tailored for homomorphic encryption, the system maintains security while improving accuracy through targeted architectural modifications rather than uniform constraints across the entire network.
Solution Approach 2:
The patent changes key architectural parameters including replacing standard activation functions with HE-friendly alternatives (such as polynomial activations), modifying pooling operations to work with encrypted data, and adjusting layer configurations. These parameter changes enable the network to achieve comparable accuracy to non-HE architectures while maintaining homomorphic encryption security properties.
2Reliability
If HE processing is used for encrypted neural network operations, then security is improved, but processing speed deteriorates being three to five orders of magnitude slower than plaintext operations
Solution Approach 1:
The patent applies preliminary actions by pre-processing the neural network architecture to be HE-friendly before deployment, including selecting appropriate activation functions and pooling operations that are computationally efficient for homomorphic encryption. This preliminary optimization reduces the computational burden during actual encrypted operations, mitigating the speed deterioration inherent in HE processing.
3Adaptability or versatility
If HE-friendly architectures with polynomial functions are used, then homomorphic encryption compatibility is improved, but model accuracy deteriorates compared to non-HE friendly architectures
Solution Approach 1:
The patent creates a universal neural network architecture that can function effectively both with homomorphic encryption and without it. By designing HE-friendly activation functions and pooling operations that can operate in both encrypted and plaintext modes, the system achieves multi-functionality where the same architecture maintains comparable accuracy across different operational contexts, eliminating the accuracy penalty typically associated with HE compatibility requirements.
Data Source
AI summary
An example system includes a processor to receive a machine learning network and a selected homomorphic encryption (HE) packing framework. The processor can generate list of HE packings for the machine learning network based on the selected HE packing framework. The processor can extend the machine learning network to include additional neurons based on the list of HE packings. The processor can also train the extended machine learning network.