Homomorphic Neural Inference Using Normalized Inverse Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Polynomial approximations in Fully Homomorphic Encryption (FHE)-friendly models are accurate only within a limited input range, which is difficult to control in complex models like deep neural networks, leading to significant accuracy issues.
Innovation Solution
A neural network is trained to learn both input features and their normalized inverses during training, and a normalized inverse function is applied to input feature vectors during inferencing to normalize values into a limited common range, enabling robust non-linear approximation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If polynomial approximations are used for non-linear functions in FHE-friendly models, then computation can be performed on encrypted data, but accuracy deteriorates when input values are outside the limited valid range
Solution Approach 1:
The patent transforms the input feature space by adding a new dimension through the normalized inverse function. Instead of relying solely on the original input values, the system creates an extended feature vector that includes normalized inverse values, effectively moving from a 1D input space to a 2D feature space that captures both original and inverse relationships, thereby improving approximation accuracy across broader input ranges
Solution Approach 2:
The patent applies parameter transformation by converting input features into their normalized inverse forms. This parameter change allows the polynomial approximation to operate on transformed values that remain within the valid range even when original inputs are outside it, thus maintaining approximation accuracy while preserving the ability to compute on encrypted data
2Measurement precision
If the input range is strictly controlled to maintain polynomial approximation accuracy, then approximation errors are reduced, but adaptability to complex models like deep neural networks deteriorates
Solution Approach 1:
The patent performs preliminary normalization of the inverse features before they are used in the polynomial approximation. By pre-processing the input data to compute and normalize inverse values within the valid range beforehand, the system ensures that the polynomial approximation operates on pre-conditioned data that guarantees accuracy, while the overall system remains adaptable to various model architectures
Solution Approach 2:
The normalized inverse function serves as an intermediary transformation layer between the raw input features and the polynomial approximation. This intermediary step converts potentially out-of-range inputs into normalized values that are suitable for approximation, thereby maintaining both accuracy and adaptability to different model types without requiring strict input range control
Data Source
AI summary
A technique for privacy-preserving homomorphic inferencing using a neural network having an activation function, such as a non-linear high-degree polynomial. The network is trained to learn input features of an input feature vector together with their associated inverses. During inferencing, an encrypted data point is received at the network. The data point comprises an input feature vector that has been extended with a set of one or more additional feature values, the set of one or more additional feature values having been generated by applying a normalized inverse function to respective one or more features in the feature vector. Homomorphic inferencing is performed on the encrypted data point using the machine learning model to generate an encrypted result, which is then returned. By applying the normalized inverse function, the high-degree polynomial can use any value of an input feature during inferencing, whether the value is within or outside of a particular input range.