FHE Model Calibration Using ECE Approximation and BFGS
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for calibrating Fully Homomorphic Encryption (FHE) based Machine Learning (ML) models face challenges due to large approximation errors from using low-degree polynomials and lack a holistic approach, especially for deeper neural networks.
Innovation Solution
A method and system that utilize pattern matching to identify evaluation functions in ML models, generate approximated functions using specific techniques, and compute Expected Calibration Error (ECE) values through Broyden-Fletcher-Goldfarb-Shanno (BFGS) optimization to calibrate FHE models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If low-degree polynomial approximations are used for ReLU and max-pooling functions in FHE based ML models, then the implementation is simpler and computation is faster, but the approximation error increases significantly making it unable to support deeper neural networks
Solution Approach 1:
The patent changes the degree parameter of the polynomial approximation from low-degree (conventional) to high-degree (our invention). Specifically, we use degree-5 or higher polynomials to approximate ReLU and max-pooling functions, which significantly reduces approximation error while maintaining computational feasibility in FHE settings.
Solution Approach 2:
The patent dynamically adjusts the approximation strategy based on the neural network depth and complexity. For deeper networks where approximation errors accumulate, the system employs higher-degree polynomials and refined approximation techniques, while for shallower networks it may use simpler approximations to maintain efficiency.
2Productivity
If conventional approximation methods are used for branching statements in FHE based ML models, then the computation remains feasible, but no holistic calibration approach is provided leading to incomplete model evaluation
Solution Approach 1:
The patent develops a universal calibration framework that works across all types of FHE based ML models regardless of encryption status or data type. The calibration method using ECE and BFGS optimization is applicable to encrypted models trained with encrypted data, encrypted models trained with unencrypted data, and unencrypted models trained with encrypted data, providing a multi-functional solution.
Solution Approach 2:
The patent implements a feedback-based calibration process where the Expected Calibration Error (ECE) is computed and used to guide the BFGS optimization algorithm. The calibration parameters are iteratively adjusted based on the ECE feedback until convergence, ensuring reliable and complete model evaluation.
3Reliability
If FHE based ML models are calibrated without proper approximation techniques, then privacy is maintained, but large approximation errors reduce the reliability of model predictions and calibration
Solution Approach 1:
The patent minimizes information loss by using high-degree polynomial approximations that more accurately represent the original non-linear functions. The degree-5 or higher polynomials capture the curvature and behavior of ReLU and max-pooling functions much better than low-degree approximations, reducing the gap between encrypted and unencrypted model predictions.
Data Source
AI summary
The present disclosure provides a technique to evaluate encrypted Machine Learning (ML) models. Conventional methods are unable to provide a holistic approach to evaluate encrypted ML models. Initially, the system receives an encrypted ML model. The ML model can be an unencrypted ML model trained with encrypted data or an encrypted ML model trained with encrypted data, or an encrypted ML model trained with unencrypted data. Further, a plurality of evaluation functions pertaining to the ML model to be calibrated are identified using a pattern matching technique. Further, an approximated function is generated for each of the plurality of evaluation functions using a corresponding approximation technique. After generating a plurality of approximated functions, an Expected Calibration Error (ECE) value is computed based on the plurality of approximated functions. Finally, the ML model is calibrated based on the computed ECE value. The ML model is perfectly calibrated if the computed ECE value is equal to zero.