Decision Tree Surrogate Networks for Efficient Homomorphic Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Decision tree-based models are inefficient for fully homomorphic encryption (FHE) due to non-linear operations and scaling issues, making it impractical to implement branching computations efficiently.
Innovation Solution
Substitute a specially-trained low-depth neural network (NN) as a surrogate for the decision tree model, performing homomorphic inferencing on the neural network instead, which avoids untenable branching computations and scaling issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If decision tree-based models are used for homomorphic inference, then data privacy is preserved through FHE, but computational efficiency deteriorates due to non-linear operations and branching computations
Solution Approach 1:
The patent creates a surrogate neural network that copies the decision boundaries and classification behavior of the original decision tree model. This copy (neural network) is then used for homomorphic inference instead of the original decision tree, enabling efficient FHE operations while preserving the privacy-preserving classification functionality.
Solution Approach 2:
The patent transforms the model representation from decision tree format (with branching logic and non-linear comparisons) to neural network format (with continuous weights and activations). This parameter change enables the use of efficient homomorphic encryption operations while maintaining the essential classification capability.
2Reliability
If decision tree branching computations are performed using FHE, then privacy is maintained, but computational complexity increases making the operation untenable
Solution Approach 1:
The patent replaces the computationally complex decision tree branching logic with a surrogate neural network copy that captures the same decision boundaries. This copy uses only linear operations and simple activations that are compatible with efficient FHE, eliminating the untenable computational complexity of homomorphic branching.
Solution Approach 2:
The patent substitutes the mechanical branching computation mechanism of decision trees with a neural network mechanism that uses continuous activations. This substitution replaces the discrete, non-linear branching operations with continuous, differentiable operations that are much more amenable to homomorphic encryption.
Data Source
Figure 1~2
Figure 3
Figure 4~6
AI summary
A method, apparatus and computer program product for homomorphic inference on a decision tree (DT) model. In lieu of HE-based inferencing on the decision tree, the inferencing instead is performed on a neural network (NN), which acts as a surrogate. To this end, the neural network is trained to learn DT decision boundaries, preferably without using the original DT model data training points. During training, a random data set is applied to the DT, and expected outputs are recorded. This random data set and the expected outputs are then used to train the neural network such that the outputs of the neural network match the outputs expected from applying the original data set to the DT. Preferably, the neural network has low depth, just a few layers. HE-based inferencing on the decision tree is done using HE inferencing on the shallow neural network. The latter is computationally-efficient and is carried without the need for bootstrapping.