Federated Decision Tree Training via Secure Histogram Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Horizontal federated learning processes face inefficiencies and long running times, particularly when using neural networks, which limits their application in fields requiring interpretability like finance and medical imaging, while decision trees offer high accuracy and interpretability but have been overlooked in federated learning research.
Innovation Solution
A federated learning method for decision trees, specifically Gradient Boosting Decision Trees, where participants search for quantile sketches of data features, construct local histograms with differential privacy, and securely aggregate them to train decision trees, reducing communication complexity and ensuring data privacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural networks are used for federated learning, then model accuracy can be improved, but training time and computational complexity increase significantly
Solution Approach 1:
The patent extracts only the essential statistical features (histograms, quantile sketches) from the training data rather than using complete neural network training. This extraction approach captures the most important information for decision tree training while eliminating the computational burden of full neural network training, thus reducing training time while maintaining sufficient model accuracy
Solution Approach 2:
The patent uses lightweight, computationally inexpensive statistical structures ( histograms, quantile sketches) as temporary intermediates during the federated learning process. These structures require minimal computational resources compared to neural networks and can be quickly constructed and discarded, enabling efficient iterative training without the heavy overhead of neural network training
2Measurement precision
If complete data is transmitted for model training, then model accuracy improves, but data privacy and security are compromised
Solution Approach 1:
The patent introduces statistical structures (histograms, quantile sketches) as intermediaries between the raw training data and the model training process. These intermediaries aggregate and anonymize the data, preserving the essential statistical patterns needed for accurate decision tree training while eliminating direct access to individual data points, thus protecting data privacy
Solution Approach 2:
The patent creates simplified copies of the training data in the form of histograms and quantile sketches that capture the statistical essence of the original data without containing sensitive information. These copies are transmitted and used for model training, allowing accurate learning while the original sensitive data remains securely stored locally at each participant
3Object-affected harmful factors
If traditional horizontal federated learning is implemented, then data privacy is protected, but training efficiency and speed are reduced
Solution Approach 1:
The patent performs preliminary computation at each participant node by constructing histograms and quantile sketches from the local training data before transmission. This preliminary aggregation and statistical summarization reduces the communication overhead and allows the server to efficiently train the decision tree model without receiving or processing large volumes of raw data, thus improving training efficiency while maintaining privacy
Solution Approach 2:
The patent transforms the training data from its original high-dimensional form into compressed statistical representations ( histograms with aggregated counts, quantile sketches with summary statistics). This parameter transformation dramatically reduces the data size and complexity while preserving the essential patterns needed for decision tree training, enabling faster and more efficient federated learning
Data Source
AI summary
Disclosed is a federated learning method for decision tree-oriented horizontal. The method comprises the following steps: all participants searching for a quantile sketch of each feature in a data feature set based on dichotomy; the participants constructing a local histogram for each feature by using locally held data features according to the quantile sketch; adding noise satisfying differential privacy to all local histograms, and sending the local histograms to a coordinator after processing through the secure aggregation method; the coordinator merging the local histograms of each feature into a global histogram, and training a root node of a first decision tree according to the histogram; the coordinator sending information of the node to other participants; and all participants updating the local histograms and repeating the above process for training to obtain the trained decision trees.