Federated Learning Model Aggregation via Quantile Sketch Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning systems face challenges in training machine learning models across disparate systems, particularly with gradient boosting techniques like XGBoost, due to difficulties in exchanging information and fusing tree models, which affects accuracy and convergence behavior.
Innovation Solution
The implementation of improved quantile sketch algorithms and data compression methods to facilitate dynamic party inclusion and exit in federated learning, while maintaining data security through differential privacy and encryption, enables efficient generation and adaptation of machine learning models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If gradient boosting techniques like XGBoost are used for federated learning, then model accuracy can be improved, but information exchange and tree model fusion become difficult, affecting convergence behavior
Solution Approach 1:
The patent introduces an intermediary mechanism that facilitates information exchange between participant systems without requiring direct communication. The system uses a coordinated approach where participant systems exchange compressed statistics and models through a structured protocol, enabling accurate model fusion while reducing the complexity of direct information exchange between disparate systems.
Solution Approach 2:
The patent segments the tree model fusion process into manageable components by processing different parts of the data and models separately. The federated learning system divides the complex task of merging gradient boosting trees into discrete steps: local model training, statistics computation, and coordinated model updates, making the overall process more tractable while maintaining accuracy.
2Adaptability or versatility
If data compression methods are implemented to facilitate dynamic party inclusion and exit, then system adaptability improves, but data security requirements increase
Solution Approach 1:
The patent applies preliminary security measures by establishing differential privacy mechanisms and encryption protocols before data exchange occurs. This preliminary protection ensures that when participant systems dynamically join or exit the federated learning network, their data remains secure throughout the transition, allowing adaptability without compromising security.
Solution Approach 2:
The system uses security intermediaries in the form of differential privacy mechanisms and encryption layers that mediate between data compression needs and security requirements. These intermediaries enable dynamic participant inclusion/exclusion while maintaining data security by ensuring that compressed statistics and models are protected during transmission and processing.
3Productivity
If quantile sketch algorithms are improved for federated learning, then model generation speed increases, but reconstruction error may increase
Solution Approach 1:
The patent optimizes the balance between speed and precision by dynamically adjusting parameters of the quantile sketch algorithms. The system modifies sketch complexity, compression ratios, and reconstruction thresholds based on the specific federated learning context, enabling faster model generation while controlling reconstruction error within acceptable ranges for the given application.
Data Source
AI summary
A method, system, and computer program product for training models for federated learning. The method determines, by a federated learning aggregator, a set of sample ratios for a set of participant systems. Each sample ratio is associated with a distinct participant system. A set of participant epsilon values are generated for the set of participant systems with each participant epsilon value being associated with a participant system of the set of participant systems. A set of surrogate data sets are received for the set of participant systems with each surrogate data set representing a data set of a participant system. The federated learning aggregator generates a set of local models. Each local model is generated based on a first global model. The method generates a second global model based on a prediction set generated by the set of participant systems using the set of local models.