Federated Doubly Stochastic Kernel Learning for Vertical Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing federated learning methods face challenges in efficiently processing large-scale, vertically partitioned data while ensuring privacy and scalability, particularly in nonlinear models, as they often rely on encryption and are limited to linear separable models.
Innovation Solution
The Federated Doubly Stochastic Kernel Learning (FDSKL) algorithm uses random features to approximate kernel mappings and employs doubly stochastic gradients for model updates, computed federatedly without revealing whole data samples, utilizing a tree-structured communication scheme for efficient computation and data privacy preservation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If encryption-based federated learning methods are used to ensure data privacy, then security is improved, but computational cost and complexity increase significantly
Solution Approach 1:
The patent introduces random features as an intermediary representation that allows computation without exposing raw data. Instead of directly sharing or encrypting original data samples, the system computes random feature mappings locally at each client and aggregates these transformed representations, thereby preserving privacy while reducing computational burden compared to full encryption schemes
Solution Approach 2:
The patent extracts only the essential information needed for model training by projecting data into random feature spaces. Rather than transmitting or processing complete encrypted datasets, the system extracts and shares only the computed random feature values, which contain sufficient statistical information for kernel-based learning while minimizing data exposure and computational overhead
2Reliability
If traditional federated learning methods are used for vertically partitioned data, then privacy is preserved, but the model is limited to linear separable cases
Solution Approach 1:
The patent transforms the data representation by changing parameters through random feature mapping. By applying random projections and kernel transformations to the vertically partitioned data, the system enables nonlinear pattern recognition capabilities while maintaining the federated privacy-preserving framework, thus extending model versatility beyond linear separability
3Measurement precision
If large scale data processing is performed in federated learning, then model accuracy is improved, but processing efficiency and scalability deteriorate
Solution Approach 1:
The patent segments the computation process into local random feature extraction at each client and centralized aggregation of these features. This segmentation allows parallel processing of large datasets across multiple clients without requiring centralized data collection, thereby maintaining both high model accuracy from large-scale data and processing efficiency through distributed computation
Solution Approach 2:
The patent uses random feature mappings as simplified copies or projections of the original high-dimensional data. These random feature representations capture essential patterns while being computationally cheaper to process, enabling efficient scaling to large datasets without sacrificing model accuracy that would require processing the complete original data
Data Source
AI summary
System and method for prediction using a machine learning model. The system includes a coordinator, an active computing device and a passive computing device in communication with each other. The active computing device has a processor and a storage device storing computer executable code. The computer executable code is configured to: obtain parameters of the machine learning model; retrieve an instance from the local data; sample a random direction of the instance; compute a dot product of the random direction and the instance, and calculate a random feature; compute predicted values of the instance in the active and passive computing devices and summarize them to obtain a final predicted value; determine a model coefficient using the random feature, the final predicted value, and a target value of the instance; update the machine learning model using the model coefficient; and predict a value for a new instance.