Partially Local Federated Learning with Split Model Parameters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing federated learning frameworks compromise user privacy by transmitting sensitive user-level data updates outside the user device, which can lead to the derivation of private information.
Innovation Solution
Implement a partially local federated learning framework that uses local training systems to maintain and update a set of local parameters on user devices, employing reconstruction and update algorithms to generate parameter updates locally, ensuring privacy by keeping sensitive data and updates within the device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If federated learning transmits parameter updates to a central server, then model training can be performed across distributed devices, but user privacy is compromised due to transmission of sensitive data outside the device
Solution Approach 1:
The model parameters are segmented into two distinct sets: local parameters maintained privately on the user device and global parameters maintained on the central server. This segmentation allows the system to achieve distributed training capabilities while protecting user privacy by keeping sensitive local parameters confined to the local device and never transmitting them to the server.
Solution Approach 2:
The patent introduces global parameters as an intermediary mechanism that enables coordination between distributed local models without requiring direct transmission of local parameters or raw data. The server uses these global parameters to guide local training while maintaining privacy boundaries, effectively mediating the federated learning process without compromising sensitive information.
2Object-affected harmful factors
If all training data is processed locally on user devices, then user privacy is protected, but the model cannot leverage distributed data across multiple users for improved accuracy
Solution Approach 1:
By dividing model parameters into local and global components, the system enables each user to contribute to model improvement through local training on private data while the aggregated global parameters capture distributed patterns across users, achieving both privacy protection and improved accuracy through selective information sharing.
Solution Approach 2:
The patent implements local quality by allowing each user device to maintain specialized local parameters optimized for their specific local data distribution and privacy requirements, while simultaneously benefiting from global parameters that encode knowledge from diverse user populations, thus achieving both local adaptability and global accuracy.
3Object-affected harmful factors
If local parameters are maintained and updated separately from global parameters, then privacy is enhanced, but system complexity increases due to managing multiple parameter sets
Solution Approach 1:
The patent merges the benefits of local and global parameter management by establishing a coordinated update mechanism where local parameter updates are computed privately and then integrated with global parameters through controlled aggregation. This combining approach maintains privacy through separate maintenance while reducing complexity through unified update protocols that leverage both parameter types synergistically.
Solution Approach 2:
The system implements feedback mechanisms where global parameters provide guidance signals back to local models, enabling coordinated updates without requiring complex direct communication between all local parameters. This feedback loop simplifies the management of multiple parameter sets by establishing a clear directional flow of information from global to local and aggregated results from local to global.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a machine learning model having a set of local model parameters and a set of global model parameters under a partially local federated learning framework. One of the methods include maintaining local data and data defining the local model parameters; receiving data defining current values of the global model parameters; determining, based on the local data, the local model parameters, and the current values of the global model parameters, current values of the local model parameters; determining, based on the local data, the current values of the local model parameters, and the current values of the global model parameters, updated values of the global model parameters; generating, based on the updated values of the global model parameters, parameter update data defining an update to the global model parameters; and transmitting the parameter update data.