Federated Model Parameter Update for Privacy-Preserving Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning-based prediction systems face challenges in effectively utilizing large amounts of user data to create a prediction model that reflects individual user differences while protecting user privacy, as training models either rely solely on single-user data or require uploading data to a central server, compromising privacy.
Innovation Solution
A method where a server sends model parameters to terminals, which update and feedback prediction losses, allowing the server to accurately update model parameters using aggregated user data without compromising privacy, and terminals can train models locally using shared parameters to improve prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If model training is performed separately on each terminal using single-user data, then user privacy is protected, but prediction accuracy deteriorates due to insufficient training data
Solution Approach 1:
The system segments the model training process into two parts: model structure initialization on the server and local model parameter training on each terminal. This segmentation allows each terminal to maintain its own training data locally for privacy protection while the server coordinates the overall training process using aggregated loss information from multiple terminals.
Solution Approach 2:
The system implements a feedback mechanism where each terminal calculates prediction loss based on its local test data and feeds back this loss information to the server. The server then uses this aggregated feedback from multiple terminals to update the model parameters, which are subsequently distributed back to terminals for further local training. This iterative feedback loop enables accurate model training without exchanging sensitive user data.
2Measurement precision
If a unified prediction model is trained on a cloud server using large-scale data, then prediction accuracy improves, but user privacy deteriorates due to data centralization
Solution Approach 1:
The invention extracts only the essential model parameters and loss information from the training process, leaving the sensitive user data localized on each terminal. The server takes out and processes only the aggregated loss values and model parameter updates, never accessing the actual user data. This extraction approach enables centralized model training benefits while maintaining distributed data privacy.
Solution Approach 2:
The system introduces prediction loss as an intermediary that mediates between user data and the central server. Instead of directly transmitting user data to the server, each terminal processes its data locally and transmits only the derived loss information. This intermediary mechanism allows the server to learn from aggregated user data patterns without ever accessing or storing individual user data, thus maintaining privacy while achieving accurate modeling.
3Measurement precision
If model parameters are updated using aggregated data from multiple terminals, then prediction accuracy improves, but system complexity increases due to coordination requirements
Solution Approach 1:
The system implements partial action by having each terminal perform only a portion of the training work locally (calculating loss on local test data) rather than completing the entire training process. The server performs the complementary partial action of aggregating loss information and updating model parameters. This division of labor reduces the computational burden on any single component while achieving accurate model training through coordinated partial actions.
Data Source
AI summary
Example prediction methods and apparatus are described. One example includes sending a first model parameter and a second model parameter by a server to a plurality of terminals. The first model parameter and the second model parameter are adapted to a prediction model of the terminal. The server receives a first prediction loss sent by at least one of the plurality of terminals. A first prediction loss sent by each of the at least one terminal is calculated by the terminal based on the prediction model that uses the first model parameter and the second model parameter. The server updates the first model parameter based on the first prediction loss sent by the at least one terminal to obtain an updated first model parameter. The server updates the second model parameter based on the first prediction loss sent by the at least one terminal to obtain an updated second model parameter.