Federated Model Training With Shared and Dedicated Parameters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing federated learning frameworks face challenges in ensuring data security and personalization during joint model training, leading to potential data leakage and loss of model personalization.
Innovation Solution
A method and system for training a privacy-preserving model using multi-task learning joint modeling, where each participant's model has a shared and dedicated portion, with shared data being perturbed and aggregated securely to ensure privacy while allowing for personalized model updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If federated learning frameworks are used for joint model training, then model generalization is improved, but data security deteriorates due to potential data leakage
Solution Approach 1:
The model is segmented into shared parameters and dedicated parameters. Shared parameters are updated through federated learning to improve generalization, while dedicated parameters remain local to each data party to ensure data security and prevent data leakage.
Solution Approach 2:
The harmful element (raw training data) is extracted from the communication process. Only model parameters are exchanged between data parties and the server, while the actual training data remains localized, eliminating data leakage risks while maintaining model generalization benefits.
2Adaptability or versatility
If federated learning frameworks are used for joint model training, then model performance on multiple tasks is improved, but data privacy deteriorates
Solution Approach 1:
The model parameters are segmented into shared and dedicated portions. The shared portion enables multi-task learning performance while the dedicated portion maintains data privacy by keeping task-specific data processing local to each data party.
Solution Approach 2:
Model parameters serve as an intermediary that carries task-specific information without exposing the underlying sensitive data. The server aggregates parameter updates from multiple data parties to improve multi-task performance while the actual private data never leaves the data parties' local environments.
3Measurement precision
If all data is centralized for model training, then model accuracy is improved, but security deteriorates
Solution Approach 1:
Training data is segmented and kept distributed across multiple data parties rather than centralized. Each data party trains the model locally on their own data, achieving accurate task-specific learning while eliminating security risks associated with data centralization.
Solution Approach 2:
The need for data centralization is extracted and replaced by parameter aggregation. Instead of collecting all data centrally to achieve accurate training, the system extracts only the necessary model parameters for aggregation, maintaining both accuracy and security.
4Object-affected harmful factors
If data is kept local for privacy preserving, then data security is improved, but model personalization deteriorates
Solution Approach 1:
The model is segmented into shared parameters updated through federated learning and dedicated parameters kept locally. This segmentation enables both data security (through local data retention) and model personalization (through dedicated local parameters adapted to specific tasks).
Solution Approach 2:
Different parts of the model have different qualities: shared parameters have generalization quality from aggregated training, while dedicated parameters have personalized quality from local task-specific training. This local quality differentiation achieves both security and personalization simultaneously.
Data Source
AI summary
Privacy preservation model training includes a plurality of iterative update rounds performed on a model held by a data party of a plurality of data parties participating in training to obtain model data, which includes first shared data and local data corresponding to a shared portion and a dedicated portion of the model, respectively. The iterative training adds a perturbation to the first shared data to perform privacy preservation on at least the first shared data. The first shared data is transmitted to a server, which determines, based on first shared data of the plurality of data parties, second shared data. The shared portion of the model is updated based on the second shared data returned by the server. A next iterative update round is performed based on an updated model or using the updated model as a final model.