Privacy Protection Model Training via Parameter Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional federated learning frameworks face challenges in model personalization while ensuring data privacy, leading to a trade-off between model generalization and personalization, where improved generalization often results in reduced personalization and privacy leakage.
Innovation Solution
A method for training a privacy protection model using multi-task learning joint modeling, where each data party has a shared and dedicated model portion, with iterative updates and data transmission to a server for aggregation, ensuring that only shared data are transmitted, maintaining privacy and improving model personalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional federated learning frameworks are used to improve model generalization through joint learning, then model prediction accuracy is improved, but data privacy is compromised due to data transmission and aggregation
Solution Approach 1:
The model is segmented into shared parameters and private parameters. Shared parameters are updated through federated learning aggregation to improve generalization, while private parameters remain local to each client to protect data privacy. This segmentation allows the system to achieve both model improvement and privacy preservation simultaneously.
Solution Approach 2:
The harmful element (private data) is extracted from the training process entirely. Instead of transmitting raw data or even full model parameters, only the necessary shared parameters are transmitted and aggregated. The private parameters are taken out and kept exclusively on local devices, eliminating the privacy leakage risk while maintaining model collaboration benefits.
2Adaptability or versatility
If model personalization is enhanced for each data owner, then model adaptability to specific data is improved, but model generalization across different data owners deteriorates
Solution Approach 1:
The model parameters are divided into shared parameters that capture general patterns across all clients and private parameters that capture client-specific characteristics. The shared parameters are updated through federated aggregation to ensure generalization, while private parameters are updated locally to ensure personalization. This segmentation resolves the contradiction by allowing both generalization and personalization to coexist in different parameter subsets.
Solution Approach 2:
Different parts of the model have different qualities and functions. The shared parameters have global quality for generalization, while the private parameters have local quality for personalization. This allows each part of the model to be optimized for its specific purpose without compromising the other, achieving both model personalization and generalization.
3Manufacturing precision
If more data is shared among data owners for joint training, then model training quality is improved, but privacy protection effectiveness is reduced
Solution Approach 1:
The private data is completely extracted from the communication process. Each client trains locally on their own data without sharing it, and only essential shared parameters are transmitted for aggregation. This extraction of harmful elements (private data) from the system allows high-quality joint training through parameter sharing while maintaining strong privacy protection.
Solution Approach 2:
Model parameters serve as an intermediary that carries training information between clients without exposing raw data. The shared parameters act as a mediator that enables knowledge transfer and joint learning while preventing direct access to sensitive private data, thus improving training quality without compromising privacy.
Data Source
AI summary
Implementations of this specification disclose methods and systems for training a privacy protection model. In an implementation, a method comprising: performing one or more times of iterative training on the model based on a training sample held by the data party to obtain model data, transmitting the first shared data to a server for the server to determine second shared data based on the first shared data, receiving the second shared data from the server, updating the shared portion of the model based on the second shared data to obtain an updated shared portion, and generating, based on the updated shared portion, an updated model for performing a next one of the plurality of iterative updates in response to determining that the next one of the plurality of iterative updates is not a last one of the plurality of iterative updates.