Collaborative Model Updating With Prediction Aggregation for Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional federated learning methods face high communication overheads and privacy leakage due to sharing high-dimensional model parameters or gradients, limiting their applicability and model performance, especially when participants have different network structures.
Innovation Solution
A method involving two-stage model updates is employed, where participants first update their models based on common samples and aggregation results, then on fixed private samples, reducing communication overheads and ensuring differential privacy through personalized training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If model parameters or gradients are shared between participants for federated learning, then collaborative model construction is achieved, but communication overhead increases and privacy leakage occurs
Solution Approach 1:
The patent extracts only the necessary information (prediction results) from the complete model parameters or gradients, transmitting only this extracted subset between participants. This reduces communication overhead while maintaining the ability to perform collaborative model construction, as the prediction results contain the essential collaborative learning information without the full parameter burden.
Solution Approach 2:
Instead of sharing actual model parameters or gradients, the patent uses prediction results as simplified copies that represent the model's output behavior. These prediction result copies enable collaborative learning without exposing the complete model structure or parameters, thereby reducing communication requirements and protecting privacy.
2Reliability
If model parameters or gradients are shared between participants, then collaborative model construction is achieved, but privacy leakage occurs
Solution Approach 1:
The patent extracts only prediction results from the complete model parameters, transmitting only this necessary subset. This extraction approach enables collaborative model construction while removing unnecessary information that could lead to privacy leakage, as only the minimal required output information is shared.
Solution Approach 2:
The patent replaces direct sharing of model parameters with prediction result copies. These copies serve as proxies that enable collaborative learning without revealing the actual model parameters, thus preventing privacy leakage while maintaining collaborative model construction capability.
3Measurement precision
If conventional federated learning is implemented with high-dimensional data sharing, then model accuracy can be improved, but communication resource consumption increases
Solution Approach 1:
The patent extracts prediction results from high-dimensional model parameters, transmitting only this condensed information. This extraction maintains model accuracy by preserving the essential collaborative learning signals while dramatically reducing communication resource consumption by eliminating redundant parameter information.
4Adaptability or versatility
If participants use different network structures in federated learning, then model adaptability is improved, but coordination complexity increases
Solution Approach 1:
The patent uses prediction results as universal copies that can represent models with different network structures. Since prediction results are structure-agnostic outputs, they enable participants with different network architectures to collaborate effectively without increasing coordination complexity, as the exchange format remains consistent regardless of underlying model structure differences.
Data Source
AI summary
The specification provides a method and a system for collaboratively updating a model by multiple parties for implementing privacy protection. A server can deliver an aggregation result of a t-th round of common samples to each participant i. Each participant i performs first update on a local ith model according to the t-th round of common samples and the aggregation result. Each participant i performs second update on the ith model obtained after the first update based on a first private sample fixed in a local sample set and a sample label thereof. Each participant i inputs a (t+1)th round of common samples that are used for a next round of iteration into the ith model obtained after the second update, and sends an output second prediction result to the server, so the server aggregates n second prediction results corresponding to n participants for a next round of iteration.