Federated Learning Data Digest Privacy Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning systems face performance degradation due to client absence, data imbalance, and non-independent-and-identically-distributed data across clients, leading to biased gradient updates and training degradation, especially when clients disconnect or change during the training process.
Innovation Solution
A federated learning framework that synthesizes representative client data at the moderator by encoding raw data into digests, using feature mixing and noise to protect privacy, and generating replacement models for absent clients to maintain training continuity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If clients disconnect or leave during federated learning training, then system flexibility is improved, but training stability and model performance deteriorate due to biased gradient updates
Solution Approach 1:
The system pre-computes and stores digests (encoded features and labels) for all clients before training begins. When a client disconnects, the moderator already has their digest data stored, allowing immediate generation of replacement models without waiting for client reconnection or data re-collection.
Solution Approach 2:
The system creates replacement models that copy the architectural structure and training configuration of absent clients' models. These replacement models are trained using the pre-stored digest data to generate gradient updates that mimic what the absent clients would have contributed, maintaining training stability without requiring the original clients.
2Object-affected harmful factors
If client data are not accessible by the moderator for privacy protection, then data security is improved, but representative data is lost when clients cease collaboration, causing biased gradient updates
Solution Approach 1:
The system extracts essential statistical information from client data in the form of digests (encoded features and labels) that capture the data distribution characteristics. These digests are sent to the moderator for storing and future use in generating replacement models, separating the necessary statistical information from the sensitive raw data.
Solution Approach 2:
The digest acts as an intermediary representation between the client's private data and the moderator's training process. The digest contains encoded features and labels that preserve the statistical properties needed for generating representative gradient updates, while the raw data remains securely stored only on the client devices.
3Duration of action of stationary object
If gradients are memorized during training to handle client absence, then training continuity is improved, but gradients become unrepresentative very quickly as iteration progresses
Solution Approach 1:
Instead of memorizing raw gradients which become outdated quickly, the system changes the stored representation to digests (encoded features and labels). These digests capture the underlying data distribution characteristics that remain valid across multiple training iterations, allowing generation of representative gradient updates even for clients absent for extended periods.
Data Source
AI summary
A federated learning method of protecting data digest includes: sending a general model to multiple clients by a moderator, generating encoded features according to raw data and training by each client, the training includes: updating the general model to generate a client model, selecting at least two encoded features and at least two labels to compute a feature weighted sum and a label weighted sum, sending a digest and update parameters of the client model to the moderator, where the digest includes a sum of the feature weighted sum and a noise and the label weighted sum, and performing the following steps by the moderator: determining an absent client and a present client, generating a replacement model according to the general model and the absent client, generating an aggregation model according to the available client and the replacement model, and training the aggregation model to update the general model.