Federated Learning With Synonym-Based Recovery for Absent Clients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing federated learning systems face performance degradation due to extended client absence or departure, leading to biased model updates and long-term training degradation, especially when data are unbalanced or non-independent-and-identically-distributed across clients, and there is a lack of representative data for model evaluation.
Innovation Solution
A federated learning system that synthesizes representative client data using synonyms generated from digests, allowing training to continue even when clients are absent by encoding private data into digests and using a synonym generator to create alternative representations for absent clients.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system uses existing client data for training, then model accuracy is maintained, but client absence causes performance degradation and biased updates
Solution Approach 1:
The system pre-processes and stores client data in a centralized database before clients may become absent. When a client departs, the pre-stored data enables the system to continue training without interruption, maintaining reliability while preventing information loss from abrupt client absence.
Solution Approach 2:
The patent introduces a centralized database as an intermediary between clients and the training system. This database acts as a buffer that preserves client data independently of active participation, allowing the system to access representative data even when clients are absent, thus resolving the contradiction between training continuity and data availability.
2Reliability
If the system memorizes gradients during training, then client absence is handled, but gradients become unrepresentative quickly
Solution Approach 1:
Instead of relying on recently computed gradients that may become unrepresentative, the system pre-stores client data in the database before gradients are needed. This allows the system to access original representative data when clients are absent, maintaining measurement precision while handling absent clients reliably.
Solution Approach 2:
The patent creates a copy of client data in the centralized database that remains static and representative over time. This data copy serves as a permanent reference that does not degrade like gradients do, allowing the system to maintain accurate representations of client data even after extended periods without client participation.
3Adaptability or versatility
If the system assumes collaborative setting, then temporary disconnection is tolerated, but extended absence causes severe performance degradation
Solution Approach 1:
The system performs preliminary actions by pre-processing and storing all client data in the database before training begins. This preparation enables the system to handle extended client absence without performance degradation, as the stored data remains available for accurate model updates regardless of how long clients remain disconnected.
Solution Approach 2:
The centralized database ensures continuous availability of client data for training purposes. Even when clients are extended absent, the system can continuously access the pre-stored representative data to maintain accurate model updates, eliminating the performance degradation that occurs in traditional FL systems during extended disconnection periods.
Data Source
AI summary
A federated learning system using synonym includes a moderator and a plurality of clients. Each client includes an encoder encoding private data into a digest. The client trains a client model according to the private data, the digest and a general model, and sends the digest and a client parameter of the client model to the moderator. The moderator communicably connected to every client includes a synonym generator, sends the general model to every client, and determines an absent client of the plurality of client. The synonym generator generates a synonym of the digest corresponding to the absent client. The moderator trains a replacement model according to the synonym and the digest corresponding to the absent client device, performs an aggregation to update the general model according to a replacement parameter of the replacement model and the client parameter of every client except the absent client.