Federated Learning Global Model Weighted Parameter Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In federated learning, important data with a small number of instances is diluted when integrated at the server, making it difficult to perform effective training, as the existing methods do not adequately consider the influence of such data during model updates.
Innovation Solution
The proposed learning system involves local devices selecting mini-batches and generating local data information, which is then used by the server to calculate integrated parameters for updating the global model, allowing for weighted averaging of local model parameters to enhance the influence of important data, and the server can request specific data distributions from local devices to ensure proper training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If federated learning integrates parameters from multiple local devices, then computational load is distributed and privacy is protected, but important data with small instances is diluted and loses influence in the global model update
Solution Approach 1:
The patent introduces importance weights as additional parameters to modify the parameter aggregation process. Each local model's parameters are weighted by an importance score that reflects the significance of its training data, particularly for rare or important classes. This changes the aggregation formula from simple averaging to weighted averaging, where the weight for each local model is determined by factors such as data distribution, class balance, and importance metrics, thereby preserving the influence of important data in the global model update
Solution Approach 2:
The patent applies asymmetric treatment to different local devices based on their data characteristics. Instead of treating all local models equally, the system identifies and assigns higher importance weights to local devices that possess rare or critical data samples. This asymmetric weighting ensures that local models trained on important but scarce data have disproportionate influence on the global model, counteracting the dilution effect in federated learning aggregation
2Loss of energy
If only model parameters are exchanged in federated learning, then communication costs are reduced and privacy is protected, but the server cannot consider data distribution and importance for effective training
Solution Approach 1:
The patent introduces an intermediary mechanism that computes and transmits importance weights and data distribution statistics from local devices to the server. These intermediaries act as auxiliary information carriers that convey metadata about data characteristics without exposing the actual training data. The server uses this intermediary information to adjust aggregation weights and perform more informed model updates, achieving adaptability to data distribution while maintaining the privacy and communication efficiency benefits of parameter-only exchange
Data Source
AI summary
According to one embodiment, a learning system includes a plurality of local devices and a server. Each of the local devices includes a processor. The processor selects a mini-batch from local data. The processor trains a local model using the mini-batch. The processor generates local data information relating to the local data included in the mini-batch and indicating information different from a label. The processor transmits a local model parameter relating to the local model and the local data information to the server. The server includes a processor. The processor calculates an integrated parameter using the local data information acquired from each of the local devices. The processor updates a global model using the integrated parameter and the local model parameter acquired from each of the local devices.