Federated Learning Gradient Weighting for Personalized Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing federated learning methods result in uniform models for all participants, failing to provide differentiated models based on diverse participant scenarios.
Innovation Solution
A method to determine model gradients by obtaining data volume and node local gradients, calculating a global gradient, and assigning actual model gradients based on participation degrees, allowing for varied models among participants.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If uniform federated learning training is implemented to obtain a unified training model, then model consistency across participants is improved, but the ability to provide differentiated models based on different scenarios is lost
Solution Approach 1:
The patent applies local quality by calculating a participation degree for each node based on its local gradient and data volume information, then using this to determine a personalized actual model gradient. This allows each participant to receive a customized model gradient tailored to their specific data characteristics and contribution level, thereby achieving model differentiation while maintaining overall training consistency.
Solution Approach 2:
The patent changes the parameter of model gradient from a uniform value to a personalized value by introducing the participation degree as a weighting factor. The actual model gradient is computed as a weighted combination of the global gradient and local gradient, where the participation degree dynamically adjusts the gradient based on each node's data volume and contribution, enabling differentiated models for different participants.
2Measurement precision
If data volume information and node local gradients are obtained to calculate global gradient, then model accuracy is improved, but system complexity increases
Solution Approach 1:
The patent implements feedback by using the node local gradient and data volume information to calculate the participation degree, which then feeds back into the actual model gradient determination. This feedback mechanism ensures that each node's specific characteristics are captured and used to refine the final model gradient, improving accuracy while keeping the complexity manageable through iterative computation.
Solution Approach 2:
The patent applies dynamics by making the model gradient determination process adaptive and iterative. The participation degree is dynamically calculated based on current data volume information and local gradients, allowing the system to adjust the actual model gradient in real-time based on each node's contribution. This dynamic approach improves accuracy by continuously optimizing the gradient based on actual training conditions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments of this specification disclose model gradient determining methods, apparatuses, devices, and media based on federated learning. Solutions include the following: obtaining data volume information of a participating node, where the data volume information is used to indicate an amount of data used by the participating node to train a basic training model based on local data; obtaining a node local gradient obtained by training the basic training model based on the local data by the participating node; determining, based on the data volume information and the node local gradient, a global gradient of a federated learning model that the participating node participates in; determining a degree of participation of the participating node based on the node local gradient of the participating node and the global gradient; and determining an actual model gradient of the participating node based on the degree of participation.