Model Gradient Updates with Actor-Critic Node Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In horizontal federated learning scenarios, the central server's average gradient aggregation method leads to poor model training results due to uneven data quality and node performance among participants, resulting in issues like over-fitting or under-fitting.
Innovation Solution
Implement an Actor-Critic network to determine node probabilities and adjust gradients based on these probabilities, using a deep reinforcement learning data fusion algorithm to optimize model training by integrating the strengths of different nodes' data contributions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If the central server uses average gradient aggregation method, then the implementation is simple, but the model training results are poor due to uneven data quality and node performance
Solution Approach 1:
The patent changes the aggregation parameter from uniform averaging to probability-weighted aggregation. The central server calculates a probability value for each node based on their historical performance and data quality, then uses these probabilities as weights in the gradient aggregation process. This parameter transformation allows the system to adaptively adjust the contribution of each node, resolving the contradiction between implementation simplicity and model training accuracy.
2Ease of operation
If all nodes participate equally in gradient aggregation, then the system is fair and simple to implement, but nodes with poor data quality degrade the model performance
Solution Approach 1:
The patent applies local quality by assigning different probability values to different nodes based on their individual characteristics. Each node's data quality, computational power, and historical contribution are evaluated locally, and this local assessment determines its probability weight in the global gradient aggregation. This resolves the contradiction by maintaining operational simplicity while ensuring that nodes with better local quality have greater influence on model performance.
3Device complexity
If the central server aggregates gradients from all nodes uniformly, then the computation is straightforward, but the training converges slowly or gets stuck in local optima
Solution Approach 1:
The patent introduces dynamics by making the aggregation weights adaptive rather than static. The probability values for each node are dynamically adjusted based on their performance in previous training rounds and the current state of the model. This dynamic weighting mechanism allows the system to automatically optimize convergence speed without significantly increasing computational complexity, as the probability calculations build upon historical data.
Data Source
AI summary
The present application provides a model gradient update method and device, for use in improving the accuracy of model training. A central server repeatedly executes a gradient update process until a stop condition is satisfied. One gradient update process comprises: receiving first gradients respectively sent by multiple nodes, the first gradients being obtained by each node using sample data to train a model to be trained of the node one or more times; obtaining a second gradient one the basis of the multiple first gradients and the probability of each node in the present gradient update process, the probability of each node in the present gradient update process being determined by an Actor-Critic network one the basis of the probability of each node in the last gradient update process; and sending the second gradient to the multiple nodes, respectively.