Distributed Learning Model Aggregation with Dynamic Noise
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed learning methods face slow convergence and poor accuracy of the global model due to the use of random numbers for security enhancement, which impairs the learning process.
Innovation Solution
A distributed learning method that incorporates a monotonically decreasing function δ(j) in the aggregation formula for generating the global model, where the global model is updated as w→g,j←A(w→1,j,… ,w→N,j)+rδ(j), with r being a random number and δ(j) decreasing over cycles, to balance security and convergence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a random number is added to enhance security in distributed learning, then security against malicious attacks is improved, but convergence speed deteriorates
Solution Approach 1:
The patent applies dynamics by making the noise addition probability variable across different learning cycles. The probability of adding noise to the global model decreases as the learning cycle progresses, allowing high security protection in early cycles when the model is most vulnerable, while enabling faster convergence in later cycles when the model has already learned significant patterns. This dynamic adjustment resolves the contradiction between maintaining security and achieving fast convergence.
2Reliability
If a random number is added to enhance security in distributed learning, then security against malicious attacks is improved, but accuracy of the global model deteriorates
Solution Approach 1:
The patent makes the noise injection strategy dynamic by reducing the noise addition probability as learning progresses. In early cycles, high noise probability provides strong security protection when the model is most susceptible to attacks. As learning advances and the model achieves better accuracy, the reduced noise probability minimizes interference with the learning process, thereby maintaining both security and model accuracy simultaneously.
3Reliability
If noise is continuously added to the global model in each learning cycle, then security is maintained, but learning efficiency deteriorates
Solution Approach 1:
The patent implements a dynamic noise addition strategy where the probability of adding noise to the global model decreases with increasing learning cycle number. This means that in early learning cycles, noise is frequently added to maintain security, but as learning progresses, noise addition becomes less frequent, thereby improving learning efficiency without compromising overall security. This dynamic approach resolves the contradiction between security maintenance and learning efficiency.
Data Source
AI summary
A distributed learning technology that can accelerate convergence while ensuring security is provided. A learning step of generating, by an i-th client, a model →wi,j in a j-th cycle using learning data Di, an aggregation step of generating, by a server, a global model →wg,j in the j-th cycle from the models →w1,j, . . . , →wN,j in the j-th cycle according to a predetermined formula, and an end condition determination step of, by the server, ending learning processing with the global model →wg,j in the j-th cycle as the global model →wg in a case where a predetermined end condition is satisfied, and otherwise, transmitting the global model →wg,j in the j-th cycle to the i-th client, and an initialization step of setting, by the i-th client, the global model →wg,j in the j-th cycle as an initial value of a model →wi,j+1 in a j+1-th cycle are included.