Asynchronous federal learning efficient dynamic adaptation method for edge network
By employing adaptive threshold gradient compression, dynamic sample size adjustment, and triple weight correction mechanisms, the device heterogeneity and dynamic network adaptability issues of asynchronous federated learning in edge networks are addressed, achieving efficient communication optimization and model performance improvement.
Patent Information
- Application Number
- CN202511809269.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-03
- Publication Date
- 2026-03-13
AI Technical Summary
Traditional synchronous federated learning faces challenges in edge networks such as heterogeneous device capabilities, unstable network connections, and tail effects, resulting in low convergence efficiency and poor system scalability. Asynchronous federated learning suffers from delayed device updates that impair global convergence, resource heterogeneity that leads to unbalanced node contributions, and a lack of adaptive coordination mechanisms for dynamic network conditions, making it difficult to balance communication efficiency, convergence speed, and model accuracy.
We propose an efficient dynamic adaptation method for asynchronous federated learning in edge networks. This method optimizes the allocation of communication and computing resources by using adaptive threshold gradient compression to select diligent nodes, dynamic sample size adjustment, and a triple weight gradient correction mechanism, thus adapting to the dynamic characteristics of edge networks.
It significantly improves communication efficiency, optimizes resource utilization, enhances model performance, ensures privacy protection, improves convergence speed and steady-state accuracy, and adapts to complex edge network scenarios.
Smart Images
Figure CN121658148A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of edge computing and federated learning technologies, specifically to an efficient dynamic adaptation method for asynchronous federated learning in edge networks. Background Technology
[0002] With the widespread adoption of Internet of Things (IoT) technology, the data generated by distributed terminal devices is growing explosively, driving the development of decentralized machine learning paradigms. In traditional cloud computing architectures, all data needs to be uploaded to the cloud for processing, leading to problems such as excessive network bandwidth consumption, high communication latency, and risks of data privacy leaks. Edge computing offloads computing tasks to edge devices, effectively reducing the load on the cloud; federated learning, through local training on edge nodes and aggregation of model parameters, saves communication costs while protecting data privacy, becoming one of the core technologies in the field of edge intelligence.
[0003] Federated learning mechanisms consist of a parameter server and multiple edge nodes: the parameter server is responsible for aggregating gradients uploaded by each node and updating the global model, while edge nodes are trained independently based on local sensitive data, sharing model parameters only with the server and not leaking raw data, thus ensuring data privacy. However, traditional synchronous federated learning (such as FedAvg) faces challenges in real-world edge networks, such as heterogeneous device capabilities, unstable network connections, and tailing effects, resulting in low convergence efficiency and poor system scalability.
[0004] To address the aforementioned issues, Asynchronous Federated Learning (AFL) emerged, allowing edge nodes to independently upload updates and eliminating idle waiting time. However, AFL still faces three key problems: (1) Delayed updates of outdated models on devices can impair global convergence; (2) Resource heterogeneity leads to unbalanced node contributions; and (3) Dynamic network conditions lack adaptive coordination mechanisms. Existing research often employs fixed-weight aggregation or static client selection strategies, which fail to flexibly adapt to the dynamic characteristics of edge networks and make it difficult to balance communication efficiency, convergence speed, and model accuracy. Summary of the Invention
[0005] The purpose of this invention is to provide an efficient dynamic adaptation method for asynchronous federated learning in edge networks, which can dynamically adapt to the heterogeneity of edge networks, optimize the allocation of communication and computing resources, and suppress the performance degradation of asynchronous updates, so as to meet the application needs of practical edge intelligence scenarios.
[0006] The technical solution to achieve the purpose of this invention is: an efficient dynamic adaptation method for asynchronous federated learning in edge networks, applied to an asynchronous federated learning system consisting of a parameter server and multiple edge nodes, comprising the following steps:
[0007] S1. Initialization Configuration: The parameter server deploys the global model and sets the training parameters, the edge nodes prepare local training data and validation data, and the system initializes the node active window;
[0008] S2, Model Parameter Broadcast: The parameter server broadcasts the current global model parameters to all edge nodes, and the edge nodes overwrite their local models after receiving the parameters.
[0009] S3. Local Training and Information Calculation: Edge nodes perform model training based on local training data, and calculate local gradients, training sample size, communication latency and local test accuracy. The communication latency is the difference between the node upload timestamp and the previous round of parameter download timestamp.
[0010] S4. Adaptive threshold gradient compression: Dynamically calculate the threshold to adapt to the gradient changes in this round, and filter diligent nodes and lazy nodes according to the threshold. Diligent nodes upload local gradients, training sample size, communication latency and local test accuracy to the parameter server. Lazy nodes accumulate gradients locally and upload them only when the threshold condition is met.
[0011] S5. Dynamic Sample Size Adjustment: After receiving the information uploaded by diligent nodes, the parameter server constructs a composite evaluation index based on the training sample size and communication latency. It normalizes the evaluation index of nodes within the node's active window to obtain a weight allocation factor, and dynamically adjusts the local training sample size of each edge node in the next round according to the weight allocation factor.
[0012] S6. Triple Weight Gradient Correction: The parameter server constructs sample weights based on the number of training samples and communication latency, latency weights based on communication latency, and model accuracy weights based on local test accuracy. The three weights are multiplied to obtain a composite weight, which is then used to dynamically correct the uploaded local gradient.
[0013] S7, Model Aggregation Update: The parameter server aggregates all corrected local gradients and executes an optimization algorithm to update the global model;
[0014] S8. Iterative training: Repeat steps S2 to S7 until the global model reaches the preset convergence condition.
[0015] Compared with the prior art, the significant advantages of this invention are:
[0016] (1) Communication efficiency is significantly improved: Through adaptive threshold gradient compression, the number of communication times is reduced to 16.6% of that of traditional methods, which greatly reduces the bandwidth usage and communication overhead of the edge network;
[0017] (2) Resource utilization optimization: The dynamic sample size adjustment strategy allocates tasks based on node computing power and network status, so that high-efficiency nodes can bear more load and low-efficiency nodes can reduce pressure, adapting to the heterogeneity of edge devices.
[0018] (3) Model performance enhancement: The triple weight correction mechanism takes into account the sample size, communication latency and model accuracy, effectively suppresses the model deviation caused by asynchronous updates, and improves the convergence speed, steady-state accuracy and robustness;
[0019] (4) Complete privacy protection: edge nodes always retain the original data and only upload model parameters and gradient information, which meets the core privacy protection requirements of federated learning. Attached Figure Description
[0020] Figure 1 This is a schematic diagram showing an overview of the EDAFL training rounds of the present invention.
[0021] Figure 2 This is a diagram illustrating federated learning. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] This invention provides an efficient dynamic adaptation method for asynchronous federated learning in edge networks, applicable to an asynchronous federated learning system consisting of a parameter server and multiple edge nodes, comprising the following steps:
[0024] S1. Initialization Configuration: The parameter server deploys the global model and sets the training parameters, the edge nodes prepare local training data and validation data, and the system initializes the node active window;
[0025] S2, Model Parameter Broadcast: The parameter server broadcasts the current global model parameters to all edge nodes, and the edge nodes overwrite their local models after receiving the parameters.
[0026] S3. Local Training and Information Calculation: Edge nodes perform model training based on local training data, and calculate local gradients, training sample size, communication latency and local test accuracy. The communication latency is the difference between the node upload timestamp and the previous round of parameter download timestamp.
[0027] S4. Adaptive threshold gradient compression: Dynamically calculate the threshold to adapt to the gradient changes in this round, and filter diligent nodes and lazy nodes according to the threshold. Diligent nodes upload local gradients, training sample size, communication latency and local test accuracy to the parameter server. Lazy nodes accumulate gradients locally and upload them only when the threshold condition is met.
[0028] S5. Dynamic Sample Size Adjustment: After receiving the information uploaded by diligent nodes, the parameter server constructs a composite evaluation index based on the training sample size and communication latency. It normalizes the evaluation index of nodes within the node's active window to obtain a weight allocation factor, and dynamically adjusts the local training sample size of each edge node in the next round according to the weight allocation factor.
[0029] S6. Triple Weight Gradient Correction: The parameter server constructs sample weights based on the number of training samples and communication latency, latency weights based on communication latency, and model accuracy weights based on local test accuracy. The three weights are multiplied to obtain a composite weight, which is then used to dynamically correct the uploaded local gradient.
[0030] S7, Model Aggregation Update: The parameter server aggregates all corrected local gradients and executes an optimization algorithm to update the global model;
[0031] S8. Iterative training: Repeat steps S2 to S7 until the global model reaches the preset convergence condition.
[0032] As a specific example, the threshold mentioned in step S4 is calculated in the following way:
[0033] Define a set of lazy nodes that satisfy the gradient contribution threshold condition, and combine the gradient descent optimization algorithm, model parameter changes, and scaling factor. The node gradient self-checking rule is derived; the gradient self-checking rule uses the approximate value of the change in model parameters, the learning rate, the scaling factor and the total number of nodes as parameters to form an adaptive threshold for gradient changes in each round.
[0034] As a specific example, the approximate value of the change in model parameters is obtained by accumulating the changes in model parameters over the first D rounds, where D is a constant, and ,in This represents the weighting coefficient for the parameter changes in each round.
[0035] As a specific example, the calculation formula for the composite evaluation index in step S5 is as follows:
[0036]
[0037] in, Indicates composite evaluation indicators, For nodes The number of local data samples, For nodes The communication delay is ε, where ε is the minimum time threshold.
[0038] As a specific example, the node New target sample number The calculation formula is:
[0039]
[0040] in This is the sample size adjustment coefficient. The normalized weight allocation factor. This represents the total number of samples that the system is expected to participate in each round of training.
[0041] As a specific example, the formula for calculating the sample weights in step S6 is:
[0042]
[0043] in, Indicates sample weights, , For hyperparameters, For nodes Upload delay time, Upload the number of samples for node i. It is the sum of all participating node samples in the current round of aggregation.
[0044] As a specific example, the formula for calculating the delay weight in step S6 is:
[0045]
[0046] in, Indicates the delay weight. The amount of data affects the control parameters. , These are parameters for delay scaling and decay rate control. For nodes Upload delay time, For nodes Number of samples uploaded.
[0047] As a specific example, the formula for calculating the model accuracy weights in step S6 is:
[0048]
[0049] in, Represents the model accuracy weights. As a scaling factor, For nodes Upload the model's accuracy on the local validation set.
[0050] As a specific example, the edge node has the characteristics of heterogeneous computing power and limited bandwidth. The parameter server adopts a GPU server, and the edge node communicates with the parameter server through the Thrift framework.
[0051] As a specific example, the optimization algorithm is gradient descent, the global model adopts the ResNet-18 architecture, and the local training data is distributed according to the Dirichlet distribution to simulate the case of non-independent and identically distributed data.
[0052] This invention proposes an efficient dynamic adaptation method for asynchronous federated learning in edge networks, the key technologies of which include:
[0053] (1) An asynchronous federated learning mechanism for collaborative learning of edge nodes is proposed, enabling nodes to perform deep learning while protecting their own data privacy.
[0054] (2) An efficient dynamic adjustment algorithm was designed, which reduced the number of gradient communication steps while ensuring the accuracy of the test set. In addition, the local training sample size was adaptively adjusted according to the current status of the edge nodes, which improved the training efficiency.
[0055] (3) In order to adapt to the problem of uneven learning samples and different learning progress of edge nodes in asynchronous model training, a triple weight correction method is proposed to solve the performance reduction problem caused by asynchronous federated learning.
[0056] Through experiments on real edge datasets and edge networks, EDAFL can reduce the number of communications to 16.6% of the original while significantly improving the model's convergence speed and loss reduction speed, effectively enhancing the model's robustness while maintaining its accuracy.
[0057] The efficient dynamic adaptation method proposed in this invention is designed for edge network nodes and includes two core functional modules: the first is a joint optimization mechanism that combines adaptive threshold gradient compression and dynamic sample size adjustment to improve the learning efficiency of edge nodes by coordinating the control of communication sparsity and local computing load; the second is a gradient correction mechanism based on triple weights, which constructs a composite weight factor based on the node training sample size, communication latency and local test accuracy to dynamically correct the asynchronous update gradient to suppress model deviation. The two work together to optimize the communication-computation link, which significantly compresses the communication scale while ensuring the convergence robustness of asynchronous federated learning.
[0058] Gradient compression refers to compressing the gradient communication between nodes and servers, that is, reducing the number of communications between a single node and the parameter server. Whether using a fixed compression rate to randomly select compression nodes or using a fixed threshold as the criterion, both methods have significant shortcomings. The former may ignore important gradient nodes with high information content, affecting the overall model training effect; the latter is prone to over-compression, causing model fluctuations and difficulty in convergence in the later stages of training. Neither method fully considers the dynamic changes in gradients during the learning process.
[0059] EDAFL achieves communication compression by dynamically calculating a threshold to adapt to gradient changes in each round. Only nodes that meet the threshold can communicate with the parameter server; nodes that do not meet the threshold accumulate gradient information locally and upload it only after accumulating sufficient information. Regardless of whether a node is eligible to communicate, a gradient check must be performed after each round of learning, a process that continues throughout the entire training cycle.
[0060] In traditional federated learning, let m be the number of learning nodes. The parameter server coordinates with m learning nodes to update model parameters. The process includes, in the (k+1)th iteration, the parameter server first updates the current model... Broadcast to all nodes; subsequently, each node Calculate local gradient The gradients are then uploaded, and finally, the parameter server aggregates all received gradients to obtain a sum. Based on this, an optimization algorithm is executed to update the model parameters.
[0061] like Figure 1 As shown, the main idea of this invention is to divide the nodes in training into "lazy" nodes and "diligent" nodes according to the magnitude of their gradients. Through a dynamic selection mechanism, only "diligent" nodes with significant gradient changes are allowed to participate in communication, while "lazy" nodes that contribute less in the current round are ignored. The set of "lazy" nodes is used... The set of "diligent" nodes is represented by... Therefore Can be changed to
[0062] (1)
[0063] in, Represents a set of nodes. This represents the sum of gradients aggregated by the parameter server. Represents the gradient of the "lazy" node. The gradient of "diligent" nodes and the set of "lazy" nodes are represented. satisfy:
[0064] (2)
[0065] Suppose that the optimization algorithm for updating the parameters is gradient descent:
[0066] (3)
[0067] in, As the learning rate, substituting equation (3) into equation (2) yields:
[0068] (4)
[0069] in, According to the AM-GM inequality, we know that satisfy:
[0070] (5)
[0071] If node Then further:
[0072] (6)
[0073] Due to sets Since the quantity cannot be known in advance, we introduce a proportionality coefficient. to indicate The total number, After rearranging (6), we can obtain:
[0074] (7)
[0075] Because parameter changes tend to smooth out during training, In this invention, it is approximated as
[0076] (8)
[0077] Among them D and All are constants. Simply put, in the relevant experiments of this invention,
[0078] Substituting equation (8) into equation (7), we get:
[0079] (9)
[0080] Equation (9) defines the gradient self-checking rule of a node after each round of learning: if the equation is not satisfied, the node communicates with the parameter server; if the equation is satisfied, the node skips communication, accumulates gradients locally, and continues to the next round of learning.
[0081] Compared to traditional federated learning, the method of this invention significantly reduces its dependence on network bandwidth and other equipment configurations. Bandwidth directly affects node transmission time; low bandwidth prolongs the learning time of a single round, thus increasing the overall time consumption. Therefore, controlling the overall time requires limiting bandwidth configuration. However, the gradient compression mechanism of this invention reduces the number of communication rounds, thereby reducing the number of communication rounds affected by bandwidth. Therefore, theoretically, its demand for bandwidth and other resources is lower than that of traditional uncompressed methods, making it more adaptable to bandwidth-constrained network environments.
[0082] like Figure 2As shown, in the asynchronous federated learning framework, the computational power and communication latency of edge nodes can reflect the training efficiency of the nodes. Given the significant heterogeneity of nodes in terms of computational resources and network status, a static task allocation strategy can easily lead to low resource utilization and training bottlenecks. To optimize the convergence efficiency of the global model and system performance, this invention proposes a composite evaluation index that integrates the training sample size and the communication latency difference (the difference between the upload and download timestamps) to dynamically quantify the overall execution performance of nodes.
[0083] In the In round training, using Represents a node The number of local data samples, Represents a node The difference between the upload timestamp and the timestamp of the parameters sent in the previous round (round K-1), i.e. So, training efficiency metrics It can then be expressed as:
[0084] (10)
[0085] in The minimum time threshold (to prevent division by zero) is used in this invention. =0.1s.
[0086] However, in asynchronous federated learning, due to the different training progress (i.e., different rounds) of each edge node, it is impossible to directly unify the current state of all nodes for a complete global aggregation. To solve this problem, EDAFL maintains a node activity window W on the parameter server. Each time the server receives a model update uploaded by an edge node, it calculates the efficiency metric of that node in real time and includes it in window W. This metric is then normalized together with the metrics of other recently uploaded nodes to construct a relative weight allocation factor. , satisfy:
[0087] (11)
[0088] Accordingly, the number of new target samples at node i is:
[0089] (12)
[0090] Wherein, γ is an adjustment coefficient that controls whether the sample size is increased or decreased. It is the total number of samples that the system expects to participate in each round of training. Simply put, in this application... =1.2.
[0091] Communication latency difference reflects the real-time network status of a node, while the training sample size characterizes its local processing load and participation. By calculating the ratio between the two, the efficiency-cost ratio of a node's task processing in the current iteration round can be objectively evaluated. After normalizing this index value for each node, its contribution weight in global aggregation can be dynamically adjusted. Based on this weighting mechanism, the system can further implement an adaptive sample allocation strategy: prioritizing the allocation of more task load to nodes with superior performance, while reducing the pressure on nodes with slow response or excessive load. This method not only efficiently utilizes heterogeneous resources but also effectively alleviates the model offset problem inherent in asynchronous update mechanisms, significantly improving the stability and convergence efficiency of the overall training process.
[0092] EDAFL targets highly flexible edge nodes. In asynchronous federated learning, nodes suffer from uneven learning samples, varying learning progress, and differences in learning accuracy. Therefore, the contribution of parameters uploaded by different nodes to the global parameters differs, making it unreasonable for these nodes to update the global parameters in the parameter server equally. This invention designs a dynamic weighting scheme for nodes based on a three-weight fusion mechanism to achieve a more refined evaluation of the quality of local updates. This scheme introduces three weights, each reflecting a sample weight. Communication delay weight And local model accuracy weights Significant impact on global model aggregation.
[0093] Sample weights This measure aims to assess the proportion of training samples contributed by a node in the current upload relative to the total number of samples. It also introduces a latency adjustment term to encourage nodes with lower communication latency to receive higher weights for effective samples. The definition is as follows:
[0094] (13)
[0095] in, This represents the time difference (i.e., "lag") between the latest upload and download of node i. Indicates the number of samples uploaded by the node. This is the sum of samples from all participating nodes in the current round of aggregation. Hyperparameter and Controlling the degree to which delay amplifies or attenuates sample weights; when the delay is low ( (Small), the data volume weight of nodes is appropriately amplified to prioritize the absorption of timely updates; when the latency is high, the contribution of its samples is suppressed through a power-law decay term to mitigate the interference of "outdated information" on model aggregation. In this application, =1, =1.5.
[0096] Used to measure the relative timeliness of node models, the core idea is: the smaller the latency, the more important its update is, and the more priority it should be given to fusion. Its definition is as follows:
[0097] (14)
[0098] in, Represents a node Upload delay time, Parameters for adjusting the degree of influence of node data volume, and The scaling magnitude and nonlinear decay rate of the time delay are controlled, simply in this invention. , .
[0099] When the amount of node data is too large, its excessive influence is penalized by the denominator; in addition, the following is adopted: The term explicitly strengthens the inverse effect of time delay to avoid the lagging model from disrupting the global convergence trend. It is worth noting that, due to and Common dependence and These two elements constitute a coupled dynamic weighting mechanism, which improves the real-time performance and stability of the model while ensuring the fusion of multi-source data.
[0100] During training, some nodes may exhibit better generalization ability when uploaded to the model due to their data distribution being closer to the target task or their model converging more fully locally. Therefore, an accuracy factor is introduced. :
[0101] (15)
[0102] in This indicates the accuracy of the model uploaded by the node on the local validation set. To adjust the proportional factor of its contribution magnitude. By directly measuring model quality, the aggregation process is guided to favor nodes with better performance, thereby enhancing the overall generalization ability of the model.
[0103] The triple weighting is adjusted as follows:
[0104] (16)
[0105] in, These are the original model parameters. These are the corrected model parameters.
[0106] During asynchronous learning, the gradients submitted by edge nodes need to be corrected by triple weights on the parameter server before they can participate in global model optimization. The corrected gradients update the global parameters according to the optimization algorithm. After optimization, the nodes obtain the latest parameters to cover the local model and then execute the next round of iterative learning.
[0107] For ease of understanding, the main parameter symbols and their meanings involved in this invention are explained below:
[0108] , representing the learning rate, is used to update the step size in gradient descent;
[0109] , represents the proportion coefficient, used to indicate the percentage of the number of "lazy" nodes;
[0110] , which represents the window size (constant) of the cumulative changes in model parameters over the previous few rounds;
[0111] , representing the weighting coefficient of the parameter change in each round;
[0112] , representing a node The number of local data samples;
[0113] , representing a node A set;
[0114] , represents the set of lazy nodes;
[0115] , representing the set of diligent nodes;
[0116] , representing a node Communication latency (difference between upload timestamp and download timestamp);
[0117] , represents the minimum time threshold (to prevent division by zero);
[0118] , representing a node Training efficiency evaluation indicators;
[0119] , representing a node Normalized weighting factor;
[0120] , representing a node The number of new target samples;
[0121] , representing the sample size adjustment coefficient;
[0122] , representing the total number of samples expected to participate in each round of training;
[0123] , representing the sample weight, a weighting factor that combines the number of samples and the time delay;
[0124] , represents a hyperparameter that controls the degree to which the delay amplifies or attenuates the sample weights;
[0125] , representing a node Upload delay time;
[0126] , representing a node The number of samples uploaded;
[0127] , representing the sum of all participating node samples in the current round of aggregation;
[0128] , representing the latency weight, which measures the timeliness of node updates;
[0129] This indicates that the amount of data affects the control parameters;
[0130] , representing the delay scaling and decay rate control parameters;
[0131] , represents the model accuracy weight, based on the accuracy of the local validation set;
[0132] , representing the scaling factor, which adjusts the contribution of the accuracy weight;
[0133] , representing a node Upload the model's accuracy on the local validation set;
[0134] , indicating the first Round global model parameters;
[0135] , indicating the first The updated global model parameters;
[0136] , representing a node In the The local gradient of the wheel;
[0137] , indicating the first The sum of gradients uploaded by all nodes in the round;
[0138] , indicating the active window of a node;
[0139] , representing a node Corrected model parameters;
[0140] , representing a node Original model parameters.
[0141] In summary, this invention proposes an efficient dynamic adaptation method for asynchronous federated learning (AFL) in edge networks, aiming to optimize the training efficiency and model performance of AFL systems in complex scenarios such as uneven data distribution, heterogeneous node computing power, and limited communication resources. EDAFL first constructs a federated learning mechanism that supports asynchronous updates, enabling edge nodes to train independently and asynchronously upload gradients while protecting local data privacy, avoiding the global waiting overhead of synchronous schemes. Based on this, EDAFL proposes an efficient dynamic adjustment algorithm consisting of two parts: first, it adjusts the number of local training samples in real time based on the node's current computing power and network state, allowing high-efficiency nodes to bear more computational load and reducing the pressure on low-efficiency nodes, thereby accelerating global model updates and improving resource utilization; second, it designs an adaptive threshold adjustment mechanism that dynamically changes the upload triggering condition according to gradient characteristics and update latency, significantly compressing the number of gradient communication calls without affecting model accuracy. To address the aggregation bias caused by different learning progress and uneven data volume among nodes in asynchronous training, EDAFL further introduces a triple weight strategy to comprehensively optimize global updates, effectively mitigating the performance degradation caused by asynchronous updates. Experimental results show that EDAFL can significantly reduce the number of communications while ensuring model accuracy and robustness, and accelerate the convergence speed and loss reduction rate. This verifies the significant advantages of combining dynamic sample adjustment and adaptive threshold control in improving the training efficiency and communication utilization of asynchronous federated learning.
[0142] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.
Claims
1. An efficient dynamic adaptation method for asynchronous federated learning in edge networks, characterized in that, The asynchronous federated learning system, applied to a parameter server and multiple edge nodes, includes the following steps: S1. Initialization Configuration: The parameter server deploys the global model and sets the training parameters, the edge nodes prepare local training data and validation data, and the system initializes the node active window; S2, Model Parameter Broadcast: The parameter server broadcasts the current global model parameters to all edge nodes, and the edge nodes overwrite their local models after receiving the parameters. S3. Local Training and Information Calculation: Edge nodes perform model training based on local training data, and calculate local gradients, training sample size, communication latency and local test accuracy. The communication latency is the difference between the node upload timestamp and the previous round of parameter download timestamp. S4. Adaptive threshold gradient compression: Dynamically calculate the threshold to adapt to the gradient changes in this round, and filter diligent nodes and lazy nodes according to the threshold. Diligent nodes upload local gradients, training sample size, communication latency and local test accuracy to the parameter server. Lazy nodes accumulate gradients locally and upload them only when the threshold condition is met. S5. Dynamic Sample Size Adjustment: After receiving the information uploaded by diligent nodes, the parameter server constructs a composite evaluation index based on the training sample size and communication latency. It normalizes the evaluation index of nodes within the node's active window to obtain a weight allocation factor, and dynamically adjusts the local training sample size of each edge node in the next round according to the weight allocation factor. S6. Triple Weight Gradient Correction: The parameter server constructs sample weights based on the number of training samples and communication latency, latency weights based on communication latency, and model accuracy weights based on local test accuracy. The three weights are multiplied to obtain a composite weight, which is then used to dynamically correct the uploaded local gradient. S7, Model Aggregation Update: The parameter server aggregates all corrected local gradients and executes an optimization algorithm to update the global model; S8. Iterative training: Repeat steps S2 to S7 until the global model reaches the preset convergence condition.
2. The efficient dynamic adaptation method for asynchronous federated learning in edge networks according to claim 1, characterized in that, The threshold mentioned in step S4 is calculated in the following way: Define a set of lazy nodes that satisfy the gradient contribution threshold condition, and combine the gradient descent optimization algorithm, model parameter changes, and scaling factor. The node gradient self-checking rule is derived; the gradient self-checking rule uses the approximate value of the change in model parameters, the learning rate, the scaling factor and the total number of nodes as parameters to form an adaptive threshold for gradient changes in each round.
3. The efficient dynamic adaptation method for asynchronous federated learning in edge networks according to claim 2, characterized in that, The approximate value of the model parameter change is obtained by accumulating the model parameter changes over the first D rounds, where D is a constant. ,in This represents the weighting coefficient for the parameter changes in each round.
4. The efficient dynamic adaptation method for asynchronous federated learning in edge networks according to claim 1, characterized in that, The calculation formula for the composite evaluation index mentioned in step S5 is as follows: ； in, Indicates composite evaluation indicators, For nodes The number of local data samples, For nodes The communication delay is ε, where ε is the minimum time threshold.
5. The efficient dynamic adaptation method for asynchronous federated learning in edge networks according to claim 4, characterized in that, The node New target sample number The calculation formula is: ； in This is the sample size adjustment coefficient. The normalized weight allocation factor. This represents the total number of samples that the system is expected to participate in each round of training.
6. The efficient dynamic adaptation method for asynchronous federated learning in edge networks according to claim 1, characterized in that, The formula for calculating the sample weights in step S6 is as follows: ； in, Indicates sample weights, , For hyperparameters, For nodes Upload delay time, Upload the number of samples for node i. It is the sum of all participating node samples in the current round of aggregation.
7. The efficient dynamic adaptation method for asynchronous federated learning in edge networks according to claim 1, characterized in that, The formula for calculating the delay weight in step S6 is as follows: ； in, Indicates the delay weight. The amount of data affects the control parameters. , These are parameters for delay scaling and decay rate control. For nodes Upload delay time, For nodes Number of samples uploaded.
8. The efficient dynamic adaptation method for asynchronous federated learning in edge networks according to claim 1, characterized in that, The formula for calculating the model accuracy weights in step S6 is as follows: ； in, Represents the model accuracy weights. As a scaling factor, For nodes Upload the model's accuracy on the local validation set.
9. The efficient dynamic adaptation method for asynchronous federated learning in edge networks according to claim 1, characterized in that, The edge nodes are characterized by heterogeneous computing power and limited bandwidth. The parameter server is a GPU server, and the edge nodes communicate with the parameter server through the Thrift framework.
10. The efficient dynamic adaptation method for asynchronous federated learning in edge networks according to claim 1, characterized in that, The optimization algorithm is gradient descent, the global model adopts the ResNet-18 architecture, and the local training data is distributed according to the Dirichlet distribution to simulate the case of non-independent and identically distributed data.