Method and system for aggregating models in federated learning and computer readable storage medium

Abstract

The invention provides a method and system for aggregating a model in federated learning and a computer readable storage medium, and the method for aggregating the model comprises the following steps: a server initializes core parameters and issues the core parameters to at least two clients, and the clients receive the core parameters and initialize a local model; the client trains a local model, calculates a global sharpness perception minimization gradient, then carries out back propagation on the model according to dual variables, updates global model parameters and dual variables, and uploads the updated global model parameters and dual variables to the server for aggregation; the server side updates the core parameters after receiving the parameters uploaded by the client side; and continuously training until the model converges. According to the method, the feature extractor is optimized by minimizing the sharpness of the global model, the model is guided to be updated towards the direction which enables the global loss to be smoother, the influence of data isomerism and long-tail data on the model is reduced, the extracted features have more representativeness and generalization ability, and therefore the overall performance of the model is improved.

Description

Technical Field

[0001] This invention relates to the field of federated learning technology, and more particularly to a method, system, and computer-readable storage medium for an aggregation model in federated learning. Background Technology

[0002] Federated learning is a distributed machine learning framework with cryptographic features that enables collaborative training among multiple participants while ensuring privacy and data security. Specifically, in federated learning, multiple participants transmit encrypted parameters to a central server and collaboratively train a shared model. The performance of this shared model is comparable to that of the model trained using the aggregated original data from each participant. Unlike traditional distributed machine learning where each computing node samples data from the same dataset on a server, in federated learning, each client generates its own data independently. This results in a misalignment between the local data distribution on each client and the overall data distribution, and the local data on each client is generally heterogeneous, exhibiting a non-independent identically distributed (Non-IID) pattern. In general, data heterogeneity includes five scenarios: biased data feature distribution (different distribution characteristics across client data); biased data label distribution (different label distributions across client data); biased data feature and label correspondence (including cases of the same label with different features and the same feature with different labels); and data skew and class imbalance (unequal quantity and class distribution of local data on each client).

[0003] In practical applications, data with a long tail distribution is most common. Experiments have shown that this long-tail data can affect the convergence speed of the model and may even prevent the model from converging. The existence of these problems means that some clients, after participating in federated training, experience less model benefit compared to locally trained models, leading to unguaranteed convergence and model parameter divergence, which may reduce the model's accuracy.

[0004] Furthermore, federated training is based on communication rounds, during which clients and servers exchange parameters updated globally, while the server never accesses its local data. On the server side, a weighted average, introduced by a recognized standard algorithm, is typically used to aggregate these updates. While this approach is quite effective in scenarios where the data is identically distributed (i.i.d.), it fails to achieve the same performance in scenarios where the data is not identically distributed (i.i.d.). In particular, different local data distributions can lead to so-called client drift, where local models converge toward different solutions within the loss landscape, making server-side aggregation more challenging. Therefore, non-identically distributed data among clients results in slower model convergence, unstable learning trends, and a catastrophic forgetting of knowledge acquired by previously participating clients in the global model. Summary of the Invention

[0005] In view of this, the technical problem to be solved by the present invention is to provide a method, system and computer-readable storage medium for aggregation models in federated learning, which can effectively improve the accuracy of the model.

[0006] The technical solution of this invention is implemented as follows:

[0007] This invention proposes a method for aggregation models in federated learning, comprising the following steps:

[0008] S1. The server initializes the core parameters and sends them to at least two clients. The core parameters include the global model parameter w, the exponential moving average model parameter v, and the dual variable u.

[0009] S2. The client receives the core parameters and initializes the local model; the client uses local data D... m After training a local model and calculating the global sharpness-aware minimum gradient, backpropagation is performed on the model based on the dual variables to update the global model parameters and dual variables. The updated global model parameters and dual variables are then uploaded to the server for aggregation.

[0010] S3. After receiving the parameters uploaded by the client, the server first calculates the new global dual variable u. t+1 And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the exponential moving average model parameters v. t+1 ;

[0011] S4. The server determines whether the model has converged. If it has not converged, it sends the updated core parameters to the client and executes S2; otherwise, the training is completed.

[0012] Furthermore, S1 specifically includes:

[0013] The server randomly selects an active client set S. t The core parameters are sent to the clients in the collection, and each client receives the core parameters and uses them to initialize its local model.

[0014] In detail, the calculation of the global sharpness-perceived minimum gradient specifically includes:

[0015] In each round of local training, the client samples a mini-batch of data and calculates the global sharpness-aware minimization loss, i.e.

[0016]

[0017] In the above formula, the hyperparameter γ represents the intensity that minimizes sharpness. It is about minimizing the empirical risk of the client-side model. It's a global loss of sharpness.

[0018] Preferably, the step of backpropagating the model based on the dual variables to update the global model parameters and the dual variables specifically includes:

[0019] Introducing dual variable u m Perform backpropagation on the model, i.e.

[0020]

[0021] Then, each client k computes the dual variables for a new round. Right now

[0022]

[0023] In the above formula, β is the penalty coefficient, and K is the local interval.

[0024] Preferably, S3 specifically includes:

[0025] After receiving all the client model parameters participating in the training, the server will first calculate the new global dual variable u. t+1 And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the exponential moving average model parameters v. t+1 ,Right now:

[0026]

[0027] v t+1 =αv t +(1-α)w t+1

[0028] This invention also proposes a system for an aggregation model in federated learning, comprising a server and at least two clients;

[0029] The server is used to initialize core parameters and send them to at least two clients. The core parameters include global model parameters w, exponential moving average model parameters v, and dual variables u.

[0030] The client accepts the core parameters and initializes the local model; the client uses local data D m After training a local model and calculating the global sharpness-aware minimum gradient, backpropagation is performed on the model based on the dual variables to update the global model parameters and dual variables. The updated global model parameters and dual variables are then uploaded to the server for aggregation.

[0031] The server, after receiving parameters uploaded by the client, first calculates the new global dual variable u. t+1And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the exponential moving average model parameters v. t+1 ;

[0032] The server is used to determine whether the model has converged. If it has not converged, the updated core parameters are sent to the client; otherwise, training is completed.

[0033] Preferably, the calculation of the global sharpness-perceived minimum gradient specifically includes:

[0034] In each round of local training, the client samples a mini-batch of data and calculates the global sharpness-aware minimization loss, i.e.

[0035]

[0036] In the above formula, the hyperparameter γ represents the intensity that minimizes sharpness. It is about minimizing the empirical risk of the client-side model. It's a global loss of sharpness.

[0037] Preferably, the step of backpropagating the model based on the dual variables to update the global model parameters and the dual variables specifically includes:

[0038] Introducing dual variable u m Perform backpropagation on the model, i.e.

[0039]

[0040] Then, each client k computes the dual variables for a new round. Right now

[0041]

[0042] In the above formula, β is the penalty coefficient, and K is the local interval.

[0043] Preferably, after receiving all client model parameters participating in the training, the server will first calculate the new global dual variable u. t+1 And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the exponential moving average model parameters v. t+1 ,Right now:

[0044]

[0045] v t+1 =αv t +(1-α)w t+1 .

[0046] This invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aggregation model method in federated learning described above.

[0047] The present invention proposes a method, system, and computer-readable storage medium for aggregation models in federated learning. It takes into account the heterogeneous and long-tailed distribution of data in the practical application of federated learning. By minimizing the sharpness of the global model, the feature extractor is optimized, guiding the model to update in a direction that makes the global loss smoother. This reduces the impact of heterogeneous data and long-tailed data on the model, making the extracted features more representative and generalizable, thereby improving the overall performance of the model. Attached Figure Description

[0048] Figure 1 This is a flowchart of the aggregation model in federated learning proposed in this embodiment of the invention;

[0049] Figure 2 This is a schematic diagram of the aggregation model in federated learning proposed in an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the aggregation model system in federated learning proposed in an embodiment of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] like Figure 1 and Figure 2 As shown in the figure, this embodiment of the invention proposes a method for aggregation models in federated learning, including the following steps:

[0053] S1. The server initializes the core parameters and sends them to at least two clients. The core parameters include the global model parameter w, the exponential moving average model parameter v, and the dual variable u.

[0054] S2. The client receives the core parameters and initializes the local model; the client then uses local data D... m After training the local model and calculating the global sharpness-aware minimum gradient, backpropagation is performed on the model based on the dual variables to update the global model parameters and dual variables. The updated global model parameters and dual variables are then uploaded to the server for aggregation.

[0055] S3. After receiving the parameters uploaded by the client, the server first calculates the new global dual variable u. t+1 And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the exponential moving average model parameters v. t+1 ;

[0056] S4. The server determines whether the model has converged. If it has not converged, it sends the updated core parameters to the client and executes S2; otherwise, the training is completed.

[0057] As can be seen, the method for aggregating models in federated learning proposed in this embodiment of the invention optimizes the feature extractor by minimizing the sharpness of the global model, guides the model to update in a direction that makes the global loss smoother, reduces the impact of data heterogeneity and long-tail data on the model, and makes the extracted features more representative and generalizable, thereby improving the overall performance of the model.

[0058] In the federated learning process, K clients k = {1,2,…,K} and an aggregation server participate in the training, where each client has an independent dataset D. k We consider a global dataset D to follow a long-tailed distribution, where D = ∪ k D k At the start of training, the server initializes the global model parameters w, the exponential moving average (EMA) model parameters v, and a dual variable u. At the start of each training round, the server randomly selects a set of active clients S. t The global model parameter w and the exponential moving average (EMA) model parameter v are sent to the clients in the set. Each client receives the model parameter w and uses it to initialize its local model.

[0059] The client uses local data D m Training the model. In each round of local training, the client samples a mini-batch of data and calculates the global sharpness-perceived minimization loss, i.e.

[0060]

[0061] In the above formula, the hyperparameter γ represents the intensity that minimizes sharpness. It is about minimizing the empirical risk of the client-side model. This is the global sharpness loss. It's worth noting that the unbiased method used to minimize the global sharpness should use the global model parameters w. t and EMA model v t In other words, we need to minimize the global dataset D. However, in the federated learning framework, the global model parameters w t It will be sent to each client m and split into This involves performing independent updates as shown in the equation above. This process introduces a non-negligible bias into the global sharpness measure; to bridge this gap, local updates should be consistent with global updates. Therefore, this application introduces a dual variable u. m Perform backpropagation on the model, i.e.

[0062]

[0063] Then, each client k computes the dual variables for a new round. Right now

[0064]

[0065] In the above formula, β is the penalty coefficient, and K is the local interval. Using these update rules, a local model that roughly satisfies the constraints can be calculated. Intuitively, by maintaining the consistency of local updates, the global trajectory loss is minimized across clients. This will mitigate the impact of significant changes in the training sample loss within the global loss function. This is achieved by forcing the client's predictions to closely approximate the EMA model v. t The prediction results help prevent overfitting to local datasets. Therefore, each client can search for a smooth loss surface consistent with the global objective, thus significantly improving overall performance. Finally, each client will calculate... Uploaded to the server for aggregation.

[0066] After receiving all the client model parameters participating in the training, the server will first calculate the new global dual variable u. t+1 And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the global EMA model parameter v. t+1 ,Right now

[0067]

[0068] v t+1 =αv t +(1-α)w t+1

[0069] In this embodiment, the goal is to integrate the local model parameters trained on multiple clients into a unified global model. Since different clients have different amounts of data, the weights of the clients can be dynamically adjusted based on the amount of data.

[0070] The server repeats the above process continuously until the model converges.

[0071] As can be seen, this invention takes into account the case where the data distribution in federated learning is long-tailed, and the proposed method can effectively improve the accuracy of the model.

[0072] This invention considers the optimization of sharpness perception minimization in federated learning, which significantly reduces the number of communication rounds and training time compared to the traditional SAM algorithm.

[0073] The method of this invention can be combined with most traditional federated learning algorithms to further improve performance.

[0074] like Figure 3 As shown, in another embodiment of the present invention, the present invention also proposes a system for an aggregation model in federated learning, including a server 1 and at least two clients 2;

[0075] Server 1 is used to initialize core parameters and distribute them to at least two clients 2. The core parameters include global model parameters w, exponential moving average model parameters v, and dual variables u.

[0076] This is used by client 2 to receive core parameters and initialize the local model; client 2 uses local data D m After training the local model and calculating the global sharpness-aware minimum gradient, backpropagation is performed on the model based on the dual variables to update the global model parameters and dual variables. The updated global model parameters and dual variables are then uploaded to the server for aggregation.

[0077] After receiving the parameters uploaded by client 2, server 1 first calculates the new global dual variable u. t+1 And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the exponential moving average model parameters v. t+1 ;

[0078] Server 1 is used to determine whether the model has converged. If it has not converged, the updated core parameters are sent to the client; otherwise, training is completed.

[0079] As can be seen, the federated learning aggregation model system proposed in this embodiment of the invention optimizes the feature extractor by minimizing the sharpness of the global model, guides the model to update in a direction that makes the global loss smoother, reduces the impact of data heterogeneity and long-tail data on the model, and makes the extracted features more representative and generalizable, thereby improving the overall performance of the model.

[0080] In a preferred embodiment of the present invention, calculating the global sharpness-perceived minimum gradient specifically includes:

[0081] In each round of local training, the client samples a mini-batch of data and calculates the global sharpness-aware minimization loss, i.e.

[0082]

[0083] In the above formula, the hyperparameter γ represents the intensity that minimizes sharpness. It is about minimizing the empirical risk of the client-side model. It's a global loss of sharpness.

[0084] In a preferred embodiment of the present invention, backpropagating the model based on the dual variables to update the global model parameters and the dual variables specifically includes:

[0085] Introducing dual variable u m Perform backpropagation on the model, i.e.

[0086]

[0087] Then, each client k computes the dual variables for a new round. Right now

[0088]

[0089] In the above formula, β is the penalty coefficient, and K is the local interval.

[0090] In a preferred embodiment of the present invention, after receiving all client model parameters participating in the training, the server will first calculate the new global dual variable u. t+1 And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the exponential moving average model parameters v. t+1 ,Right now:

[0091]

[0092] v t+1 =αv t +(1-α)w t+1

[0093] In one embodiment of the present invention, the technical effect of processing long-tail data in federated learning is further illustrated by means of simulation experiments.

[0094] The algorithm steps in this embodiment are as follows: At the beginning of each training round, the server randomly selects a subset of active clients from the entire client set. The global server passes the parameters to the active clients for local training. The client calculates the global sharpness-aware minimization gradient, then updates the local model using the stochastic gradient descent algorithm, and finally updates the local dual variables for the local model. After completing the local training process, the local model parameters are transmitted to the global server for aggregation. The server updates the global model parameters based on the uploaded parameters.

[0095] In the simulation experiments below, the system parameters were set to default settings as follows. The datasets we used were CIFAR-10 / 100, CINIC-10, and AG News. We considered two common data heterogeneity scenarios: an extreme non-independent identically distributed (IID) setting and a Dirichlet setting, controlled by the variable 'r'. Furthermore, to further enhance data heterogeneity and simulate real-world scenarios, the original balanced datasets were shaped into a long-tailed distribution for the Dirichlet setting. In this experiment, the imbalance factor (i.e., the ratio of the number of instances in the most numerous class to the number of instances in the least numerous class) was set to 2 for all datasets.

[0096] Our comparative method mainly considers the following two aspects:

[0097] Traditional federated learning algorithms, such as FedAvg, FedNTD, and FedDyn.

[0098] The algorithms most relevant to us for processing long-tail data include FedSAM, FedSpeed, and FedSMOO.

[0099] We use the training accuracy, the number of communication rounds required to reach the target accuracy, and the training time as performance comparison standards for all methods. Based on the above methods and specific parameter settings, we evaluate the performance of the algorithms by changing the dataset and the degree of imbalance. Each experiment is repeated 5 times, and the average of the 5 experiments is taken as the experimental result. In the experimental comparison table below, the method for processing long-tail data in federated learning according to this invention is abbreviated as GloSAM.

[0100] As shown in Table 1, we compared the accuracy of our algorithm with six other algorithms across different datasets and varying degrees of imbalance. The comparison shows that our method achieves the best results in all cases, improving the accuracy of FedAvg by approximately 5%. Furthermore, our method requires fewer communication rounds and has shorter computation time. It is noteworthy that FedAvg and FedSAM fail to reach the target accuracy within a given number of communication rounds. State-of-the-art SAM-based methods, namely FedSpeed ​​and FedSMOO, require more than 3× more computation time than GloSAM on the CINIC-10 dataset and more than 2× more on the AG News dataset.

[0101] Table 1: Accuracy of different methods on different datasets and with different degrees of imbalance

[0102]

[0103] As shown in Table 2, we compared the number of communication rounds and training time required for our algorithm to reach the target accuracy with six other algorithms. GloSAM achieves a training speed approximately 0.7 times faster than FedAvg per round. In other words, GloSAM only takes 0.3 times more time than FedAvg to achieve a significant accuracy improvement. However, other SAM-based methods, such as FedSAM, FedSpeed, and FedSMOO, require more than 0.5 times more time due to the double calculation of gradients via the SAM optimizer. In terms of total time cost, GloSAM achieves the shortest training time cost among the benchmark methods because it effectively reduces the number of communication rounds.

[0104] Table 2: Number of communication rounds and training time required to achieve target accuracy using different methods

[0105]

[0106] Compared with the prior art, the present invention has the following obvious and prominent substantive features and significant technological advancements:

[0107] This invention takes into account the case where the data distribution in federated learning is non-independent and identically distributed, and the proposed method can effectively improve the accuracy of the model.

[0108] This invention considers the optimization of sharpness perception minimization in federated learning, which significantly reduces the number of communication rounds and training time compared to the traditional SAM algorithm.

[0109] The method of this invention can be combined with most traditional federated learning algorithms to further improve performance.

[0110] This invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aggregation model method in federated learning described above.

[0111] In summary, the present invention provides a method for aggregating and generalizing a global model in federated learning. Considering the heterogeneous and long-tailed distribution of data in practical applications of federated learning, the feature extractor is optimized by minimizing the sharpness of the global model, guiding the model to update in a direction that makes the global loss smoother, reducing the impact of data heterogeneity and long-tailed data on the model, making the extracted features more representative and generalizable, thereby improving the overall performance of the model.

[0112] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, and of course, it can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memory, special components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0113] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0114] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is only used to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for aggregation models in federated learning, characterized in that, Includes the following steps: S1. The server initializes the core parameters and sends them to at least two clients. The core parameters include the global model parameter w, the exponential moving average model parameter v, and the dual variable u. S2. The client receives the core parameters and initializes the local model; the client uses local data D... m After training a local model and calculating the global sharpness-aware minimum gradient, backpropagation is performed on the model based on the dual variables to update the global model parameters and dual variables. The updated global model parameters and dual variables are then uploaded to the server for aggregation. S3. After receiving the parameters uploaded by the client, the server first calculates the new global dual variable u. t+1 And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the exponential moving average model parameters v. t+1 ; S4. The server determines whether the model has converged. If it has not converged, it sends the updated core parameters to the client and executes S2; otherwise, the training is completed.

2. The method for aggregation models in federated learning as described in claim 1, characterized in that, S1 specifically includes: The server randomly selects an active client set S. t The core parameters are sent to the clients in the collection, and each client receives the core parameters and uses them to initialize its local model.

3. The method for aggregation models in federated learning as described in claim 1, characterized in that, The calculation of the global sharpness-perceived minimum gradient specifically includes: In each round of local training, the client samples a mini-batch of data and calculates the global sharpness-aware minimization loss, i.e. In the above formula, the hyperparameter γ represents the intensity that minimizes sharpness. It is about minimizing the empirical risk of the client-side model. It's a global loss of sharpness.

4. The method for aggregation models in federated learning as described in claim 3, characterized in that, The step of backpropagating the model based on the dual variables to update the global model parameters and dual variables specifically includes: Introducing dual variable u m Perform backpropagation on the model, i.e. Then, each client k computes the dual variables for a new round. Right now In the above formula, β is the penalty coefficient, and K is the local interval.

5. The method for aggregation models in federated learning as described in any one of claims 1-4, characterized in that, S3 specifically includes: After receiving all the client model parameters participating in the training, the server will first calculate the new global dual variable u. t+1 And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the exponential moving average model parameters v. t+1 ,Right now: v t+1 =αv t +(1-a)w t+1 6. A system for an aggregation model in federated learning, characterized in that, Includes a server and at least two clients; The server is used to initialize core parameters and send them to at least two clients. The core parameters include global model parameters w, exponential moving average model parameters v, and dual variables u. The client accepts the core parameters and initializes the local model; the client uses local data D m After training a local model and calculating the global sharpness-aware minimum gradient, backpropagation is performed on the model based on the dual variables to update the global model parameters and dual variables. The updated global model parameters and dual variables are then uploaded to the server for aggregation. The server, after receiving parameters uploaded by the client, first calculates the new global dual variable u. t+1 And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the exponential moving average model parameters v. t+1 ; The server is used to determine whether the model has converged. If it has not converged, the updated core parameters are sent to the client; otherwise, training is completed.

7. The system for aggregation models in federated learning as described in claim 6, characterized in that, The calculation of the global sharpness-perceived minimum gradient specifically includes: In each round of local training, the client samples a mini-batch of data and calculates the global sharpness-aware minimization loss, i.e. In the above formula, the hyperparameter γ represents the intensity that minimizes sharpness. It is about minimizing the empirical risk of the client-side model. It's a global loss of sharpness.

8. The system for aggregation models in federated learning as described in claim 7, characterized in that, The step of backpropagating the model based on the dual variables to update the global model parameters and dual variables specifically includes: Introducing dual variable u m Perform backpropagation on the model, i.e. Then, each client k computes the dual variables for a new round. Right now In the above formula, β is the penalty coefficient, and K is the local interval.

9. The system for aggregation models in federated learning as described in any one of claims 6-8, characterized in that, After receiving all the client model parameters participating in the training, the server will first calculate the new global dual variable u. t+1 And aggregate the global model parameters w based on the new dual variables. t+1 Finally, update the exponential moving average model parameters v. t+1 ,Right now: v t+1 =αv t +(1-a)w t+1 。 10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method for aggregation models in federated learning as described in any one of claims 1-5.

Images