Lightweight federated learning method and system based on parameter clustering and model decomposition

Abstract

The invention belongs to the technical field of federated learning, and particularly relates to a lightweight federated learning method and system based on parameter clustering and model decomposition. The method comprises the following steps: S1, initializing server model parameters and client model parameters; s2, performing local training of the model on each client, and storing the model increment of each round of training; s3, compressing the local model of each client, encoding the model increment at the same time, and after the encoding is completed, uploading the code to a server by each client; s4, the server decodes and aggregates all the uploaded coded information, encodes the aggregated information again, and downloads the codes to all the clients after encoding is completed; and S5, each client decodes the code downloaded by the server, and updates the model parameters of the respective client according to the decoding result. The method has the characteristic that the communication efficiency and the model performance maintenance can be considered.

Description

Technical Field

[0001] This invention belongs to the field of federated learning technology, specifically relating to a lightweight federated learning method and system based on parameter clustering and model decomposition. Background Technology

[0002] As artificial intelligence (AI) technology deeply penetrates key sectors such as finance, healthcare, and electric vehicles, its data processing paradigm is facing a fundamental transformation. In these fields, AI not only improves the efficiency of data processing but also drives intelligent decision-making. However, traditional centralized AI architectures are gradually revealing their limitations when faced with cross-institutional data privacy, distributed data heterogeneity, and the asymmetric constraints of computing power and communication resources on edge devices. Against this backdrop, Federated Learning (FL), as a privacy-preserving distributed machine learning paradigm, aims to distribute data processing and model training tasks across multiple devices or nodes rather than concentrating them on a single server. This distributed processing model is closely linked to a client-server (CS) architecture, where clients are responsible for local data processing and model training, while the server aggregates model updates from various clients.

[0003] However, the implementation of federated learning faces high communication overhead, especially as model complexity and client scale increase simultaneously. Communication cost becomes a core bottleneck restricting system efficiency. The root of this problem lies in the fact that after each round of model training, the client needs to upload its local model parameters or gradient updates (the gradient is the partial derivative of the model error with respect to the parameters, a key signal driving model parameter optimization and causing the loss function to converge) to the server. For complex deep neural networks, which may contain millions or even hundreds of millions of parameters, uploading this data for each training round places a significant burden on network bandwidth and device resources. The problem manifests itself in the following aspects:

[0004] (1) Bandwidth bottleneck restricts training efficiency

[0005] Federated learning is typically deployed on distributed devices in different geographical locations, with significantly varying network environments. In areas with poor network conditions (such as remote mountainous areas or places with insufficient mobile network coverage), the transmission speed of model parameters is extremely slow, resulting in a significant increase in the time required for each round of communication, thus significantly extending the entire training process.

[0006] (2) Limited terminal equipment resources

[0007] Many client devices participating in federated learning (such as smartphones, IoT devices, and edge computing nodes) have limitations in computing power and battery life. Faced with the huge demand for model parameter transmission, these devices consume enormous resources, and frequent communication not only leads to a surge in energy consumption but may also affect the normal operation of the devices.

[0008] (3) Data privacy and security risks

[0009] Although federated learning avoids direct data transfer by training on local data, the large number of model parameters uploaded frequently still poses a risk of attack or leakage. Malicious attackers could analyze the transmitted model parameters to infer certain characteristics of the local data, thus threatening user privacy.

[0010] (4) Increased model complexity exacerbates communication burden

[0011] As deep learning models such as ResNet and CNN become increasingly complex, the number of parameters also increases exponentially. In this context, the amount of data transmitted in each round of communication in federated learning grows exponentially, leading to enormous bandwidth consumption and a heavy communication burden.

[0012] Therefore, it is very important to design a new communication mechanism that balances communication efficiency and model performance preservation. Summary of the Invention

[0013] This invention aims to overcome the problem that existing federated learning methods, when implemented in practice, struggle to balance compression ratio and model accuracy, especially when dealing with high-dimensional tensor parameters and heterogeneous device network environments, which can easily lead to a decrease in model convergence speed or a reduction in generalization ability. The invention provides a lightweight federated learning method and system based on parameter clustering and model decomposition that can balance communication efficiency and model performance.

[0014] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0015] A lightweight federated learning method based on parameter clustering and model decomposition includes the following steps:

[0016] S1 initializes the server model parameters and client model parameters;

[0017] S2 performs local training of the model on each client and saves the model increment for each training round;

[0018] S3 compresses the local model on each client and encodes the model increments. After encoding, each client uploads the encoded data to the server.

[0019] S4, the server decodes and aggregates all uploaded encoded information, and then re-encodes the aggregated information. After the encoding is completed, the server downloads the encoded information to all clients.

[0020] In S5, each client decodes the encoding sent down from the server and updates its own client model parameters based on the decoding result.

[0021] Preferably, step S1 includes the following steps:

[0022] S11, Initialize client model parameters W i Client-side model residual R i ;

[0023] S12, initialize server model parameters W and server model residuals R;

[0024] S13, set the learning rate h, batch size B, and training epochs T.

[0025] Preferably, step S2 includes the following steps:

[0026] For the local dataset D i For each batch B, the following steps are performed:

[0027] S21, the client model parameters W before training i Save to backup The specific formula is as follows:

[0028]

[0029] S22, using local dataset D i Train the client model and assign the trained client model parameters to W. i The specific formula is as follows:

[0030] Let W i =SGD(W i D i );

[0031] Wherein, SGD(·) represents local training;

[0032] S23, using the trained client model parameters W i and backup client model parameters Calculate the client-side model training increment ΔW i The specific formula is as follows:

[0033]

[0034] Preferably, step S3 includes the following steps:

[0035] S31, the client model residual R i Add client-side model training increment △W i In the middle, that is, △W i =△W i +R;

[0036] S32, Incremental ΔW for client-side model training i The singular value decomposition operation is performed using the following formula:

[0037] △W i SVD =SVD d (△W i );

[0038] Among them, △W i SVD Indicates the incremental parameters of the client model after singular value decomposition; SVD d (·) denotes singular value decomposition;

[0039] S33, Incremental parameter △W for the client model i SVD The sparse ternary quantization operation is performed using the following formula:

[0040]

[0041] in, STC(·) represents the incremental parameters of the client model after sparse ternary quantization and singular value decomposition; STC(·) represents the sparse ternary quantization operation.

[0042] S34, Calculate the residual R before and after sparse quantization. i SVD The specific formula is as follows:

[0043]

[0044] S35, for residual R i SVD Perform singular value aggregation and assign the aggregated residual parameters to R. i The specific formula is as follows:

[0045] R i =SVD a (R i SVD );

[0046] Among them, SVD a (·) indicates singular value aggregation;

[0047] S36, to The specific formula for Huffman coding is as follows:

[0048]

[0049] Where encode(·) represents the Huffman coding operation;

[0050] S37. After encoding is complete, the client uploads the encoded data to the server.

[0051] Preferably, step S4 includes the following steps:

[0052] S41, The server processes all uploaded encoded information (msg) i The decoding operation is performed using the following formula:

[0053]

[0054] Where decode(·) represents the Huffman decoding operation;

[0055] S42, for uploads from all clients. Perform singular value aggregation, calculate the weighted average, and finally add the server model residual R. The specific formula is as follows:

[0056]

[0057] Where N represents the number of clients; △W represents the incremental parameters of the server model;

[0058] S43, perform singular value decomposition on the incremental parameter ΔW of the server model, the specific formula is as follows:

[0059] △W SVD =SVD d (△W);

[0060] Among them, △W SVD This represents the incremental parameters of the server model after singular value decomposition;

[0061] S44, for △W SVD The sparse ternary quantization operation is performed using the following formula:

[0062]

[0063] in, This represents the incremental parameters of the server model after sparse ternary quantization and singular value decomposition.

[0064] S45, Calculate the residual R of the server model after singular value decomposition. SVD The specific formula is as follows:

[0065]

[0066] S46, for R SVD Perform SVD aggregation and assign the aggregated residual parameters to R, as shown in the following formula:

[0067] R = SVD a (R SVD );

[0068] S47, to The specific formula for Huffman coding is as follows:

[0069]

[0070] S48, after encoding is completed, the server will download the encoded data to all clients.

[0071] Preferably, step S5 includes the following steps:

[0072] S51 decodes the encoded message sent from the server. The specific formula is as follows:

[0073]

[0074] S52, after decoding in singular value aggregation step S51 The aggregated model increment parameters are assigned to △W i The specific formula is as follows:

[0075]

[0076] S53, use the incremental parameters △W of the client model obtained in step S52. i Updating backup The updated model parameter values ​​are then assigned to W. i The specific formula is as follows:

[0077]

[0078] This invention also provides a lightweight federated learning system based on parameter clustering and model decomposition, including:

[0079] The server is used to aggregate the model increments uploaded by the clients and then send them back to the client models;

[0080] The client device is a distributed node where the data resides, responsible for training on the local dataset;

[0081] The initialization module is used to initialize the server model parameters and the client model parameters;

[0082] The local training module is used to train the model locally on each client and save the model increment for each training round.

[0083] The local model compression module is used to compress the local model of each client and encode the model increment. After the encoding is completed, each client uploads the encoding to the server.

[0084] The aggregation module enables the server to decode and aggregate all uploaded encoded information, and then re-encode the aggregated information. After the encoding is completed, the server downloads the encoded information to all clients.

[0085] The local model update module enables each client to decode the encoding sent down from the server and update its own client model parameters based on the decoding result.

[0086] Compared with the prior art, the beneficial effects of this invention are: (1) Cooperative compression mechanism of parameter clustering and model decomposition: This invention combines sparse ternary quantization with singular value decomposition for the first time to form a complete compression and transmission mechanism: After the model training is completed, the client first calculates the model increment based on the changes in model parameters before and after training; then performs singular value decomposition on the model increment, thereby reducing the number of parameters to be transmitted; finally, the decomposed model increment is clustered with parameters and used in conjunction with Huffman coding, so that the parameters that originally required 32 bits to be transmitted can be reduced to only one or two bits to be transmitted; this cooperative compression mechanism can reduce the communication transmission volume to 5 times that of the traditional federated average algorithm. (2) Singular Value Decomposition of High-Dimensional Tensors: In view of the limitation that traditional SVD is only applicable to two-dimensional matrices, a cross-dimensional reshaping decomposition method for high-dimensional tensors is proposed to achieve efficient compression of complex model structures (such as four-dimensional CNN convolution kernels); When the dimension of the tensor to be decomposed is four-dimensional, the four-dimensional tensor (such as convolutional layer parameters [out_channels, in_channels, kernel_height, kernel_width]) is first reshaped into multiple two-dimensional matrices, each matrix corresponding to a dimension of flattening; then SVD decomposition is performed on each two-dimensional matrix to obtain the left singular vector matrix (U). The singular value matrix (S) and the right singular vector matrix are generated. Next, the singular values ​​are pruned according to a given energy value to determine the number of singular values ​​to retain. Based on the retained number, the left singular vector matrix (U) of each dimension is pruned, resulting in the left singular vector matrix (U_o) for the output channel dimension, the left singular vector matrix (U_i) for the input channel dimension, the left singular vector matrix (U_h) for the height dimension, and the left singular vector matrix (U_w) for the width dimension. Finally, the core tensor is obtained by multiplying the original four-dimensional tensor by the left singular vector of each dimension. Thus, a four-dimensional tensor is decomposed into U_o, U_i, U_h, U_w, and cor. Five small models of e_tensor; (3) Accuracy compensation mechanism based on model residuals: In order to further improve the accuracy of the model, this invention designs a residual compensation mechanism for sparse ternary quantization model after singular value decomposition; after each sparse ternary quantization, the residuals before and after quantization are calculated, and singular value aggregation is performed on the residuals to compensate for the information lost due to parameter clustering; this residual compensation mechanism can compensate for the loss of accuracy of the model caused by sparse ternary quantization while reducing the amount of communication; for example, in practical applications, even when the amount of communication is greatly reduced, the test accuracy of the model can be converged to a level comparable to that of the uncompressed model through the residual compensation mechanism. Attached Figure Description

[0087] Figure 1This is a flowchart of a lightweight federated learning method based on parameter clustering and model decomposition according to the present invention.

[0088] Figure 2 This is a comparative diagram showing the accuracy versus number of iterations for three algorithms (FedAvg, STC, and STVD).

[0089] Figure 3 This is a comparative diagram showing the cumulative uploaded data volume versus the number of iterations for three algorithms (FedAvg, STC, and STVD).

[0090] Figure 4 This is a comparative diagram showing the cumulative downloaded data volume versus the number of iterations for three algorithms (FedAvg, STC, and STVD). Detailed Implementation

[0091] To more clearly illustrate the embodiments of the present invention, specific implementation methods will be described below with reference to the accompanying drawings. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative effort.

[0092] like Figure 1 As shown, this invention provides a lightweight federated learning method based on parameter clustering and model decomposition, comprising the following steps:

[0093] S1 initializes the server model parameters and client model parameters;

[0094] S2 performs local training of the model on each client and saves the model increment for each training round;

[0095] S3 compresses the local model on each client and encodes the model increments. After encoding, each client uploads the encoded data to the server.

[0096] S4, the server decodes and aggregates all uploaded encoded information, and then re-encodes the aggregated information. After the encoding is completed, the server downloads the encoded information to all clients.

[0097] In S5, each client decodes the encoding sent down from the server and updates its own client model parameters based on the decoding result.

[0098] Specifically, step S1 includes the following steps:

[0099] S11, Initialize client model parameters W i Client-side model residual R i ;

[0100] S12, initialize server model parameters W and server model residuals R;

[0101] S13, set the learning rate h, batch size B, and training epochs T.

[0102] Specifically, step S2 includes the following steps:

[0103] For the local dataset D i For each batch B, the following steps are performed:

[0104] S21, the client model parameters W before training i Save to backup Right now

[0105] S22, using local dataset D i Train the client model, let Wherein, SGD(·) represents local training;

[0106] S23, using the trained client model parameters W i and backup client model parameters Calculate the client-side model training increment ΔW i The specific formula is as follows:

[0107]

[0108] Specifically, step S3 includes the following steps:

[0109] S31, the client model residual R i Add client-side model training increment △W i In the middle, that is, △W i =△W i +R;

[0110] S32, Incremental ΔW for client-side model training i The singular value decomposition operation is performed using the following formula:

[0111] △W i SVD =SVD d (△W i );

[0112] Among them, △W i SVD Indicates the incremental parameters of the client model after singular value decomposition; SVD d (·) denotes singular value decomposition;

[0113] S33, Incremental parameter △W for the client modeli SVD The sparse ternary quantization operation is performed using the following formula:

[0114]

[0115] in, STC(·) represents the incremental parameters of the client model after sparse ternary quantization and singular value decomposition; STC(·) represents the sparse ternary quantization operation.

[0116] S34, Calculate the residual R before and after sparse quantization. i SVD The specific formula is as follows:

[0117]

[0118] S35, for residual R i Perform singular value aggregation and assign the aggregated residual parameters to R. i The specific formula is as follows:

[0119] R i =SVD a (R i SVD );

[0120] Among them, SVD a (·) indicates singular value aggregation;

[0121] S36, to The specific formula for Huffman coding is as follows:

[0122]

[0123] Where encode(·) represents the Huffman coding operation;

[0124] S37. After encoding is complete, the client uploads the encoded data to the server.

[0125] Specifically, step S4 includes the following steps:

[0126] S41, The server processes all uploaded encoded information (msg) i The decoding operation is performed using the following formula:

[0127]

[0128] S42, for uploads from all clients. Perform singular value aggregation, calculate the weighted average, and finally add the server model residual R. The specific formula is as follows:

[0129]

[0130] Where N represents the number of clients; △W represents the incremental parameters of the server model;

[0131] S43, perform singular value decomposition on the incremental parameter ΔW of the server model, the specific formula is as follows:

[0132] △W SVD =SVD d (△W);

[0133] Among them, △W SVD This represents the incremental parameters of the server model after singular value decomposition;

[0134] S44, for △W SVD The sparse ternary quantization operation is performed using the following formula:

[0135]

[0136] in, This represents the incremental parameters of the server model after sparse ternary quantization and singular value decomposition.

[0137] S45, Calculate the residual R of the server model after singular value decomposition. SVD The specific formula is as follows:

[0138]

[0139] S46, for R SVD Perform SVD aggregation and assign the aggregated residual parameters to R, as shown in the following formula:

[0140] R = SVD a (R SVD );

[0141] S47, to The specific formula for Huffman coding is as follows:

[0142]

[0143] S48, after encoding is completed, the server will download the encoded data to all clients.

[0144] Specifically, step S5 includes the following steps:

[0145] S51 decodes the encoded message sent from the server. The specific formula is as follows:

[0146]

[0147] S52, after decoding in singular value aggregation step S51 The aggregated model increment parameters are assigned to △W i The specific formula is as follows:

[0148]

[0149] S53, use the incremental parameters △W of the client model obtained in step S52. i Updating backup The updated model parameter values ​​are then assigned to W. i The specific formula is as follows:

[0150]

[0151] This invention also provides a lightweight federated learning system based on parameter clustering and model decomposition, including:

[0152] The server is used to aggregate the model increments uploaded by the clients and then send them back to the client models;

[0153] The client device is a distributed node where the data resides, responsible for training on the local dataset;

[0154] The initialization module is used to initialize the server model parameters and the client model parameters;

[0155] The local training module is used to train the model locally on each client and save the model increment for each training round.

[0156] The local model compression module is used to compress the local model of each client and encode the model increment. After the encoding is completed, each client uploads the encoding to the server.

[0157] The aggregation module enables the server to decode and aggregate all uploaded encoded information, and then re-encode the aggregated information. After the encoding is completed, the server downloads the encoded information to all clients.

[0158] The local model update module enables each client to decode the encoding sent down from the server and update its own client model parameters based on the decoding result.

[0159] To verify the advantages of this invention in terms of communication volume, the communication scheme of this invention is compared with existing federated learning methods.

[0160] In the FedAvg algorithm (a traditional federated learning algorithm that aggregates model parameter updates from the client after each training round), the client uploads complete model parameters or gradient updates after each training round, and its communication overhead is linearly related to the size of the model parameters. Assuming each parameter requires q bits to be transmitted after being converted to binary encoding, and the number of parameters to be transmitted is N, the communication overhead (the sum of upload and download) for each training round is 2Nq. The STVD algorithm of this invention performs singular value decomposition, sparse ternary quantization, and Huffman coding on the model increments obtained from each round of local training, making the amount of data uploaded and downloaded in each round much smaller than the complete model parameters. Assuming the model parameters require q1 bits to be transmitted after sparse ternary quantization and the number of model parameters after singular value decomposition is N1, the communication overhead for each training round is 2N1q1.

[0161] Due to the combined effect of parameter clustering in sparse ternary quantization and Huffman coding, non-zero data that originally required 32 bits now only requires two bits, and zero data only requires one bit. With the total number of model parameters N remaining constant, this significantly reduces the number of bits q encoded for each parameter, making q1 smaller than q. Simultaneously, the inclusion of singular value decomposition transforms the data model to be transmitted from a large, complete model framework into 3-5 smaller models, resulting in a substantial reduction in the number of parameters N1 compared to the original N. Therefore, the communication overhead of the STVD algorithm is significantly lower than that of the traditional FedAvg and STC algorithms.

[0162] like Figures 2 to 4 As shown, when training the Cifar100 dataset using a DNN network, with the support of the STVD compression algorithm, after 20,000 training rounds, the cumulative transmission volume of the STVD compression algorithm is 3.96% of that of the FedAvg algorithm and 57.7% of that of the STC algorithm.

[0163] This invention provides an efficient communication mechanism for federated learning in distributed artificial intelligence, significantly reducing communication overhead while ensuring model performance, thus providing a solid technical foundation for the field of artificial intelligence. This not only promotes the development of intelligent and autonomous driving functions in electric vehicles but also provides strong technical support for the intelligent transformation of multiple industries such as finance, healthcare, and intelligent manufacturing.

[0164] The above description is merely a detailed explanation of preferred embodiments and principles of the present invention. For those skilled in the art, there may be changes in specific implementation methods based on the ideas provided by the present invention, and these changes should also be considered within the scope of protection of the present invention.

Claims

1. A lightweight federated learning method based on parameter clustering and model decomposition, characterized in that, Includes the following steps: S1 initializes the server model parameters and client model parameters; S2 performs local training of the model on each client and saves the model increment for each round of training; S3 compresses the local model on each client and encodes the model increments. After encoding, each client uploads the encoded data to the server. S4, the server decodes and aggregates all uploaded encoded information, and then re-encodes the aggregated information. After the encoding is completed, the server downloads the encoded information to all clients. In S5, each client decodes the encoding sent down from the server and updates its own client model parameters based on the decoding result.

2. The lightweight federated learning method based on parameter clustering and model decomposition according to claim 1, characterized in that, Step S1 includes the following steps: S11, Initialize client model parameters W i Client-side model residual R i ; S12, initialize server model parameters W and server model residuals R; S13, set the learning rate h, batch size B, and training epochs T.

3. The lightweight federated learning method based on parameter clustering and model decomposition according to claim 2, characterized in that, Step S2 includes the following steps: For the local dataset D i For each batch B, the following steps are performed: S21, the client model parameters W before training i Save to backup Right now S22, using local dataset D i Train the client-side model and assign the trained model parameters to W. i The specific formula is as follows: W i =SGD(W i ,D i ); Wherein, SGD(·) represents local training; S23, using the trained client model parameters W i and backup client model parameters Calculate the client-side model training increment ΔW i The specific formula is as follows:

4. The lightweight federated learning method based on parameter clustering and model decomposition according to claim 3, characterized in that, Step S3 includes the following steps: S31, the client model residual R i Add client-side model training increment △W i The specific formula is as follows: △W i =△W i +R i ; S32, Incremental ΔW for client-side model training i The singular value decomposition operation is performed using the following formula: in, Indicates the incremental parameters of the client model after singular value decomposition; SVD d (·) denotes singular value decomposition; S33, Incremental parameters for the client model The sparse ternary quantization operation is performed using the following formula: in, STC(·) represents the incremental parameters of the client model after sparse ternary quantization and singular value decomposition; STC(·) represents the sparse ternary quantization operation. S34, Calculate the residuals before and after sparse quantization. The specific formula is as follows: S35, for residuals Perform singular value aggregation and assign the aggregated residual parameters to R. i The specific formula is as follows: Among them, SVD a (·) indicates singular value aggregation; S36, to The specific formula for Huffman coding is as follows: Where encode(·) represents the Huffman coding operation; S37. After encoding is complete, the client uploads the encoded data to the server.

5. The lightweight federated learning method based on parameter clustering and model decomposition according to claim 4, characterized in that, Step S4 includes the following steps: S41, The server processes all uploaded encoded information (msg) i The decoding operation is performed using the following formula: S42, for uploads from all clients. Perform singular value aggregation, calculate the weighted average, and finally add the server model residual R. The specific formula is as follows: Where N represents the number of clients; △W represents the incremental parameters of the server model; S43, perform singular value decomposition on the incremental parameter ΔW of the server model, the specific formula is as follows: △W SVD =SVD d (△W); Among them, △W SVD This represents the incremental parameters of the server model after singular value decomposition; S44, for △W SVD The sparse ternary quantization operation is performed using the following formula: in, This represents the incremental parameters of the server model after sparse ternary quantization and singular value decomposition. S45, Calculate the residual R of the server model after singular value decomposition. SVD The specific formula is as follows: S46, for R SVD Perform SVD aggregation and assign the aggregated residual parameters to R, as shown in the following formula: R=SVD a (R SVD ); S47, to The specific formula for Huffman coding is as follows: S48, after encoding is completed, the server will download the encoded data to all clients.

6. The lightweight federated learning method based on parameter clustering and model decomposition according to claim 1, characterized in that, Step S5 includes the following steps: S51 decodes the encoded message sent from the server. The specific formula is as follows: S52, after decoding in singular value aggregation step S51 The aggregated model increment parameters are assigned to △W i The specific formula is as follows: S53, use the incremental parameters △W of the client model obtained in step S52. i Updating backup The updated model parameter values ​​are then assigned to W. i The specific formula is as follows:

7. A lightweight federated learning system based on parametric clustering and model decomposition, used to implement the lightweight federated learning method based on parametric clustering and model decomposition as described in any one of claims 1-6, characterized in that, The lightweight federated learning system based on parameter clustering and model decomposition includes: The server is used to aggregate the model increments uploaded by the clients and then send them back to the client models; The client device is a distributed node where the data resides, responsible for training on the local dataset; The initialization module is used to initialize the server model parameters and the client model parameters; The local training module is used to train the model locally on each client and save the model increment for each training round. The local model compression module is used to compress the local model of each client and encode the model increment. After the encoding is completed, each client uploads the encoding to the server. The aggregation module enables the server to decode and aggregate all uploaded encoded information, and then re-encode the aggregated information. After the encoding is completed, the server downloads the encoded information to all clients. The local model update module enables each client to decode the encoding sent down from the server and update its own client model parameters based on the decoding result.

Images