A federated continual learning method based on an improved unet network

Abstract

The application discloses a kind of federal continuous learning methods based on diffusion model, comprising the following steps: step 1, server distributes parameter to each client;Step 2, local real data is split into training set and validation set by client, and model parameter is updated;Step 3, if the current task is initial task, train local classifier, then go to step 6, if the current task is not initial task, client synthesizes historical data;Step 4, mix synthetic data with local real data;Step 5, train local classifier using cross-entropy loss and kl divergence;Step 6, save local classifier parameters according to validation set result;Step 7, train diffusion model;Step 8, client training is finished, and parameter is uploaded to server.Step 9, server first carries out average aggregation to model parameter.Again, all local synthetic samples are used to train the classifier.Step 10, training is finished.The method improves the accuracy of federal system and reduces the forgetting of previous knowledge.

Description

Technical Field

[0001] This invention relates to the field of federated learning, and specifically to a federated continuous learning method based on an improved UET network, with the goal of protecting data privacy and mitigating the loss of historical knowledge. Background Technology

[0002] In federated learning (FL), each client retains its data locally and only transmits training updates to a central server for aggregation. Traditional FL assumes that the data held by the clients is static, but in reality, new data categories may emerge on the client's end over time. For example, in mobile edge computing (MEC) networks, edge devices (UEs) continuously generate datasets, leading to new data categories.

[0003] Based on this scenario, the concept of continuous learning was introduced into the federated framework. Federated Continuous Learning (FCL) allows local clients to continuously collect new data and dynamically add new classes as needed. However, due to data privacy, limited local client storage, and task differences, past network knowledge will be forgotten. Furthermore, the imbalanced data distribution among clients in FL exacerbates the catastrophic forgetting problem. Clearly, this forgetting hinders the progress of FL. Therefore, how to enable devices to participate in federated learning in new environments while retaining old knowledge is a problem that needs further research.

[0004] Currently, federated learning primarily addresses these challenges by utilizing additional datasets or previous task data. However, this is difficult to achieve in certain data-sensitive scenarios and fails to capture the full complexity of the original data. Consider using a generator to synthesize historical datasets, aiming to simulate the global distribution of data across each client. However, when the model is trained on a series of lengthy tasks, there is a potential risk of deteriorating generated data quality. Maximizing the maintenance and preservation of representations of old task data in the current task becomes a crucial issue. Therefore, a research method that can both guarantee the image quality of the synthesized historical dataset and efficiently aggregate client knowledge is of significant research importance. Summary of the Invention

[0005] The purpose of this invention is to effectively mitigate the catastrophic forgetting problem in federated continuous learning (FCL) in data privacy-sensitive scenarios without storing the client's local private data or any additional datasets. It proposes a federated continuous learning method that utilizes a diffusion model as the generative model. Based on the characteristics of the UET network, a new "classifier" module is added, giving the UET additional functionality for label classification. Therefore, other classifiers, such as AlexNet, are not needed, reducing model size and optimizing server memory resources.

[0006] This invention provides a federated continuous learning method based on an improved UET model, the specific training method including the following steps:

[0007] Step 1: The server distributes all parameters of the UET network to each client.

[0008] Step 2: The client divides the local real data into training and validation sets according to a certain ratio, and updates the UET network parameters, thus starting local training;

[0009] Step 3: If the current task is the initial task, freeze the parameters of the UET network "decoder" module, train the local classifier using cross-entropy loss based on the training set containing only local real data, especially the "classifier" parameters of UET, and then proceed to step 6.

[0010] If the current task is not the initial task, the client freezes the parameters of the UET "classifier" module, uses a diffusion model based on UET network denoising as the generator, and synthesizes samples using the labels of local historical tasks under the guidance of the gradient of the UET classifier of the previous task in the local area.

[0011] Step 4: Split the synthetic samples into a training set and a validation set, and add them to the local training set. The local training set is updated from containing only real data to a mixture of real data and synthetic samples. The validation set is updated in the same way.

[0012] Step 5: Freeze the parameters of the "decoder" module of the UET network. First, train the local classifier using cross-entropy loss based on the local training set containing local real data and synthetic samples. Then, train the local classifier using KL divergence based on the local synthetic samples and the server model as the teacher model.

[0013] Step 6: Test with the local validation set and save the optimal local classifier parameters;

[0014] Step 7: Locally train the generator. During the initial communication, freeze the "classifier" module of UET and train the diffusion model using local real data, especially the "decoder" parameters of UET. If this is not the initial task, also use synthetic samples for training;

[0015] Step 8: After the client completes this round of training, it uploads all parameters of the UET network, including classifier and generator parameters, to the server. If this is not the initial task, it also uploads the locally synthesized samples obtained in Step 3.

[0016] Step 9: The server performs a global average aggregation of all parameters of the UET model. If this is not the initial task, all the obtained local synthetic samples are used to train the classifier parameters using the cross-entropy loss function.

[0017] Step 10: Federated training ends, and the final prediction model is output.

[0018] Preferably, the server distributes the parameters of the global model to the client.

[0019] Preferably, the backbone network of the global model shown is a UET network, with a classifier label output layer added after the intermediate layers of the UET network. Currently, the UET network is responsible for predicting noise during the diffusion model sampling process. This invention, based on the characteristics of the UET network, updates the model structure and gives the UET additional functionality for label classification. Therefore, other classification models, such as AlexNet, are not needed, reducing the model size and optimizing server memory resources. The designed new UET model satisfies the needs of both noise prediction and label classification, reducing the number of parameters that need to be trained, thereby reducing the model's complexity and memory requirements.

[0020] Preferably, the classifier label output layer includes a normalization layer, a SiLU activation function, an AttentionPool2d layer, and a fully connected layer connected in sequence.

[0021] Preferably, the training method for the local classifier in step 3 is as follows:

[0022]

[0023] Where, ω i These are the model parameters updated in the i-th client round. It is the client i's local real data. It is the actual label category corresponding to the data. The model predicts the label category, and the loss is calculated using the cross-entropy formula.

[0024] Preferably, the training method for the local classifier in step 5 is as follows: first, the local classifier is trained on real data and synthetic samples using formula (2) to learn new knowledge; then, local synthetic samples and formula (3) are used to limit the local model from deviating from the initial global model, so as to better utilize global knowledge.

[0025]

[0026] Where, x i The client i's local training set data includes real data and synthetic samples; formula (3) is the KL divergence. It is a locally synthesized sample from client i. It is the predicted output category of the global model from the previous round of communication aggregation for the current client's local synthetic sample.

[0027] Preferably, in step 6, the step of training the diffusion model is as follows:

[0028] ① Add noise to the original image x0 at a given time step l to generate xl .

[0029] ② Calculate the loss: Using the current UET network parameters to predict the noise, calculate the loss between the predicted noise and the actual added noise, i.e. Where, ∈ θ These are the parameters of the UET network excluding the "classifier" module; x l y is the image after adding noise at step l, and y is the category label.

[0030] ③ Backpropagation and parameter update: The gradient is calculated through backpropagation, and the network parameters are updated using the optimizer.

[0031] ④ Repeat steps 1-3 above until the predetermined number of training rounds is reached. The goal is to optimize the UET network's ability to predict noise and learn richer feature representations of the image.

[0032] Preferably, in step 9, the global average aggregation is performed according to formula (4), and the classifier parameters are trained using all local synthetic samples and the cross-entropy loss function according to formula (5):

[0033]

[0034] In formula (4), ω g It is a global classifier model. is the "classifier" parameter for client i at task t; ∈ is the diffusion model, specifically the parameters of the UET network excluding the "classifier" module. t,i It is the actual number of data entries for client i at task t. It is the actual amount of data from all clients at task t; in formula (5), CE is the cross-entropy loss. This represents all locally synthesized data.

[0035] Preferably, steps 2-9 constitute a federated training iteration cycle.

[0036] As a preferred option, the number of federated training iteration cycles is the number of communication rounds set for federated learning.

[0037] Preferably, the goal of the method is to minimize the global loss function:

[0038]

[0039] in, Let represent the loss of the "classifier" model of client i over all task data, with the goal of maximizing the model performance of each client across all tasks.

[0040] Furthermore, the method described above designs a new UET model that meets the needs of predicting noise and label classification, effectively reducing the model size.

[0041] The beneficial effects of this invention are:

[0042] 1. Without violating data privacy and security, use the diffusion model as the generation and replay model for historical data to improve the quality of generated images and reduce the risk of generator mode crashes.

[0043] 2. By performing image generation and classification simultaneously through the updated UET network, the computational and resource overhead in the training and inference processes is simplified. Combining these two tasks also allows the model to learn more comprehensive and richer feature representations.

[0044] 3. On the server side of federated learning, using small batches of synthetic samples to train classifiers and generators can more effectively learn global knowledge and reduce the risk of forgetting. Attached Figure Description

[0045] Figure 1 This is a system architecture diagram of the federated continuous learning method based on the improved UET network proposed in this invention.

[0046] Figure 2 This is a schematic diagram of an improved UET network.

[0047] Figure 3 This is a flowchart of the training process for the FedDiffusion method. Detailed Implementation

[0048] The following examples are provided for reference. Figure 1 , Figure 1 This presents the system architecture for the federated continuous learning method based on an improved UET network proposed in this invention. In this embodiment, the public image dataset (Cifar-10) can be divided into different numbers of tasks and different numbers of clients.

[0049] The original UET network, combined with a diffusion model, predicts noise and consists of three parts: the first part is the encoding module, which mainly consists of convolutional layers, temporal embedding layers, skip connection layers, group normalization layers, downsampling layers, etc.; the second part is the intermediate layer, which extracts higher-level features; and the third part is the decoding module, which consists of upsampling layers and a predicted noise output layer.

[0050] Based on the above network structure, this invention updates the UET network as a global model and adds label classification functionality. (See attached document.) Figure 2The encoding and intermediate layers share two functions. Afterwards, the output can be split, either into a classifier label output layer or a decoding model from an existing network. These two directions handle different functions and produce different outputs. The specific modifications to the classifier module are as follows: First, the input features are normalized, and the number of channels is set to 512; then, the SiLU activation function is applied to increase non-linearity; next, attention pooling layers are used for spatial dimension attention calculation and feature aggregation; finally, the fully connected layer receives the features output from the attention mechanism and transforms them into the desired output dimension.

[0051] Taking CIFAR-10 data as an example, in the attention pooling layer, the mean of the input tensor is first concatenated along the last dimension; then, a position embedding of shape (512, 65) is added to the tensor to capture spatial location information; next, the tensor is projected onto the representation of query, key, and value through a convolutional layer with 1 input channel, 1536 output channels, and a kernel size of 1; subsequently, a multi-head attention mechanism is applied for feature extraction, with 8 attention heads; finally, the output is projected through another convolutional layer with 1 input channel, 100 output channels, and a kernel size of 1.

[0052] This modification allows image generation and image classification tasks to share low-level feature representations, enabling the model to learn more comprehensive and richer feature representations and reducing the risk of overfitting. Furthermore, the UET network depth remains consistently four layers, and the model structure is identical for both the client and server.

[0053] The specific pseudocode implementation steps for FedDiffusion are as follows:

[0054]

[0055]

[0056] The present invention will be further described below, with additional examples provided.

[0057] Example 1

[0058] This embodiment provides a federated continuous learning method based on an improved UET network, including the following steps:

[0059] Step 1. Applying this method during the initialization phase, the server distributes the parameters of the global model to each client.

[0060] It should be noted that the global model and local model mentioned in this embodiment are the models initialized by the parameter server for federated learning.

[0061] Step 2. The client first divides the local real data into training and validation sets according to a certain ratio, and then uses the global model to directly update and overwrite the local classifier and generator parameters, thus starting local training.

[0062] Step 3: If the current task is the initial task, freeze the parameters of the "decoder" module of the UET network, and train the local classifier based on the local real data according to formula (1), especially the "classifier" parameters of UET.

[0063]

[0064] Where, ω i These are the model parameters updated in the i-th client round. It is the client i's local real data. It is the actual label category corresponding to the data. The model predicts the label category, and the loss is calculated using the cross-entropy formula.

[0065] Step 4. Based on the test results of the validation set, save the "classifier" parameter of the local UET with the highest accuracy.

[0066] Step 5. During the initial communication, freeze the "classifier" module of UET and train the diffusion model using a local training set containing only real data. The specific training steps of the DiffusionTrain algorithm are as follows:

[0067] ① Add noise to the original image x0 at a given time step l to generate x l ;

[0068] ② Calculate the loss: Using the current UET network parameters to predict the noise, calculate the loss between the predicted noise and the actual added noise, i.e. Where, ∈ θ These are the parameters of the UET network excluding the classifier module; x l is the image after adding noise at step l, and y is the category label;

[0069] ③ Backpropagation and parameter update: The gradient is calculated through backpropagation, and the network parameters are updated using the optimizer;

[0070] ④ Repeat steps 1-3 above until the predetermined number of training rounds is reached. The goal is to optimize the UET network's ability to predict noise.

[0071] Step 6. After the client finishes this round of training, it uploads all parameters of the UET network, including classifier parameters and generator parameters, to the server.

[0072] Step 7. The server first performs a global average aggregation of all parameters of the UET model, referring to formula (4):

[0073]

[0074] In Formula 3, ω g It is a global classifier model. This is the "classifier" parameter for client i at task t. t,i This refers to the amount of data on client i during task t. This represents the total amount of data across all clients at time t.

[0075] Step 8. This example is the initial task training steps. A task has one or more communication attempts; steps 2-7 constitute one communication cycle.

[0076] Example 2

[0077] The difference between this embodiment and Embodiment 1 is that, in this embodiment,

[0078] Step 3: If the current task is not the initial task, first use the diffusion model to synthesize historical data. Use the previously obtained local labels to synthesize a fixed number of samples for each category condition.

[0079] First, sample an initial noisy image x from a standard normal distribution. L For each time step t from T to 1, the trained UET network is used to predict the noise in the current image, and the denoised image x is calculated based on the predicted noise. t At time step t=0, the generated image x0 is the final output image. The specific pseudocode implementation steps for image generation are as follows:

[0080]

[0081]

[0082] Step 4. Split the synthetic samples into a training set and a validation set, and insert them into the local training set. The local training set is updated from containing only real data to a mixture of real data and synthetic samples. The validation set is processed in the same way.

[0083] Step 5. Freeze the parameters of the "decoder" module of the UET network and cancel the gradient update of the "decoder". First, based on local real data and synthetic samples, use cross-entropy loss and Equation (2) to train the "classifier" parameters of UET to learn new knowledge; then, based on local synthetic samples, use the server model as the teacher model, use KL divergence and Equation (3) to train the "classifier" parameters of UET. This step trains the parameters for one or more rounds. The training method is as follows:

[0084]

[0085] Where, x i The client i's local training set data includes real data and synthetic samples; formula (3) is the KL divergence. It is a locally synthesized sample from client i. It is the predicted output category of the global model from the previous round of communication aggregation for the current client's local synthetic sample.

[0086] Step 6. Based on the test results of the validation set, save the "classifier" parameter of the local UET with the highest accuracy.

[0087] Step 7. During the initial communication, freeze the "classifier" module of UET and train the diffusion model using a local training set containing both local real data and synthetic data. The specific training steps of the DiffusionTrain algorithm are as follows:

[0088] ① Add noise to the original image x0 at a given time step l to generate x l ;

[0089] ② Calculate the loss: Using the current UET network parameters to predict the noise, calculate the loss between the predicted noise and the actual added noise, i.e. Where, ∈ θ These are the parameters of the UET network excluding the classifier module; x l is the image after adding noise at step l, and y is the category label;

[0090] ③ Backpropagation and parameter update: The gradient is calculated through backpropagation, and the network parameters are updated using the optimizer;

[0091] ④ Repeat steps 1-3 above until the predetermined number of training rounds is reached. The goal is to optimize the UET network's ability to predict noise.

[0092] Step 8. After the client finishes training this round, it uploads all parameters of the UET network, including classifier parameters and generator parameters, to the server, and also uploads the local synthetic samples obtained in Step 3.

[0093] Step 9. The server first performs global average aggregation of all parameters of the UET model, referring to formula (4); if the current task is not the initial task, then use all the obtained local synthetic samples to train the classifier parameters using formula (5).

[0094]

[0095] In Formula 4, ω g It is a global classifier model. This is the "classifier" parameter for client i at task t. t,i This refers to the amount of data on client i during task t. It represents the total amount of data from all clients at task t; in Equation 5, CE is the cross-entropy loss. This represents all locally synthesized data.

[0096] Step 10. This example has at least two tasks. The client's local data is completely overwritten sequentially when a new task arrives. Each task has one or more communication attempts; steps 2-9 constitute one communication cycle.

[0097] The effectiveness of this method is primarily evaluated by the average accuracy (ACC) of all task data in the model, with the goal of achieving high accuracy.

[0098] The calculation formula is shown below.

[0099]

[0100] Among them, a i,T It is the average model accuracy of the global model after training on task T, based on all client i test data under the current task.

[0101] The above embodiments mainly illustrate how the model can quickly learn global knowledge while retaining old knowledge to achieve equilibrium when a new task arrives. The example of this invention minimizes the loss function using the above scheme, and its expression is: in, Let represent the loss of the "classifier" model of client i over all task data, with the goal of maximizing the model performance of each client across all tasks.

[0102] In this example, there are three requirements for building tasks for the client. First, for the client, if a class already exists in its historical tasks, the client will not pick up that class again. This ensures that for any client, a class will not appear more than once in its private sequence, depending on the definition of incremental class learning. Second, the client data distribution is different, allocated using a Dirichlet distribution, with a non-independent identically distributed (Non-IID) label distribution. Third, the client data will be split into training and validation sets in an 8:2 ratio.

[0103] The effects of the present invention will be further explained below with reference to simulation experiments.

[0104] 1. Experimental setup

[0105] The dataset used in the simulation experiments of this invention is CIFAR-10, a color image dataset that more closely approximates universal objects. Each image is 32x32 pixels in size, with 6000 images per category, totaling 50,000 training images and 10,000 test images. The 10 categories are randomly divided into 5 tasks, each containing 2 categories, 10,000 training images, and 2000 test images. The experiment simulates 5 local clients holding privacy data, collaboratively training a federated learning global model in response to 5 different tasks arriving sequentially. In each task, images are assigned to clients according to a Dirichlet distribution (α = 0.5). Each client synthesizes 128 samples for each historical category. The client trains locally 15 times, communicates 30 times, and the diffusion model is trained 15,000 times, with a step count of 4000.

[0106] 2. Results Analysis

[0107] The simulation experiment of this invention employs this invention and two existing related technologies (FedLwF-2T training method and FedCIL training method). The two existing technologies employed are:

[0108] The existing FedLwF-2T training method refers to the federated continuous learning and non-forgetting training method proposed by Usmanova et al. in "Adistillation-based approach integrating continual learning and federated learning for pervasive services, 2021", abbreviated as FedLwF-2T training method.

[0109] The existing FedCIL training method refers to the federated learning generative replay method proposed by Daiqing Qi et al. in "Better generative replay for continual federated learning, 2023", abbreviated as FedCIL training method.

[0110] To verify the effectiveness of the simulation experiments of this invention, the average accuracy of the model on the current task and previous tasks after completing all tasks (i.e., after learning the fifth task, the average accuracy of the model on the test sets of the first to fifth tasks) was measured as the evaluation metric to evaluate the training results of the federated continuous learning for the three methods. All results are plotted in the following table:

[0111] FedLwF-2T 39.46 FedCIL 47.68 Ours 51.84

[0112] As shown in the table above, the average classification accuracy of the federated learning global network across all tasks in this invention remains at 51.84 after training on five tasks, which is higher than the other two existing methods. This demonstrates that this invention, by using an improved method based on a UET network, prevents the federated learning global network from forgetting knowledge from old tasks during the task flow. This invention combines the characteristics of a diffusion model and continuous learning, ensuring stable image quality and reducing the client's forgetting of knowledge from old tasks.

[0113] The simulation experiments above show that the method of the present invention uses an improved UET network for federated continuous learning training, which solves the problems of catastrophic forgetting of old task knowledge, sensitivity to real data, and generator pattern collapse in existing methods. It is an effective federated continuous learning training method.

[0114] The above describes the implementation process of this invention. Any changes made in accordance with the technology of this invention that do not exceed the scope of the technical solution of this invention shall fall within the protection scope of this invention.

Claims

1. A federated continuous learning method based on an improved UET network, characterized in that, The method includes the following steps: Step 1: The server distributes all parameters of the UET network to each client, updates the UET network as a global model, and adds a classifier label output layer after the intermediate layer of the UET network. The output of the intermediate layer is split between the classifier label output layer and the existing decoder of the original UET network. Step 2: The client divides the local real data into training and validation sets according to a certain ratio, and updates the UET network parameters, thus starting local training; Step 3: If the current task is the initial task, freeze the decoder module parameters in the UET network, train the local classifier in the UET network using cross-entropy loss based on the training set containing only local real data, and then proceed to step 6. If the current task is not the initial task, the client freezes the parameters of the classifier module in the UET network, uses the diffusion model based on the UET network for denoising as the generator, uses the labels of the local historical tasks, and synthesizes samples under the guidance of the gradient of the UET classifier of the previous task in the local area, and continues to step 4. Step 4: Divide the synthetic samples into a training set and a validation set, and put them into the local training set. The local training set is updated from containing real data to a mixture of real data and synthetic samples. The validation set is done in the same way. Step 5: Freeze the decoder module parameters in the UET network. First, train the local classifier using cross-entropy loss based on the local training set. Then, train the local classifier using KL divergence based on the local synthetic samples and the server model as the teacher model. Step 6: Test with the local validation set and save the optimal local classifier parameters; Step 7: Train the generator locally. During the first communication, freeze the classifier module in the UET network and train the decoder parameters in the diffusion model using local real data. If the current task is not the initial task, use synthetic samples for training. Step 8: The client ends the current round of training and uploads all parameters of the UET network, including classifier parameters and generator parameters, to the server. If the current task is not the initial task, the local synthetic sample obtained in step 3 is also uploaded. Step 9: The server performs global average aggregation of all parameters of the UET model. If the current task is not the initial task, the classifier parameters are trained using all the obtained local synthetic samples and the cross-entropy loss function. Step 10: Federated training ends, and the final prediction model is output.

2. The federated continuous learning method based on an improved UET network according to claim 1, characterized in that, The server distributes parameters of the global UET network model to the client.

3. The federated continuous learning method based on an improved UET network according to claim 2, characterized in that, The classifier label output layer includes a normalization layer, a SiLU activation function, an AttentionPool2d layer, and a fully connected layer connected in sequence.

4. The federated continuous learning method based on an improved UET network according to claim 1, characterized in that, The training method for the local classifier in step 3 is as follows: （1） in, It is a client Local real data, It is the actual label category corresponding to the data. The model predicts the label category, and the loss is calculated using the cross-entropy formula. When it is the t-th task, the client The classifier parameters.

5. A federated continuous learning method based on an improved UET network according to claim 1, characterized in that, The training method for the local classifier in step 5 is as follows: First, train the local classifier on real data and synthetic samples using formula (2) to learn new knowledge; then, on local synthetic samples, use formula (3) to limit the local model from deviating from the initial global model, so as to better utilize global knowledge: （2） （3） in, It is a client Local training set data, including real data and synthetic samples; formula (3) is the KL divergence, It is a client Locally synthesized samples, It is the predicted output category of the global model from the previous round of communication aggregation for the current locally synthesized sample on the client. It is a global classifier model. When it is the t-th task, the client The classifier parameters.

6. The federated continuous learning method based on an improved UET network according to claim 1, characterized in that, In step 7, the goal is to optimize the noise prediction capability of the UET network. The steps for training the diffusion model are as follows: (1) Given a time step l and the original image Add noise to generate ; (2) Calculate the loss: Use the current UET network parameters to predict the noise, and calculate the loss between the predicted noise and the actual added noise; In steps Image with added noise; (3) Backpropagation and parameter update: The gradient is calculated through backpropagation, and the network parameters are updated using the optimizer; (4) Repeat steps (1)-(3) above until the predetermined number of training rounds is reached.

7. A federated continuous learning method based on an improved UET network according to claim 1, characterized in that, In step 9, the global average aggregation is performed according to formula (4), and the classifier parameters are trained using all local synthetic samples and the cross-entropy loss function according to formula (5). , （4） （5） In formula (4), It is a global classifier model. It is a task Time Client The classifier parameters; It's a diffusion model, specifically the parameters of the UET network excluding the classifier module. It is a task Time Client The actual number of data, It is a task The actual amount of data from all clients at that time; in formula (5), CE is the cross-entropy loss, This represents all locally synthesized data. It is a client Locally synthesized samples.

8. A federated continuous learning method based on an improved UET network according to claim 1, characterized in that, Steps 2-9 constitute the federated training iteration cycle.

9. A federated continuous learning method based on an improved UET network according to claim 7, characterized in that, The number of federated training iteration cycles is the number of communication rounds set for federated learning.

Images