An emotion recognition method based on federated learning driven by cluster knowledge distillation

Abstract

The application discloses an emotion recognition method based on federated learning driven by clustering knowledge distillation. The application adopts a clustering knowledge distillation method to solve the problem of model heterogeneity, allowing each client to select a suitable training model according to the actual local situation. A multi-leader distributed consensus mechanism based on voting is used to realize the clustering process, so that the clustering clusters completed in each communication round are different. Introducing a random factor can better alleviate the problems of slow convergence speed and low robustness caused by data heterogeneity. In the clustering knowledge distillation process, firstly, the leader knowledge distillation in the cluster obtains the convergence direction of the entire cluster while accelerating the convergence speed of the model, and secondly, the follower knowledge distillation between clusters obtains the global convergence direction by fusing the convergence directions of different clusters, ensuring that the follower will not deviate from the global convergence direction due to data heterogeneity. The application has fast convergence speed, high recognition rate and strong robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of emotion recognition and distributed learning, and particularly relates to an emotion recognition method based on federated learning driven by clustering knowledge distillation. BACKGROUND

[0002] Emotion is a complex psychological state that is generated by the combination of subjective cognitive experience and is an important indicator of mental health. Therefore, recognizing emotion is crucial for monitoring and diagnosing mental health. With the gradual maturity of emotion recognition technology, it plays a key role in depression diagnosis, mental health assessment, and treatment of special populations. Emotion reflects the advanced activity of the brain, and the current data sources for emotion recognition can be divided into physiological sources and non-physiological sources. Although emotion recognition based on physiological sources such as facial expressions, speech, and text has been relatively mature, these data are prone to be subjectively manipulated, leading to inaccurate recognition. In contrast, physiological signals such as electroencephalogram (EEG) that reflect sympathetic nervous system activity are less susceptible to intentional control by subjects and are a low-cost, easy-to-collect objective physiological signal. In addition, such signals can help people with functional disorders to achieve emotion classification, so emotion recognition based on EEG has gained increasing attention.

[0003] The current frontier algorithm for emotion recognition is deep neural networks, and according to the widely recognized principle, a large-scale dataset is needed to build an effective and reliable emotion recognition model. However, transmitting raw patient data to a central server will face the problem of leaking patient data privacy and violating privacy laws, such as the General Data Protection Regulation (GDPR) passed by the European Union in May 2018 and the Data Security Law of the People's Republic of China passed in June 2021. In 2017, to solve the problem of privacy protection and data silos in devices, Google proposed federated learning (FL). As a powerful method for handling distributed data, federated learning does not require the transmission of data from each institution to the central server, but instead achieves collaborative training by aggregating the model parameters or gradients of each institution, thereby training a global model that performs well on most clients.

[0004] Most of the current research on federated learning is based on ideal assumptions, ignoring the basic challenges faced by federated learning in practical application scenarios: (1) Participating clients have highly heterogeneous local private data. For example: in the medical scenario of emotion recognition based on electroencephalogram (EEG), due to the lack of popularization of EEG collection and ethical issues, the EEG databases of various medical institutions often have problems such as small sample size, extremely unbalanced sample distribution, and serious cultural differences among subjects in different medical institutions. This will greatly reduce the convergence speed of each client model. (2) Participating clients have individualized local models. In traditional federated learning, a shared global model is deployed to all clients. However, in actual scenarios, the differences in computing resources, personal use, data distribution, and other aspects of each client force them to deploy heterogeneous models. SUMMARY

[0005] Therefore, the present application provides a federated learning emotion recognition method based on clustering knowledge distillation, which effectively solves the problem of heterogeneous data and models in the actual local medical institutions of EEG-based emotion recognition, and has fast convergence speed, high recognition rate, and strong robustness.

[0006] The federated learning emotion recognition method based on clustering knowledge distillation of the present application comprises:

[0007] Step 1, initialize the parameters of the model deployed by each client, and construct a federated learning model;

[0008] Step 2, train the constructed federated learning model, specifically:

[0009] Step 21, each client trains a local model using local data;

[0010] Step 22, use a voting-based multi-leader distributed consensus to cluster all clients, specifically:

[0011] S221, initialize all n clients as followers; randomly select c clients as candidates; the c is the number of clusters in the final clustering;

[0012] S222, each candidate broadcasts a voting request, which contains the output obtained by performing a forward propagation on the candidate's own model using a proxy dataset as input; the proxy dataset is a public dataset owned by each client;

[0013] S223, after receiving the voting request of the candidate, the follower calculates the similarity between the output obtained by performing a forward propagation on the local model using the proxy dataset and the output of the candidate, and if the similarity is greater than or equal to a set threshold, the local model gives the candidate a vote.

[0014] S224, when all followers complete voting on all candidates, count the total number of votes V obtained by the candidate, if the candidate becomes the leader; otherwise, the candidate becomes a follower, and re-executes S222-S224, if the number of leaders selected by voting is still less than c, return to step 21;

[0015] S225, according to the similarity of the output of the leader and the follower, assign followers to each leader: for each leader, select the first largest similarity as the follower belonging to the leader, thus all clients are divided into c clusters centered on the leader; Wherein a follower can belong to multiple leaders at the same time;

[0016] S226, every t communication round, select c clients with the least training times from n clients as leaders without voting, and assign followers to these leaders according to the size of the similarity, get c clusters;

[0017] Step 227, taking all followers in the cluster as the teacher network and the leader of the cluster as the student network, knowledge distillation is performed;

[0018] Step 228, taking all leaders to which the follower belongs as the teacher network and the follower as the student network, knowledge distillation is performed; Wherein, when aggregating the outputs of multiple leaders, an entropy reducer module is used, that is, the average output of the leader is passed through a softmax layer with parameter t, and the entropy of the fused output of the leaders is reduced by reducing the value of t;

[0019] Step 229, repeat steps 221-228 until each local model converges, and get trained local models;

[0020] Step 3, each client uses the local model trained in step 2 to perform emotion recognition.

[0021] Preferably, in S33, cosine similarity or Euclidean distance is used for similarity calculation.

[0022] Preferably, in the knowledge distillation process of step 227 and step 228, the loss function is the weighted sum of the cross-entropy loss between the output of the student network and the original label of the sample and the cross-entropy loss between the output of the student network and the output of the teacher network.

[0023] Preferably, in S223, the threshold is an adaptive threshold, that is: in each communication round, the average of the first largest similarity of all candidates is taken as the threshold of the similarity of the communication round.

[0024] Advantages:

[0025] (1) The traditional federated learning transmits model parameters or gradients between clients, and then aggregates through the server to achieve collaborative training. This method requires that the models of each client be the same, otherwise they cannot be aggregated. However, the federated learning framework proposed in the present application transmits the output of the model between clients. Since the nature of the model is a mapping from the input space to the output probability space, as long as the input labels are the same, the output can be aggregated, regardless of the model. In summary, the federated learning framework based on knowledge distillation proposed in the present application can achieve personalized deployment of the model.

[0026] (2) The traditional federated learning framework cannot converge to the global optimal solution in a data heterogeneous environment, because the local data heterogeneity causes the model trained by each client to deviate from the global optimal direction. However, the method proposed in the present application uses a clustering knowledge distillation approach. First, the leader within the cluster distills knowledge to enable the leader to obtain the convergence direction of the entire cluster while accelerating the convergence speed of the model. Second, the knowledge distillation of the followers between clusters obtains the global convergence direction by fusing the convergence directions of different clusters, ensuring that the followers will not deviate from the global convergence direction due to data heterogeneity.

[0027] (3) The drift management mechanism proposed in the present application forces clients with a drift tendency to participate in training by counting the number of times each client participates in training, ensuring that clients that are deviating from the global convergence direction can participate in training to correct their convergence direction with the knowledge of other clients, so that they gradually conform to the global convergence direction.

[0028] (4) The entropy reducer module proposed in the present application adjusts the entropy of the knowledge of the leaders aggregated during the model update of the followers by using a softmax function with a temperature coefficient, reducing it to provide appropriate regularization to improve the convergence speed of the model. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 A flowchart of a federated learning method based on clustering knowledge distillation applied to emotion recognition is provided for the embodiments of the present application;

[0030] Figure 2 The first two stages of a federated learning method based on clustering knowledge distillation applied to emotion recognition are provided for the embodiments of the present application, which are S1: self-training stage and S2: leader-follower election stage;

[0031] Figure 3The last two stages of the federated learning method based on cluster knowledge distillation applied to emotion recognition provided by the embodiment of the present application are S3: parallel intra-cluster knowledge distillation for leaders and S4: parallel inter-cluster knowledge distillation for followers.

[0032] Figure 4 The performance of the federated learning method based on cluster knowledge distillation applied to emotion recognition (FedKDC) provided by the embodiment of the present application and other advanced federated learning frameworks on SEED and SEED-IV datasets. (a), (b), and (c) are the performance of the SEED dataset under independent and identically distributed, alpha = 3, and alpha = 1 Dirichlet distribution, respectively; (d), (e), and (f) are the performance of the SEED-IV dataset under independent and identically distributed, alpha = 3, and alpha = 1 Dirichlet distribution.

[0033] Figure 5 The box plot of the federated learning method based on cluster knowledge distillation applied to emotion recognition (FedKDC) provided by the embodiment of the present application and other advanced federated learning frameworks in the cross-cultural experimental paradigm. DETAILED DESCRIPTION

[0034] The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0035] The present application provides an emotion recognition method based on federated learning driven by cluster knowledge distillation, which adopts a new personalized federated learning (Personalized FL, PFL) framework. To solve the problem of model heterogeneity, the present application adopts a cluster knowledge distillation method to realize personalized model deployment, allowing each client to select a suitable training model according to the actual local situation. In this method, the cluster knowledge distillation method only needs to transfer the probability distribution of the model output between clients. Since the essence of a classification model is a mapping from the input space to the output probability space, the outputs from different models can be directly aggregated under the condition that the input categories are the same. This scheme provides a guarantee for the deployment of personalized models. Generally speaking, the parameter quantity of the model output is much smaller than the parameter quantity of the model, so compared with the traditional federated learning that transfers model parameters, the federated learning using cluster knowledge distillation further reduces the communication cost and improves the scalability of the framework. In addition, compared with global knowledge distillation, intra-cluster knowledge distillation has higher convergence efficiency and robustness. In addition, the characteristic of knowledge distillation that needs to transfer the probability distribution of the model output further reduces the communication cost and improves the scalability of the PFL framework.

[0036] In the process of clustering knowledge distillation, the application adopts a multi-leader distributed consensus mechanism based on voting to realize the clustering process. The traditional distributed consensus mechanism "Raft" is used to ensure the consistency of data in a distributed system. In the distributed consensus algorithm, each client can switch between three states: 1) follower: the initial state of each client. 2) candidate: a random selection from the followers, trying to become a leader through voting. 3) leader: the candidate who gets the majority of the client votes can become the leader. Otherwise, the candidate is reselected. The multi-leader distributed consensus mechanism proposed by the application forms multiple clusters centered on the leader by electing multiple leaders and the followers corresponding to each leader, thereby providing prior conditions for subsequent clustering knowledge distillation. Moreover, the clustering clusters completed by the application in each communication round are different, so that the introduction of random factors can better alleviate the problems of slow convergence speed and low robustness caused by data heterogeneity.

[0037] To solve the client drift problem in collaborative training, the application proposes a "drift management" mechanism to eliminate outlier clients and solve the classic challenges in federated learning. In the non-independent and identically distributed (non-IID) data heterogeneous environment, the accuracy of traditional federated learning will decrease significantly due to client drift. Because in traditional federated learning, the local model of each client will converge to the local optimal value during training, and the global model will move towards the global optimal value after aggregating the local models. However, when the local data is non-independent and identically distributed, the convergence direction of each local model will deviate, so that the aggregated global model will deviate from the global optimal value. In the multi-leader distributed consensus mechanism based on voting proposed by the application, in the data heterogeneous environment, the models of some clients will deviate from the local optimal value and cannot reach consensus with other clients, so they cannot become leaders or followers and are excluded from the training process. The "drift management" mechanism will count the number of times each client participates in training, and after a fixed communication round, it will force the client with the least training times to participate in training. Through knowledge distillation, the convergence direction of the drifting client is corrected with the knowledge of other clients, thereby alleviating client drift.

[0038] Finally, an entropy reducer module is added in the process of clustering knowledge distillation to reduce the entropy of the aggregated output of the leaders. In the process of clustering knowledge distillation, the follower is the student network, and the leader to which it belongs is the teacher network. When the output of the leader is aggregated, the aggregated output will have high entropy because the leaders belong to different clusters. The probability distribution with too high entropy has too much redundant information and cannot effectively regularize the student network, so the high entropy of the aggregated leader knowledge in the follower client model update needs to be adjusted to reduce it to a proper regularization effect to improve the model convergence speed.

[0039] The method flowchart of the application is shown in Figure 1 and specifically includes the following steps:

[0040] S0, initialize the federated learning model: n clients are set in total, and each client (medical institution) deploys a personalized local model according to its own computing resources and data requirements. Each client stores a private local EEG dataset and a shared public EEG dataset (proxy dataset). The purpose of the application is to collaboratively train among multiple medical institutions through federated learning without sharing local datasets, thereby obtaining a local model that can accurately perform emotion recognition.

[0041] S1, self-training phase: each client trains the local model using local data.

[0042] As shown in the first stage shown in Figure 2 , like traditional federated learning, each client uses its own local EEG dataset and local model to perform self-training, and in this process, each client is completely independent and has no interaction. The number of training rounds is artificially set, generally 1-3 rounds.

[0043] S2, leader and follower election phase: use a voting-based multi-leader distributed consensus to cluster all clients, and finally form multiple clusters centered on leaders. S3 and S4 are both model update phases;

[0044] As shown in the leader and follower election phase in Figure 2 . In this phase, all n clients are first initialized as followers, and Figure 2In the second step, c clients are randomly selected as candidates, where c is determined by human, and the size of c is the number of clusters. At this time, all clients are divided into c candidates and n-c followers. In the third step, each candidate will broadcast a voting request containing the output of its own model after a forward propagation on the proxy dataset. The proxy dataset refers to the public dataset owned by each client. The followers receive the voting request and decide whether to vote according to the similarity between the output of the local model on the proxy dataset and the output of the candidate. The formula for calculating the similarity is Where s fi and s c1 are the output probability distribution of the model of the i-th follower on the proxy dataset and the output probability distribution of the first candidate on the proxy dataset. If the similarity is greater than or equal to the similarity threshold E, it is considered that the follower and the candidate have reached a consensus, and the follower i will vote for the candidate. On the contrary, if the similarity is less than E, the follower will vote against. In this way, until all followers vote for each candidate, the fourth step is to record the total number of votes each candidate gets as V, if the candidate has reached a consensus with enough followers, and the candidate will become a leader. Otherwise, if the candidate cannot reach a consensus with enough followers, it will become a follower again, and randomly select another candidate to become a follower, and repeat the previous voting process. Only one reselection is allowed in each communication round. If the second vote still fails, the system will return to the self-training phase.

[0045] When all c leaders are successfully elected, the system will assign followers to each leader according to the similarity between the output of the leader and the output of the follower on the proxy dataset. Specifically, all followers are sorted in descending order according to the similarity between the output on the proxy dataset and the output of the leader on the proxy dataset, and the first followers are selected as followers belonging to the leader. At this point, the invention divides all clients into c clusters centered on the leader according to the consensus. It is worth noting that the invention allows a follower to belong to multiple leaders at the same time. These followers that belong to multiple leaders at the same time will play a role in communicating knowledge in different clusters during subsequent model updates.

[0046] To solve the client drift phenomenon, during clustering, the invention proposes a drift management mechanism to correct part of the clients that are deviating from the global update direction. For example Figure 2The parameter t for drift management is recorded in the fifth step in the method. During the training process, the system will count the number of times each client participates in the training. Every t communication rounds, a direct election will be held. In the direct election, the invention selects c clients with the least training times from n clients as leaders without voting, and assigns followers to them according to the similarity, thereby completing clustering. The drift management mechanism ensures that the client deviating from the global convergence direction can participate in the training and correct his convergence direction with the knowledge of other clients, so that it gradually conforms to the global convergence direction.

[0047] S3, parallel intra-cluster knowledge distillation for all leaders: taking all followers in the cluster as teacher networks and the leaders as student networks to perform knowledge distillation;

[0048] In the S3 step, the invention performs parallel intra-cluster knowledge distillation for all leaders based on the clustering achieved in the S2 step. As shown in Figure 3 , Figure 3 (a) shows an abstract two-dimensional plane representation of the relationship between all clients after clustering is completed. In the figure, different colored circles represent leaders, and triangles represent followers. A total of 10 clients are set in the figure, and the number of clusters is set to 3, so there are 3 followers in each cluster. The situation of the followers can be mainly divided into three categories: 1) outlier followers, which cannot reach a consensus with other leaders due to low similarity, and therefore do not belong to any cluster. Such followers cannot participate in this round of model updating; 2) followers belonging to only one leader; 3) followers belonging to multiple leaders.

[0049] The step S3 is shown in the upper half of Figure 3 (b), which shows the process of intra-cluster knowledge distillation using leader 1 as an example. Here, leader 1 acts as the student network in knowledge distillation, and its parameters will participate in backpropagation. All followers belonging to leader 1 are considered as teacher networks, and their outputs after acting on the proxy dataset are aggregated by averaging and then participate in the calculation of the loss function. The model acting as a teacher network does not participate in backpropagation, so the teacher network will not update the parameters. The loss function here is divided into two parts: the first part is the cross-entropy loss between the student network and the original label of the sample, and the second part is the cross-entropy loss between the output of the student network and the output of the teacher network. The latter is used as a regularization term and is weighted and summed with the former to form the total loss function.

[0050] S4, parallel inter-cluster knowledge distillation for all followers: taking all leaders to which the followers belong as teacher networks and the followers as student networks to perform knowledge distillation. After the four steps are completed, it is considered that a communication round is completed, and the next communication round starts from S1 and is executed in a loop until each local model converges.

[0051] S4 describes the steps as Figure 3 The lower half of (b) shows the process of model updating of follower n. Here, follower n participates in the backpropagation as a student network to update the parameters. All leaders to which follower n belongs are teacher networks. They take the proxy dataset as input and get the output. The output is aggregated by averaging and passed to the entropy reducer. The entropy reducer is used to reduce the entropy of the aggregated output. The output aggregated in S3 comes from the same cluster, so their similarity is high. Therefore, the output aggregated by averaging will not have a high entropy problem. However, the output aggregated in S4 comes from different leaders, i.e. from different clusters. Therefore, it cannot be guaranteed that the entropy of the output is within a reasonable range. If the entropy of the output is too high, it will seriously affect the penalty effect of the regularization term. Therefore, the entropy reducer module is added when aggregating the output from different leaders in the present application.

[0052] S5, the federated learning framework trained by the above steps enables the personalized local model of each client to achieve high accuracy in the EEG-based emotion recognition task. In medical institutions with unbalanced EEG data distribution, the features extracted from the EEG data are input into the trained local model to output the emotion recognition classification of the subject.

[0053] To verify the performance of the federated learning framework proposed in the present application in a medical environment where the model data is heterogeneous, the present application deploys three different lightweight networks DNN, CNN1 and CNN2 to each client to simulate a model-heterogeneous environment. In addition, two experimental paradigms are designed to simulate different data-heterogeneous environments, namely the intra-cultural and cross-cultural experimental paradigms. In the intra-cultural experimental paradigm, the present application uses the EEG dataset SEED and SEED-IV. The subjects of these two datasets are Chinese. SEED is a three-classification, and SEED-IV is a four-classification. The present application assigns local data to each client according to the principle of subject independence, and adjusts the parameters of the Dirichlet distribution to change the unevenness of the data categories, thereby simulating a data-heterogeneous scenario. In the cross-cultural experimental paradigm, the present application uses the EEG dataset SEED-FRA, SEED-GER and SEED to simulate the influence of data heterogeneity generated by subjects of different cultures on classification accuracy. Among them, SEED-FRA subjects are French, SEED-GER subjects are German, and SEED subjects are Chinese. These three datasets are all three-classifications: positive, neutral and negative.

[0054] As Figure 4As shown is the performance of a federated learning method based on clustering knowledge distillation (FedKDC) provided by the application applied to emotion recognition and other advanced federated learning frameworks on SEED and SEED-IV datasets. (a), (b), (c) are the performances of the SEED dataset under independent and identically distributed, α = 3, and α = 1 Dirichlet distribution, respectively; (d), (e), (f) are the performances of the SEED-IV dataset under independent and identically distributed, α = 3, and α = 1 Dirichlet distribution, respectively. It can be seen that in the two datasets under different distribution conditions, the framework proposed by the application can achieve the highest average model performance (AMP).

[0055] As shown is the performance of a federated learning method based on clustering knowledge distillation (FedKDC) provided by the application applied to emotion recognition and other advanced federated learning frameworks on SEED and SEED-IV datasets. (a), (b), (c) are the performances of the SEED dataset under independent and identically distributed, α = 3, and α = 1 Dirichlet distribution, respectively; (d), (e), (f) are the performances of the SEED-IV dataset under independent and identically distributed, α = 3, and α = 1 Dirichlet distribution, respectively. It can be seen that in the two datasets under different distribution conditions, the framework proposed by the application can achieve the highest average model performance (AMP). Figure 5 As shown is the box plot of the federated learning method based on clustering knowledge distillation (FedKDC) provided by the embodiment of the application applied to emotion recognition and other advanced federated learning frameworks in a cross-cultural experimental paradigm. Three cases are set here, which are 1) FG-C: the data of French and German subjects as the training set, and the data of Chinese subjects as the test set; 2) CG-F: the data of Chinese and German subjects as the training set, and the data of French subjects as the test set; 3) CF-G: the data of Chinese and French subjects as the training set, and the data of German subjects as the test set. It can be seen in the figure that in the cross-cultural setting, the model proposed by the application can still achieve the highest average accuracy (acc). This embodies the effectiveness of the model proposed by the application.

[0056] To sum up, the above is only a preferred embodiment of the application, and is not used to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for emotion recognition based on federated learning driven by cluster knowledge distillation, characterized in that, The method comprises the following steps: Step 1, initializing the parameters of the model deployed by each client, and constructing a federated learning model; Step 2, training the constructed federated learning model, specifically: Step 21, each client uses local data to train a local model; wherein each client stores a private local EEG data set and a shared public EEG data set, which is also called a proxy data set; Step 22, clustering all clients using a voting-based multi-leader distributed consensus, specifically: Step 221: All n Clients are initialized as followers; randomly selected c clients as candidates; c is the number of clusters in the final clustering; Step 222, each candidate broadcasts a voting request, which contains the output of the candidate's own model after one forward propagation with the proxy data set as input; the proxy data set is a public data set owned by each client; Step 223, after receiving the voting request of the candidate, the follower calculates the similarity between the output of the local model after one forward propagation on the proxy data set and the output of the candidate, and if the similarity is greater than or equal to a set threshold, the local model gives the candidate a vote of approval; Step 224, after all followers have finished voting for all candidates, count the total number of votes that a candidate has received If , then the candidate becomes the leader; otherwise, the candidate becomes a follower, and steps 222-224 are executed again. If the number of leaders selected by voting is still less than c, return to step 21. Step 225, assign followers to each leader based on the similarity between the leader and followers' outputs: for each leader, select the leader with the greatest similarity to it. Followers are followers of the leader, so all clients are divided into c A leader-centered cluster, where a follower can belong to multiple leaders at the same time; Step 226, every communication round, from one client, select one client, select one client, select c one client, select Step 227, taking all followers in the cluster as the teacher network and the leader of the cluster as the student network to perform knowledge distillation; Step 228, taking all leaders to which the follower belongs as the teacher network and the follower as the student network to perform knowledge distillation; wherein an entropy reducer module is used when aggregating the outputs of multiple leaders, i.e. the average output of the leaders is passed through a softmax layer with parameter t, and the entropy of the fused output of the leaders is reduced by reducing the value of t; Step 229, repeat steps 221-228 until each local model converges to obtain trained local models; Step 3, each client uses the local model trained in step 2 to perform emotion recognition.

2. The method of claim 1, wherein, In the knowledge distillation process of steps 227 and 228, the loss function is the weighted sum of the cross-entropy loss between the output of the student network and the original label of the sample and the cross-entropy loss between the output of the student network and the output of the teacher network.

3. The method of claim 1, wherein, In step 223, the threshold is an adaptive threshold, i.e., in each communication round, the similarity of the top candidate is averaged as the threshold of the similarity of the communication round.

Images