Semi-asynchronous federated learning method based on clustering client scheduling in heterogeneous mobile edge computing network
By adopting the semi-asynchronous federated learning method of clustered client scheduling in heterogeneous mobile edge computing networks, training delays and data heterogeneity caused by heterogeneity are solved, and more efficient model training and faster convergence time are achieved.
Patent Information
- Application Number
- CN202510234928.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-28
- Publication Date
- 2025-07-04
AI Technical Summary
In heterogeneous mobile edge computing networks, training delays and data heterogeneity caused by heterogeneity affect model convergence problems, and existing federated learning methods are difficult to effectively solve.
Using a semi-asynchronous federated learning method based on clustering client scheduling, the clients are clustered through edge servers, and the clients participating in model updates are selected according to the clustering clusters and training gradients, and weighted aggregation is performed. A semi-asynchronous federated learning framework is designed to reduce latency and improve training efficiency.
It improves training efficiency, shortens the model convergence time, enhances model stability, adapts to the wide application of heterogeneous networks, and reduces the delay of each round of updates.
Smart Images
Figure CN120258170A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of mobile edge computing and federated learning, and specifically to a semi-asynchronous federated learning method based on clustering client scheduling in a heterogeneous mobile edge computing network. Background Art
[0002] In a mobile edge computing network, the number of mobile clients / devices (such as in-vehicle devices, smartphones, etc.) connected to the network is exploding. These mobile clients / devices generate a large amount of diverse data, which is beneficial for the training of machine learning models. However, using this private data for training may lead to problems that cannot be ignored, such as privacy leakage and network congestion. As a solution that focuses on information processing at the device end, federated learning has become a promising solution to address these challenges. It can not only protect the data privacy of clients but also give full play to the advantages of large-scale distributed training. Federated learning is a collaborative machine learning training framework in which multiple clients jointly train a shared machine learning model using their respective private data. Different from the traditional centralized machine learning framework (where all training data is stored centrally), federated learning only exchanges update information (such as model gradients) between clients and the global server, without the need to upload a large amount of private data, thus avoiding unnecessary data transmission, ensuring data privacy, and effectively alleviating network congestion.
[0003] Although federated learning has the above advantages in a mobile edge computing network, the clients at the edge are usually heterogeneous, and there are differences in the hardware structure and computing power of each client. Therefore, they show strong heterogeneity in performance. This heterogeneity is mainly reflected in the following aspects: on the one hand, there are significant differences in the performance of the central processing unit (CPU) and transmission rate of mobile devices, resulting in different training times and update information upload times for each device. There may be some slower devices becoming "laggards", thus prolonging the update time for each round; on the other hand, there are differences in the data scale and data quality of different clients, which makes the training performance of each client different. If clients with poor training performance participate in the global update, it will have a negative impact on the convergence of the model. In addition, due to the limited resources and limited participation time of mobile devices, as well as the limitation of the bandwidth resources of the mobile edge computing network, only some clients are allowed to participate in each round of global update.
[0004] Therefore, in order to address the "laggard" problem brought by system heterogeneity, it is particularly necessary to study a semi-asynchronous federated learning framework. At the same time, in order to reduce the negative impact of data heterogeneity on model training, it is also necessary to study a client scheduling method for semi-asynchronous federated learning in a heterogeneous mobile edge computing network to ensure that the global model can achieve effective convergence within a limited time. Summary of the Invention
[0005] The objective of the present invention is to provide a semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, which uses an asynchronous mechanism and client information to improve the training efficiency, and includes the following steps:
[0006] 1) Establish a heterogeneous mobile edge computing network for federated learning, including a base station equipped with an edge server and N mobile clients;
[0007] 2) The edge server obtains all mobile client information and clusters the mobile clients to obtain clustering clusters
[0008] 3) The edge server sends an initial global machine learning model to all mobile clients;
[0009] 4) Each mobile client independently docks the received global machine learning model and stores it in the reception buffer;
[0010] When there is the latest global model in the reception buffer of each mobile client and the previous round of local training has ended, a new round of training can start, calculate the training gradient, and store the calculated gradient in the transmission buffer;
[0011] 5) The edge server selects mobile clients participating in the aggregation update of the global machine learning model according to the clustering clusters contribution degree and training gradient;
[0012] The mobile client uploads the training gradient to the edge server;
[0013] 6) The edge server performs weighted aggregation on the received training gradients to obtain global pseudo-gradients, and updates the global machine learning model according to the global pseudo-gradients;
[0014] 7) The edge server determines whether the updated global machine learning model meets the accuracy requirements. If so, jump to step 8); otherwise, the edge server distributes the current global machine learning model to all mobile clients participating in the training of the global machine learning model and returns to step 4);
[0015] 8) Use the current global machine learning model to execute tasks.
[0016] Furthermore, the heterogeneous mobile edge computing network includes a base station equipped with a server and N mobile clients;
[0017] The index set of the clients is represented as N = {1, 2,..., k,... N}, and the index of the base station is represented as {0}.
[0018] Furthermore, there is system heterogeneity among the N mobile clients;
[0019] The CPU computing performance, transmission speed, and data volume of N mobile clients satisfy the following formula:
[0020]
[0021]
[0022] where D k represents the private data owned by client i, i and j represent the data categories, I represents the total number of data categories, and |·| represents the quantity of ·; f i is the CPU computing performance of client i, and r i is the transmission speed between client i and the base station.
[0023] Furthermore, in step 2), the edge server clusters the mobile clients through the k-medoids clustering algorithm according to the data category distribution in the mobile client information.
[0024] Furthermore, in step 2), the steps of clustering the mobile clients include:
[0025] 2.1) Calculate the category distribution p k,i of each mobile client, that is:
[0026]
[0027] where n k,i represents the data volume of client k with data category i;
[0028] 2.2) Based on the category distribution of the mobile clients, construct the category distribution vector P k of all clients, that is:
[0029] P k = [p k,0 p k,1 ... p k,I-1 , k ∈ N(6)
[0030] 2.3) Calculate the similarity between different mobile clients That is:
[0031]
[0032] 2.4) Cluster the mobile clients according to the similarity between the mobile clients to obtain the clustering clusters
[0033] Furthermore, in step 4), the trained global machine learning model is as follows:
[0034]
[0035] Among them, W k (t, τ + 1) is the model obtained by client k in the local training of the (τ + 1)-th round; τ ∈ {0, 1, 2…, E k - 1}; η k is the local learning rate, represents the gradient of the loss function; B k (t, τ) represents the subset of the private data D k ; W k (t, τ) is the model obtained by client k in the local training of the τ-th round.
[0036] Furthermore, in step 4), the training gradient is as follows:
[0037] g k (t) = W(t, 0) - W k (t, E k )(9)
[0038] In the formula, g k (t) is the training gradient; W(t, 0) is the initial model; W k (t, E k ) is the model after E k rounds of training.
[0039] Furthermore, in step 5), the mobile clients selected to participate in the global machine learning model training simultaneously satisfy the following constraint conditions:
[0040] Constraint condition 1: The training gradient age a k = t - s t ≤ M; M is the staleness tolerance;
[0041] Constraint condition 2: The variance of the number of mobile clients scheduled in each cluster is the smallest;
[0042] The variance of the number of mobile clients Var(S t ) is as follows:
[0043]
[0044] where u i,t represents the number of clients scheduled in cluster i, represents the average number of clients scheduled in each cluster;
[0045] Constraint condition 3: The contribution degree of the mobile clients scheduled from each cluster reaches the maximum, that is:
[0046]
[0047] where Fk,t Denotes the loss function value of client k, N k,t Denotes the number of times client k has been scheduled before the t-th global update; C i,t = C i ∩R t , Denotes the clients in cluster C i that have completed local training. q k Denotes the contribution degree of client k; k' is the maximum contribution degree;
[0048] Furthermore, in step 6), the global pseudo-gradient is as follows:
[0049]
[0050] In the formula, G t is the global pseudo-gradient.
[0051] Furthermore, in step 6), the updated global machine learning model is as follows:
[0052] W(t + 1) = W(t) - ηG t (14)
[0053] In the formula, W(t) and W(t + 1) are the global machine learning models before and after update; η is the learning rate.
[0054] Furthermore, in step 8), the current global machine learning model is a visual model, and the task is an image recognition task;
[0055] The input of the visual model is image information, and the output is a recognition label.
[0056] The technical effect of the present invention is beyond doubt, and the beneficial effects of the present invention are as follows:
[0057] 1) Considering system heterogeneity and data heterogeneity, the present invention has a wider application range compared with other methods.
[0058] 2) By designing a semi-asynchronous federated learning framework, it allows clients to train the model according to their own rhythm, thereby reducing the delay of each round of update and improving the training efficiency.
[0059] 3) Based on the data heterogeneity of clients, the present invention proposes a clustering-based client scheduling algorithm. By considering the data similarity between clients for scheduling, it can effectively ensure data balance and make the global model more stable. At the same time, comprehensively considering the contribution of clients to global updates, maximizing the benefits of each round of update, significantly reducing the total number of update rounds required for convergence, and thus shortening the convergence time BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Figure 1It is a model diagram of a semi - asynchronous federated learning system;
[0061] Figure 2 It is a flow chart of client clustering;
[0062] Figure 3 It is a semi - asynchronous federated learning flow chart based on clustered client scheduling. Specific implementation manners
[0063] The present invention will be further described below in conjunction with embodiments, but it should not be understood that the scope of the above - mentioned subject matter of the present invention is limited to the following embodiments. Without departing from the above - mentioned technical idea of the present invention, various substitutions and changes made according to ordinary technical knowledge and customary means in the art should be included within the protection scope of the present invention.
[0064] Embodiment 1:
[0065] See Figures 1 to 3 , a semi - asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, which uses an asynchronous mechanism and client information to improve the training efficiency, and includes the following steps:
[0066] 1) Establish a heterogeneous mobile edge computing network for federated learning, including a base station equipped with an edge server and M mobile clients;
[0067] 2) The edge server obtains all mobile client information and clusters the mobile clients according to to obtain clustering clusters
[0068] 3) The edge server distributes the initial global machine learning model to all mobile clients;
[0069] 4) Each mobile client independently docks the received global machine learning model and stores it in the reception buffer;
[0070] When there is the latest global model in the reception buffer of each mobile client and the previous round of local training has ended, a new round of training can start, calculate the training gradient, and store the calculated gradient in the transmission buffer;
[0071] 5) The edge server selects mobile clients participating in the aggregation update of the global machine learning model according to the clustering clusters contribution degree and training gradient;
[0072] The mobile client uploads the training gradient to the edge server;
[0073] 6) The edge server performs weighted aggregation on the received training gradients to obtain global pseudo - gradients, and updates the global machine learning model according to the global pseudo - gradients;
[0074] 7) The edge server determines whether the updated global machine learning model meets the accuracy requirements. If so, it jumps to step 8). Otherwise, the edge server distributes the current global machine learning model to all mobile clients participating in the training of the global machine learning model and returns to step 4).
[0075] 8) Use the current global machine learning model to execute tasks.
[0076] The heterogeneous mobile edge computing network includes a base station equipped with a server and N mobile clients;
[0077] The index set of the clients is represented as M = {1, 2,..., k,... N}, and the index of the base station is represented as {0}.
[0078] There is system heterogeneity among the N mobile clients;
[0079] The CPU computing performance, transmission speed, and data volume of the N mobile clients satisfy the following formula:
[0080]
[0081] where D k represents the private data owned by client i, i and j represent the data categories, I represents the total number of data categories, and |·| represents the quantity of ·; f i is the CPU computing performance of client i, and r i is the transmission speed between client i and the base station.
[0082] In step 2), the edge server clusters the mobile clients through the k-medoids clustering algorithm according to the data category distribution in the mobile client information.
[0083] In step 2), the steps of clustering the mobile clients include:
[0084] 2.1) Calculate the category distribution p k，i of each mobile client, that is:
[0085]
[0086] where n k,i represents the quantity of data with category i owned by client k;
[0087] 2.2) Based on the category distribution of the mobile clients, construct the category distribution vector P k of all clients, that is:
[0088] P k = [p k,0 p k,1 ... p k,I-1, k ∈ N(6)
[0089] 2.3) Calculate the similarity between different mobile clients That is:
[0090]
[0091] 2.4) Cluster the mobile clients according to the similarity between them to obtain clustering clusters
[0092] In step 4), the trained global machine learning model is as follows:
[0093]
[0094] Among them, W k (t, τ + 1) is the model obtained by client k in the (τ + 1)-th round of local training; τ ∈ {0, 1, 2…, E k -1}; η k is the local learning rate, represents the gradient of the loss function; B k (t, τ) represents the subset of the private data D k ; W k (t, τ) is the model obtained by client k in the τ-th round of local training.
[0095] In step 4), the training gradient is as follows:
[0096] g k (t) = W(t, 0) - W k (t, E k )(9)
[0097] In the formula, g k (t) is the training gradient; W(t, 0) is the initial model; W k (t, E k ) is the model after E k rounds of training.
[0098] In step 5), the mobile clients selected to participate in the global machine learning model training simultaneously satisfy the following constraint conditions:
[0099] Constraint 1: The training gradient age a k = t - s t ≤ M; M is the staleness tolerance;
[0100] Constraint 2: The variance of the number of mobile clients scheduled in each cluster is the smallest;
[0101] The variance of the number of mobile clients Var(S t ) is as follows:
[0102]
[0103] where u i,t represents the number of clients scheduled in cluster i, represents the average number of clients scheduled per cluster;
[0104] Constraint 3: The contribution of the mobile clients scheduled from each cluster reaches the maximum, that is:
[0105]
[0106] where F k,t represents the loss function value of client k, and N k,t represents the number of times client k has been scheduled before the t-th global update; C i,t = C i ∩R t , represents the clients in cluster C i who have completed local training. q k represents the contribution of client k; k' is the maximum contribution;
[0107] In step 6), the global pseudo-gradient is as follows:
[0108]
[0109] In the formula, G t is the global pseudo-gradient.
[0110] In step 6), the updated global machine learning model is as follows:
[0111] W(t + 1)= W(t)-ηG t (14)
[0112] In the formula, W(t) and W(t + 1) are the global machine learning models before and after update; η is the learning rate.
[0113] In step 8), the current global machine learning model is a vision model, and the task is an image recognition task;
[0114] The input of the vision model is image information, and the output is a recognition label (such as person, car, etc.).
[0115] Example 2:
[0116] A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, which uses an asynchronous mechanism and client information to improve training efficiency, includes the following steps:
[0117] 1) Establish a heterogeneous mobile edge computing network for federated learning, including a base station equipped with an edge server and N mobile clients;
[0118] 2) The edge server obtains information of all mobile clients and clusters the mobile clients to obtain clustering clusters.
[0119] 3) The edge server distributes an initial global machine learning model to all mobile clients;
[0120] 4) Each mobile client independently docks the received global machine learning model and stores it in the reception buffer;
[0121] When there is the latest global model in the reception buffer of each mobile client and the previous round of local training has ended, a new round of training can start, calculate the training gradient, and store the calculated training gradient in the transmission buffer;
[0122] 5) The edge server selects mobile clients participating in the aggregation update of the global machine learning model according to the clustering clusters contribution degree and training gradient;
[0123] The mobile clients upload the training gradient to the edge server;
[0124] 6) The edge server performs weighted aggregation on the received training gradients to obtain global pseudo-gradients, and updates the global machine learning model according to the global pseudo-gradients;
[0125] 7) The edge server determines whether the updated global machine learning model meets the accuracy requirement. If so, jump to step 8); otherwise, the edge server distributes the current global machine learning model to all mobile clients participating in the training of the global machine learning model and returns to step 4);
[0126] 8) Use the current global machine learning model to execute tasks.
[0127] Embodiment 3:
[0128] A semi-asynchronous federated learning method based on clustering client scheduling in a heterogeneous mobile edge computing network, the technical content is the same as that of Embodiment 2. Further, the heterogeneous mobile edge computing network includes a base station equipped with a server and N mobile clients;
[0129] The index set of the clients is represented as {N = 1, 2,..., k,... N}, and the index of the base station is represented as {0}.
[0130] Embodiment 4:
[0131] A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, the technical content is the same as any one of Embodiments 2-3. Further, there is system heterogeneity among N mobile clients;
[0132] The CPU computing performance, transmission speed, and data volume of N mobile clients satisfy the following formula:
[0133]
[0134] where D k represents the private data owned by client i, i and j represent the categories of data, I represents the total category of data, and |·| represents the quantity of ·; f i is the CPU computing performance of client i, and r i is the transmission speed between client i and the base station.
[0135] Embodiment 5:
[0136] A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, the technical content is the same as any one of Embodiments 2-4. Further, in step 2), the edge server clusters the mobile clients through the k-medoids clustering algorithm according to the data category distribution in the mobile client information.
[0137] Embodiment 6:
[0138] A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, the technical content is the same as any one of Embodiments 2-5. Further, in step 2), the steps of clustering the mobile clients include:
[0139] 2.1) Calculate the category distribution p k,i of each mobile client, that is:
[0140]
[0141] where n k,i represents the data volume of client k with category i;
[0142] 2.2) Based on the category distribution of the mobile clients, construct the category distribution vector P k of all clients, that is:
[0143] P k = [p k,0 p k,1 ... p k,I-1 , k ∈ N(6)
[0144] 2.3) Calculate the similarity between different mobile clients That is:
[0145]
[0146] 2.4) Cluster the mobile clients according to the similarity between mobile clients to obtain clustering clusters
[0147] Embodiment 7:
[0148] A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, the technical content is the same as any one of Embodiments 2-6. Further, in step 4), the trained global machine learning model is as follows:
[0149]
[0150] Among them, W i (t, τ+1) is the model obtained by the client through local training in the τ-th round; τ ∈ {0, 1, 2…, E k -1}; η k is the local learning rate, represents the gradient of the loss function; B k (t, τ) represents the data subset of the private data D k .
[0151] Embodiment 8:
[0152] A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, the technical content is the same as any one of Embodiments 2-7. Further, in step 4), the training gradient is as follows:
[0153] g k (t) = W(t, 0) - W k (t, E k )(9)
[0154] In the formula, g k (t) is the training gradient.
[0155] Embodiment 9:
[0156] A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, the technical content is the same as any one of Embodiments 2-8. Further, in step 5), the mobile clients participating in the training of the global machine learning model simultaneously satisfy the following constraint conditions:
[0157] Constraint 1: The training gradient age a k = t - s t ≤ M; M is the staleness tolerance;
[0158] Constraint 2: The variance of the number of mobile clients scheduled in each cluster is minimized;
[0159] The variance of the number of mobile clients is as follows:
[0160]
[0161] where u i,t represents the number of clients scheduled in cluster i, and represents the average number of clients scheduled in each cluster;
[0162] Constraint 3: The contribution of the mobile clients scheduled from each cluster reaches the maximum, that is:
[0163]
[0164] where F k,t represents the loss function value of client k, and N k,t represents the number of times client k has been scheduled before the t-th global update; C i,t = C i ∩R t , represents the clients in cluster C i that have completed local training.
[0165] Example 10:
[0166] A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, the technical content is the same as any one of Examples 2-9. Further, in step 6), the global pseudo-gradient is as follows:
[0167]
[0168] In the formula, G t is the global pseudo-gradient.
[0169] Example 11:
[0170] A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, the technical content is the same as any one of Examples 2-10. Further, in step 6), the updated global machine learning model is as follows:
[0171] W(t + 1) = W(t) - ηG t (14)
[0172] Example 12:
[0173] A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network, the technical content is the same as any one of Examples 2-11. Further, in step 8), the current global machine learning model is a visual model, and the task is an image recognition task;
[0174] The input of the visual model is image information, and the output is recognition labels (such as people, vehicles, etc.).
[0175] Embodiment 13:
[0176] A semi-asynchronous federated learning method based on clustering client scheduling in a heterogeneous mobile edge computing network, the steps are as follows:
[0177] 1) Establish a heterogeneous mobile edge computing network and use a semi-asynchronous federated learning framework for model training. This heterogeneous mobile edge computing network includes a base station equipped with a server and N mobile clients / devices. The base station and these N devices cooperate to train a global machine learning model. The index set of the clients is represented as N = {1, 2,..., k,..., N}, and the index of the base station is represented as {0}.
[0178] There is system heterogeneity among these N clients. The CPU computing performance of different clients is different, and the transmission speed that can be obtained during communication with the base station may also be different. The specific manifestations are as follows:
[0179]
[0180] Where f i is the CPU computing performance of client i, and r i is the transmission speed between client i and the base station. ≠ means that the computing capabilities of the clients are different. At the same time, there is also data heterogeneity among the clients, and the data scale and data quality are different. On the one hand, the amount of data owned by each client is different, which is expressed as follows:
[0181]
[0182] On the other hand, the private data owned by the clients itself has class imbalance, which is expressed as follows:
[0183]
[0184] Where D k represents the private data owned by client i, i and j represent the data classes, I represents the total number of data classes, and |·| represents the number of ·.
[0185] 2) Training of semi-asynchronous federated learning. In the semi-asynchronous federated learning framework, the edge base station communicates and exchanges information with the clients to jointly train a machine learning model. Different from the traditional synchronous training method, the semi-asynchronous framework allows different clients to train according to their own rhythms without being restricted by the time of the edge base station or other clients. At the beginning of the t-th round of global update, some clients may have completed the latest local training, and their set is represented as Rt The edge server will schedule a subset S from the clients that have recently completed local training according to the clustering C and the contribution degree G t . The clients in the subset will upload the latest model to the edge server. At the same time, the unscheduled clients will continue with local training and will not be interrupted by the edge server's scheduling. Subsequently, the edge server will aggregate and update the received model parameters to obtain a new global model and broadcast it to all clients.
[0186] The client stores the global model sent by the edge server in the receive buffer. Before the start of the t-th round of global update, once the receive buffer contains a new global model and the most recent local update has been completed, the client will retrieve the latest global model W(t) from the receive buffer. The global model W(t) is often an old version, which we denote as W(s t ), s t representing the version index of the global model. Before the start of local training, client k, needs to obtain an initial model W l (t,0) = W(s t ), and then will perform E k rounds of local training model updates. The specific update rules are as follows:
[0187]
[0188] where W i (t,τ + 1) is the model obtained by the client in the τ-th round of local training, τ ∈ {0, 1, 2…, E k - 1}, η k is the local learning rate, denotes the gradient of the loss function, B k (t,τ) represents a mini-batch data subset of the private data D k . After completing E k rounds of local training, the client will calculate the difference and gradient of the model parameters before and after training:
[0189] g k (t) = W(t,0) - W k (t,E k ).
[0190] 3) The client stores the gradient in the send buffer and waits for the edge server to schedule it. At the same time, we can also calculate the age of the gradient:
[0191] a k = t - s t .
[0192] Assume that the number of CPU cycles required for client k to compute a mini-batch of data is Then the time required for client k to complete local training can be calculated based on the CPU clock cycle frequency and the number of local training rounds executed. The specific formula is as follows:
[0193]
[0194] where f k represents the CPU processing power and clock cycle frequency, and E k represents the number of remaining local training rounds that client k needs to execute when it is scheduled.
[0195] When the client is scheduled by the edge server and there are updated gradients in the send buffer, the client will send the updated gradients to the edge server. Let the size of the gradient parameter be According to the information transmission model, the time required for client k to upload the gradient parameter can be calculated as follows:
[0196]
[0197] where r k,t is the transmission rate of client k in the t-th round of global update, which can be calculated using the Shannon formula as follows:
[0198]
[0199] where G k represents the channel gain, p k represents the communication power, and N0 represents Gaussian noise.
[0200] 4) Edge server synchronous federated aggregation algorithm. At the t-th round of global update, after receiving the gradient parameters of all clients in k |, the edge server will perform weighted aggregation based on the data volume |D
[0201]
[0202] to obtain the global pseudo-gradient:
[0203] W(t + 1) = W(t) - ηG t .
[0204] 5) Client clustering algorithm. Before starting to train the global model using federated learning, the edge server needs to cluster according to the data class distribution of each client. Only by finding the correct clustering can effective client scheduling be carried out. First, we need to calculate the class distribution of each client:
[0205]
[0206] where n k,i represents the amount of data of category i owned by client k. Then, the category distribution vectors of all clients are obtained:
[0207] P k = [p k,0 p k,1 ... p k,I-1 , k ∈ N.
[0208] The similarity between clients is measured by the Euclidean distance of the category distribution vectors:
[0209]
[0210] Then, according to all the similarities, the edge server will use the k-medoids clustering algorithm to cluster all clients. Set the number of clusters to c, and all clusters will be obtained through clustering
[0211] 6) Client scheduling algorithm. At the beginning of each round of global update, the edge server schedules clients to participate in this round of global update according to information such as clustering and contribution degree. The scheduling steps include: deleting stale gradients, determining the clustering scheduling quantity, and selecting participating clients.
[0212] 6.1) Deleting stale gradients. At the t-th round of global update, the gradients in the client buffer may be stale. If the stale gradients are uploaded to the edge server for aggregated update, it will affect the convergence of the model. Therefore, it is necessary to ensure that the uploaded gradients are not too stale. We define the stale tolerance M, which represents the maximum gradient age that the edge server can accept. Only when the gradient age sent by the client satisfies this tolerance, that is, a k ≤ M, can it be scheduled by the edge server and participate in this round of global update.
[0213] 6.2) Determining the clustering scheduling quantity. Due to the data heterogeneity among clients, the edge server needs to ensure the balanced data distribution of each client during each round of scheduling. Given that the data distributions of clients within each cluster are similar, the edge server should ensure that the variance of the number of scheduled clients in each cluster is minimized to maintain data balance. The variance calculation formula is as follows:
[0214]
[0215] where u i,t represents the number of scheduled clients in cluster i, represents the average number of scheduled clients in each cluster. To minimize the variance of the number of clients within a cluster, clients need to be scheduled sequentially from each cluster.
[0216] 6.3) Determine the scheduling clients. To maximize the benefits of global updates in each round, the edge server needs to select the best-performing clients from each cluster for scheduling. Research shows that clients with larger training loss values usually bring better results. In addition, clients that have been scheduled too many times may transmit duplicate information and have limited contribution to global updates. Therefore, the contribution of clients can be measured by the loss function value and the number of scheduling times as follows:
[0217]
[0218] where F k,t represents the loss function value of client k, and N k,t represents the number of times client k has been scheduled before the t-th global update.
[0219] After knowing the contribution of each client in the current round of global update, the edge server can schedule the client with the largest contribution from each cluster to maximize the benefits of the current round of global update. The scheduling method can be expressed as:
[0220]
[0221] where C i,t = C i ∩R t , representing the clients in cluster C i that have completed local training.
[0222] In summary, the present invention provides a semi-asynchronous federated learning framework based on a clustering client scheduling method for model training in a heterogeneous mobile edge computing network. The steps include: 1) establishing a heterogeneous edge computing network system for federated learning, including a base station equipped with an edge server and multiple mobile clients; 2) constructing a semi-asynchronous federated learning framework; 3) the edge server clustering according to the data distribution of the clients to obtain cluster information; 4) the clients asynchronously performing local training and sending training contribution information to the edge server; 5) the edge server scheduling the clients to upload updated gradients according to the cluster information and contribution degree for aggregated update to obtain a new global model. The present invention takes into account the heterogeneity of the system, designs a semi-asynchronous federated learning framework, allows the clients to train at their own pace, reduces the impact of procrastinators, and reduces the latency of each round of global update. At the same time, by balancing the data distribution of each global update and scheduling the better-performing clients to upload gradients, the benefits of each round of global update are maximized, thereby accelerating the convergence of the global model and reducing the convergence latency.
Claims
1. A semi-asynchronous federated learning method based on clustering client scheduling in a heterogeneous mobile edge computing network, characterized in that Improving training efficiency using asynchronous mechanisms and client information, including the following steps: 1) Establish a heterogeneous mobile edge computing network for federated learning, including a base station equipped with an edge server and N mobile clients; 2) The edge server obtains all the mobile client information and clusters the mobile clients to obtain clustering clusters 3) The edge server distributes the initial global machine learning model to all mobile clients; 4) Each mobile client independently docks the received global machine learning model and stores it in the reception buffer; When the reception buffer of each mobile client has the latest global model and the previous round of local training has ended, a new round of training can begin, calculate the training gradient, and store the calculated gradient in the transmission buffer; 5) The edge server selects mobile clients participating in the aggregated update of the global machine learning model according to the contribution degree and training gradient of the clustering clusters The mobile client uploads the training gradient to the edge server; 6) The edge server performs weighted aggregation on the received training gradients to obtain the global pseudo-gradient, and updates the global machine learning model according to the global pseudo-gradient; 7) The edge server determines whether the updated global machine learning model meets the accuracy requirements. If so, jump to step 8). Otherwise, the edge server distributes the current global machine learning model to all mobile clients participating in the training of the global machine learning model and returns to step 4); 8) Use the current global machine learning model to execute tasks.
2. The semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network according to claim 1, wherein The heterogeneous mobile edge computing network includes a base station equipped with a server and N mobile clients; The index set of the clients is represented as N = {1, 2,..., k,... N}, and the index of the base station is represented as {0}.
3. A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network according to claim 2, characterized in that, There is system heterogeneity among the N mobile clients; The CPU computing performance, transmission speed, and data volume of the N mobile clients satisfy the following formula: Among them, D k represents the private data owned by client i, where i and j represent the data categories, I represents the total number of data categories, and |·| represents the quantity of ·; f i is the CPU computing performance of client i, and r i is the transmission speed between client i and the base station.
4. A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network according to claim 1, characterized in that, In step 2), the edge server clusters the mobile clients through the k-medoids clustering algorithm according to the data category distribution in the mobile client information.
5. A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network according to claim 1, characterized in that, In step 2), the steps for clustering the mobile clients include: 2.1) Calculate the category distribution p of each mobile client k,i , namely: where n k,i represents the amount of data of category i owned by client k; D k represents the private data owned by client i; 2.2) Construct the category distribution vector P of all clients based on the category distribution of the mobile clients k , that is: P k = [p k,0 p k,1 ... p k,I-1 , k ∈ N(6) 2.3) Calculate the similarity between different mobile clients That is: 2.4) Cluster the mobile clients according to the similarity between the mobile clients to obtain clustering clusters 6. The semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network according to claim 1, characterized in that, In step 4), the global machine learning models trained by each client are as follows: Among them, W k (t, τ + 1) is the model obtained by client k in the (τ + 1)-th round of local training; τ ∈ {0, 1, 2, …, E k - 1}; η k is the local learning rate, denotes the gradient of the loss function; B k (t, τ) represents the subset of the private data D k ; W k (t, τ) is the model obtained by client k in the τ-th round of local training.
7. A semi-asynchronous federated learning method based on clustering client scheduling in a heterogeneous mobile edge computing network according to claim 1, characterized in that In step 4), the training gradient is as follows: g k g(t) = W(t,0) - W k g(t,E k )(9) where g k (t) is the training gradient; W(t, 0) is the initial model; W k (t, E k ) is the model after E k rounds of training.
8. A semi-asynchronous federated learning method based on clustering client scheduling in a heterogeneous mobile edge computing network according to claim 1, characterized in that In step 5), the mobile clients participating in the training of the global machine learning model simultaneously satisfy the following constraint conditions: Constraint 1: Training gradient age a k = t - s t ≤ M; M is the obsolescence tolerance; Constraint 2: The variance of the number of mobile clients scheduled in each cluster is the smallest; Variance of the number of mobile clients Var(S t ) is as follows: where u i,t represents the number of clients scheduled in cluster i, and represents the average number of clients scheduled per cluster; Constraint 3: The contribution of the mobile clients scheduled from each cluster reaches the maximum, that is: Among which F k,t represents the loss function value of client k, and N k,t represents the number of times client k has been scheduled before the t-th global update; C i,t = C i ∩R t , represents the clients in cluster C i who have completed local training; q k represents the contribution degree of client k; k' is the maximum contribution degree.
9. A semi-asynchronous federated learning method based on clustering client scheduling in a heterogeneous mobile edge computing network according to claim 1, characterized in that In step 6), the global pseudo-gradient is as follows: where G t is the global pseudo-gradient; g k (t) is the gradient; In step 6), the updated global machine learning model is as follows: W(t + 1) = W(t) - ηG t (14) In the formula, W(t) and W(t + 1) are the global machine learning models before and after the update; η is the learning rate.
10. A semi-asynchronous federated learning method based on clustered client scheduling in a heterogeneous mobile edge computing network according to claim 1, characterized in that, In step 8), the current global machine learning model is a vision model, and the task is an image recognition task; The input of the vision model is image information, and the output is the recognition label.