A client training method, device and equipment based on layered federated learning

Abstract

The application discloses a client training method and device based on layered federated learning, and equipment. The method comprises the following steps: according to the estimated context reward between each client in the overlapping area and each overlapping edge server, the edge servers in the overlapping area are allocated to the clients to determine the client set in each edge server; according to the estimated context reward between each client in the client set and the corresponding edge server and the training quality of each client, the client participating in the training is determined, and the client participating in the training is trained according to the received training model; and the estimated context reward and the training quality between the client and the corresponding edge server are updated according to the training result of the client. The method effectively solves the attribution problem of the clients in the overlapping area of the edge servers, and designs a reasonable selection mechanism for the edge servers when training the clients, so that the training time and the training quality of the clients can be effectively guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing and federated learning, and particularly relates to a client training method and device based on hierarchical federated learning and equipment. BACKGROUND

[0002] Artificial intelligence is currently indispensable in many new applications, such as natural language processing, self-driving cars, etc. Neural networks can support these applications on the premise that we have a large amount of data. As the amount of data collected by terminal devices increases, the number of terminal devices is also increasing, so federated learning has emerged. In hierarchical federated learning, terminal devices no longer interact directly with the central server, but interact with the edge server corresponding to them, which greatly reduces the communication time and provides more stable network connections.

[0003] In hierarchical federated learning, each edge server is only responsible for a region, so different edge servers must have overlapping regions. Limited by the bandwidth and computing power of the client, the client in the overlapping region can only participate in the training of one edge server in one round, so how to select a suitable edge server for the client in the overlapping region is a problem that needs to be solved, and after determining the attribution of the client in the overlapping region, how the edge server trains the client to effectively guarantee the training time and training quality is also a technical problem that needs to be solved. SUMMARY

[0004] The main purpose of the present application is to provide a client training method, device and equipment based on hierarchical federated learning, which aims to solve the above technical problems.

[0005] In a first aspect, the present application provides a client training method based on hierarchical federated learning, the method comprising the following steps:

[0006] According to the estimated contextual rewards between each client in the overlapping region and each overlapping edge server, the edge servers for the clients in the overlapping region are allocated to determine the client set in each edge server;

[0007] According to the estimated contextual rewards between each client in the client set and the corresponding edge server and the training quality of each client, the client participating in the training is determined, and the client participating in the training is trained according to the received training model;

[0008] According to the training results of the client, the estimated contextual rewards and the training quality between the client and the corresponding edge server are updated.

[0009] In some embodiments, the assigning an edge server to a client in an overlapping area according to an estimated context reward between the client and each of the overlapping edge servers comprises:

[0010] determining a set of overlapping edge servers in an overlapping area of an edge server where the client is located;

[0011] calculating an estimated context reward between the client and each of the edge servers in the set of edge servers according to a context vector between the client and each of the edge servers and a context parameter of each of the edge servers:

[0012]

[0013] according to the formula assigning the client to the edge server with the largest estimated context reward;

[0014] wherein i is the client, t is a training round, a is an edge server to which the client i is assigned in the tth training round, P t (i) is a set of edge servers of the client i, r i,a,t is the estimated context reward between the client i and the edge server a in the tth training round, X i,a,t is a context vector between the client i and the edge server a in the tth training round, is a context parameter of the edge server a after the (t-1)th training round, CB r,i is a confidence bound of the estimated context reward.

[0015] In some embodiments, the determining a client to participate in training according to an estimated context reward between each of the clients and a corresponding edge server and a training quality of each of the clients comprises:

[0016] sorting the clients in the set of clients according to the estimated context reward and determining whether a number of the clients in the set of clients is greater than a first preset number;

[0017] if yes, sorting the first preset number of clients with the highest estimated context reward according to a UBC training quality of the clients, otherwise, sorting all the clients in the set of clients according to the UBC training quality of the clients;

[0018] determining whether a number of the clients sorted according to the UBC learning quality is greater than a second preset number;

[0019] if yes, selecting the second preset number of clients with the highest UBC training quality to participate in training, otherwise, selecting all the clients in the set of clients to participate in training;

[0020] According to the participating training clients in the edge server, a corresponding participating training client set is generated;

[0021] The first preset number is greater than the second preset number.

[0022] In some embodiments, the training quality of the corresponding client is updated according to the training result of the client, including:

[0023] According to the loss value of the training model after the client completes the current round of training and the loss value of the training model before the training, the training quality of the current round of training of the client is determined:

[0024] q i,t = (L s -L e )*D i,t

[0025] Wherein, q i,t is the training quality of the client i in the tth round of training, L s is the loss value of the training model in the (t-1) th round of training, L e is the loss value of the training model in the tth round of training, and D i,t is the data set of the client i in the tth round of training.

[0026] The value of the training quality q i,t of the client i in the tth round of training is scaled to a preset range to obtain

[0027] According to the number of times of participating in training of the client and the training quality of the current round of training of the client, the overall training quality of the client up to the current training round is determined.

[0028]

[0029] Wherein, is the overall training quality of the client i in the edge server a up to the tth round of training, is the training quality of the client i in the tth round of training after numerical scaling, z i,t is the number of times of participating in training of the client i up to the tth round of training, α is the weight parameter of, and β is the weight parameter of, and α<β.

[0030] According to the overall training quality of the client up to the current training round, the UCB training quality of the client is updated:

[0031]

[0032] Wherein,

[0033]

[0034] wherein, is the UCB training quality of client i in the edge server a range at the tth training round, u i,t is the overall training quality of client i at the tth training round is the difference between the UCB training quality and the overall training quality, K is the number of clients in the edge server a range participating in the current training round.

[0035] In some embodiments, the estimated contextual reward of a client is updated according to the training result of the client, including:

[0036] The actual contextual reward of a client is determined according to the total time consumption of the client in the current training round:

[0037]

[0038] wherein, r' i,a,t is the actual contextual reward of client i in the edge server a at the tth training round, v is any client participating in the training in the edge server a at the tth training round, Δt t,i is the total time consumption of client i at the tth training round, S a,t is the set of clients participating in the training corresponding to the edge server a;

[0039] The contextual parameters of the corresponding edge server are updated according to the contextual vector and the actual contextual reward of the client:

[0040]

[0041] wherein,

[0042]

[0043]

[0044] wherein, θ a,t is the updated contextual parameters of the edge server a at the tth training round, A a,t is the first update parameter of the contextual parameters of the edge server a at the tth training round, b a,t is the second update parameter of the contextual parameters of the edge server a at the tth training round, wherein, includes x i,a,t and r' i,a,t , is the historical data of the edge server a up to the tth training round, I is the unit matrix;

[0045] The estimated contextual reward between the client and the edge server is updated according to the updated contextual parameters of the edge server.

[0046] In some embodiments,

[0047] Before the participating clients are trained according to the estimated contextual rewards of each client in the client set and the corresponding edge server and the training quality of each client, and the participating clients are trained according to the received training model, the method further comprises:

[0048] The central server initializes the training model and distributes the training model to each edge server;

[0049] Each edge server distributes the initialized training model to the corresponding participating client.

[0050] In some embodiments, the method further comprises:

[0051] After each round of training is completed, the participating client sends the training model update parameter to the corresponding edge server;

[0052] The edge server aggregates and updates the training model according to the received training model update parameter.

[0053] In some embodiments, the method further comprises:

[0054] Every interval of a preset number of training rounds, each edge server uploads the latest aggregated training to the central server for aggregation.

[0055] In a second aspect, the present application also provides a client training device based on hierarchical federated learning, the device comprising:

[0056] The allocation module is configured to allocate edge servers to the clients in the overlapping area according to the estimated contextual rewards between each client in the overlapping area and each overlapping edge server, to determine a client set in each edge server;

[0057] The determination module is configured to determine participating clients according to the estimated contextual rewards of each client in the client set and the corresponding edge server and the training quality of each client, and to train the participating clients according to the received training model;

[0058] The update module is configured to update the estimated contextual rewards between the client and the corresponding edge server and the training quality according to the training result of the client.

[0059] In a third aspect, the present application also provides a computer device comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the client training method based on hierarchical federated learning as described above.

[0060] The present application provides a client training method, device and equipment based on hierarchical federated learning, which allocates edge servers for clients in an overlapping area by determining a set of clients in each edge server according to the estimated contextual rewards between each client and each overlapping edge server in the overlapping area; determines the clients participating in training according to the estimated contextual rewards between each client in the set of clients and the corresponding edge server and the training quality of each client, and makes the clients participating in training train according to the received training model; and updates the estimated contextual rewards between the clients and the corresponding edge servers and the training quality according to the training results of the clients, effectively solving the attribution problem of the clients in the overlapping area of the edge servers, and designing a reasonable selection mechanism for the edge servers when training clients, so that the training time and training quality of the clients can be effectively guaranteed. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0062] Figure 1 A flowchart of a client training method based on hierarchical federated learning is provided for the embodiments of the present application.

[0063] Figure 2 An overview of the client training method based on hierarchical federated learning is provided.

[0064] Figure 3 A specific flowchart of the client training method based on hierarchical federated learning is provided.

[0065] Figure 4 A structural schematic block diagram of a client training device based on hierarchical federated learning is provided for the embodiments of the present application.

[0066] Figure 5 A structural schematic block diagram of a computer device related to an embodiment of the present application is provided.

[0067] The implementation, functional features and advantages of the present application will be further described with reference to the drawings. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall into the scope of the present application.

[0069] The flowchart shown in the drawings is only an example and does not necessarily include all the contents and operations / steps, nor does it have to be executed in the order described. For example, some operations / steps can be further decomposed, combined or partially merged, so the actual execution order may be changed according to the actual situation.

[0070] The embodiments of the present application provide a client training method, device and equipment based on hierarchical federated learning. The client training method based on hierarchical federated learning can be applied to a server in a computer device.

[0071] Some embodiments of the present application will be described in detail below with reference to the drawings. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0072] Please refer to Figure 1 , Figure 1 A flowchart of a client training method based on hierarchical federated learning provided by the embodiments of the present application is shown.

[0073] As Figure 1 shown, the method includes steps S1 to S3.

[0074] Step S1, according to the estimated contextual rewards between each client in the overlapping area and each overlapping edge server, assigning an edge server to the client in the overlapping area to determine a client set in each edge server;

[0075] Step S2, determining the client participating in training according to the estimated contextual rewards between each client in the client set and the corresponding edge server and the training quality of each client, and making the client participating in training train according to the received training model;

[0076] Step S3, updating the estimated contextual rewards between the client and the corresponding edge server and the training quality according to the training result of the client.

[0077] The embodiments of the present application will be described in further detail below with reference to the drawings.

[0078] As Figure 2As shown, in this embodiment, there is one center server, s edge servers and n clients. Each edge server covers a certain number of clients, which can be represented by φ a,t The set of clients within the coverage of edge server a in the t-th training round can be represented by φ However, the clients in the overlapping area can only participate in the training of one edge server in each training round, so the edge server for the clients in the overlapping area needs to be assigned.

[0079] As shown in Figure 2 and Figure 3 The training model is uploaded to the central server or each edge server before the start of each training round. When the training model is in the central server, the central server initializes the parameters ω0of the training model and distributes them to each edge server. The edge server sets the model parameters to ω0and initializes the parameters

[0080] In this embodiment, the t-th training round is taken as an example for illustration.

[0081] Specifically, each edge server assigns an edge server to each client in the overlapping area according to the estimated context reward between the client and each overlapping edge server to determine the set of clients in each edge server. The specific steps include:

[0082] Determine the set of edge servers overlapping in the edge server overlapping area where the client is located; calculate the estimated context reward between the client and each edge server in the set of edge servers according to the context vector between the client and each edge server in the set of edge servers and the context parameters of each edge server:

[0083]

[0084] According to the formula Assign the client to the edge server with the maximum estimated context reward;

[0085] Where i is the client, t is the training round, a is the edge server to which the client i is assigned in the t-th training round, P t (i) is the set of edge servers of client i, r i,a,tis the estimated context reward between client i and edge server a in the tth training round, X i,a,t is the context vector between client i and edge server a in the tth training round, is the context parameter of edge server a after the (t-1)th training round, CB r,i is the confidence bound of the estimated context reward.

[0086] It is worth noting that, for example, client i is a client in the edge server overlapping area, which includes edge servers al, a2, a3, then the corresponding edge server set of the client is {al, a2, a3}. Each client in the overlapping area can be added to the processing set 2 in the format of (i {al, a2,...}), where i is the number of the client, and {al, a2,...} is the set of edge servers that the client i is in at the same time.

[0087] It is understood that the context vector X between the client and the edge server in the embodiment is composed of several context information, which can include computing power, bandwidth, and distance from the server, that is, X = [computing power, bandwidth, distance], and we can define the context vector of client i observed by edge server a in the tth round as X i,a,t The upper and lower vectors are known values that can be obtained. Each edge server has a context parameter bound to the edge server, and the context parameter of the edge server is updated according to the training of the clients in the edge server in the last training round, which reflects the training effect to a certain extent, θ a,t is the context parameter of edge server a after the tth training round. The context reward between the client and the edge server in the tth training round can be estimated according to the context information of the client in the tth training round and the context parameter of the edge server obtained after the last training, for example, the estimated context reward in the tth training round is

[0088] CB r,i is the confidence bound of the estimated context reward, which is calculated according to the context vector of client i and the noise during training, etc. The specific calculation method is known to those skilled in the art and will not be repeated here.

[0089] The context reward can reflect the total delay between client i and edge server a, and the larger the context reward, the smaller the delay, so the client can be assigned to the edge server with the largest context reward, i.e., the smallest total delay, for training to improve training efficiency.

[0090] It is worth noting that after assigning clients in overlapping areas to edge servers, the clients are assigned to the client set of the edge server and removed from the sets of other edge servers, thus...

[0091] Specifically, the clients participating in training are determined based on the estimated contextual rewards of each client in the client set with the corresponding edge server and the training quality of each client. These clients include:

[0092] The clients in the client set are sorted according to the estimated context reward, and it is determined whether the number of clients in the client set is greater than a first preset number. If so, the clients with the highest estimated context reward are sorted according to their UBC training quality, up to the first preset number. Otherwise, all clients in the client set are sorted according to their UBC training quality. It is then determined whether the number of clients sorted according to their UBC training quality is greater than a second preset number. If so, the clients with the highest UBC training quality, up to the second preset number, are selected for training. Otherwise, all clients in the client set are selected for training. The first preset number is greater than the second preset number.

[0093] As an example, in this embodiment, the first preset quantity is set to 8, and the second preset quantity is set to 4. Edge server a assigns its corresponding client set φ based on the estimated context reward. a,t The clients in the list are sorted, and the 8 clients with the highest estimated context reward are selected. If φ a,t If the number of clients is less than 8, all clients are selected. Then, the selected clients are sorted according to their UCB learning quality, and the 4 clients with the highest UCB learning quality are selected for training. If the number of selected clients is less than 4, all clients participate in training, and S is defined. a,t Let be the set of clients participating in the training in the t-th round of edge server a.

[0094] After determining the set of clients participating in the training, each edge server distributes the initialized training model to the corresponding client. Upon receiving the training model, the client first calculates the loss value L of the received model. s Then, independently using its own training dataset D i Complete the preset training for E epochs, update the local model parameters ω, and calculate the loss value L during the training process. e .

[0095] As an example, the method for calculating the loss value of the training model is as follows: Where ω is the parameter of the training model, L i(ω) the model loss value calculated by the client i, L is the training loss function, Di is the data set of the client i participating in the training, and ε is the sample in the training data set. The loss value of the training model before training and the loss value of the training model after training can be calculated according to the formula.

[0096] Specifically, the training quality of the corresponding client is updated according to the training result of the client, including:

[0097] According to the loss value of the training model after the training and the loss value of the training model before the training of the client after completing the training of the round, the training quality of the client in the round is determined.

[0098] q i,t = (L s -L e )*D i,t

[0099] Wherein, q i,t is the training quality of the client i in the tth round of training, L s is the loss value of the training model in the (t-1) th round of training, L e is the loss value of the training model in the tth round of training, and D i,t is the data set of the client i in the tth round of training.

[0100] After each client completes the local training, the model parameters and the training quality q i,t of the tth round are uploaded to the corresponding edge server, and the edge server receives the training quality of the tth round of all the clients and processes it, scales the value of the training quality of the client in the tth round to a preset range, and obtains the training quality of the client in the tth round after numerical scaling The preset range in this embodiment is [0, 1], and the specific scaling process is as follows:

[0101]

[0102]

[0103]

[0104] Wherein, S a,t is the generated set of participating training clients corresponding to the edge server a.

[0105] Further, the edge server aggregates the overall training quality of all clients in the set of participating training clients corresponding thereto. Respectively, z i,t and represents the number of times that the client i participates in training and the overall learning quality up to the t-th round. Assuming that the client i is within the range of the edge server a at the t-th round, for the client i, the number of times that it participates in training z i,t The updating process is:

[0106]

[0107] The overall learning quality and The updating process of is that the edge server determines the overall training quality of the client up to the current training round according to the number of times that the client participates in training and the training quality of the client in the current round of training:

[0108]

[0109] wherein, is the overall training quality of the client i within the range of the edge server a up to the t-th round of training, is the training quality of the client i after numerical scaling at the t-th round of training, z i,t is the number of times that the client i participates in training up to the t-th round of training, a is a weight parameter of, a weight parameter of, and a < β;

[0110] The UCB training quality of the client is updated according to the overall training quality of the client up to the current training round:

[0111]

[0112] wherein,

[0113]

[0114] wherein, is the UCB training quality of the client i within the range of the edge server a at the t-th round of training, u i,t is the overall training quality of the client i at the t-th round of training the difference between the UCB training quality and the overall training quality, and K is the number of clients within the range of the edge server a participating in the current round of training.

[0115] Further, the client uploads the data to the corresponding edge server after the training is completed, and the time at which the client uploads the data to the edge server is determined as the time at which the client completes the training.

[0116] Specifically, the estimated context reward of the corresponding client is updated according to the training result of the client, including:

[0117] The start time of the current round of training is t t,s , and the time at which the edge server receives the feedback parameter of the client i is tt,i,r , then Δt t,i = t t,i,r -t t,s is the total time consumption of the t-th round of training of the client i.

[0118] The actual context reward of the client is determined according to the total time consumption of the current round of training of the client:

[0119]

[0120] wherein r′ i,a,t is the actual context reward of the client i in the server a in the t-th round of training, v is any one of the clients participating in training in the edge server a in the t-th round of training, S a,t is the set of the clients participating in training.

[0121] The context parameters of the corresponding edge server are updated according to the context vector of the client and the actual context reward:

[0122]

[0123] wherein,

[0124]

[0125]

[0126] wherein θ a,t is the updated context parameters of the edge server a in the t-th round of training, A a,t is the first update parameter of the context parameters of the edge server a in the t-th round of training, b a,t is the second update parameter of the context parameters of the edge server a in the t-th round of training, wherein, includes x i,a,t and r′ i,a,t , is the historical data of the edge server a up to the t-th round of training, and I is the unit matrix.

[0127] It is worth noting that after obtaining the updated context parameters of the edge server, the estimated context reward between the client and the edge server can be updated according to the updated context parameters of the edge server, i.e., the context reward in the next round (t+1) of training is obtained according to the formula

[0128] It is worth noting that after each round of training is completed, the client participating in training sends the training model update parameter to the corresponding edge server; and the edge server aggregates and updates the training model according to the received training model update parameter.

[0129] Preferably, T​a After each round of training, all edge servers upload the latest model to the central server, for example, every 3 rounds of training, all edge servers upload the latest model to the central server. After the central server aggregates the models of all edge servers, it reissues them to each edge server.

[0130] In one specific embodiment, using the MNIST dataset, experiments are conducted at a noise ratio of 0 to 100%, where the noise ratio refers to the proportion of clients with 50% incorrect labels. s = 4 edge servers and n = 40 clients are used. The context information used here is X i,a,t = [dist, cmp_capcity, bandwidth] T , where dist is the distance between client i and edge server a (which can be calculated by providing positioning information), cmp_capcity is the computing power of the client, and bandwidth is the bandwidth. Each edge server independently selects K = 4 clients to participate in training every T a = 3 rounds, and the maximum number of training rounds is T max = 150, and the accuracy requirement is 98%.

[0131] Step 1: The central server initializes the model parameters ω0 and distributes them to each edge server. The edge server sets the model parameters to ω0 and initializes the parameters

[0132] Step 2: Determine which edge server each client in the overlapping area specifically belongs to according to the method described above, so that

[0133] Step 3: Each edge server a independently selects the top 8 clients from its client set φ a,t according to the context reward value, if the number of clients in φ a,t is less than 8, then all are selected, and then the top 4 clients are selected for training using the UCB learning quality of the selected clients, if the number of selected clients is less than 4, then all clients participate in training, we define the set of clients participating in training in the edge server as S a,t , and the edge server a issues the model to each client in S a,t .

[0134] Step 4: Before training, each client calculates the loss value L s , then independently completes E cycles of training using its own data, and updates the local model parameters ω and recalculates the loss value L eand then calculates the learning quality of this time.

[0135] Step 5: The edge server aggregates the model parameters from all clients and updates z i,t , According to the time of receiving each client uploaded model and historical data, the context parameter θ a,t is updated according to the above method.

[0136] Step 6: If the current round t mod 3 = 0, each edge server uploads the latest model parameters to the central server for aggregation, and after the central server completes the aggregation and issues the model, the server updates the local model.

[0137] Step 7: Repeat steps 2 to 6 until the training round t = 150 or the model accuracy reaches the preset 98%.

[0138] The embodiments of the present application reasonably determine the attribution of the clients in the overlapping area through the context information of the clients, and consider the delay and quality factors when selecting the clients. The method considers the instability of the clients, and can adjust the selection in time when the clients change. Then the historical quality information is used to select a certain number of clients for the edge server to train by using the greedy algorithm, so that the training time and training quality of the clients can be effectively guaranteed.

[0139] Please refer to Figure 4 , Figure 4 A schematic block diagram of a client training device based on hierarchical federated learning provided by the embodiments of the present application is shown.

[0140] As shown in Figure 4 , the device comprises:

[0141] The allocation module is configured to allocate the edge servers for the clients in the overlapping area according to the estimated context rewards between each client in the overlapping area and each overlapping edge server, to determine the client set in each edge server.

[0142] The determination module is configured to determine the client participating in training according to the estimated context rewards between each client in the client set and the corresponding edge server and the training quality of each client, and make the client participating in training train according to the received training model.

[0143] The update module is configured to update the estimated context rewards and the training quality between the client and the corresponding edge server according to the training result of the client.

[0144] The allocation module is further configured to:

[0145] determining a set of edge servers overlapped in an edge server overlapping area where the client is located;

[0146] calculating an estimated context reward between the client and each edge server in the set of edge servers according to a context vector between the client and each edge server and a context parameter of each edge server;

[0147]

[0148] according to the formula assigning the client to an edge server with the largest estimated context reward;

[0149] wherein i is the client, t is the training round, a is the edge server to which the client i is assigned in the tth training round, P t (i) is the set of edge servers of the client i, r i,a,t is the estimated context reward between the client i and the edge server a in the tth training round, X i,a,t is the context vector between the client i and the edge server a in the tth training round, is the context parameter of the edge server a after the (t-1)th training round, CB r,i is a confidence bound of the estimated context reward.

[0150] The determining module is further configured to:

[0151] sort the clients in the set of clients according to the estimated context reward, and determine whether the number of clients in the set of clients is greater than a first preset number;

[0152] If yes, sort the first preset number of clients with the highest estimated context reward according to the UBC training quality of the clients, otherwise, sort all the clients in the set of clients according to the UBC training quality of the clients;

[0153] determine whether the number of clients sorted according to the UBC learning quality of the clients is greater than a second preset number;

[0154] If yes, select the second preset number of clients with the highest UBC training quality to participate in training, otherwise, select all the clients in the set of clients to participate in training;

[0155] generate a corresponding set of clients participating in training according to the clients participating in training in the edge servers;

[0156] The first preset number is greater than the second preset number.

[0157] The updating module is further configured to:

[0158] determine the training quality of the client in the current round of training according to the loss value of the training model after the client completes the current round of training and the loss value of the training model before the training:

[0159] q i,t = (L s -L e )*D i,t

[0160] wherein q i,t is the training quality of the client i in the tth round of training, L s is the loss value of the training model in the (t-1)th round of training, L e is the loss value of the training model in the tth round of training, and D i,t is the data set of the client i in the tth round of training;

[0161] scaling the value of the training quality q i,t of the client i in the tth round of training to a preset range to obtain

[0162] determine the overall training quality of the client up to the current round of training according to the number of times the client participates in the training and the training quality of the client in the current round of training;

[0163]

[0164] wherein is the overall training quality of the client i in the edge server a range up to the tth round of training, is the training quality of the client i after numerical scaling in the tth round of training, z i,t is the number of times the client i participates in the training up to the tth round of training, a is a weight parameter of a weight parameter of

[0165] update the UCB training quality of the client according to the overall training quality of the client up to the current round of training:

[0166]

[0167] wherein

[0168]

[0169] wherein is the UCB training quality of the client i in the edge server a range in the tth round of training, u i,t is the overall training quality of the client i in the tth round of training The difference between the UCB training quality, K is the number of clients participating in the current round of training within the range of edge server a.

[0170] The updating module is further configured to:

[0171] Determine the actual context reward of the client according to the total time consumed by the client in the current round of training:

[0172]

[0173] Wherein, r' i,a,t is the actual context reward of the client i in the edge server a in the tth round of training, v is any client participating in training in the edge server a in the tth round of training, Δt t,i is the total training time consumed by the client i in the tth round of training, S a,t is the set of participating training clients corresponding to the edge server a;

[0174] Update the context parameters of the corresponding edge server according to the context vector and the actual context reward of the client:

[0175]

[0176] Wherein,

[0177]

[0178]

[0179] Wherein, θ a,t is the updated context parameter of the edge server a in the tth round of training, A a,t is the first update parameter of the context parameter of the edge server a in the tth round of training, b a,t is the second update parameter of the context parameter of the edge server a in the tth round of training, wherein, Including x i,a,t and r' i,a,t , is the historical data of the edge server a up to the tth round of training, I is the unit matrix;

[0180] Update the estimated context reward between the client and the edge server according to the updated context parameter of the edge server.

[0181] The device is further configured to:

[0182] Initialize the training model through the central server, and distribute the training model to each edge server;

[0183] Each edge server distributes the initialized training model to the corresponding participating training client.

[0184] The device is further configured to:

[0185] After each round of training is completed, the client participating in the training sends the training model update parameters to the corresponding edge server.

[0186] The edge server aggregates and updates the training model according to the received training model update parameters.

[0187] The device is further configured to:

[0188] Every interval of a preset number of training rounds, each edge server uploads the latest aggregated training to the central server for aggregation.

[0189] It should be noted that, for the convenience and brevity of description, the specific working processes of the above-described device, modules and units can refer to the corresponding processes in the foregoing embodiments, which will not be described here.

[0190] The client training device based on hierarchical federated learning provided by the above-described embodiments can be implemented in the form of a computer program, which can run on a computer device as shown in Figure 5 .

[0191] Please refer to Figure 5 , Figure 5 is a structural schematic block diagram of a computer device provided by an embodiment of the present application. The computer device can be a server.

[0192] It should be noted that, in this document, the terms "comprise", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or system including the element.

[0193] The above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A client training method based on layered federated learning, characterized in that, The method comprises the following steps: allocating edge servers for clients in the overlapping area according to the estimated context rewards between each client and each overlapping edge server in the overlapping area, to determine a client set in each edge server; determining clients participating in training according to the estimated context rewards between each client in the client set and the corresponding edge server and the training quality of each client, and making the clients participating in training train according to the received training model; updating the estimated context rewards between the clients and the corresponding edge servers and the training quality of the clients according to the training results of the clients; wherein the step of allocating edge servers for clients in the overlapping area according to the estimated context rewards between each client and each overlapping edge server in the overlapping area comprises: determining a set of overlapping edge servers in the edge server overlapping area where the client is located; calculating the estimated context rewards between the client and each edge server in the set of edge servers according to the context vector between the client and each edge server in the set of edge servers and the context parameters of each edge server: According to the formula Clients are assigned to edge servers where the estimated context reward is maximized. Where i represents the client and t represents the training round. For the edge server assigned to client i during the t-th round of training, For client i, the set of edge servers For the client i and the edge server during the t-th round of training The estimated contextual reward between them For the client i and the edge server during the t-th round of training Context vectors between For the first Edge server after round training Context parameters, Confidence bounds for estimating contextual rewards; wherein the step of determining clients participating in training according to the estimated context rewards between each client in the client set and the corresponding edge server and the training quality of each client comprises: sorting the clients in the client set according to the estimated context rewards, and determining whether the number of clients in the client set is greater than a first preset number; if yes, sorting the first preset number of clients with the highest estimated context rewards according to the UBC training quality of the clients, otherwise, sorting all the clients in the client set according to the UBC training quality of the clients; determining whether the number of clients sorted according to the UBC learning quality of the clients is greater than a second preset number; if yes, selecting the second preset number of clients with the highest UBC training quality to participate in training, otherwise, selecting all the clients in the client set to participate in training; generating a corresponding client set participating in training according to the clients participating in training in the edge server; wherein the first preset number is greater than the second preset number.

2. The client training method based on layered federated learning according to claim 1, characterized in that, updating the training quality of the corresponding client according to the training result of the client comprises: determining the training quality of the client in the current round of training according to the loss value of the training model after the client completes the current round of training and the loss value of the training model before training: wherein, is the training quality of the client i at the t-th round of training, is the loss value of the training model at the t-1-th round of training, is the loss value of the training model at the t-th round of training, is the data set of the client i at the t-th round of training; The training quality of client i during the t-th round of training The value is scaled to a preset range to obtain ; determining the overall training quality of the client up to the current training round according to the number of times the client participates in training and the training quality of the client in the current round of training; wherein, is the overall training quality of the client i within the edge server up to the t-th round of training, is the training quality of the client i after numerical scaling up to the t-th round of training, is the number of times the client i participated in training up to the t-th round of training, is the weight parameter of is the weight parameter of is the weight parameter of is the weight parameter of ; updating the UCB training quality of the client according to the overall training quality of the client up to the current training round: wherein wherein, is the UCB training quality of client i in the edge server range at the t-th round of training, is the overall training quality of client i at the t-th round of training the difference between the UCB training quality and the overall training quality, K is the number of clients participating in the current round of training in the edge server range.

3. The client training method based on layered federated learning according to claim 2, characterized in that, updating the estimated context rewards of the corresponding client according to the training result of the client comprises: determining the actual context rewards of the client according to the total time consumption of the client in the current round of training: wherein, is the actual context reward of client i in server at the t-th round of training, v is the actual context reward of any one of the clients participating in training in edge server at the t-th round of training, is the total training time of client i at the t-th round of training, is the set of clients participating in training corresponding to edge server a; updating the context parameters of the corresponding edge server according to the context vector of the client and the actual context rewards: wherein in, The edge server updated during the t-th round of training Context parameters, For the edge server during the t-th round of training The first update parameter of the context parameters, For the edge server during the t-th round of training The second update parameter of the context parameters, where, include and , Let I be the historical data of edge server a up to the t-th round of training, and let I be the identity matrix; updating the estimated context rewards between the client and the edge server according to the updated context parameters of the edge server.

4. The client training method based on layered federated learning according to claim 1, wherein, Before the participating clients are determined according to the estimated contextual rewards of each client in the client set and the corresponding edge server and the training quality of each client, and the participating clients are caused to train according to the received training model, the method further comprises: the central server initializes the training model, and distributes the training model to each edge server; each edge server distributes the initialized training model to the corresponding participating client.

5. The client training method based on layered federated learning according to claim 4, characterized in that, The method further comprises: After each round of training is completed, the participating client sends the training model update parameter to the corresponding edge server; the edge server aggregates and updates the training model according to the received training model update parameter.

6. The client training method based on layered federated learning according to claim 5, characterized in that, The method further comprises: Every interval of a preset number of training rounds, each edge server uploads the latest aggregated training to the central server for aggregation. 7.A client training apparatus based on layered federated learning, characterized in that, Comprise: an allocation module configured to allocate edge servers to clients in an overlapping area according to estimated contextual rewards between each client in the overlapping area and each overlapping edge server, to determine a client set in each edge server; a determination module configured to determine participating clients according to the estimated contextual rewards of each client in the client set and the corresponding edge server and the training quality of each client, and cause the participating clients to train according to the received training model; an update module configured to update the estimated contextual rewards between the client and the corresponding edge server and the training quality according to the training result of the client; wherein the allocation module is further configured to: determine a set of overlapping edge servers in the edge server overlapping area where the client is located; calculate the estimated contextual rewards between the client and each edge server in the set of edge servers according to the context vector between the client and each edge server in the set of edge servers, in combination with the context parameters of each edge server: According to the formula Clients are assigned to edge servers where the estimated contextual reward is maximized. Where i represents the client and t represents the training round. For the edge server assigned to client i during the t-th round of training, For client i, the set of edge servers For the client i and the edge server during the t-th round of training The estimated contextual reward between them For the client i and the edge server during the t-th round of training Context vectors between For the first Edge server after round training Context parameters, Confidence bounds for estimating contextual rewards; wherein the determination module is further configured to: sort the clients in the client set according to the estimated contextual rewards, and determine whether the number of clients in the client set is greater than a first preset number; if yes, sort the first preset number of clients with the highest estimated contextual rewards according to the UBC training quality of the clients, otherwise, sort all the clients in the client set according to the UBC training quality of the clients; determine whether the number of clients sorted according to the UBC learning quality is greater than a second preset number; if yes, select the second preset number of clients with the highest UBC training quality to participate in training, otherwise, select all the clients in the client set to participate in training; generate a corresponding participating client set according to the participating clients in the edge server; wherein the first preset number is greater than the second preset number.

8. A computer device, comprising: The computer device comprises a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the client training method based on hierarchical federated learning according to any one of claims 1-6 are implemented.

Images