Information age sensitive federal learning client selection method

Abstract

The invention belongs to the technical field of machine learning, and discloses an information age sensitive federated learning client selection method, which comprises a federated learning system, and the federated learning system comprises the following steps: S01, a server sends a global model obtained by the last round of training to all clients, selecting client subsets which need to collect fresh data in the round, and paying corresponding rewards to the client subsets; s02, the client carries out training of a local model, on the basis of the global model sent by the server, the local model is trained through a stochastic gradient descent method, and then the obtained model is uploaded to the server; s03, after the server collects the local models of all the clients, carrying out weighted aggregation on the local models to obtain a global model of the round, and sending the global model to each client for the next round of training; according to the method, the timeliness of the model obtained by training is effectively improved through the selection method, and the budget limitation is not exceeded.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and in particular relates to an information age-sensitive federated learning client selection method. Background Art

[0002] Federated learning is a distributed machine learning paradigm that has emerged in recent years. It allows many clients to train collaboratively without uploading their local data, thereby obtaining a machine learning model with good performance. In most existing federated learning studies, each client is assumed to have a dataset in advance and always uses the same dataset to train its local model in each round of learning. However, in data-driven federated learning scenarios, clients need to go to specific locations to collect data to build their local datasets for model training. Data-driven and information age-sensitive federated learning systems can be applied to a variety of scenarios, including but not limited to financial market analysis, network security, intelligent transportation systems, and smart cities. The data of these locations will continue to be generated over time, and the distribution of data may shift over time. In this case, if the client keeps using an outdated dataset for training, the resulting model may not be generalized to the latest data. Therefore, the server hopes that clients will use datasets that are as fresh as possible to participate in federated learning training.

[0003] In the process of collecting data on the client side, the clients will inevitably incur some additional costs, such as equipment maintenance and management costs for data collection equipment, and the fresher the data set collected, the higher the cost required. In data-driven federated learning scenarios, the number of clients is very large, and the server budget is usually limited. It is impossible to let all clients collect fresh data in each round of federated learning. However, if only a small number of clients are allowed to collect fresh data in order to save budget, most of the clients participating in federated learning will use outdated data sets for training, resulting in poor performance of the resulting model. At the same time, how to select the clients that need to collect fresh data in each round will affect the selection of clients in subsequent rounds, because the information age of the client data set will increase as federated learning proceeds until it is selected by the server to collect fresh data. The above problems have brought great challenges to data-driven and information-age-sensitive federated learning systems.

[0004] Therefore, how to design a client selection algorithm to maximize the utility of the global model under a limited budget is crucial for federated learning systems that consider data freshness. Summary of the invention

[0005] In view of the above-mentioned technical problems, the purpose of the present invention is to provide an information age-sensitive federated learning client selection method, which effectively improves the timeliness of the trained model through the selection method and ensures that the budget limit is not exceeded.

[0006] The technical solution of the present invention is:

[0007] An information age-sensitive federated learning client selection method includes a federated learning system, the federated learning system includes two components: a server and a client, and performs federated learning with data freshness considered, and introduces an information age indicator to measure the data freshness, where the information age of the data is the time elapsed since the client collected the data;

[0008] The federated learning system includes the following steps:

[0009] S01: The server sends the global model obtained from the previous round of training to all clients, selects a subset of clients that need to collect fresh data in this round, and pays them corresponding rewards;

[0010] S02: The client trains the local model by using the stochastic gradient descent method based on the global model sent by the server, and then uploads the obtained model to the server;

[0011] S03: After collecting the local models of all clients, the server performs weighted aggregation on them to obtain the global model of this round, and sends it back to each client for the next round of training.

[0012] Preferably, the local dataset of client i in round t is The round number of the most recent update is , then its information age That is, the difference between the current round number and the round number, that is, , if the client is selected by the server in each round, the information age of the client becomes 0, otherwise the information age increases by 1.

[0013] Preferably, the total budget for the server to select clients to update data in each round is limited, and only some clients can be selected. represents the strategy of server selection of clients; the system optimization goal is defined as the following problem P1:

[0014]

[0015] in, represents the global model after T rounds, represents the optimal global loss, Indicates whether client i is selected in round t, if yes, it is 1, otherwise it is 0. The next constraint shows the dynamic change of the client information age, and the last constraint is the budget limit.

[0016] Preferably, in each round, the server calculates the Whittle index based on parameters such as the information age of the client, and selects the client based on the index.

[0017] Preferably, the calculation formula of the Whittle index of the client is as follows:

[0018]

[0019] in is the system parameter.

[0020] Preferably, in step S02, the client uses a small batch stochastic gradient descent method to update the local model, and the formula is as follows:

[0021]

[0022] in , It is when executing the 𝑘th step of the mini-batch stochastic gradient descent method from The sampled data, is the learning rate in the tth round, is the loss function of client i in round t. At the same time, is the global model of the previous round sent by the server, The local model obtained by client i after completing training in round t.

[0023] Preferably, in step S03, the server performs weighted aggregation on the local models of all the clients received, and the formula is as follows:

[0024]

[0025] in is the size of the local dataset of client i, and n is the sum of the sizes of local datasets of all clients.

[0026] Compared with the prior art, the advantages of the present invention are:

[0027] (1) The present invention provides an information age-sensitive federated learning client selection method, which ensures the timeliness of the local training model by selecting a suitable client in each round to update its local data set. It can not only ensure that the reward paid by the server to the selected client in each round does not exceed the budget, but also meet the server's requirements for the freshness of the client's local data as much as possible, thereby obtaining a global model with higher timeliness and better applicability to the current data distribution.

[0028] (2) Based on reinforcement learning theory, the client selection problem is modeled as a dynamic multi-armed bandit model with constraints. Each client corresponds to a dynamic rocker. The server selects a client to shake the corresponding rocker. The problem is solved by Lagrange dual method and Markov decision process theory, and the formula of Whittle index is obtained. Based on this, a client selection method is designed. At the same time, the present invention takes into account the timeliness of the client's local data set, which can be more suitable for practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0030] Figure 1 A schematic diagram of the information age-sensitive federated learning system of the present invention;

[0031] Figure 2 It is a schematic diagram of the information age-sensitive federated learning client selection method of the present invention;

[0032] Figure 3 is a schematic diagram of the existence of the platform Steinberg equilibrium under different system parameters of the present invention;

[0033] Figure 4 Schematic diagram of the existence of the worker-Steinberg equilibrium under different parameter a values ​​of the present invention;

[0034] Figure 5 It is a schematic diagram of the existence of the Worker-Steinberg equilibrium under different parameter s values ​​of the present invention. DETAILED DESCRIPTION

[0035] The above scheme is further described below in conjunction with specific examples. It should be understood that these examples are used to illustrate the present invention and are not limited to the scope of the present invention. The implementation conditions adopted in the examples can be further adjusted according to specific conditions, and the unremarked implementation conditions are usually the conditions in routine experiments.

[0036] Reference Figure 1-5 ,The technical solution of the present invention is as follows: an information age-sensitive federated learning client selection method, comprising a federated learning system, the federated learning system comprising two components: a server and a client, performing federated learning with data freshness in mind, and introducing an information age indicator to measure data freshness, the information age of data being the time elapsed since the client collected the data;

[0037] The federated learning system includes the following steps:

[0038] S01: The server sends the global model obtained from the previous round of training to all clients, selects a subset of clients that need to collect fresh data in this round, and pays them corresponding rewards;

[0039] S02: The client trains the local model by using the stochastic gradient descent method based on the global model sent by the server, and then uploads the obtained model to the server;

[0040] S03: After collecting the local models of all clients, the server performs weighted aggregation on them to obtain the global model of this round, and sends it back to each client for the next round of training.

[0041] Client i's local dataset in round t The round number of the most recent update is , then its information age That is, the difference between the current round number and the round number, that is, , if the client is selected by the server in each round, the information age of the client becomes 0, otherwise the information age increases by 1.

[0042] In each round, the server has a limited total budget for selecting clients to update data, and can only select some clients. represents the strategy of server selection of clients; the system optimization goal is defined as the following problem P1:

[0043]

[0044] in, represents the global model after T rounds, represents the optimal global loss, Indicates whether client i is selected in round t, if yes, it is 1, otherwise it is 0. The next constraint shows the dynamic change of the client information age, and the last constraint is the budget limit.

[0045] In each round, the server calculates the Whittle index based on parameters such as the client's information age and selects the client based on this index.

[0046] The calculation formula of the Whittle Index on the client side is as follows:

[0047]

[0048] in is the system parameter.

[0049] Wherein, in step S02, the client uses a small batch stochastic gradient descent method to update the local model, and the formula is as follows:

[0050]

[0051] in , It is when executing the 𝑘th step of the mini-batch stochastic gradient descent method from The sampled data, is the learning rate in the tth round, is the loss function of client i in round t. At the same time, is the global model of the previous round sent by the server, The local model obtained by client i after completing training in round t.

[0052] In step S03, the server performs weighted aggregation on the local models of all the clients received, and the formula is as follows:

[0053]

[0054] in is the size of the local dataset of client i, and n is the sum of the sizes of local datasets of all clients.

[0055] In a further design, the software information age-sensitive federated learning client selection method of the present invention calculates the Whittle index based on parameters such as the information age of the clients before each round of federated learning begins, and then selects the clients based on the index;

[0056] like Figure 2 As shown, the process of this embodiment is as follows:

[0057] (1) The information age of each client is 0 at the beginning. Then the server calculates the Whittle index of each client according to the Whittle index calculation formula, and then sorts the Whittle index of all clients from high to low;

[0058] (2) The server then uses a greedy algorithm to select clients from high to low into a winning set and pays them corresponding rewards to collect fresh data until the remaining budget cannot afford the next client. Figure 2 In the table, the checked mark indicates that the client is selected in this round. In the next round, the information age of the client selected in the previous round is still 0, while other clients still use the old data set of the previous round, so their information age increases by 1;

[0059] (3) The selected client updates the local data set, which is Figure 1 The clients with fresh data in the selected list have no changes in their local datasets. Figure 1The server sends the global model of the previous round to all clients, and then the clients train the local model of this round on the local dataset based on this model;

[0060] (4) After all clients have trained their local models, they will upload them to the server in the cloud. After receiving the local models of all clients, the server aggregates these models to obtain the global model of this round, and then sends the global model back to all clients for the next round of training.

[0061] The experimental test description of this case is as follows Figures 3 to 5 As shown in the figure, this client selection method can not only make the accuracy and loss of the trained global model better than the existing methods, but also minimize the information age of the client while ensuring the budget limit.

[0062] The above examples are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. An information age-sensitive federated learning client selection method, characterized in that: A federated learning system is included, wherein the federated learning system includes two components: a server and a client, and the federated learning is performed while considering the freshness of data, and an information age indicator is introduced to measure the freshness of data, where the information age of data is the time elapsed since the client collected the data; The federated learning system includes the following steps: S01: The server sends the global model obtained from the previous round of training to all clients, selects a subset of clients that need to collect fresh data in this round, and pays them corresponding rewards; S02: The client trains the local model by using the stochastic gradient descent method based on the global model sent by the server, and then uploads the obtained model to the server; S03: After collecting the local models of all clients, the server performs weighted aggregation on them to obtain the global model of this round, and sends it back to each client for the next round of training.

2. The information age-sensitive federated learning client selection method according to claim 1, characterized in that: Client i's local dataset in round t The round number of the most recent update is , then its information age That is, the difference between the current round number and the round number, that is, , if the client is selected by the server in each round, the information age of the client becomes 0, otherwise the information age increases by 1.

3. The information age-sensitive federated learning client selection method according to claim 1, characterized in that: In each round, the server has a limited total budget for selecting clients to update data, and can only select some clients. represents the strategy of server selection of clients; the system optimization goal is defined as the following problem P1: in, represents the global model after T rounds, represents the optimal global loss, Indicates whether client i is selected in round t, if yes, it is 1, otherwise it is 0. The next constraint shows the dynamic change of the client information age, and the last constraint is the budget limit.

4. The information age-sensitive federated learning client selection method according to claim 1, characterized in that: In each round, the server calculates the Whittle index based on parameters such as the client's information age and selects the client based on this index.

5. The information age-sensitive federated learning client selection method according to claim 4, characterized in that: The calculation formula of the Whittle Index of the client is as follows: in is the system parameter.

6. The information age-sensitive federated learning client selection method according to claim 1, characterized in that: In step S02, the client uses a small batch stochastic gradient descent method to update the local model, and the formula is as follows: in , It is when executing the 𝑘th step of the mini-batch stochastic gradient descent method from The sampled data, is the learning rate in the tth round, is the loss function of client i in round t. At the same time, is the global model of the previous round sent by the server, The local model obtained by client i after completing training in round t.

7. The information age-sensitive federated learning client selection method according to claim 1, characterized in that: In step S03, the server performs weighted aggregation on the local models of all clients received, and the formula is as follows: in is the size of the local dataset of client i, and n is the sum of the sizes of local datasets of all clients.

Images