Federal learning client selection method based on Bandit

Abstract

The invention relates to a federal learning client selection method based on Bandit. The method comprises the steps that S1, a server initializes Beta distribution for a reward score of each client; s2, the server generates a reward value based on Beta distribution of each client, selects a target client and distributes current global model parameters; s3, the target client carries out training locally by using the current global model parameters and feeds back local model parameters and training precision to the server after training is completed, and the server aggregates the local model parameters and updates the global model parameters; s4, the server calculates an accumulated regret value of each target client according to the training precision and updates Beta distribution; and S5, the server sends the updated global model parameters to a new round of selected target client, and the steps S2-S4 are repeated until a preset global model convergence condition is met or a maximum training round is reached. According to the invention, the accuracy of model training can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly to a method for selecting federated learning clients based on Bandit. Background Art

[0002] In the digital age, data privacy and security issues have become the focus of widespread global attention. Traditional centralized learning methods require aggregating user data to a central server, which not only brings huge data transmission costs but also poses a serious risk of data privacy leakage. Therefore, federated learning, as a privacy-preserving distributed machine learning paradigm, has emerged. Federated learning allows multiple clients (such as mobile devices, browsers, or distributed servers) to collaboratively train a shared model while keeping data locally, thus effectively protecting user privacy and reducing data transmission costs. It has become one of the important solutions for the combination of privacy protection and machine learning. However, federated learning faces many challenges in practical applications, especially the heterogeneity problem of client data. The data distributions generated by different clients may vary significantly (non-independent and identically distributed, Non-IID), including uneven data volumes, differences in feature spaces, and inconsistencies in label distributions. This data heterogeneity leads to instability in the model training process, affecting the convergence speed, generalization ability, and final performance of the model. Therefore, how to effectively select clients with higher data quality to participate in model training in a federated learning environment has become a key issue for improving model performance and optimizing resource utilization efficiency.

[0003] Currently, client selection in federated learning usually adopts random selection or polling strategies. These traditional methods cannot effectively evaluate the data quality of clients, easily leading to low training efficiency and even dragging down the overall performance of the model. In addition, factors such as the computing power of clients, network connection stability, and data transmission latency also significantly affect the effect of model training. Therefore, there is an urgent need for an intelligent and dynamic client selection mechanism that comprehensively considers data quality, computing resources, and network environment to improve the training efficiency and final performance of federated learning. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for selecting federated learning clients based on Bandit, which models the client selection problem as a Bandit model and uses the Thompson sampling algorithm to balance historical feedback and overall benefits, thereby effectively selecting client devices that can provide high-quality training data to participate in training. It can effectively improve the accuracy of model training and make the global model trained by federated learning improve in accuracy.

[0005] To achieve the above object, the present invention provides the following solution:

[0006] A Bandit-based method for selecting federated learning clients, including:

[0007] S1. The server initializes a Beta distribution for the reward scores of each client;

[0008] S2. The server generates reward values based on the Beta distribution of each client, selects a target client based on the reward values, and distributes the current global model parameters;

[0009] S3. The target client uses the current global model parameters for local training, and after completing the training, feeds back the local model parameters and training accuracy to the server. The server aggregates the local model parameters and updates the global model parameters;

[0010] S4. The server calculates the cumulative regret value of each target client according to the training accuracy, and updates the Beta distribution based on the cumulative regret value;

[0011] S5. The server sends the updated global model parameters to the newly selected target clients, and repeats S2 to S4 until the preset global model convergence condition is met or the maximum number of training rounds is reached.

[0012] Optionally, in S1, the server initializing a Beta distribution for the reward scores of each client includes:

[0013]

[0014] where Beta(α i , β i ) represents the Beta distribution of client i, C represents the set of all clients, α i is the positive feedback parameter of the Beta distribution of client i, and β i is the negative feedback parameter of the Beta distribution of client i.

[0015] Optionally, in S2, the server generating reward values based on the Beta distribution of each client includes:

[0016] The server performs Thompson sampling on each client and samples a reward value from the corresponding Beta distribution:

[0017] r i ~Beta(α i , β i );

[0018] where r i is the reward value of client i.

[0019] Optionally, in S2, selecting a target client based on the reward value includes:

[0020]

[0021] where C k is the set of target clients, and k is the number of target clients.

[0022] Optionally, in S3, the target client performs mini-batch gradient descent training locally using the current global model parameters and updates the local model parameters to:

[0023]

[0024] where is the updated local model parameter of client i, ω t represents the global model parameter, η is the learning rate, is the gradient operator, represents the loss function of the data D i on client i.

[0025] Optionally, in S3, the server aggregates the local model parameters and updates the global model parameters, including:

[0026]

[0027] where C k is the set of target clients, k is the number of target clients, and ω t+1 represents the updated global model parameter.

[0028] Optionally, in S4, the server calculates the cumulative regret value of each target client according to the training accuracy, including:

[0029] Q = R T1 *R T2 ;

[0030]

[0031] where Q is the cumulative regret, R T1 is the cumulative regret of accuracy, R T2 is the cumulative regret of response latency, represents the average accuracy return of client i up to the t-th round of training, represents the accuracy reward return of client i in the t-th round of training, represents the average latency return of client i up to the t-th round of training, represents the latency reward return of client i in the t-th round of training, T is the total number of training rounds, i ∈ C k , Ck is the set of target clients.

[0032] Optionally, in S4, updating the Beta distribution based on the cumulative regret value includes:

[0033] If the cumulative regret value of the target client does not increase, the server increases the positive feedback parameter of the corresponding Beta distribution;

[0034] If the cumulative regret value of the target client increases, the server increases the negative feedback parameter of the corresponding Beta distribution.

[0035] The beneficial effects of the present invention are:

[0036] (1) By using the Thompson sampling method, the present invention preferentially selects clients with higher data quality to participate in training, models the data quality of clients using the Beta distribution, and dynamically updates the distribution parameters according to the performance feedback during training, enabling the client selection strategy to adapt to the dynamic changes of the data distribution, thereby achieving a more flexible and accurate selection mechanism.

[0037] (2) By using the cumulative regret mechanism to balance historical feedback and overall benefits, the present invention enables the client selection strategy to be continuously optimized during training, effectively avoiding the interference of short-term performance fluctuations on the client selection results, and ensuring the stability and long-term optimal effect of the client selection process.

[0038] (3) Aiming at the problems of dynamic exit and joining of clients during the federated learning process, the present invention can timely adjust the data distribution of clients participating in training, dynamically balance the data quality and quantity, avoid the decline of model accuracy caused by data imbalance, and significantly improve the stability and final accuracy of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0040] Figure 1 is a flowchart of a method for selecting clients in federated learning based on Bandit according to an embodiment of the present invention;

[0041] Figure 2 is a change curve of the global model accuracy and recall rate in different federated learning training stages according to an embodiment of the present invention;

[0042] Figure 3 is a schematic diagram of the federated learning structure according to an embodiment of the present invention. Detailed implementation manner

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts belong to the scope of protection of the present invention.

[0044] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0045] This embodiment provides a method for selecting a federated learning client based on Bandit, as Figure 1 shown, including:

[0046] S1. The server initializes a Beta distribution for the reward scores of each client;

[0047] S2. The server generates a reward value based on the Beta distribution of each client, selects a target client based on the reward value, and distributes the current global model parameters;

[0048] S3. The target client uses the current global model parameters to perform local training, and after completing the training, feeds back the local model parameters and training accuracy to the server. The server aggregates the local model parameters and updates the global model parameters;

[0049] S4. The server calculates the cumulative regret value of each target client according to the training accuracy, and updates the Beta distribution based on the cumulative regret value;

[0050] S5. The server sends the updated global model parameters to the newly selected target client, and repeats S2 to S4 until the preset global model convergence condition is met or the maximum number of training rounds is reached.

[0051] Specifically, the federated learning structure adopted in this embodiment is as Figure 3As shown in the figure, it consists of a server and multiple clients. The server is located in the central position and is responsible for key operations such as the initialization of the global model, the selection of clients, the distribution and aggregation of model parameters, and the update of the Beta distribution. Each client is distributed around the server. They each have local data and train their local models according to the instructions of the server. By adopting the above-mentioned federated learning structure and the method for selecting federated learning clients in this embodiment, the data distribution of the clients participating in the training can be adjusted in a timely manner, the data quality and quantity can be dynamically balanced, the decline in model accuracy caused by data imbalance can be avoided, and the stability and final accuracy of model training can be significantly improved.

[0052] Further, in S1, the server initializes the Beta distribution for the reward score of each client, including:

[0053] The server initializes a Beta distribution for each client i∈C (where C represents the set of all clients and |C| = N), indicating the prior unknown of the data quality of the client:

[0054]

[0055] where Beta(α i ,β i ) represents the Beta distribution of client i, C represents the set of all clients, α i is the positive feedback parameter of the Beta distribution of client i, and β i is the negative feedback parameter of the Beta distribution of client i.

[0056] Specifically, the initial parameters of the Beta distribution are set to the same initial values. In this embodiment, the initial parameters α and β are initialized to 1, indicating the prior unknown and uniform distribution of the data quality of the client.

[0057] Further, in S2, the server generates the reward value based on the Beta distribution of each client, including: The server performs Thompson sampling on each client and samples the reward value from the corresponding Beta distribution:

[0058] r i ~Beta(α i ,β i );

[0059] where r i is the reward value of client i.

[0060] Specifically, the server models the reward distribution of the client using the Beta distribution without uploading the client's data for analysis. The server uses the Thompson sampling method to sample the clients, which can balance the exploration of under-selected clients and the preference for clients with better historical performance.

[0061] Further, in S2, selecting the target client based on the reward value includes: The server selects the Top-k client set with the highest reward value according to the sampled reward value

[0062]

[0063] where C k is the target client set, and k is the number of target clients.

[0064] The server allocates the current global model parameter ω to the selected Top-k clients t , and instructs the clients to perform mini-batch gradient descent training.

[0065] Further, in S3, the target client uses the current global model parameter to perform mini-batch gradient descent training locally, and updating the local model parameter includes:

[0066] Each client i ∈ C k completes E rounds of mini-batch gradient descent training locally and updates its local model parameter

[0067]

[0068] where is the updated local model parameter of client i, ω t represents the global model parameter, η is the learning rate, is the gradient operator, represents the loss function of the data D i on client i. The client feeds back the local model parameter and the corresponding training accuracy Acc i to the server.

[0069] Further, in S3, the server aggregates the local model parameters and updates the global model parameter including:

[0070] The server uses the Federated Averaging algorithm (FedAvg) to aggregate the local parameters of the Top-k clients and updates the global model parameter to:

[0071]

[0072] where Ck is the set of target clients, k is the number of target clients, ω t+1 represents the updated global model parameters.

[0073] Further, in S4, the server calculates the cumulative regret value of each target client according to the training accuracy, including:

[0074] The server calculates the cumulative regret R i and latency Delay i ) of each client i ∈ C k at time t. The accuracy reward return is: i (t). Among them, the accuracy reward return is:

[0075]

[0076] The updated latency reward return is:

[0077]

[0078] where |C k | represents the number of times client i has been selected before the t-th round, i ∈ K. represents the average accuracy return of client i up to the t-th round, represents the average latency return of client i up to the t-th round.

[0079] Then calculate the accuracy cumulative regret as:

[0080]

[0081] The response latency cumulative regret is:

[0082]

[0083] To comprehensively consider the accuracy and response latency of the client, the cumulative regret Q is calculated as:

[0084] Q = R T1 *R T2 .

[0085] Further, in S4, updating the Beta distribution based on the cumulative regret value includes:

[0086] If the cumulative regret value of the target client does not increase, the server increases the positive feedback parameter of the corresponding Beta distribution;

[0087] If the cumulative regret value of the target client increases, the server increases the negative feedback parameter of the corresponding Beta distribution.

[0088] Specifically, if the cumulative regret does not increase, it indicates that its data quality is high, and the server will increase the α parameter of its Beta distribution:

[0089] α i ←α i +1;

[0090] If the cumulative regret increases, it indicates that the client data quality is low or there are latency issues, and the server will increase the β parameter of its Beta distribution:

[0091] β i ←β i +1.

[0092] Furthermore, in S5, the server will send the updated global model parameter ω t+1 to the newly selected Top-k clients in the new round, and repeat S2 to S4 until the global model loss function converges or reaches the predetermined number of training iterations T.

[0093] As Figure 2 , shows the change curves of the global model accuracy and recall rate in different federated learning training stages. It can be seen from the figure that in the initial stage of training, both the accuracy and recall rate of the global model are relatively low. As the training progresses, the accuracy and recall rate gradually increase. In the middle stage of training, the recall rate increases more significantly, while the increase in accuracy is relatively gentle. When the training reaches the later stage, both the accuracy and recall rate tend to be stable and close to the optimal value. This indicates that this method can effectively optimize the performance of the global model, gradually improve the accuracy and recall ability of the model during the training process, and finally reach a better balance state, reflecting the effectiveness and superiority of this method in improving the performance of the federated learning model.

[0094] At the same time, this method also has the following beneficial effects:

[0095] By using the Thompson sampling method to preferentially select clients with higher data quality to participate in training, using the Beta distribution to model the client data quality, and dynamically updating the distribution parameters according to the performance feedback during training, the client selection strategy can adapt to the dynamic changes of the data distribution, thus realizing a more flexible and accurate selection mechanism.

[0096] By using the cumulative regret mechanism to balance the historical feedback and the overall benefit, the client selection strategy is continuously optimized during the training process, effectively avoiding the interference of short-term performance fluctuations on the client selection results, and ensuring the stability and long-term optimal effect of the client selection process.

[0097] Regarding the problem of the dynamic exit and joining of clients in the federated learning process, it can timely adjust the data distribution of the clients participating in the training, dynamically balance the data quality and quantity, avoid the decline of model accuracy caused by data imbalance, and significantly improve the stability and final accuracy of model training.

[0098] The embodiments described above are only descriptions of the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for selecting a federated learning client based on Bandit, characterized in that: include: S1. The server initializes the Beta distribution for each client’s reward score; S2. The server generates a reward value based on the Beta distribution of each client, selects a target client based on the reward value, and distributes the current global model parameters; S3, the target client performs training locally using the current global model parameters, and feeds back the local model parameters and training accuracy to the server after the training is completed. The server aggregates the local model parameters and updates the global model parameters. S4, the server calculates the cumulative regret value of each target client according to the training accuracy, and updates the Beta distribution based on the cumulative regret value; S5. The server sends the updated global model parameters to the target clients selected in a new round, and repeats S2 to S4 until the preset global model convergence condition is met or the maximum number of training rounds is reached.

2. The method for selecting a federated learning client based on Bandit according to claim 1, characterized in that: In S1, the server initializes the Beta distribution for each client's reward score, including: Among them, Beta(α i ,β i ) represents the Beta distribution of client i, C represents the set of all clients, α i is the positive feedback parameter in the Beta distribution of client i, β i is the negative feedback parameter in the Beta distribution of client i.

3. The method for selecting a federated learning client based on Bandit according to claim 2, characterized in that: In S2, the server generates a reward value based on the Beta distribution of each client, including: The server performs Thompson sampling on each client, sampling reward values ​​from the corresponding Beta distribution: r i ~Beta(α i ,b i ); Among them, r i is the reward value of client i.

4. The method for selecting a federated learning client based on Bandit according to claim 3, characterized in that: In S2, selecting a target client based on the reward value includes: Among them, C k is the target client set, and k is the number of target clients.

5. The method for selecting a federated learning client based on Bandit according to claim 1, characterized in that: In S3, the target client performs small batch gradient descent training locally using the current global model parameters, and updates the local model parameters to: in, is the updated local model parameter of client i, ω t represents the global model parameters, η is the learning rate, is the gradient operator, Indicates data D on client i i The loss function of .

6. The method for selecting a federated learning client based on Bandit according to claim 5, characterized in that: In S3, the server aggregates the local model parameters and updates the global model parameters, including: Among them, C k is the target client set, k is the number of target clients, ω t+1 Represents the updated global model parameters.

7. The method for selecting a federated learning client based on Bandit according to claim 1, characterized in that: In S4, the server calculates the cumulative regret value of each target client according to the training accuracy, including: Q=R T1 *R T2 ; Among them, Q is the cumulative regret, R T1 Accumulated regret for accuracy, R T2 Accumulated regret for response delay, represents the average accuracy return of client i until the tth round of training, represents the accuracy reward return of client i in the tth round of training, represents the average latency return of client i until the tth round of training, represents the delayed reward return of client i in the tth round of training, T is the total number of training rounds, i∈C k , C k The target client collection.

8. The method for selecting a federated learning client based on Bandit according to claim 7, characterized in that: In S4, updating the Beta distribution based on the cumulative regret value includes: If the cumulative regret value of the target client does not increase, the server increases the corresponding positive feedback parameter of the Beta distribution; If the cumulative regret value of the target client increases, the server increases the corresponding negative feedback parameter of the Beta distribution.

Images