Data non-independent identically distributed federal learning system and method

Abstract

The invention discloses a data non-independent identically distributed federal learning system and method. According to the method, the correction item is introduced into client model updating, so that the updating direction is corrected, the model variance is reduced, the convergence speed of a global model is remarkably increased, and meanwhile, the model precision is improved. In addition, the correction item adding strategy of the client is optimized through the game theory, privacy protection and model performance are balanced, and the total training cost is reduced. In the aspect of privacy protection, the correction item replaces traditional noise addition, and the privacy leakage risk of the server side is effectively prevented. In terms of cost control, the calculation and transmission cost is reduced, and the training efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of federated learning, and particularly to a federated learning system and method that simultaneously consider model performance and training cost in a scenario where data is non-independent and identically distributed. Background Art

[0002] Federated Learning is becoming an essential distributed learning paradigm in large-scale machine learning. FedAvg is the most widely used federated learning framework currently. In the FedAvg federated learning framework, each client device sends the locally trained local model to the server, and then the server obtains the updated global model by taking the weighted average of the local model parameters.

[0003] In the Cross-silo federated learning scenario, the gradient information of the client is inevitably exposed to the server side. The server with a "curious" nature can use inversion attacks to steal the sensitive data of users, thus causing a high privacy cost for the client. Therefore, the client will add noise when transmitting model parameters or gradients to defend against the server's data theft behavior.

[0004] During the long-term training process, clients with a selfish nature usually add too much noise to protect their own data privacy. This privacy protection behavior will instead significantly reduce the convergence speed of the global model. However, the heterogeneous distribution of client data will further exacerbate the reduction of the global model's convergence speed. The lower model convergence speed will result in a large amount of computational cost, transmission cost, and model performance cost for the client. To achieve a fast convergence speed and thus reduce the total training cost of the client, the present invention proposes an adaptive correction term addition scheme to achieve the fast convergence of the global model while reducing the total training cost of all clients. Summary of the Invention

[0005] In view of the deficiencies of the prior art, the present invention proposes a non-independent and identically distributed federated learning system and method for data.

[0006] In the first aspect of the present invention, a non-independent and identically distributed federated learning system for data is provided, including:

[0007] A server and multiple client devices, each client device stores a local dataset, and the datasets of different clients are non-independent and identically distributed;

[0008] The server is configured to randomly select multiple clients in each training round and send the global model and global correction term information to the selected clients;

[0009] Each client device is used to update the local model using the local dataset according to the received global model and global correction term information, and add correction terms during the update process to correct the update direction and reduce model variance;

[0010] The client device is also used to send the updated local model and local correction term information back to the server;

[0011] The server is also used to receive the gradient information and correction term information of the client, and update the global model and global correction term information;

[0012] By means of adaptively adding correction terms, the privacy protection of the client and the model convergence speed are balanced, and the total training cost of the client is reduced.

[0013] In a second aspect of the present invention, a data non-independent and identically distributed federated learning method is provided, which includes a server and multiple client devices. Each client device stores a local dataset, and the datasets of different clients are non-independent and identically distributed. The method includes the following steps:

[0014] In each training round, the server randomly selects multiple clients and sends the global model and global correction term information to the selected clients;

[0015] Each client device updates the local model using the local dataset according to the received global model and global correction term information, and adds correction terms during the update process to correct the update direction and reduce model variance;

[0016] Each client device sends the updated local model and local correction term information back to the server;

[0017] The server receives the gradient information and correction term information of the client, and updates the global model and global correction term information;

[0018] By means of adaptively adding correction terms, the privacy protection of the client and the model convergence speed are balanced, and the total training cost of the client is reduced.

[0019] The present invention proposes a federated learning framework with adaptively added correction terms, which effectively solves the problems of slow model convergence and insufficient privacy protection in the scenario of non-independent and identically distributed data. By introducing correction terms in the client model update, the update direction is corrected, the model variance is reduced, the convergence speed of the global model is significantly accelerated, and the model accuracy is improved at the same time. Experiments show that compared with traditional methods, the model convergence speed of the present invention is increased by 84.21%-110.53% and the accuracy is improved by 5.45%-20.83% under the same training conditions.

[0020] In addition, the present invention optimizes the correction term addition strategy of the client through game theory, balances privacy protection and model performance, and reduces the total training cost. In terms of privacy protection, the correction term replaces traditional noise addition, effectively preventing the risk of privacy leakage on the server side. In terms of cost control, it reduces the computational and transmission costs and improves the training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is an illustration of model drift, showing the impact of data heterogeneity on the global model;

[0022] Figure 2 It is a graph of the accuracy increase of the present invention and five other comparative algorithms at different degrees of data heterogeneity;

[0023] Figure 3 It is a comparison chart of the total costs of the present invention and five other comparative algorithms at different degrees of data heterogeneity. DETAILED DESCRIPTION OF THE INVENTION

[0024] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings. On the contrary, the present invention covers any alternatives, modifications, equivalent methods and solutions made within the spirit and scope of the present invention defined by the claims. Further, in order to enable the public to have a better understanding of the present invention, in the following detailed description of the present invention, some specific details are described in detail. Those skilled in the art can fully understand the present invention without the description of these details.

[0025] Aiming at the problem of slow model convergence speed, the present invention corrects the update direction of the client model by adding a correction term, reducing the variance of the client model. Aiming at the problem of user data privacy leakage, the present invention replaces the traditional privacy protection scheme of adding noise by adding a correction term. Through experiments and proofs, it is found that the two have similar privacy protection effects in defending gradient attacks.

[0026] Adding too many correction terms can provide good protection for users' private data, but it will lose some feature information of local data, resulting in a decline in the generalization ability of the global model; adding fewer correction terms can enable the global model to better absorb the feature information of local data while reducing the variance between client models, but the user's private data cannot be well protected. Therefore, the present invention designs a federated learning system and method for adaptively adding correction terms, and each client decides on the decision variable to update the local model.

[0027] The specific technical solution adopted by the present invention is as follows:

[0028] S1. Establish a system model:

[0029] The scenario considered in the present invention is a federated learning scenario consisting of 1 server and N client devices. Each client has its own local dataset. Moreover, the datasets among different users are non-independent and identically distributed, mainly reflected in different degrees of label skew and different numbers of samples.

[0030] The trained model w is composed of L consecutive model layers. In each training round, the server randomly selects M clients to train the global model w r-1 (where r represents the index of the training round). Each client first makes a decision (where indicates whether client n adds a correction term on the l-th model layer in training round r), and then uses the Stochastic Gradient Descent (SGD) method to update the local model Finally, the client sends the updated local model to the server for aggregation operation to obtain a new round of global model w r .

[0031] S2. Training framework of federated learning with correction term alignment:

[0032] The present invention considers a federated learning scenario with data heterogeneity. Therefore, having local data with different distributions will cause the phenomenon of "model drift" in each local model. As Figure 1 shown, it will ultimately lead to problems such as slow convergence of the global model and low model test accuracy. To alleviate the impact brought by data heterogeneity, a correction term is added to the traditional federated learning framework to improve the efficiency of model training.

[0033] Step 1:

[0034] In each training round, the server randomly selects M active clients for training. After selection, the server sends the latest global model and the latest global correction term information to the selected client devices.

[0035] Step 2:

[0036] After receiving the global model and the global correction term information, the active clients start training the local model according to the local dataset. When updating the gradient of the local model, the correction term information is added to correct the update direction of the local model and reduce the gap between local models.

[0037] In one embodiment, the local correction term information is represented by the symbol and the global correction term information is represented by sc r-1 . The specific update formula is as follows:

[0038]

[0039] where η l represents the local learning rate, indicates which model layers of the local model the client decides to add correction term information to in order to balance the training efficiency of the model and the privacy security of users. k represents the index of local iteration.

[0040] After completing the local iteration, the client updates the local correction term information according to the updated local model. The update formula is as follows:

[0041]

[0042] where K represents the total number of local iterations, represents the number of local training samples of client n.

[0043] Step 3:

[0044] After the server receives the gradient information of all clients, it aggregates the gradient information to obtain the updated global model. The aggregation formula is as follows:

[0045]

[0046] where η g represents the global model learning rate.

[0047] After the server receives the local correction term information of all clients, it aggregates the correction term information of all clients and then updates the global correction term information. The update formula is as follows:

[0048]

[0049] Furthermore, in order to illustrate that adding the correction term does not affect the convergence of the global model, the present invention gives the correlation formula between adaptively adding the correction term and the model convergence speed as follows:

[0050]

[0051] where R represents the number of rounds required to reach the target accuracy, represents the average value of user decisions, C 0 represents the difference between the control variance information and the optimal gradient. ζ represents the coefficient of the maximum distance between the model parameters after adding the control variance and those without adding the control variance. σ represents the perturbation value brought by the stochastic gradient descent method. D represents the sum of the absolute distances between the initial model and the optimal model parameters. F represents the difference between the loss of the initial model and the loss of the optimal model. ∈ represents the minimum tolerance error between the final model accuracy and the target model accuracy.

[0052] Step 4:

[0053] If the global model does not reach the target model performance and the total number of training rounds has not been reached, repeat Step1-3. Stop training when the target accuracy and the total number of training rounds are reached, and calculate the total cost of all users.

[0054] The above multi-round federated learning rounds will incur training costs for the clients, including energy consumption costs (training costs, transmission costs), privacy costs, and model accuracy loss costs. The present invention considers the real training scenario, models the three costs respectively, and finally obtains the total cost of each client in each training round.

[0055] S3. Define the cost function:

[0056] S3.1 Model accuracy loss cost

[0057] The difference between the model accuracy required by the client and the test accuracy of the global model is measured by the loss of the global model accuracy. The desired global model accuracy for each client is a fixed value. The test accuracy of the global model in each round is Therefore, the model accuracy loss cost of client i is defined as:

[0058]

[0059] S3.2 Privacy cost

[0060] The client uses local data to update the local model and uploads the updated model to the parameter server for aggregation. A server with a "curious" nature may obtain gradient information based on the model uploaded by the client and use the gradient information to reconstruct the sensitive data of the client, resulting in a potential privacy leakage risk. The present invention uses the method of adding a correction term to protect the data privacy of users. According to the number of model layers added with the correction term and the degree of user data restoration, the exponential regression method is used to model the privacy leakage risk function. The specific formula is as follows:

[0061]

[0062] where a and b are the parameters of the exponential regression fitting function, represents the proportion of the decision made by client n in training round r to the total number of model layers.

[0063] S3.3 Energy consumption cost

[0064] S3.3.1 Training cost

[0065] In each cross-silo FL process, M active clients perform K local iterations according to the local model. In this embodiment, the energy consumption for each local iteration is set as the following formula:

[0066]

[0067] where κ represents the hardware attributes of the client device, represents the workload of each data sample, f n represents the computing frequency of the client n device.

[0068] S3.3.1 Transmission Cost

[0069] Each client uses its local data to perform local model training on the previous global model. After local training, the updated model parameters or gradients are uploaded to the server for aggregation operations. In order to better alleviate the problems brought by data heterogeneity, the client needs to transmit additional correction terms to the server to reduce the variance between all client models. Therefore, the transmission cost of each client training round is set as shown in the following formula:

[0070]

[0071] where B represents the communication bandwidth of each client, h represents the channel gain of the client, represents the transmission power of client n, represents the proportion of the correction term parameters added to the local model by client n, represents the number of model parameters in the l-th layer of the client model.

[0072] Based on the above analysis, the energy consumption cost of each client n in the training round r can be obtained as:

[0073]

[0074] S3.4 Total Cost:

[0075] Generally speaking, the total cost of each client n in the training round r can be calculated by the following formula:

[0076]

[0077] where represents the set of decisions made by all other clients except client n in the previous round.

[0078] S4. Solving for the User's Optimal Solution Using Game Theory:

[0079] In multiple FL training rounds, multiple clients repeatedly perform local training and aggregate the model to obtain the optimal global model. In each FL training round, each client selfishly decides how to add its correction term to protect its privacy. Therefore, the interaction of multiple clients in the FL learning process is modeled as a game theory problem, and the training in multiple stages is modeled as a multi-stage game. Then, the stage game and its corresponding Nash equilibrium (NE), and the multi-stage game and its corresponding subgame perfect Nash equilibrium (SPNE) are defined respectively.

[0080] I. Game 1: Single-stage game:

[0081] ①. Players: The clients selected in a single round.

[0082] ②. Strategies: The correction term addition strategies of each client in the current training round

[0083] ③. Objective function: Each client minimizes the total cost in the current training round.

[0084] The Nash equilibrium solution of the single-stage game: If no client can improve its own benefit by changing its decision again, it means that the corresponding Nash equilibrium solution is obtained for this single-stage game.

[0085] II. Game 2: Definition of multi-stage game

[0086] ①. Players: All clients participating in the entire federated learning.

[0087] ②. Strategies: The set of correction term addition strategies of each client in all training rounds

[0088] ③. Objective function: Minimize the sum of the total costs of all training rounds of each client, that is

[0089] The Nash equilibrium solution of the multi-stage game: In any subgame from training round r′ to training round R in Game 2, there exists an equilibrium solution for each subgame Then the Nash equilibrium solution of Game 2 is defined as:

[0090] S5. Solving the single-stage game

[0091] In the federated learning scenario, the correction term addition strategies of clients are coupled with each other, affecting the convergence performance of the global model, and thus affecting the total training cost of each client. It is difficult to directly solve the Nash equilibrium solution of Game 1. Therefore, the present invention models Game 1 as an ordinary potential game problem, constructs a potential function for Game 1, and obtains the solution of each client by solving the solution of the minimum value of the potential function.

[0092] Furthermore, to prove that Game 1 is an ordinary potential game problem, this application gives the definition of the potential function: an ordinary potential game has the following property. The Nash equilibrium solution of an ordinary potential game coincides with the minimum value of the potential function, that is, as the decision changes, the changing trends of the user's cost function and the potential function are consistent. If Then Therefore, it is necessary to construct a potential function that can satisfy the above properties, so as to prove that the defined game is an ordinary potential game.

[0093] This application gives the following definition of the potential function:

[0094]

[0095] where represents the transmission energy consumption of each client to transmit all model parameters, represents the decisions of M active clients.

[0096] where and the cost function have a consistent changing trend, and have a consistent changing trend. Therefore, it can be shown that Game 1 is a potential game.

[0097] By analyzing the corresponding Hessian matrix of the potential function and proving that it is a positive definite matrix, it can be shown that the potential function has one and only one Nash equilibrium solution.

[0098] This application further details the process of the adaptive addition of correction terms strategy, and the specific process is as follows:

[0099] 1. Each client makes a decision that minimizes the potential function based on the decision made in the previous training round.

[0100] 2. Update the decisions of the latest active clients.

[0101] 3. Determine whether there is a change in the latest decision set; if there is a change, repeat steps 1-2; otherwise, output the decision set.

[0102] After obtaining the Nash equilibrium solution of the single-stage game, the solution of the multi-stage game is composed of the Nash equilibrium solutions of each single-stage game, that is

[0103] Preferably, the DNN model trained by the federated learning framework in S2 is Resnet18, and the dataset is completed on the real-world image classification dataset CIFAR10. Among them, RestNet18 consists of a convolutional layer (CONV) layer, a max pooling layer (POOL) layer, 8 basic blocks, and 1 fully connected layer (FC) layer. These 8 basic blocks are divided into 4 parts, with 1 basic block in each part. The basic blocks in the first, second, third, and fourth parts are respectively two 3*3*64, two 3*3*128, two 3*3*256, and two 3*3*512 convolutional operations; the CIFAR10 dataset has 60,000 32×32 color images, of which 50,000 are used as model training samples, and the training dataset follows the Dirichlet distribution, and 10,000 are used as model evaluation samples. These 60,000 color pictures are divided into 10 categories, with 6,000 in each category. Each color image is labeled as one of the ten categories, such as "cat" or "dog".

[0104] The federated learning environment of this embodiment consists of 1 server and 100 client devices, and the CPU performance of each client is set to κ = 10 -27 , the number of cpu cycles required to process each bit is 40 cycle / bit, the frequency of the cpu is 3 GHz, and the size of each batch is 64×3×32×32 bits. The communication channel between the client and the server is default to be used independently, and the scale M of the transmitted model parameters is 5×10 6 parameters, and the transmission power is The channel bandwidth is B = 1 MHz, and the channel background noise N 0 = -114 dBm / MHz.

[0105] Next, the implementation and implementation results of the comparison algorithm are shown:

[0106] The present invention also implements five baseline algorithms: FedAvg, FedProx, SCAFFOLD, FedPVR, and FedMut. Among them, the FedAvg algorithm updates the model by performing SGD (stochastic gradient descent method) locally on the client; the FedProx algorithm adds a regularization term to align the local model with the global model while performing SGD locally on the client; the SCAFFOLD and FedPVR algorithms add control variance information on the basis of gradient information on the basis of SGD. The former adds control variance information to all layers, and the latter only adds control variance information to the classification layer of the model; the FedMut algorithm performs mutation operations on the global model to make the global model jump out of the narrow local optimal solution and fall into a broader global optimal solution.

[0107] In order to study the influence of different data distributions on the model training speed, Figure 2shows the training efficiency of different algorithms under different degrees of data heterogeneity when following the Dirichlet distribution. The parameters are set to 0.1, 0.5, and 1 respectively. 0.1 indicates a serious degree of data distribution heterogeneity among different clients. Compared with 0.1, 0.5 has a weaker degree of heterogeneity. 1 means that the data among clients follows the same distribution. The more serious the degree of heterogeneity, the more obvious the "model drift" phenomenon, and the lower the training efficiency of the model. From Figure 2 It can be observed that the present invention can not only converge faster in the case of heterogeneous data distribution, but also converge faster in the case of homogeneous data distribution, and the final accuracy is higher than that of other baseline algorithms. The reason is that by adding correction terms in a fine-grained manner, the present invention can not only ensure that the "model drift" phenomenon does not occur, but also enable the global model to fully learn the feature information of local data. From Figure 3 It can be observed that the training cost of the present invention is also much less than that of all other algorithms. This is because the present invention requires fewer training rounds, and the training cost per round is always lower than that of other algorithms.

[0108] The above-described embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the present application for the corresponding technical solutions, and should all be included in the protection scope of the present application.

Claims

1. A data non-independent and identically distributed federated learning system, characterized in that: include: A server and multiple client devices, each client device stores a local data set, and the data sets of different clients are not independent and identically distributed; The server is used to randomly select multiple clients in each training round and send global model and global correction item information to the selected clients; Each client device is used to update the local model using the local data set according to the received global model and global correction item information, and add correction items during the update process to correct the update direction and reduce the model variance; The client device is also used to send the updated local model and local correction item information back to the server; The server is also used to receive the gradient information and correction item information from the client, and update the global model and the global correction item information; By adaptively adding correction items, the client's privacy protection and model convergence speed are balanced, reducing the total training cost of the client.

2. A data non-independent and identically distributed federated learning system according to claim 1, characterized in that: The strategy for adding the correction item is determined by the client independently based on the training status of the local model and the privacy protection requirements.

3. A data non-independent and identically distributed federated learning system according to claim 1 or 2, characterized in that: The total training cost of the client includes model accuracy loss cost, privacy cost and energy consumption cost. By modeling and optimizing these costs, the client training cost is minimized.

4. The data non-independent and identically distributed federated learning system according to claim 1, characterized in that: When the server updates the global model, it aggregates the client's local model parameters using a weighted average approach, where the weight is determined based on the client's local data volume or other relevant factors.

5. The data non-independent and identically distributed federated learning system according to claim 3, characterized in that: The energy consumption cost of the client includes training cost and transmission cost, wherein the training cost is related to the hardware properties, workload and computing frequency of the client device; the transmission cost is related to the communication bandwidth, channel gain and transmission power.

6. The data non-independent and identically distributed federated learning system according to claim 1, characterized in that: The training process of the client is modeled as a game theory problem, and the optimal corrective term addition strategy of the client is determined by solving the Nash equilibrium of the game.

7. The data non-IID federated learning system according to claim 6, characterized in that: The game problem includes a single-stage game and a multi-stage game, wherein the Nash equilibrium of the single-stage game is obtained by constructing a potential function and solving its minimum value, and the Nash equilibrium of the multi-stage game is formed by combining the Nash equilibrium of each single-stage game.

8. A method for federated learning of data with non-independent and identical distribution, comprising a server and multiple client devices, each client device storing a local data set, and data sets of different clients are non-independent and identically distributed, characterized in that: The method comprises the following steps: In each training round, the server randomly selects multiple clients and sends the global model and global correction item information to the selected clients; Each client device uses the local data set to update the local model based on the received global model and global correction item information, and adds correction items during the update process to correct the update direction and reduce the model variance; Each client device sends the updated local model and local correction item information back to the server; The server receives the gradient information and correction item information from the client, and updates the global model and global correction item information; By adaptively adding correction items, the client's privacy protection and model convergence speed are balanced, reducing the total training cost of the client.

Images