Shapley Value-Based Method for Processing Federated Learning Mobile Device Distribution Data

Abstract

A method for processing distributed data of mobile devices in federated learning based on the Shapley value. Multiple mobile devices are used to construct a federated learning cluster. In each round of federated learning, the central node applies the Monte-Carlo sampling method to estimate the current federated Shapley value of each federated learning mobile device, and takes the projection of the value in the direction of the change of the global model parameters relative to the initial parameters as its importance and contribution to the model. Selecting federated learning mobile devices to participate in the model training of this round based on the federated Shapley value can effectively accelerate the model convergence speed and improve the final accuracy of the model. The present invention can measure the influence of the data sets of each mobile terminal on the model training process, so as to select devices with high contribution degrees to participate in the training in each round, reduce the data communication overhead, accelerate the convergence speed, and improve the model performance.

Description

Technical Field

[0001] The present invention relates to a technology in the field of distributed data processing, specifically a method for processing distributed data of mobile devices based on Federated Learning and Shapley Value. Background Art

[0002] With the continuous development of mobile networks, the amount of data generated in real time by devices at different network layers is increasing, and the format is becoming more and more complex, requiring an artificial intelligence model to automatically manage the entire network. Traditional model training methods require a central server to collect data from each mobile device and then train the model in a centralized manner. However, transmitting a large amount of data will bring high communication overhead, and uploading user data will violate the privacy of mobile devices.

[0003] The federated learning framework is proposed as a distributed machine learning method to solve the above problems. In each round of training, mobile devices upload model updates instead of user data, reducing communication overhead while avoiding the leakage of customer private data. Limited by the communication bandwidth and computing resources of the central server, only some mobile devices can be selected by the central server to participate in the model training process in each round of federated learning, which greatly limits the amount of training data involved in each model update, thereby reducing the convergence speed and final performance of the model.

[0004] Many works have proved that different training data samples are also of different importance for model training. Selecting only some important samples to participate in training can reduce the training time while ensuring the final accuracy of the model. Some existing works have proposed training data selection methods in a centralized learning scenario, including methods based on LOO (Leave-one-out), methods based on Influence Function, and methods based on Data Shapley. Compared with the first two methods, using the Shapley value as the selection criterion has three satisfactory properties: Efficiency, Symmetry, and Additivity, so it is widely considered to be the fairest and most reasonable selection method. Summary of the Invention

[0005] In view of the defects that the computing power resources of existing mobile devices have no direct relationship with the model performance, the data characteristics of mobile devices and the current model of the central node are not considered, and mobile devices are selected only based on computing power, which cannot theoretically accelerate model convergence and improve model accuracy, the present invention proposes a method for processing distributed data of mobile devices in federated learning based on the Shapley value, which more reasonably applies the Shapley value to federated learning to solve the problem of mobile device selection; a method for estimating the federated Shapley value is proposed to avoid exponential model retraining, and combined with the traditional Monte-Carlo sampling method to further simplify the computational complexity of the Shapley value, so as to measure the influence of the data sets of each mobile terminal on the model training process, and thus select devices with high contribution in each round to participate in training, reduce data communication overhead, accelerate the convergence speed, and improve the model performance.

[0006] The present invention is realized through the following technical solutions:

[0007] The present invention relates to a method for processing distributed data of mobile devices in federated learning based on the Shapley value. A federated learning cluster is constructed by multiple mobile devices. In each round of federated learning, the central node uses the Monte-Carlo sampling method to estimate the current federated Shapley value (Fed-Shapley) of each federated learning mobile device, and projects it in the direction of the change of the global model parameters relative to the initial parameters as its importance and contribution to the model. Selecting federated learning mobile devices based on the federated Shapley value to participate in the model training of this round can effectively accelerate the model convergence speed and improve the final accuracy of the model.

[0008] The Shapley value is: Where: is the federated Shapley value of the federated learning mobile device k in the t-th round; C is the set of all federated learning mobile devices; S is the subset of mobile devices; is the parameter of the global model in the t-th round when only the subset S of mobile devices participates in the federated learning training process, and its value needs to be obtained by retraining the model.

[0009] The federated Shapley value (Fed-Shapley) is estimated in the following way: Where: is the federated Shapley value of the federated learning mobile device k in the t-th round; C is the set of all federated learning mobile devices; S and Q represent subsets of federated learning mobile devices; is the parameter of the global model in the t-th round when only the subset S of federated learning mobile devices participates in the federated learning training process; It represents the parameter change of the model at the t-th round after removing the device subset Q from the total set C of federated learning mobile devices during the training process. Its value can be obtained through the estimation method of the present invention:

[0010]

[0011] Where: C t is the set of mobile devices currently participating in model training; n k is the size of the dataset of the k-th federated learning mobile device; N(C t \Q) is the total dataset size of the device subset C t \Q; m is the number of times the mobile device locally updates the model; I is the identity matrix; η is the learning rate; represents that when the model parameter is the loss function of the model on the dataset D k of device k; is the model after the mobile device k updates i times on its local dataset during the t-th round of federated learning process;

[0012] represents the parameters of the global model only after removing the federated learning mobile device subset Q in the t-th round. Because the calculation of the federated Shapley value needs to traverse each subset of the mobile device set C, an estimation method with a lower time complexity can be obtained by using the Monte-Carlo sampling method.

[0013] The Monte-Carlo sampling mentioned above refers to: randomly selecting multiple permutations that include all federated learning mobiles, and calculating the marginal contribution of each federated learning mobile to the set of mobile devices before it in each permutation in order. Finally, taking the average of the marginal contributions of each federated learning mobile device is the importance of each device, that is, the criterion for mobile device selection.

[0014] The marginal contribution mentioned above refers to: the change in the global model parameters after adding this federated learning mobile device to the training.

[0015] The federated learning mobile device selection algorithm mentioned above is based on the classical concept of game theory, the Shapley Value, and has three similar fairness theorems: when the dataset of device k has no impact on the model performance, its value is 0; when for two devices k1, k2, adding their datasets to any subset results in the same model performance, then k1 and k2 have the same value; the value of the dataset obtained by any multiple evaluation methods is equal to the value of the dataset obtained by combining these evaluation methods.

[0016] The model training specifically includes: 1) The central node distributes the global model to the selected federated learning mobile devices; 2) The federated learning mobile devices update the model according to the local data samples and upload the updated model parameters to the central node; 3) The central node aggregates the model parameters uploaded by each federated learning mobile device into a new round of global model.

[0017] The method specifically includes:

[0018] Step 1. At the beginning stage of the federated learning process, the central node applies the Monte-Carlo sampling method to select p permutations A that contain all the federated learning mobile devices i , i = 0, 1,..., p - 1. For each mobile device A i [j] in each permutation, the central node initializes the estimation of the influence of the device subset composed of this device and the devices before it on the model, that is Q = A i [0:j], j = 0, 1,..., |C|, i = 0,.., p.

[0019] Step 2. In each round of the training process, the participating federated learning mobile device k not only uploads the locally updated model but also uploads the parameter correction terms corresponding to multiple local iterations, specifically: where: m is the number of times the mobile device locally updates the model; I is the identity matrix; η is the learning rate; is the model after the mobile device k updates i times on the local dataset in the t-th round of the federated learning process; is the model on the dataset D k the second derivative of the loss function.

[0020] Step 3. The central node updates the local estimation of the influence of the device subset on the model according to the correction terms uploaded by each device. The update formula is

[0021] Step 4. For each mobile device k, the central node estimates its federated Shapley value and projects it onto the change direction of the global model as a criterion to select the clients participating in the next round of training. The estimation method is to calculate the marginal contribution of this device to the device subset Q before it in p permutations, and the mean value is the estimated value of the federated Shapley value of this device. The marginal contribution is Q is the set composed of all the mobile devices before the mobile device k and the device k in each permutation.

[0022] The present invention relates to a system for implementing the above method, including: a sampling unit, a Shapley value calculation unit, a mobile device selection unit, a distribution unit, a mobile device calculation unit, a collection unit, and a central node calculation unit, where: at the beginning stage of federated learning, the sampling unit, according to multiple full permutations including all devices obtained by sampling, for each device in each permutation, the center initializes the influence of the device subset composed of the device and the devices before it in the permutation on the model, and obtains the initial estimation results of the influence of each device subset on the model; in each round of training, the Shapley value calculation unit calculates the mean marginal contribution of each mobile device based on the influence of the device subsets in each permutation calculated by the sampling unit in the previous round, and obtains the estimated results of the federated Shapley values of each mobile device; the mobile device selection unit calculates the projection value of each device in the direction of the change of the global model parameters as the selection criterion based on the federated Shapley values of each device, and obtains the set of mobile devices participating in model training in this round; the distribution unit distributes the model of the current central node according to the selected set of mobile devices; the mobile device calculation unit performs local model update and calculation of local correction terms based on the received model information, and obtains the updated model parameters and the corresponding correction terms in this round; the collection unit transmits the model parameters and correction terms of each participating device back to the central node; the central node calculation unit performs parameter aggregation processing based on the received updated model parameters, and obtains a new round of model parameters; the sampling unit updates the influence of multiple mobile device subsets in each permutation on the model according to the received correction terms of each participating device.

[0023] Technical effects

[0024] By selecting mobile devices based on the Shapley value in each round of federated learning and estimating the Shapley value of a single mobile device with low complexity, the present invention significantly improves the final accuracy of the global model in federated learning and reduces the model training time compared with the prior art. Description of the drawings

[0025] Figure 1 It is a flowchart of the present invention;

[0026] Figure 2 It is a schematic diagram of the system of the present invention;

[0027] Figure 3 It is the change of the estimation error of the global model parameter change with the number of training rounds when different numbers of devices are removed in the embodiment;

[0028] Figure 4 It is the relationship between the estimation error of the federated Shapley value and the number of training rounds when the model loss function is a convex function in the embodiment, and when the device dataset distributions are the same and the variances are all small, the distributions are different but the variances are all small, and the distributions are different and the variances are large;

[0029] Figure 5 The relationship between the estimation error of the federal Shapley value and the number of training rounds after applying the improved method of the present invention with a large variance in the embodiments;

[0030] Figure 6 The relationship between the error of the present invention's estimation of the federal Shapley value and the number of training rounds after applying the Monte-Carlo sampling method in the embodiments, as well as the change in error after applying the improved method;

[0031] Figure 7a The relationship between the estimation error of the federal Shapley value and the number of training rounds when the model loss function is a non-convex function in the embodiments (the case when the device datasets are independently and identically distributed);

[0032] Figure 7b The relationship between the estimation error of the federal Shapley value and the number of training rounds when the model loss function is a non-convex function in the embodiments (the case when the device datasets are not independently and identically distributed);

[0033] Figure 8 The training curves when different mobile devices are selected to participate in model training according to the federal Shapley value in the embodiments. Detailed implementation manners

[0034] This embodiment includes 8 federated learning mobile devices, and the relevant information of their data is shown in Figure 7. The implementation steps are as follows:

[0035] Step 1: At the beginning stage of the federated learning process, the central node applies the Monte-Carlo sampling method to select p permutations A that contain all the federated learning mobile devices i , where i = 1, 2,..., p. For each device A[j] in each permutation, the central node initializes the estimation of the influence of the device subset composed of this device and the previous devices on the model, that is i Q = A [0:j], i = 1,..., p, j = 1,..., 8.

[0036] Step 2: In each round of the federated learning training process, each participating device k not only uploads the model updated m times on the local data but also uploads the parameter correction terms corresponding to multiple local iterations The correction term is where m is the number of times the device updates the model; I is the identity matrix; η is the learning rate;

[0037] is the model after being updated i times on the mobile device data; D k is the data of the mobile device.

[0037] Step 3. The center updates the estimation of the influence of each stored device subset on the model according to the correction items uploaded by each mobile device, that is Q = A i [0:j[, i = 1,..., p, j = 1,..., 8. The update formula is where: k is a single mobile device; C t is the set of mobile devices participating in the current round of federated learning; n k is the dataset scale of device k; N(C t \Q) is the total aggregation scale of the federated mobile device subset C t \Q; is the correction item uploaded by device k; is the model parameters aggregated by the central node when not considering the model parameter update uploaded by the device subset Q; is the model obtained by the central node aggregating the parameter updates uploaded by all mobile devices participating in training in this round.

[0038] Step 4. The center estimates the federated Shapley value of each mobile device k according to the influence of the stored mobile device subset on the model, and projects it onto the change direction of the global model as a standard to select the clients participating in training in the next round. The estimation method is to obtain the marginal contribution of this device to its previous device subset Q in p permutations, and its average value is the estimated value of the federated Shapley value of this mobile device, that is where: is the federated Shapley value of device k in the current training round; p is the number of permutations obtained by Monte-Carlo sampling; j i，k is the position of mobile device k in the i-th permutation; A i [0:j i，k is the mobile device subset composed of device k and the devices before it in the i-th permutation. The projection value is where is the federated Shapley value of mobile device k, is the model parameters of the current federated learning, is the initial model parameters of the federated learning.

[0039] As shown in Figure 7, the relevant information of the datasets and training models involved in the experimental part.

[0040]

[0041] As Figure 3 shown, in Scenario 1, when the model is logistic regression and the loss function is a convex function, the changes in the model parameters after removing different device subsets Q by this method The variation relationship of the estimation error with the number of training rounds. It proves the theoretical analysis of this embodiment: when the loss function is a convex function, the upper bound of the estimation error of the change of the model parameters by this method has a linear relationship with the number of training rounds t.

[0042] As Figure 4 shown, in Scenario 1, when the model loss function is a convex function and when the device dataset distributions are the same and the variances are all small, the distributions are different but the variances are all small, and the distributions are different and the variances are large, the relationship between the estimation error of the federated Shapley value and the number of training rounds. It proves together with Figure 3 that the greater the distribution difference of the device datasets, the greater the change in the model parameters, further causing the average estimation error of the federated Shapley value to rise from 0.004 to 0.15. When this embodiment replaces a small part of the device datasets with datasets with larger variances, the average estimation error of the federated Shapley value rises to 4.0. This extremely large error comes from the inaccurate estimation of the impact of the federated mobile devices on the model when the number of removed devices is too large, that is, the estimation is very inaccurate.

[0043] To solve the above problems, in this embodiment, when calculating the federated Shapley value of each device through the formula , the case where the number of the removed device subset Q, that is, |Q|, is large is ignored. After improving the estimation method, the estimation error of the federated Shapley value by this method is as Figure 5 shown.

[0044] To find the error caused only by this method's estimation method, this embodiment first considers all possible marginal contributions when calculating the federated Shapley value of each device. It can be seen from Figure 6 that devices with larger data variances also have larger estimation errors. Then, this embodiment combines the estimation method with Monte-Carlo sampling to reduce the time complexity. This method tries different sampling numbers, such as |C| 2 , |C| 3 , where |C| is the number of devices, which is 8 in this embodiment. It can be found from Figure 6 that compared with the error caused by the estimation method, the error brought by sampling can be ignored. To solve the problem of large estimation errors caused by large data variances, this embodiment adopts the improved method described above and tries different |Q| of the mobile devices themselves as thresholds. It can be seen from Figure 6 that the estimation error of the average federated Shapley value drops from 0.6 to 0.2, and the estimation error of the federated Shapley value of mobile devices with large variances drops from 2.5 to 0.3.

[0045] As shown in Figures 7(a) and 7(b), in Scenario 2, when the model is a convolutional neural network, the loss function is non-convex, and the device datasets are independently and identically distributed or not independently and identically distributed, the variation of the estimation error of the device federated Shapley value with the number of training rounds in this method is presented. It verifies the theoretical analysis of this embodiment: when the loss function is non-convex, the estimation error has an exponential relationship with the number of training rounds t.

[0046] As Figure 8 shown, for the experimental effect after applying the federated Shapley value to the selection of participating devices, in this embodiment, devices with larger and smaller projection values of the federated Shapley value in the model update direction are selected for the federated learning model, and the changes in model performance and performance are compared. The experimental results prove that selecting devices with larger Shapley values to participate in training can accelerate model convergence and improve the final accuracy, while selecting devices with smaller Shapley values to participate in training will damage the model's performance and prolong its training time.

[0047] After specific actual experiments, when 8 mobile devices participate in federated learning, the data of each mobile device is the FEMNIST (handwritten digit recognition) dataset and is not independently and identically distributed, and each device participating in model training updates the model 2 times per round, with the learning rate of the model being 0.02, the experimental data that can be obtained is as follows: compared with randomly selecting mobile devices to participate in model training, selecting devices with larger projection values of the federated Shapley value in the model update direction to participate in each round of federated learning can increase the final accuracy of the model from 0.95 to 0.99, and the number of training rounds required for the model to reach the target accuracy (0.95) is reduced from 30 rounds to 13 rounds.

[0048] Compared with the prior art, by selecting devices with larger projection values of the federated Shapley value in the model update direction to participate in each round of federated learning, this method can improve the final accuracy of the model, which is increased from 0.95 to 0.99 in the embodiment, and reduce the number of training rounds required for the model to be trained to the target accuracy. In the embodiment, the number of training rounds required for the model to be trained to the target accuracy (0.95) is reduced from 30 rounds to 13 rounds.

[0049] The above specific implementation can be locally adjusted in different ways by those skilled in the art without departing from the principles and purposes of the present invention. The protection scope of the present invention is subject to the claims and is not limited by the above specific implementation. All implementation solutions within its scope are subject to the constraints of the present invention.

Claims

1. A method for processing distributed data of mobile devices in federated learning based on the Shapley value, characterized in that Construct a federated learning cluster with multiple mobile devices. In each round of federated learning, the central node applies the Monte-Carlo sampling method to estimate the current federated Shapley value of each federated learning mobile device, and takes the projection of it in the direction of the change of the global model parameters relative to the initial parameters as its importance and contribution to the model. Selecting federated learning mobile devices to participate in the model training of this round based on the federated Shapley value can effectively accelerate the model convergence speed and improve the final accuracy of the model; The Shapley value is as follows: , where: is the federated Shapley value of the federated learning mobile device k in the th round; is the set of all federated learning mobile devices; is a subset of mobile devices; When only the mobile device subset participates in the federated learning training process, is the parameter of the global model in the th round, and its value needs to be obtained by retraining the model; The federal Shapley value mentioned above is estimated as follows: , where: is the federal learning mobile device at the round of the federal Shapley value; is the set of all federal learning mobile devices; represents a subset of federal learning mobile devices; is the parameter of the global model at the round when only the subset of federal learning mobile devices participates in the federal learning training process; represents the change in the model's parameters at the round after removing the device subset from the total set of federal learning mobile devices , which is estimated specifically as: , where: is the set of mobile devices currently participating in model training; is the th size of the dataset of the federal learning mobile device; is the total dataset size of the device subset ; is the number of times the mobile device locally updates the model; is the identity matrix; is the learning rate; represents the loss function of the model on the dataset of the device when the model parameter is ; is the model after the mobile device updates times on the local dataset during the round of the federal learning process; represents the parameter of the global model after removing only the subset of federal learning mobile devices in the round; because the calculation of the federal Shapley value requires traversing each subset of the set of mobile devices , a Monte-Carlo sampling method can be used to estimate a lower time complexity estimation method.

2. The method for processing distributed data of mobile devices in federated learning based on the Shapley value according to claim 1, wherein The Monte-Carlo sampling mentioned above refers to: randomly selecting multiple permutations containing all federated learning mobiles, and after calculating the marginal contribution of each federated learning mobile in each permutation to the set of mobile devices before it in the permutation in order, taking the average value of the marginal contributions of each federated learning mobile device as the importance of each device, that is, the standard for mobile device selection.

3. The method for processing distributed data of mobile devices in federated learning based on the Shapley value according to claim 2, wherein The marginal contribution mentioned above refers to: the change of the global model parameters after adding this federated learning mobile device to the training.

4. The method for processing distributed data of mobile devices in federated learning based on the Shapley value according to claim 1, wherein, The described federated learning mobile device selection algorithm is based on the classical concept of game theory, the Shapley Value, and has three fairness theorems similar to it: When the data set of the device has no impact on the model performance, its value is 0; When for two devices , adding their data sets to any subset results in the same model performance, then and have the same value; The value of the data set obtained by any multiple evaluation methods is equal to the value of the data set obtained by combining these evaluation methods.

5. The method for processing distributed data of mobile devices in federated learning based on the Shapley value according to claim 1, wherein The model training mentioned above specifically includes: 1) The central node distributes the global model to the selected federated learning mobile devices; 2) The federated learning mobile devices update the model according to the local data samples and upload the updated model parameters to the central node; 3) The central node aggregates the model parameters uploaded by each federated learning mobile device into a new round of global model.

6. The method for processing distributed data of mobile devices in federated learning based on the Shapley value according to any one of claims 1 to 5, characterized in that specifically Including: Step 1. At the beginning stage of the federated learning process, the central node applies the Monte-Carlo sampling method to select p permutations that contain all the federated learning mobile devices , for each mobile device in each permutation , the central node initializes the estimation of the influence of the device subset composed of this device and the previous devices on the model, that is ; Step 2: In each round of the training process, the participating federated learning mobile device k not only uploads the locally updated model, but also uploads the parameter correction terms corresponding to multiple local iterations, specifically: , where: is the number of times the mobile device locally updates the model; is the identity matrix; is the learning rate; is the -th round of the federated learning process, and the mobile device updates times on the local dataset; is the model on the dataset the second derivative of the loss function; Step 3: The central node updates the estimation of the impact of the device subset on the local model based on the correction items uploaded by each device. The update formula is ; Step 4. For each mobile device k, the center estimates its federated Shapley value and projects it onto the change direction of the global model as the criterion to select the clients participating in the next round of training; the estimation method is to calculate the marginal contribution of this device to the previous device subset in the permutations, and the mean value is the estimated value of the federated Shapley value of this device; the marginal contribution is , where is the set consisting of all mobile devices before mobile device in each permutation.

7. A system for implementing the above-mentioned method for processing distributed data of mobile devices in federated learning based on the Shapley value according to any one of claims 1 to 6, characterized in that Including: A sampling unit, a Shapley value calculation unit, a mobile device selection unit, a distribution unit, a mobile device calculation unit, a collection unit, and a central node calculation unit, where: The sampling unit, at the beginning stage of federated learning, based on multiple full permutations containing all devices obtained by sampling, for each device in each permutation, the center initializes the influence of the device subset composed of this device and the devices before it in the permutation on the model, and obtains the initial estimation results of the influence of each device subset on the model; The Shapley value calculation unit, in each round of the federated learning process, calculates the average value of the marginal contributions of each mobile device based on the influence of the device subsets in each permutation calculated by the sampling unit in the previous round, and obtains the estimated value results of the federated Shapley values of each mobile device; The mobile device selection unit calculates the projection value of each device in the direction of the change of the global model parameters as the selection criterion based on the federated Shapley values of each device, and obtains the set of mobile devices participating in the model training in this round; The distribution unit distributes the model of the current central node according to the selected set of mobile devices; The mobile device calculation unit performs local model update and calculation of local correction terms based on the received model information, and obtains the updated model parameters and the corresponding correction terms for this round; The collection unit sends back the model parameters and correction terms of each participating device to the central node; The central node calculation unit performs parameter aggregation processing based on the received updated model parameters to obtain a new round of model parameters; The sampling unit updates the influence of multiple mobile device subsets on the model in each permutation according to the received correction terms of each participating device.

Images