Personalized federated learning method based on asynchronous update of model parameters

Abstract

The present invention relates to a federated learning method, in particular to a personalized federated learning method based on asynchronous updating of model parameters. First, a personalized federated learning framework is constructed, which is divided into two parts on the client side, one part updates all model parameters and interacts with the server. The other part fixes the basic layer parameters to update the personality layer and establishes a personalized model; secondly, under this framework, the basic layer of common features and the personality layer of personality features are divided by introducing a distance metric function; finally, a new asynchronous update strategy for model parameters is designed, and the basic layer parameters are transmitted in each round of training from the client to the server, and the personality layer parameters are transmitted every t rounds to realize the asynchronous transmission of the basic layer and personality layer parameters. Simulation is carried out on a public data set, and the effectiveness of the method is proved by experimental results.

Description

Technical Field

[0001] The present invention relates to a federated learning method, in particular to a personalized federated learning method based on asynchronous updating of model parameters. Background Art

[0002] In recent years, with the continuous development of artificial intelligence technology, protecting data privacy and security has become an important development trend. The emergence of federated learning provides an effective way to solve machine learning problems under data privacy conditions. In 2016, Google first proposed federated learning and introduced it in detail through a pioneering paper Konen J, Mcmahan HB, Ramage D, et al. Federated Optimization: Distributed Machine Learning for On-Device Intelligence [J]. 2016. The core idea of ​​federated learning is to train a maximum likelihood model on a server or edge device. Figure 1 This is a diagram of the horizontal federation framework, which consists of four steps. First, each client trains its local model. Second, each client sends the updated model parameters to the server. Third, the server aggregates the parameters sent by each client. Finally, the server sends the aggregated updated parameters to each client. Each client starts the next iteration, and the cycle repeats until the entire training process converges.

[0003] Later, with the continuous deepening of research, in the field of federated learning, Mcmahan HB, Moore E, D Ramage, et al. Communication-Efficient Learning of Deep Networks from Decentralized Data [J]. 2016. proposed the FedAvg algorithm, which uses model averaging on the server to aggregate and update the parameters uploaded by the client, and build a global model from a subset of clients with non-independent and identically distributed data; Li X, Huang K, Yang W, et al. On the Convergence of FedAvg on Non-IID Data [J]. 2019. Further proved the convergence of the FedAvg algorithm on Non-IID data; Tian L, Sahu AK, Zaheer M, et al. Federated Optimization for Heterogeneous Networks. 2019. Proposed the Fedprox algorithm, adding an approximation term on the basis of FedAvg to optimize the global model, thereby allowing the differences in local updates of the client; Xie M, Long G, Shen T, et al. Multi-Center Federated Learning[J]. 2020. A federated learning framework with multiple central servers is proposed. Clients with similar model parameters are divided into a central server according to the Euclidean distance to solve the problem of data distribution differences.

[0004] In terms of reducing communication, Reisizadeh A, Mokhtari A, Hassani H, et al. FedPAQ: A communication-efficient federated learning method with periodic averaging and quantization [C] / / International Conference on Artificial Intelligence and Statistics. PMLR, 2020: 2021-2031. proposed the FedPAQ algorithm, which has periodic averaging and quantization functions to reduce communication costs; Dai X, Yan X, Zhou K, et al. Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning [J]. 2019. proposed a general framework that can trade off communication efficiency and gradient accuracy; Wang J, Joshi G. Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms [J]. 2018. proposed to perform multiple rounds of local optimization before sending the local model to the server to reduce communication costs.

[0005] Although the above methods have made great contributions in protecting data privacy and reducing communication, they ignore the performance of local client models during the iteration process. Deng Y, Kamani MM, Mahdavi M. AdaptivePersonalized Federated Learning.2020 proved that when the non-independent distribution of data increases, the generalization error of the global model on the client local data also increases significantly, making it difficult for the trained global model to adapt to the specific data tasks of each client. Therefore, it is urgent for federated learning to perform personalized modeling locally on the client. Summary of the invention

[0006] An object of the present invention is to solve the defects mentioned in the above background technology by proposing a personalized federated learning method based on asynchronous updating of model parameters.

[0007] The technical solution adopted by the present invention is as follows: including weighted aggregation of model parameters according to the data size of each client at the server side, expressed as:

[0008]

[0009] where n k is the sample data size of the kth client, n is the total number of training samples, is the model parameter of the kth client at time t+1; the client uses the gradient descent method to update all parameters of the local model, and the formula is as follows;

[0010]

[0011] The loss function is defined as:

[0012]

[0013] where f k (w) is the loss function of the kth client

[0014] As a preferred technical solution of the present invention: described.

[0015] The server global model is W t , the client local model is W k .

[0016] As a preferred technical solution of the present invention: described.

[0017] For a network model with L layers, the model parameters W of the server are t The expansion is expressed as {(W t )1,(W t )2,...,(W t ) L}.

[0018] As a preferred technical solution of the present invention: described.

[0019] The client model parameters are expressed layer by layer as {(W k )1,(W k )2,...,(W k ) L}.

[0020] As a preferred technical solution of the present invention: described.

[0021] It also includes calculating the cosine similarity between the server and the client at the lth layer, which is expressed as follows:

[0022]

[0023] Among them, (W t ) l is the model parameter of the server layer l, (W k )l Represents the model parameters of the client's lth layer. If P is closer to 1, it means that the parameters of the server's lth layer are more similar to those of the client's lth layer.

[0024] As a preferred technical solution of the present invention: described.

[0025] When there are k clients, the cosine similarity is calculated layer by layer with the following formula:

[0026]

[0027] As a preferred technical solution of the present invention: described.

[0028] Also includes seeking l The average value for K clients is:

[0029]

[0030] judge:

[0031]

[0032] Among them, comparing the change values, if the absolute value of the change of layer l for l-1 is much greater than the absolute value of the change of layer l-1 for l-2; then output l, and according to l, the model W at time t t Divide into base layers and personality layer

[0033] The beneficial effects of the present invention are:

[0034] (1) In order to effectively mine the basic layer and individual layer representing common and individual characteristics in the model, a model parameter stratification method is proposed. By constructing a distance measurement function, the differences between common characteristics and individual characteristics are fully described; at the same time, the threshold is set by the distribution distance between layers to effectively divide the basic layer representing common characteristics and the individual layer representing individual characteristics; experiments have shown that this method can effectively divide the basic layer and the individual layer.

[0035] (2) In order to improve the accuracy of personalized modeling, an asynchronous update strategy for model parameters is proposed. First, based on the principle of updating the base layer first and then the personalized layer, the server global model is updated. In the process of transmitting parameters from the server to the client, the base layer is transmitted in each round of training, and the personalized layer is transmitted every t rounds; secondly, the client local model uses private data to update the personalized layer to establish a personalized model. Finally, compared with existing methods, this strategy can not only improve the accuracy of personalized models, but also significantly reduce communication costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of the horizontal federation framework of the present invention;

[0037] Figure 2 Personalize the federated learning framework in the preferred embodiment of the present invention;

[0038] Figure 3 A diagram of a hierarchical asynchronous update strategy for a DNN model in a preferred embodiment of the present invention;

[0039] Figure 4 A schematic diagram of a data set in a preferred embodiment of the present invention;

[0040] Figure 5 It is a cosine similarity graph of the DNN global model and the client model in the preferred embodiment of the present invention;

[0041] Figure 6 This is a personalized model accuracy diagram for different numbers of personalized layers of the DNN model in the preferred embodiment of the present invention;

[0042] Figure 7 It is a cosine similarity graph of the CNN global model and the client model in the preferred embodiment of the present invention;

[0043] Figure 8 This is a personalized model accuracy diagram for different numbers of personalized layers of the CNN model in the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0044] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Reference Figure 1-8 The preferred embodiment of the present invention provides a personalized federated learning method based on asynchronous updating of model parameters. In this embodiment, a federated average algorithm is first proposed, which is developed to obtain the central prediction model of Google applications and can be embedded in mobile phones to protect user privacy. The model parameters are weighted and aggregated on the server side according to the data size of each client, which is expressed as:

[0046]

[0047] where n k is the sample data size of the kth client, n is the total number of training samples, is the model parameter of the kth client at time t+1. The client uses the gradient descent method to update all parameters of the local model. The formula is as follows:

[0048]

[0049] The loss function is defined as:

[0050]

[0051] where f k (w) is the loss function of the kth client.

[0052] Transfer learning is to use the knowledge gained in solving one problem to solve another related problem, rather than learning from scratch. Inspired by model transfer: the base layer in the model is suitable for the common features of different tasks, which means that a part of the model parameters (base layer parameters) represent the common features of different data sets; the individual layer in the model learns the individual features related to a specific data set. Furthermore, transfer learning can achieve personalized customization of federated learning. The server sends part of the parameters of the global model to the client, and the client freezes the model parameters transmitted by the server and uses local data to train the remaining model parameters to achieve personalization. Based on the above inspiration, this embodiment proposes a model layering algorithm, and uses model migration to achieve personalized customization locally on the client.

[0053] The personalized federated learning framework of this embodiment is as follows Figure 2 As shown, where N m Represented as the mth client, (x i ,y i ) m is the private data of the mth client; W m is its model parameter, where Represented as the base layer model parameters of the mth client, are the parameters of the personality layer model, and each client has a different data distribution.

[0054] ClientN m Take this as an example to explain the personalized federated learning framework of this embodiment in detail. Figure 2 ,Client N m There are two parts, one part freezes the base layer parameters And update the personality layer parameters The other part does not freeze the base layer parameters, but updates all parameters W m , and decides whether to send all model parameters to the server at the current moment according to the model parameter asynchronous update strategy. The server aggregates the model parameters sent by the client and then sends them to each client, and this cycle continues.

[0055] In this embodiment, the server global model is W t , the client local model is W k For a network model with L layers, the model parameters W of the server are t The expansion is expressed as {(W t )1,(W t )2,...,(W t ) L}, and the client model parameters are expressed layer by layer as {(W k )1,(W k )2,...,(W k ) L}, calculate the cosine similarity between the server and the client at layer l, expressed as follows:

[0056]

[0057] In the above formula, (W t ) l is the model parameter of the server layer l, (W k ) l Represents the model parameters of the client's layer l. If P is closer to 1, the parameters of the server's layer l are more similar to those of the client's layer l. For k clients, the cosine similarity is calculated layer by layer as follows:

[0058]

[0059] Seeking l The average value for K clients is:

[0060]

[0061] judge:

[0062]

[0063] Here, comparing the change values, if the absolute value of the change of layer l for l-1 is much greater than the absolute value of the change of layer l-1 for l-2, then output l, and according to l, the model W at time t t Divide into base layers and personality layer

[0064] like Figure 3 As shown, the server and client models are DNN. Through the above, the model can be divided into a basic layer W that learns the common features of the task. B , the personality layer W of the learning task personality characteristics P , where the parameter size of the base layer is represented by S B, the size of the personality layer parameter is represented by S P .at the same time Figure 3 An example of federated learning with four clients {A, B, C, D} and a central server is also given to illustrate the asynchronous update strategy of model parameters in this embodiment. The horizontal axis is the number of communication rounds, and the vertical axis represents the four clients {A, B, C, D} and a central server. The gray shaded part at the bottom represents the model parameters involved in the update in each round, and the model parameters outside the gray box at the bottom represent the personality layers of the four clients. In the five rounds of communication (t-4,...,t-1,t), the model parameters of the base layer and the personality layer are only updated simultaneously in the last two rounds. For convenience, the inventor uses freq=2 / 5. Compared with traditional federated learning, it reduces 3*S P Therefore, for neural network models with very large model parameters, this method can significantly reduce the communication cost.

[0065] For the client to fix the basic layer parameters sent by the server, use private data to update the personalized layer model parameters, and establish a personalized model locally, using the following formula:

[0066]

[0067] in is the personality layer of the kth client at time t;

[0068] The server sends the basic layer parameters to each client at time t; is the local data of the kth client; b is the batch size.

[0069] The algorithm is mainly divided into two parts, one of which is executed by the server, such as Algorithm 1.

[0070]

[0071]

[0072] In Algorithm 1, the server initializes the network model parameter W0, sets←freq means that when freq = 2 / 10, sets = {9, 10}, when freq = 4 / 10, sets = {7, 8, 9, 10}, and defines the flag according to the number of rounds; when t = 1, W is executed first. t k ←ClientUpdate(k,W t); secondly, use formula (2) to weight the server model according to the size of the client data; then calculate the cosine similarity of the server model and the client model layer by layer according to formulas (4) and (5), and distinguish the boundary between the basic layer and the personality layer according to formula (7); finally, the basic layer model after distinction When t≠1, only the base layer uploaded by the client is updated, and the updated base layer is returned to the client for the client to update at the next moment and personalize the client's local model.

[0073] The other part is executed by the client, as shown in Algorithm 2.

[0074]

[0075]

[0076] In Algorithm 2, when t=1 or flag=true, the client uses local data to update the model parameters and sends them all to the server; when t≠1, in order to cooperate with the server, the client uses local data to update the base layer and uploads the updated base layer to the server; at the same time, the client uses local data to fix the base layer sent by the server and trains the personality layer to generate a personalized model locally.

[0077] Experimental data: The MNIST dataset is a training dataset with 60,000 handwritten digits. Figure 4 As shown, each image is a grayscale image of 28×28 pixels. In order to generate non-independent and identically distributed data on the client, this embodiment first sorts the data according to the image digital label (0, 1, ..., 9), and divides every 300 data into a group based on the sorting, for a total of 200 groups. Two groups are randomly selected as the local data of a client, that is, the local data of each client is 600.

[0078] Experimental details: (1) First, this embodiment uses the traditional FedAvg algorithm to obtain the accuracy of the global model when facing different client data tasks. (2) Comparative analysis with experiment (1) verifies the effectiveness of the personalized model obtained based on the hierarchical strategy proposed in this embodiment. (3) At the same time, in order to ensure the accuracy of the global model and reduce communication costs, this embodiment also conducts an asynchronous update experiment of model parameters and compares the communication cost with the FedAvg algorithm. The experimental parameters are shown in Table 1.

[0079] Table 1 Experimental parameters

[0080]

[0081]

[0082] Here, this embodiment uses two network structures, one is a 6-layer DNN network structure, and the other is a 10-layer convolutional neural network. DNN and convolutional neural network are common model structures in image classification experiments. The experiments in this embodiment are all conducted on the basic structures of these two networks, which ensures the fairness and comparability of the experiments.

[0083] In the DNN network, the model parameters of the ten clients are compared with the global model parameters layer by layer using cosine similarity. The results are as follows: Figure 5 Then, the accuracy of the personalized model under different numbers of base layers / personalized layers is compared, and the results are as follows Figure 6 , verify the hierarchical strategy based on cosine similarity proposed in this embodiment; finally, as shown in Table 2, a comparison experiment of model accuracy and communication cost of asynchronous update with different frequencies is carried out in the DNN network.

[0084] Table 2 Comparative analysis of DNN model accuracy and communication cost at different frequencies of asynchronous update

[0085]

[0086] Note: freq is the frequency. When it is 2 / 10, it means that in the 10-round federated learning process, all layers are uploaded in the last two rounds. The same is true when it is 4 / 10. The communication cost is the amount of parameter transmission between the server and the client. This embodiment uses the traditional FedAvg algorithm as the benchmark for communication cost.

[0087] Similarly, in the CNN network, the cosine similarity comparison between the client model parameters and the global model parameters is performed layer by layer. The results are as follows: Figure 7 Then, the accuracy of the personalized model under different numbers of base layers / personalized layers is compared, and the results are as follows Figure 8 , which further verifies the effectiveness of the cosine similarity-based hierarchical strategy proposed in this embodiment; finally, as shown in Table 3, a comparison experiment of model accuracy and communication cost of asynchronous update at different frequencies is carried out in the CNN network to increase the scalability of the algorithm.

[0088] Table 3 Comparative analysis of CNN model accuracy and communication cost with different frequencies of asynchronous update

[0089]

[0090]

[0091] (1) Comparative analysis of personalized modeling

[0092] In the DNN network structure, such as Figure 5As shown, the ten curves represent ten clients. At the 3rd and 4th layers, the cosine similarity decreases significantly, so the first three layers are taken as the basic layers (that is, the number of personality layers in Table 2 is 2). Figure 6 It is known that when the number of personality layers is 2, the average accuracy of the personalized model is as high as 90.7%, which is 21.5% higher than FedAvg (the number of personality layers is 0), 7.2% higher than the number of personality layers is 1, and 12.3% higher than the number of personality layers is 3; similarly, in the CNN network structure, Figure 7 As shown in Figure 2, when the cosine similarity drops significantly at 7-8 layers, the first seven layers are taken as the basic layers. Figure 8 It is known that the highest accuracy of the personalized model is 94.8%. The effectiveness of the hierarchical strategy algorithm proposed in this embodiment is verified by conducting simulation experiments on two different network structures.

[0093] (2) Comparative analysis of global models and communication costs

[0094] This embodiment uses the traditional federated average algorithm as the benchmark for communication cost. As shown in Tables 2 and 3, after 10 rounds of federated cycles, the global model accuracy obtained by the federated average algorithm in the DNN and CNN networks is 70.44% and 77.72% respectively, and the communication cost is 1. In the DNN network, when the freq is 2 / 10, the accuracy of the global model is 71.30%, and the communication cost is reduced by 28% compared with the federated average algorithm; when the freq is 4 / 10, the accuracy of the global model is 73.15%, and the communication cost is reduced by 20%. Similarly, in the CNN network, the accuracy of the global model is 79.50%, and the communication cost is reduced by 11%; when the freq is 4 / 10, the accuracy of the global model is 82.59%, and the communication cost is reduced by 15%. In the network model, the number of parameters is usually very large. This embodiment conducts experiments through two different network structures to verify that the asynchronous update strategy proposed in this embodiment effectively reduces the communication cost while ensuring the accuracy of the global model.

[0095] Federated learning often ignores the impact of different layers of feature parameters in the model on personalized modeling, resulting in low model accuracy and high communication costs. This embodiment proposes a personalized federated learning method based on asynchronous update of model parameters. By constructing a distance metric function, the base layer representing common features and the personalized layer representing individual features are divided based on the comparison of the cosine similarity change values ​​between layers; according to the asynchronous update strategy of model parameters, a high-quality personalized model is established locally on the client. By conducting multiple simulation experiments and achieving good results, the effectiveness of the method in this embodiment is proved.

[0096] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0097] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A personalized federated learning method based on asynchronous updating of model parameters, characterized by: This includes weighted aggregation of model parameters based on the data size of each client on the server side, expressed as: Where n k is the sample data size of the kth client, n is the total number of training samples, is the model parameter of the kth client at time t+1; the client uses the gradient descent method to update all parameters of the local model, and the formula is as follows; The loss function is defined as: where f k (w) is the loss function of the kth client; It also includes calculating the cosine similarity between the server and the client at the lth layer, which is expressed as follows: Among them, (W t ) l is the model parameter of the server layer l, (W k ) l Represents the model parameters of the client's lth layer. If P is closer to 1, it means that the parameters of the server's lth layer are more similar to those of the client's lth layer. When there are k clients, the cosine similarity is calculated layer by layer with the following formula: Also includes seeking l The average value for K clients is: judge: Among them, comparing the change values, if the absolute value of the change of layer l for l-1 is much greater than the absolute value of the change of layer l-1 for l-2; then output l, and according to l, the model W at time t t Divided into base layer WB t and personality layer W Pt .

2. The personalized federated learning method based on asynchronous updating of model parameters according to claim 1, characterized in that: The server global model is W t , the client local model is W k .

3. The personalized federated learning method based on asynchronous updating of model parameters according to claim 2, characterized in that: For a network model with L layers, the model parameters W of the server are t The expansion is expressed as {(W t )1,(W t )2,...,(W t ) L }.

4. The personalized federated learning method based on asynchronous updating of model parameters according to claim 3, characterized in that: The model parameters of the client are expressed layer by layer as {(W k )1,(W k )2,...,(W k ) L }.

Images