Federal learning model training method and federal learning system based on personalized transformer

Abstract

The application discloses a federated learning system based on personalized Transformer. Another technical scheme of the application provides a federated learning model training method based on personalized Transformer. The application learns a unique self-attention layer for each client, and the parameters of other layers are shared among the clients. In addition, the application trains a hypernetwork on the server to generate a projection matrix in the self-attention layer, so as to obtain queries, keys and values related to the clients. The hypernetwork effectively maintains the parameter sharing among the clients while maintaining the flexibility of the personalized Transformer. Therefore, the application greatly improves the inference and prediction ability of federated learning in a heterogeneous data environment.

Description

TECHNICAL FIELD

[0001] The present application relates to a personalized Transformer-based federated learning model training method and a federated learning system, and belongs to the field of artificial intelligence. BACKGROUND

[0002] Federated learning is a kind of distributed machine learning based on parallel computing, which is composed of a server and several clients. Each client trains a model using its own local data and only sends the trained model back to the server, and the server aggregates the received models to obtain a global model. In this process, the local data of the client is not shared. In this way, federated learning overcomes a series of problems in data privacy and communication overhead. However, when there is data heterogeneity and system heterogeneity between clients, a single global model will be difficult to cope with the local distribution which is completely different from the global distribution. Therefore, personalized federated learning, which can provide personalized solutions for each client, emerges as the times require as an extension of federated learning.

[0003] Most current federated learning frameworks are based on convolutional neural networks, which usually focus on the high-frequency local area of the input. This pattern may be very sensitive to data heterogeneity. Since the features extracted using self-attention have globality, Transformer is often more robust to non-independent and identically distributed data and distribution shift. Based on this, a recent work proposes to use Transformer as the network architecture of federated learning, pointing out that Transformer can more effectively handle heterogeneous data than convolutional neural networks, but the impact of federated learning algorithm on the self-attention mechanism in the aggregation process has not been studied.

[0004] In summary, the main problems faced by current federated learning are:

[0005] (1) Existing methods cannot well handle the problem of data heterogeneity and system heterogeneity between clients, and most of them are based on convolutional neural networks, which are sensitive to non-independent and identically distributed data;

[0006] (2) Existing methods do not have a unified federated learning framework for image and language tasks;

[0007] (3) Existing methods will have a great impact on the self-attention mechanism of Transformer in the aggregation process, and lack a more suitable personalized federated learning framework for Transformer structure. SUMMARY

[0008] The purpose of the present application is to consider the characteristics of self-attention in the Transformer, retain client individualization features, and better capture data heterogeneity and handle the distribution offset problem between clients.

[0009] In order to achieve the above purpose, the technical scheme of the present application discloses a federated learning system based on personalized Transformer, comprising a server and a plurality of clients, characterized in that according to different execution tasks, each client selects a corresponding Transformer structure as a local model, and the model parameter θ i of the client i is decomposed into θ i ={W i ,ξ i}, wherein W i is the projection matrix of the self-attention layer in the Transformer, and ξ i is the parameter of the remaining layers in the Transformer except the self-attention layer.

[0010] The server stores a super network and an embedding vector corresponding to each client;

[0011] In the kth communication round of federated learning:

[0012] The client i downloads the updated aggregated parameter θ of the k-1th communication round from the server The projection matrix W i of the client i is generated by using the super network located in the server , wherein: i is the parameter of the super network; z i is the input of the super network, which is the learnable embedding vector of the client i;

[0013] The client i uses the local data B i to locally train the initialized model , wherein: is the updated model parameter of the client i in the kth communication round; and α is the learning rate; is the gradient operator with respect to the parameter θ i ; represents l(·,·) is the cross-entropy loss function, f(θ i ;·) is the individualized model of the client i, is the feature of the training sample, is the label of the training sample;

[0014] The updated model parameter θ is decomposed into , wherein W ik is the projection matrix of the self-attention layer in the local model, are parameters of the layers in the local model except the self-attention layer;

[0015] The model parameters W i k are reserved locally at the client i for storing the personalized information of the client i; the parameter difference AW i = W i k - W i k-1 and are uploaded to the server for parameter updating and aggregation respectively to obtain the beneficial information on other clients, and the global parameters of the kth communication round Wkare obtained. where N is the number of clients, m i is the local data volume of the client i, and M is the total data volume of all clients.

[0016] Preferably, for the super network parameters on the server and the embedding vector z i of the client i, the following two equations are used for updating respectively:

[0017]

[0018]

[0019] where β is the learning rate, is the gradient operator with respect to the parameter , and is the gradient operator with respect to the parameter z i .

[0020] Preferably, the super network is composed of three fully connected networks, and the last fully connected network is different for each module in the Transformer.

[0021] Another technical solution of the present application is to provide a federated learning model training method based on personalized Transformer, characterized in that it comprises the following steps:

[0022] Step one: in the kth round of training, first randomly select a client from all clients to form a subset C t For the client i in the set C t , update its model parameters W where:

[0023] W i k，0is the projection matrix of the self-attention layer in the Transformer, which is generated by the super network on the server: where, is the updated super network parameter in the k-1th round, is the updated embedding vector of the client i in the k-1th round;

[0024] is the parameter of the remaining layers in the Transformer except the self-attention layer;

[0025] download the global parameters aggregated in the k-1th round from the server

[0026] Step two: for the initialized client i, use the stochastic gradient descent method to locally train its model t rounds according to the training batch B sampled from the local data i , the model parameter is updated with , where: is the updated model parameter of the client i in the kth communication round t rounds, and a is the learning rate; is the gradient operator with respect to the parameter i ; represents l(·,·) is the cross-entropy loss function, f(θ i ;·) is the personalized model of the client i, is the feature of the training sample, is the label of the training sample;

[0027] Step three: after local training, calculate the difference AW i between the parameters W i before and after local training of the client i, i k，t i k ，0 and send AW i and the updated parameter to the server together;

[0028] Step four: the server aggregates the received parameters to obtain the global parameter in the kth round where m i is the local data volume of the client i, and M is the total data volume of all clients;

[0029] Step five: for the super network parameter on the server and the embedding vector z i of the client i, update them by the following two equations respectively: ​

[0030]

[0031]

[0032] wherein, β is a learning rate;

[0033] Step six: repeat steps one to five until the specified training iteration round is reached.

[0034] The application trains and retains a unique self-attention layer parameter for each client, while other network layer parameters in the model are shared with the remaining clients, so that the model has better ability to process data heterogeneity. In addition, in order to effectively share parameters between clients while maintaining the flexibility of personalized Transformer models, the application trains a hypernetwork on the server wherein, is the parameter of the hypernetwork, z as the input of the hypernetwork is the learnable embedding vector of the corresponding client, and the output is the projection matrix of the self-attention layer to obtain the query Q=HW related to the client Q , the key K=HW K and the value V=HW V wherein, H is the input embedding matrix. This hypernetwork structure not only reduces the number of parameters that the entire model needs to learn, but also provides personalized models with good performance for new clients that do not participate in the training phase.

[0035] On the other hand, thanks to the personalized Transformer structure, the application also provides a federated learning system suitable for both image and language tasks, which includes a central server and multiple clients. According to different execution tasks, each client selects a corresponding Transformer structure as the local model, and in the kth communication round, the model is trained according to the local data, and the parameters ξi excluding the self-attention layer are sent to the server. In the central server, a hypernetwork and an embedding vector corresponding to each client are stored. The hypernetwork is composed of three fully connected networks, and the last fully connected network of the hypernetwork is different for each module in the Transformer in order to improve the performance of the hypernetwork. The server generates the self-attention layer parameters W i of each client in the kth communication round, and performs weighted aggregation on the parameters ξ i sent by each client to obtain the global parameters

[0036] Compared with the prior art, the application has the following beneficial effects:

[0037] (1) The first exploration of the application is the role of self-attention mechanism in personalized federated learning, which can more effectively process non-independent and identically distributed data.

[0038] (2) The application proposes a new federated learning framework based on Transformer, which learns a new self-attention mechanism for each client and shares the remaining parameters, while preserving personalized information and improving model robustness.

[0039] (3) The application provides a convenient environment for performing image and language tasks using the same federated network architecture, and in both tasks, the application can achieve better performance than other personalized federated learning methods. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The personalized Transformer-based federated learning model framework is illustrated;

[0041] Figure 2 The model framework of each client personalized Transformer is illustrated;

[0042] Figure 3 The personalized Transformer-based federated learning model flowchart is illustrated;

[0043] Figure 4 The accuracy of the new client after fine-tuning the personalized parameters in the model of each algorithm is illustrated. DETAILED DESCRIPTION

[0044] The application will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the application and not to limit the scope of the application. In addition, it should be understood that after reading the content taught by the application, those skilled in the art can make various modifications or modifications to the application, and these equivalent forms also fall within the scope defined by the claims attached to this application.

[0045] The personalized Transformer-based federated learning method disclosed in the embodiment comprises the following steps:

[0046] Step 1: In the initialization t = 0 start stage, given a central server and N clients C = {c1, c2,..., cN}, then extract the local data set N from the total data set according to the distribution (Dirichlet distribution with parameter α = 0.3 is used in the application) for each client where m i is the number of samples of the data set and initialize the local model f(θi and embedding vectors z i and the hypernetwork on the server parameters in the server

[0047] Step two: For the client i in the set C, the local data needs to be preprocessed before local training, including data shuffling and normalization processing, and then the local data is divided into training set and test set according to the ratio of 5:1, the data distribution of the training set and the test set needs to be consistent, and the two data sets are randomly sampled into n batches to obtain Where n is the batch size.

[0048] Step three: After the local data of the client i is divided into n batches, the sample data in the batch needs to be processed again: for sequence data, it is mapped into an embedding matrix; for image data, the original picture is blocked and flattened into a one-dimensional vector, and then a linear transformation is performed on each vector to compress its dimension to d, so that an embedding matrix is also obtained. In order to add the position information of the sequence and the blocked picture, position encoding needs to be introduced in each vector.

[0049] Step four: In the kth round of training, first randomly select a client from all clients to form a subset C t For the client i in the set C t , initialize the model parameters of the client i in two steps W i k，0 is generated by the hypernetwork on the server: Where is the updated hypernetwork parameter in the k-1th round, is the embedding vector of the corresponding client updated in the k-1th round;2) Download the global parameters aggregated in the k-1th round from the server

[0050] Step five: for the initialized client i, input the training set batch processed in step three into the model with the structure of Transformer, and use the stochastic gradient descent method to locally train t rounds, the model parameters can be updated by , where: is the model parameter of the client i updated in the kth communication round t, and a is the learning rate; is the gradient operator of the parameter θ i ; denotes l(·,·) is the cross-entropy loss function, and f(θi ;·) represents the personalized model for client i. Features of the training samples, The labels are for the training samples.

[0051] Step 6: After local training is complete, calculate the parameters W of client i before and after local training. i The difference ΔW i =W i k，t -W i k ，0 and ΔW i and updated parameters Send them to the server together.

[0052] Step 7: The server processes the received subset C t Uploaded parameters Aggregate the data to obtain the global parameters for the k-th round. m i Let M be the local data volume of client i, and M be the total data volume of all clients.

[0053] Step 8: For the hypernetic parameters on the server and the embedding vector z of each client i The parameter ΔW uploaded by the client can be used through the following two formulas. i Update Where β is the learning rate. For parameters gradient operator, For the parameter z i The gradient operator.

[0054] Step 9: After completing the k-th round of training, the server will update the global parameters. and super network The generated parameter W i k The data is sent to all clients, and each client's local test set is used for testing. The test accuracy of the personalized model updated in the kth round is calculated on each client, and the global average test accuracy is calculated. This is used to judge the performance of the personalized model trained in the kth round.

[0055] Step 10: Repeat steps 4 through 9 until the specified number of training iterations is reached.

[0056] Step 11: After training, set the global parameters Hypernetwork parameters and the embedding vectors of each client (K is the number of training iteration rounds) are recorded and can be used for subsequent manipulation.

[0057] Step twelve: For new clients that do not participate in the training process, the global parameters recorded in step eleven and super network parameters and the randomly initialized corresponding embedding vectors z can be directly used to train their personalized models, and good results can be obtained after a few rounds of fine-tuning. We call the personalized Transformer-based federated learning model obtained through the above steps FedTP, and compare its accuracy with that of some popular personalized federated learning methods on image data sets CIFAR-10 and CIFAR-100 under two different data distributions (label imbalance distribution refers to each client having a fixed number of labeled data samples, and Dirichlet distribution refers to each client being assigned a certain proportion of labeled samples according to a specific Dirichlet distribution) and having 100 clients, where each model is trained for 1500 rounds, and 10% of the clients are randomly selected to participate in training each round. The experimental results are shown in Table 1. As can be seen from the table, FedTP outperforms other models in all cases.

[0058] Table 1: Accuracy of FedTP and other models with 100 clients

[0059]

[0060] To eliminate the influence of model structure differences, we transfer the same Transformer structure to other personalized federated learning methods and further compare their results, as shown in Table 2. To distinguish these models from those in Table 1, we add "-T" to the models after changing the network structure. The results in Table 2 show that even if the network structure of other models is changed to the same Transformer as FedTP, our FedTP can still achieve the best results.

[0061] Table 2: Accuracy of FedTP and other Transformer-based models with 100 clients

[0062]

[0063] In addition, we tested the generalization ability of pFedMe, pFedHN, FedRod, and FedTP on the CIFAR-100 dataset, where 80% of the clients were used for training and the remaining 20% of the clients were not visible during training. For new clients, we compared the generalization ability of each model by fine-tuning the personalized parameters of the model and the accuracy of the new client based on the global model and personalized parameters, as shown inFigure 4 As shown in Figure 4 It can be seen from the figure that FedTP can achieve good results through one-step fine-tuning. With the increase of fine-tuning rounds, FedTP can always achieve better results than other methods, which confirms that FedTP has good generalization.

Claims

1. A personalized Transformer-based federated learning system comprising one server and several clients, characterized in that, According to different execution tasks, each client selects a corresponding Transformer structure as a local model, and the model parameters θ i of the client i are decomposed into θ i ={W i ,ξ i}, where W i is a projection matrix of a self-attention layer in the Transformer, and ξ i is a parameter of a layer other than the self-attention layer in the Transformer; The server stores a super network and an embedding vector corresponding to each client; In the kth communication round of federated learning: Client i downloads from the server the updated aggregation of parameters for the k-1th communication round Utilizing a hypernetwork located at the server Generating the projection matrix W of client i i where: is a parameter of the hypernetwork; z i is the input to the hypernetwork, is a learnable embedding vector of client i; Client i utilizes local data B i Local training on initialized model wherein: is the updated model parameter of client i in the kth communication round; and a is the learning rate. is the gradient operator with respect to parameter i ; denotes l(·,·) is the cross-entropy loss function, and f(θ i ;·) is the personalized model of client i, is the feature of the training sample, is the label of the training sample. updating the model parameters decomposed into where W i k is a projection matrix of the self-attention layer in the local model, is the parameters of the rest layers in the local model except the self-attention layer. The model parameters W i k Reserved locally at the client i for storing the personalized information of the client i; the parameter difference ΔW i = W i k -W i k-1 And Uploaded to the server, respectively, for parameter update and aggregation to obtain the advantageous information on other clients, obtaining the global parameters of the kth communication round Where N is the number of clients, m i The local data volume of the client i, and M is the total data volume of all clients.

2. The personalized Transformer-based federated learning system of claim 1, wherein, For the supernetwork parameters on the server and the embedding vector z of client i i are updated by the following two equations respectively: In the formula, β is the learning rate. For parameters gradient operator, For the parameter z i The gradient operator.

3. The personalized Transformer-based federated learning system of claim 1, wherein, The super network is composed of three fully connected networks, and the last fully connected network is different for each module in the Transformer.

4. A personalized Transformer-based federated learning model training method, characterized in that, The method comprises the following steps: Step one: In the k-th round of training, first randomly select a clients from all clients to form a subset C t For client i in the set C t Initialize updating its model parameters Where: W i k,0 is the projection matrix of the self-attention layer in the Transformer, which is generated by the super network on the server: where, is the super network parameter updated in the k-1th round, is the embedding vector of the client i updated in the k-1th round. are the parameters of the remaining layers in the Transformer except the self-attention layer; download the global parameters obtained from the server in the k-1th round of aggregation Step two: for the initialized client i, locally train its model for t rounds using stochastic gradient descent, according to the training batch B sampled from the local data i , the model parameters are updated as , where: is the updated model parameter of the client i in the kth communication round and the tth round; and a is the learning rate; is the gradient operator with respect to the parameter i ; represents l(·,·) is the cross-entropy loss function, f( i ;·) is the personalized model of the client i, is the feature of the training sample, is the label of the training sample; Step three: after the local training is completed, the client i sends the difference AW between the parameters W before and after the local training of the client i to the server i i i k,t i k,0 i the updated parameters​​​​​ Step four: the server aggregates the received parameters to obtain the global parameters of the kth round where m i is the local data volume of the client i, and M is the total data volume of all clients Step five: For the supernetwork parameters on the server and the embedding vector z of client i i are updated by the following two equations respectively: In the formula, β is a learning rate; Step six: repeat steps one to five until the specified training iteration round is reached.

Images