Large model federation training method under privacy protection limitation and GPU bottleneck

Abstract

The invention discloses a large model federated training method under privacy protection limitation and GPU bottleneck, belongs to the field of large model training and federated learning, and aims to solve the problem of low accuracy caused by privacy protection limitation and GPU bottleneck limitation of existing large language model training. According to the method, in an initial round, # imgabs0 # and # imgabs1 # of each client side are initialized by using Kaiming initialization and zero initialization respectively, and in a subsequent round, initialization of each client side adopts a two-step process: firstly, server aggregation LoRA parameters are combined with client side model weights; secondly, the client performs training by using the updated model weight and the LoRA parameters to generate a new round of LoRA parameters; the process is cyclically carried out until the model converges.

Description

Technical Field

[0001] The present invention relates to a large model federated training method under privacy protection constraints and GPU bottlenecks, belonging to the fields of large model training and federated learning. Background Art

[0002] Currently, the rapid development of pre-trained foundation models, especially large language models (such as ChatGPT, Llama, Claude), is driving the artificial intelligence technology into a new stage. At the same time, many excellent large language models have emerged in China, such as Tongyi Qianwen, KimiChat, Doubao, etc. However, training these large language models needs to overcome two key challenges: on the one hand, training large language models requires centralized deployment of high-memory and high-performance GPUs; on the other hand, the training data of large language models usually contains a large amount of privacy data that is widely distributed and highly diverse, and these data cannot be centrally shared due to legal, ethical, and security restrictions.

[0003] First, although domestic graphics cards (such as Huawei Ascend GPU, Moore Threads' S series) have made significant progress, their memory capacity is relatively small and it is difficult to support the training requirements of large-scale models. Second, the dependence on privacy data in the large model training process has increased significantly, and the existing data distribution and privacy protection requirements have further exacerbated the difficulty of data sharing and centralized training.

[0004] In summary, the training of large language models is restricted by privacy protection and GPU bottlenecks, resulting in low accuracy. Summary of the Invention

[0005] Aiming at the problem that the existing large language model training is restricted by privacy protection and GPU bottlenecks, resulting in low accuracy, the present invention provides a large model federated training method under privacy protection constraints and GPU bottlenecks.

[0006] On the one hand, the present invention provides the above-mentioned large model federated training method under privacy protection constraints and GPU bottlenecks, and the method includes the following steps:

[0007] S1. At the initial iteration of t = 0, the initialization parameters at the start of the local training of the client LoRA for the t-th iteration of each client are initialized using the kaiming initialization method and the zero initialization method respectively;

[0008] S2. In subsequent iterations of t ≥ 1, the weight and parameter update of the client adopts a two-step process:

[0009] S21. Aggregate the LoRA parameters of the server side for the t-th iteration with the client model weights for the (t - 1)-th iteration Merge to obtain the client model weights for the t-th round of iteration

[0010]

[0011] Among them, the server-side aggregated LoRA parameters for the t-th round of iteration are aggregated by the server from all clients' respectively, and are the parameters after the local training of the client LoRA at the end of the (t - 1)-th round of iteration;

[0012] S22. The initialization parameters at the start of the local training of the client LoRA for the t-th round of iteration are obtained according to the following formula:

[0013]

[0014] TruncatedSVD(·) is the singular value decomposition function;

[0015] S3. The client uses the updated client model weights for the t-th round of iteration and the initialization parameters at the start of the local training of the client LoRA for the t-th round of iteration to perform training and obtain the parameters after the local training of the client LoRA at the end of the t-th round of iteration

[0016] S4. After the end of the t-th round of training, the server aggregates all the parameters after the local training of the client LoRA for the t-th round of iteration to obtain the server-side aggregated LoRA parameters for the next round and update the global model weights

[0017]

[0018]

[0019] S5. Send the server-side aggregated LoRA parameters for the next round to all clients, and loop to enter the next round of training until the model converges.

[0020] Preferably, the client model weights for the t-th round of iteration are initialized with \(W_s^0\) at the initial round of iteration \(t = 0\).

[0021] Preferably, the server-side global model weights for the t-th round of iteration are initialized with the pre-trained model weights at the initial round of iteration \(t = 0\).

[0022] On the other hand, the present invention provides the above-mentioned large model federated training method under privacy protection restrictions and GPU bottlenecks, and the method includes the following steps:

[0023] S1. At the initial iteration of t = 0, the initialization parameters at the start of the client LoRA local training for the t-th iteration of each client are initialized using LoRA-GA;

[0024] S2. In subsequent iterations where t ≥ 1, the weight and parameter update of the client adopts a two-step process:

[0025] S21. Aggregate the LoRA parameters of the server side for the t-th iteration with the client model weights for the (t - 1)-th iteration to obtain the client model weights for the t-th iteration

[0026]

[0027] wherein, the aggregated LoRA parameters of the server side for the t-th iteration are respectively aggregated by the server from all clients' and are the parameters after the client LoRA local training for the (t - 1)-th iteration ends;

[0028] S22. The initialization parameters at the start of the client LoRA local training for the t-th iteration are obtained according to the following formula:

[0029]

[0030] In the formula, TruncatedSVD(·) is the singular value decomposition function;

[0031] is the temporary variable after the client LoRA-GA initialization for the t-th iteration;

[0032] S3. The client uses the updated client model weights for the t-th iteration and the initialization parameters at the start of the client LoRA local training for the t-th iteration to perform training and obtain the temporary variable after the client LoRA training for the t-th iteration ends

[0033] S4. After each round of training ends, the parameters after the client LoRA local training for the t-th iteration are obtained through the following formula

[0034] ​

[0035] S5. After each round of training, the server aggregates the parameters after the local training of the client LoRA for all the t-th round of iterations. Obtain the aggregated LoRA parameters on the server side for the next round. And update the global model weights.

[0036]

[0037]

[0038] S6. Send the aggregated LoRA parameters on the server side for the next round To all clients, and loop into the next round of training until the model converges.

[0039] Preferably, the client model weights for the t-th round of iterations Are initialized with \(W_s^0\) at the initial round of iteration \(t = 0\).

[0040] Preferably, the server-side global model weights for the t-th round of iterations Are initialized with the pre-trained model weights at the initial round of iteration \(t = 0\).

[0041] Advantages of the present invention: The present invention proposes an innovative solution that combines the Parameter Efficient Fine-Tuning (PEFT) method with federated learning, aiming to achieve efficient training and fine-tuning of large models.

[0042] Federated learning is a distributed machine learning technique where the server only receives the gradients uploaded by users to update the global model. Since the parameter server does not directly access the user data, it is generally considered that the privacy of the user data is protected.

[0043] The present invention fine-tunes the model parameters, improving the accuracy of model training. Compared with the traditional FedAvgLoRA, the first scheme of the present invention improves the average accuracy by 7.24%, while the second scheme of the present invention achieves a 21.68% improvement.

[0044] The effectiveness and competitiveness of using the method of the present invention in training large language models, especially in complex tasks that require processing data in professional fields, are particularly significant in terms of performance improvement. Description of the Drawings

[0045] Figure 1 Is the schematic diagram of a large model federated training method under privacy protection restrictions and GPU bottlenecks in Embodiment 1, FedRiLA;

[0046] Figure 2 It is the schematic diagram of a large model federated training method under privacy protection restrictions and GPU bottlenecks described in Embodiment 2, FedRiLA-GA. Specific implementation manners

[0047] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0048] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.

[0049] Next, the present invention will be further described in conjunction with the accompanying drawings and specific embodiments, but it is not a limitation of the present invention.

[0050] Specific implementation manner 1: Next, in conjunction with Figure 1 This implementation manner is described. The large model federated training method under privacy protection restrictions and GPU bottlenecks described in this implementation manner includes the following steps:

[0051] S1. At the initial iteration of t = 0, the initialization parameters at the start of the local training of the client LoRA for the t-th round of iteration of each client Are initialized using the kaiming initialization method and the zero initialization method respectively;

[0052] S2. In subsequent iterations where t ≥ 1, the weight and parameter update of the client adopts a two-step process:

[0053] S21. Aggregate the LoRA parameters of the server side for the t-th round of iteration With the client model weight for the (t - 1)-th round of iteration To obtain the client model weight for the t-th round of iteration

[0054]

[0055] Among them, the aggregated LoRA parameters of the server side for the t-th round of iteration Are aggregated by the server from all clients' Respectively, And are the parameters after the local training of the client LoRA for the (t - 1)-th round of iteration ends;

[0056] S22. Initialization parameters at the start of the local training of the client LoRA in the t-th round of iteration Obtained according to the following formula:

[0057]

[0058] TruncatedSVD(·) is the singular value decomposition function;

[0059] S3. The client uses the updated client model weights in the t-th round of iteration and the initialization parameters at the start of the local training of the client LoRA in the t-th round of iteration to perform training and obtain the parameters after the local training of the client LoRA in the t-th round of iteration ends

[0060] S4. After the t-th round of training ends, the server aggregates all the parameters after the local training of the client LoRA in the t-th round of iteration to obtain the aggregated LoRA parameters on the server side for the next round and update the global model weights

[0061]

[0062]

[0063] S5. Send the aggregated LoRA parameters on the server side for the next round to all clients, and loop to enter the next round of training until the model converges.

[0064] The client model weights in the t-th round of iteration are initialized with \(W_s^0\) in the initial round of iteration when \(t = 0\).

[0065] The server-side global model weights in the t-th round of iteration are initialized with the pre-trained model weights in the initial round of iteration when \(t = 0\).

[0066] Name the trained model in this embodiment as FedRiLA, and the implementation program code is:

[0067]

[0068] Specific Embodiment 2: The following is combined with Figure 2 to illustrate this embodiment. The method for federated training of large models under privacy protection constraints and GPU bottlenecks described in this embodiment includes the following steps:

[0069] S1. At the initial iteration of t = 0, the initialization parameters at the start of the client LoRA local training for the t-th iteration of each client are initialized using LoRA-GA;

[0070] S2. In subsequent iterations where t ≥ 1, the weight and parameter update of the client adopts a two-step process:

[0071] S21. Aggregate the LoRA parameters of the server side for the t-th iteration with the client model weights for the (t - 1)-th iteration to obtain the client model weights for the t-th iteration

[0072]

[0073] Among them, the aggregated LoRA parameters of the server side for the t-th iteration are respectively aggregated by the server from all clients' and are the parameters after the client LoRA local training for the (t - 1)-th iteration ends;

[0074] S22. The initialization parameters at the start of the client LoRA local training for the t-th iteration are obtained according to the following formula:

[0075]

[0076] In the formula, TruncatedSVD(·) is the singular value decomposition function;

[0077] is a temporary variable after the client LoRA-GA initialization for the t-th iteration;

[0078] LoRA-GA (abbreviation for Low-Rank Adaptation with Gradient Approximation, low-rank adaptation using gradient approximation method), LoRA (abbreviation for Low-Rank Adaptation) is a parameter-efficient fine-tuning method that reduces costs by fine-tuning an auxiliary low-rank model with fewer parameters. Although LoRA significantly reduces the computational and memory requirements per iteration, its convergence speed is much slower than full-scale fine-tuning. Therefore, LoRA and GA form a new initialization method LoRA-GA, which significantly improves efficiency and performance by aligning the gradient of the low-rank matrix product with the gradient of full-scale fine-tuning in the first step without changing the architecture and training algorithm.

[0079] S3. The client uses the client model weights of the updated t-th round of iteration and the initialization parameters at the start of the local training of the client LoRA in the t-th round of iteration to perform training and obtain the temporary variables after the local training of the client LoRA in the t-th round of iteration

[0080] S4. After each round of training, obtain the parameters after the local training of the client LoRA in the t-th round of iteration through the following formula

[0081]

[0082] S5. After each round of training, the server aggregates all the parameters after the local training of the client LoRA in the t-th round of iteration to obtain the aggregated LoRA parameters of the server in the next round and update the global model weights

[0083]

[0084]

[0085] S6. Send the aggregated LoRA parameters of the server in the next round to all clients, and loop back to the next round of training until the model converges.

[0086] The client model weights of the t-th round of iteration are initialized with \(W_s^0\) at the initial round of iteration \(t = 0\).

[0087] The global model weights of the server in the t-th round of iteration are initialized with the pre-trained model weights at the initial round of iteration \(t = 0\).

[0088] This embodiment incorporates GA on the basis of Embodiment 1, and the solution of this embodiment is named FedRiLA-GA

[0089] Specific Embodiment 3: In this embodiment, verification tests are carried out on the solutions of Embodiment 1 and Embodiment 2.

[0090] Classification experiments were conducted on the DomianNet and CIFAR100 datasets. Table 1 shows the experimental results of FedRiLA and FedRiLA-GA on the DomianNet and CIFAR100 datasets. FedRiLA and FedRiLA-GA have significant improvements in both IID and Non-IID scenarios. For example, on the DomainNet dataset, compared with FedAvgLoRA, FedRiLA increased the average accuracy by 7.24%, while FedRiLA-GA achieved a 21.68% improvement. These results verify the effectiveness of our method.

[0091] Table 1 Classification Results of DomainNet and CIFAR-100

[0092]

[0093] Table 2 summarizes the experimental results of fine-tuning large language models on various tasks. On this task, FedRiLA shows significant performance advantages in mathematics, code, and dialogue tasks. This result further verifies the effectiveness and competitiveness of FedRiLA in training large language models (LLMs), especially in complex tasks that require processing data in professional fields, where its performance improvement is particularly significant.

[0094] Table 2 Evaluation Results of Llama Large Model in the Medical Field

[0095]

[0096] Although the present invention has been described with reference to specific embodiments in this document, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed, as long as they do not deviate from the spirit and scope of the present invention defined by the appended claims. It should be understood that different dependent claims and the features described in this document can be combined in a manner different from that described in the original claims. It should also be understood that the features described in connection with individual embodiments can be used in other described embodiments.

Claims

1. A large model federated training method under privacy protection constraints and GPU bottleneck, characterized by: The method comprises the following steps: S1, at the initial round iteration of t = 0, the initialization parameters at the beginning of the client LoRA local training of the tth round iteration of each client Use kaiming initialization method and zero initialization method for initialization respectively; S2. In subsequent iterations of t≥1, the weight and parameter updates of the client are performed in two steps: S21, aggregate the LoRA parameters of the server side of the tth iteration The client model weight at the t-1th iteration Merge to get the client model weight of the tth iteration Among them, the server-side aggregated LoRA parameters of the tth iteration are The server will send all clients' They are aggregated separately, The parameters after the client LoRA local training of the t-1th iteration is completed; S22, initialization parameters at the start of the client LoRA local training of the tth iteration Get it by pressing: TruncatedSVD(·) is the singular value decomposition function; S3: The client uses the updated client model weights for the tth iteration and the initialization parameters at the beginning of the client LoRA local training of the tth iteration Perform training to obtain the parameters of the client LoRA local training after the tth iteration S4. After the tth round of training, the server aggregates the parameters of all the client LoRA local training after the tth round of iteration. Get the next round of server-side aggregated LoRA parameters And update the global model weights S5: aggregate the LoRA parameters on the server side for the next round The data is sent to all clients and the next round of training is repeated until the model converges.

2. According to claim 1, a large model federated training method under privacy protection constraints and GPU bottlenecks is characterized in that: Client model weights at the tth iteration At the initial round iteration at t=0, W_s^0 is used for initialization.

3. According to claim 1, a large model federated training method under privacy protection constraints and GPU bottlenecks is characterized in that: Server-side global model weights at the tth iteration The pre-trained model weights are initialized at the initial round iteration t=0.

4. The large model federated training method under privacy protection constraints and GPU bottleneck is characterized by: The method comprises the following steps: S1, at the initial round iteration of t = 0, the initialization parameters at the beginning of the client LoRA local training of the tth round iteration of each client Initialize using LoRA-GA; S2. In subsequent iterations of t≥1, the weight and parameter updates of the client are performed in two steps: S21, aggregate the LoRA parameters of the server side of the tth iteration The client model weight at the t-1th iteration Merge to get the client model weight of the tth iteration Among them, the server-side aggregated LoRA parameters of the tth iteration are The server will send all the clients' Aggregated, The parameters after the client LoRA local training of the t-1th iteration is completed; S22, initialization parameters at the start of the client LoRA local training of the tth iteration Get it by pressing: Where, TruncatedSVD(·) is the singular value decomposition function; Temporary variables after LoRA-GA initialization for the client in the tth iteration; S3: The client uses the updated client model weights for the tth iteration and the initialization parameters at the beginning of the client LoRA local training of the tth iteration Perform training and obtain the temporary variables after the client LoRA training of the tth iteration is completed S4. After each round of training, the parameters of the client LoRA local training after the tth round of iteration are obtained by the following formula S5. After each round of training, the server aggregates the parameters of all client LoRA local training after the tth iteration. Get the next round of server-side aggregated LoRA parameters And update the global model weights S6: aggregate the LoRA parameters of the next round of servers The data is sent to all clients and the next round of training is repeated until the model converges.

5. The method for large-model federated training under privacy protection constraints and GPU bottleneck according to claim 4, characterized in that: Client model weights at the tth iteration At the initial round iteration at t=0, W_s^0 is used for initialization.

6. The method for large-model federated training under privacy protection constraints and GPU bottleneck according to claim 4, characterized in that: Server-side global model weights at the tth iteration The pre-trained model weights are initialized at the initial round iteration t=0.

Images