Large language model training method and system based on elastic federated low-rank adaptive fine-tuning

Abstract

The present invention discloses a large language model training method and system based on elastic federated low-rank adaptive fine-tuning. It adopts federated aggregation based on singular value decomposition (SVD) to reconstruct the client's local LoRA module into a local incremental matrix, and then aggregate it into a global incremental matrix. The next round of LoRA modules is generated by SVD, effectively eliminating the aggregation error of traditional FedLoRA. The present invention dynamically assigns LoRA rank to each weight matrix, achieving a balance between statistics and hardware efficiency. In order to further reduce communication overhead and maintain model accuracy, the present invention introduces an adaptive matrix freezing mechanism to determine whether to freeze based on the training stability of the weight matrix. The present invention solves the challenges of traditional FedLoRA in training efficiency, communication overhead, model accuracy, etc.

Description

Technical Field

[0001] The present invention relates to the fields of computer networks and artificial intelligence, and in particular to a large language model training method and system, and in particular to a large language model training method and system based on low-rank adaptation (LoRA) fine-tuning, cross-device federated learning (Cross-device Federated Learning), singular value decomposition (SVD) and parameter freezing technology. Background Art

[0002] Large Transformer-based language models, such as GPT and LLaMA, have revolutionized the field of natural language processing, demonstrating unprecedented capabilities across a wide range of tasks, including text generation, sentiment analysis, and chatbots. As the number of parameters in these large models scales to billions or even trillions, full parameter fine-tuning becomes challenging to implement due to the enormous storage and computational resources required. Currently, low-rank adaptive fine-tuning (LoRA) is commonly employed to reduce model complexity by approximating the high-dimensional structure of large models with low-dimensional structures. While LoRA offers significant advantages, its fine-tuning process typically requires collecting large amounts of client-side data, which may contain sensitive information such as personal images or medical records, potentially leading to serious privacy breaches. Cross-client federated learning allows multiple clients to collaboratively train a global model without sharing local data, offering a novel approach to data privacy protection. However, while existing Federated Low-Rank Adaptive Fine-tuning (FedLoRA) techniques achieve efficient parameter fine-tuning, they often suffer from performance degradation and excessive communication overhead during model aggregation, severely impacting training efficiency. Specifically, first, FedLoRA expresses the updated value of the weight matrix in the model as the product of low-rank matrices A and B, and independently aggregates the A and B matrices through methods such as FedAvg. However, this independent aggregation method will produce aggregation errors because it ignores the intrinsic correlation between the A and B matrices during the model update process. Secondly, the fixed LoRA rank is difficult to balance statistical efficiency and hardware efficiency when processing different weight matrices. Too high a rank value may cause the model to overfit, while too low a rank value may slow down the model convergence speed. Moreover, due to the huge number of parameters in large models, FedLoRA often generates high communication overhead in resource-constrained client environments, further reducing training efficiency. Finally, existing model freezing methods are usually performed at fine granularity (such as parameters, vectors) or coarse granularity (Transformer layer), without fully considering the structural characteristics of large Transformer-based models, and cannot effectively balance the computational requirements and model performance of large models. Summary of the Invention

[0003] The present invention aims to solve the problems of aggregation error, training efficiency caused by fixed rank, high communication overhead, etc. in the existing FedLoRA system, and provides a large language model training method and system based on parameter-efficient low-rank adaptation (LoRA) fine-tuning technology, singular value decomposition (SVD), and parameter freezing technology.

[0004] The technical solution adopted by the method of the present invention is: a large language model training method based on elastic federated low-rank adaptive fine-tuning, the participating entities include a parameter server and a client; and the method includes the following steps:

[0005] Step 1: Client selection and global model parameter distribution;

[0006] At the beginning of the tth round of training, the parameter server randomly selects a part of the clients And the global model parameters Distributed to these clients; where the client set They are respectively The result after low-rank decomposition;

[0007] Step 2: Local low-rank adaptation fine-tuning;

[0008] After receiving the global model parameters, the selected client performs E-step stochastic gradient descent using local data to generate a locally updated low-rank adaptation increment matrix And send it back to the parameter server;

[0009] Step 3: Federated aggregation and dynamic rank assignment based on singular value decomposition;

[0010] The parameter server receives the low-rank adaptive incremental matrix of all clients, and aggregates the low-rank adaptive incremental matrix of each client into a global weighted incremental matrix through an aggregation algorithm based on singular value decomposition. Decomposition via singular value decomposition And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r Represents the first r rows and columns of the left singular matrix, S r Represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix;

[0011] Step 4: Adaptive matrix freezing;

[0012] The parameter server evaluates the training stability of each global weight increment matrix and decides whether to freeze certain global weight increment matrices based on the freezing threshold δ to reduce the amount of parameters that need to be transmitted. At the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

[0013] As a preference, in step 3, the aggregation algorithm based on singular value decomposition, for each selected client i, the parameter server is based on its local low-rank adaptation matrix module Calculate the corresponding local low-rank adaptation increment matrix The parameter server performs weighted averaging on the local low-rank adaptation incremental matrices of all clients to obtain the aggregated global low-rank adaptation incremental matrix. And use singular value decomposition to decompose the matrix to get U, S, V T ; Among them, the singular values ​​in S are arranged in descending order; according to the rank assignment algorithm, each singular value σ in S is traversed sequentially to find the minimum rank Make the front The ratio of the sum of squares of singular values ​​to the total sum of squares is not less than the rank assignment threshold θ; After that, through U, S, V T Perform truncation and multiplication to generate a new global low-rank adaptation matrix.

[0014] Preferably, in step 4, first, the training stability of each global weight increment matrix is ​​calculated by calculating its update score To evaluate, where M is the window size of the historical round number, η is the learning rate; the global weight increment matrix after aggregation is i represents the client, j represents the index of the Transformer layer, and k represents the type of the weight increment matrix; if the update score of a global weight increment matrix If the value is lower than the set threshold δ, the global weight increment matrix is ​​determined to be stable, and a freeze operation is performed to exclude it from subsequent updates. At the same time, the freeze period is dynamically adjusted. If the global weight increment matrix continues to be stable, its freezing period is extended; otherwise, the freezing period is shortened to allow it to re-participate in the update and prevent the model from falling into a suboptimal state.

[0015] The technical solution adopted by the system of the present invention is: a large language model training system based on elastic federated low-rank adaptive fine-tuning, the participating entities include a parameter server and a client; and includes the following modules:

[0016] Client selection and global model parameter distribution module;

[0017] Used to randomly select a part of clients at the beginning of the tth round of training. And the global model parameters Distributed to these clients; where the client set They are respectively The result after low-rank decomposition;

[0018] Local low-rank adaptation fine-tuning module;

[0019] After the selected client receives the global model parameters, it uses the local data to perform E-step stochastic gradient descent to generate a locally updated low-rank adaptation increment matrix And send it back to the parameter server;

[0020] Federation aggregation and dynamic rank allocation module based on singular value decomposition;

[0021] The parameter server receives the low-rank adaptive incremental matrix of all clients, and aggregates the low-rank adaptive incremental matrix of each client into a global weighted incremental matrix through an aggregation algorithm based on singular value decomposition. Decomposition via singular value decomposition And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r Represents the first r rows and columns of the left singular matrix, S r Represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix;

[0022] Adaptive matrix freezing module;

[0023] The parameter server evaluates the training stability of each global weight increment matrix and decides whether to freeze certain global weight increment matrices based on the freezing threshold δ to reduce the amount of parameters that need to be transmitted. At the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

[0024] As a preference, in the federated aggregation and dynamic rank allocation module based on singular value decomposition, the aggregation algorithm based on singular value decomposition, for each selected client i, the parameter server is based on its local low-rank adaptation matrix module Calculate the corresponding local low-rank adaptation increment matrix The parameter server performs weighted averaging on the local low-rank adaptation incremental matrices of all clients to obtain the aggregated global low-rank adaptation incremental matrix. And use singular value decomposition to decompose the matrix to get U, S, VT ; Among them, the singular values ​​in S are arranged in descending order; according to the rank assignment algorithm, each singular value σ in S is traversed sequentially to find the minimum rank Make the front The ratio of the sum of squares of singular values ​​to the total sum of squares is not less than the rank assignment threshold θ; After that, through U, S, V T Perform truncation and multiplication to generate a new global low-rank adaptation matrix.

[0025] Preferably, in the adaptive matrix freezing module, first, the training stability of each global weight increment matrix is ​​calculated by calculating its update score To evaluate, where M is the window size of the historical round number, η is the learning rate; the global weight increment matrix after aggregation is i represents the client, j represents the index of the Transformer layer, and k represents the type of the weight increment matrix; if the update score of a global weight increment matrix If the value is lower than the set threshold δ, the global weight increment matrix is ​​determined to be stable, and a freeze operation is performed to exclude it from subsequent updates. At the same time, the freeze period is dynamically adjusted. If the global weight increment matrix continues to be stable, its freezing period is extended; otherwise, the freezing period is shortened to allow it to re-participate in the update and prevent the model from falling into a suboptimal state.

[0026] The technical solution adopted by the device of the present invention is: a large language model training device based on elastic federated low-rank adaptive fine-tuning, comprising:

[0027] one or more processors;

[0028] A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the large language model training method based on elastic federated low-rank adaptive fine-tuning.

[0029] The technical solution adopted by the product of the present invention is: a large language model training product based on elastic federated low-rank adaptive fine-tuning, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the large language model training method based on elastic federated low-rank adaptive fine-tuning.

[0030] Compared with the prior art, the beneficial effects of the present invention include:

[0031] (1) This paper proposes an efficient and flexible federated fine-tuning framework E for large models. 2LoRA is the first attempt to combine singular value decomposition (SVD) to optimize model aggregation and LoRA low-rank adaptation in federated learning. It can complete efficient global model updates by eliminating aggregation errors, reduce communication overhead by dynamically adjusting the rank of the weight matrix, and adopt an adaptive matrix freezing mechanism to balance training efficiency and model accuracy, solving the challenges of traditional FedLoRA in training efficiency, communication overhead, model accuracy, etc.

[0032] (2) E of the present invention 2 LoRA aggregates the client's local LoRA incremental matrix into a global incremental matrix through an SVD-optimized aggregation process, and generates the corresponding low-rank matrix through SVD decomposition, effectively eliminating the errors introduced by traditional FedLoRA and its variants in the model aggregation process, ensuring the consistency and efficiency of the global model.

[0033] (3) E of the present invention 2 LoRA has designed a rank assignment strategy based on the importance of singular values, which dynamically assigns an appropriate LoRA rank to each weight matrix. By selecting update directions with larger singular values, it effectively balances statistical efficiency and hardware efficiency, avoiding overfitting or slow convergence caused by fixed rank values, thereby improving the overall performance and training speed of the model.

[0034] (4) E of the present invention 2 LoRA implements an adaptive matrix freezing mechanism that evaluates the training stability of weight matrices and dynamically freezes those that have stabilized, significantly reducing communication overhead and data transmission while maintaining model accuracy and performance. This mechanism enables parameter freezing at an appropriate granularity for the first time in the federated LoRA framework, improving training efficiency.

[0035] (5) The present invention improves the training efficiency and reduces the communication overhead during the federated fine-tuning process by comprehensively applying SVD optimized aggregation, rank allocation based on the importance of singular values, and an adaptive matrix freezing mechanism, thereby significantly improving the overall efficiency of federated large model fine-tuning. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The technical solution of the present invention is further illustrated below using embodiments and specific implementation methods. In addition, some drawings are also used in the process of illustrating the technical solution. For those skilled in the art, other drawings and the intention of the present invention can be obtained based on these drawings without making any creative efforts.

[0037] Figure 1 A schematic diagram of a method according to an embodiment of the present invention;

[0038] Figure 2Schematic diagram of the experimental results of system communication overhead comparison in an embodiment of the present invention;

[0039] Figure 3 Schematic diagram of the system memory overhead comparison experiment results in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to facilitate ordinary technicians in this field to understand and implement the present invention, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0041] First, the formal definition of the t-th round symbol in this embodiment is given:

[0042] Singular value decomposition function SVD(·);

[0043] Client Collection

[0044] Client parameters

[0045] LoRA incremental matrix of the client

[0046] Aggregation weight of the client

[0047] Selected client set

[0048] LoRA updates received by the parameter server

[0049] Global LoRA updates aggregated by parameter servers

[0050] Rank assignment threshold θ;

[0051] Matrix freezing threshold δ;

[0052] Please see Figure 1 This implementation provides a large language model training method based on elastic federated low-rank adaptive fine-tuning. Participating entities include a parameter server and a client; the method includes the following steps:

[0053] Step 1: Client selection and global model parameter distribution;

[0054] At the beginning of the tth round of training, the parameter server randomly selects a part of the clients And the global model parameters Distributed to these clients; where the client set They are respectively The result after low-rank decomposition;

[0055] Step 2: Local Low-Rank Adaptation (LoRA) fine-tuning;

[0056] After receiving the global model parameters, the selected client performs E-step stochastic gradient descent (SGD) on the local data to generate the locally updated low-rank adaptation (LoRA) incremental matrix And send it back to the parameter server;

[0057] Step 3: federated aggregation and dynamic rank assignment based on singular value decomposition (SVD);

[0058] The parameter server receives the Low Rank Adaptation (LoRA) update incremental matrix of all clients, and aggregates the Low Rank Adaptation (LoRA) incremental matrix of each client into a global weight incremental matrix through an aggregation algorithm based on singular value decomposition (SVD) Decomposition by singular value decomposition (SVD) And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation (LoRA) matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r Represents the first r rows and columns of the left singular matrix, S r Represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix;

[0059] Step 4: Adaptive matrix freezing;

[0060] The parameter server evaluates the training stability of each global weight increment matrix and decides whether to freeze certain global weight increment matrices based on the freezing threshold δ to reduce the amount of parameters that need to be transmitted. At the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

[0061]

[0062]

[0063] In Algorithm 1, represents the round in which the weight matrix k in the jth layer was most recently frozen, Indicates the corresponding freezing period; The low-rank matrix representing the value of the weight matrix k in layer j when it was last frozen.

[0064] The traditional FedLoRA system treats the LoRA modules B and A as independent low-rank matrices and aggregates the local LoRA modules from different clients separately. However, when the client updates the LoRA module locally, it jointly optimizes B and A. The difference between local optimization and global aggregation strategies will introduce additional aggregation errors. According to theoretical derivation, this error will increase with the increase of factors such as the number of clients participating in training, the number of local training rounds, and the heterogeneity of client data, which greatly affects the training efficiency of the FedLoRA system. To eliminate this aggregation error, the SVD-based federated aggregation scheme first reconstructs the corresponding incremental matrix using the locally updated LoRA module of the client, and then performs weighted aggregation on the incremental matrices of all local clients. Subsequently, SVD is used to decompose the aggregated incremental matrix and restore the LoRA modules B and A as the trainable parameters for the next round, effectively eliminating the additional error introduced by the traditional FedLoRA system during aggregation.

[0065] Fixed LoRA rank often struggles to balance statistical efficiency and hardware efficiency. Specifically, while a larger rank can reduce the number of rounds required for model convergence, it can lead to overfitting, increased communication traffic, and increased local computational load on the client. A smaller rank can reduce communication traffic, local computational load on the client, and avoid overfitting, but it can increase the number of training rounds and affect model performance.

[0066] To this end, in one embodiment, E 2 LoRA has designed a dynamic rank assignment strategy based on the importance of singular values. Since larger singular values ​​indicate more important directions in the model update, retaining updates in these important directions while ignoring updates in less important directions can help the model find the optimal point more efficiently. Specifically, we filter out the most important model update directions based on the size of the singular values, and the number of these singular values ​​represents the dynamically adjusted LoRA rank.

[0067] Since the above two processes are highly coupled, they are put together in Algorithm 2. First, for each selected client i, the server uses its local LoRA module Calculate the corresponding local weight increment matrix Subsequently, the server performs a weighted average of the local weight increment matrices of all clients to obtain the aggregated global weight increment matrix And use SVD to decompose the matrix to get U, S, V R It is worth noting that the singular values ​​in S are arranged in descending order. According to the rank assignment algorithm, the sequential traversal For each singular value σ in, find the minimum rank Make the front The proportion of the sum of squares of singular values ​​to the total sum of squares is not less than θ. After that, through U, S, V R Truncation and multiplication are performed to restore the new LoRA module.

[0068]

[0069]

[0070] In one embodiment, to further reduce communication overhead, E 2 LoRA introduces an adaptive matrix freezing mechanism. First, the training stability of each global weight increment matrix is ​​calculated by calculating its update score To evaluate, where M is the window size of the historical round number, η is the learning rate, and the aggregated global weight increment matrix i represents the client, j represents the index of the Transformer layer, and k represents the type of the weight matrix, which can be Q / K / V. If the update score of a global weight matrix If the value is lower than the set threshold δ, the global weight increment matrix is ​​determined to be stable, and a freeze operation is performed to exclude it from subsequent updates. At the same time, a window mechanism similar to TCP congestion control is used to dynamically adjust the freeze period. If the global weight increment matrix remains stable, its freezing period is extended; otherwise, its freezing period is shortened to allow it to re-participate in the update and prevent the model from falling into a suboptimal state. 2 LoRA can reduce communication volume and computational overhead while maintaining the accuracy and stability of the model.

[0071] This embodiment also provides a large language model training device based on elastic federated low-rank adaptive fine-tuning, wherein the participating entities include a parameter server and a client; and the device includes the following modules:

[0072] Client selection and global model parameter distribution module;

[0073] Used to randomly select a part of clients at the beginning of the tth round of training. And the global model parameters Distributed to these clients; where the client set They are respectively The result after low-rank decomposition;

[0074] Local Low-Rank Adaptation (LoRA) fine-tuning module;

[0075] After the selected client receives the global model parameters, it uses local data to perform E-step stochastic gradient descent (SGD) to generate a locally updated low-rank adaptation (LoRA) incremental matrix And send it back to the parameter server;

[0076] Singular Value Decomposition (SVD)-based federated aggregation and dynamic rank assignment module;

[0077] The parameter server receives the Low Rank Adaptation (LoRA) incremental matrix of all clients, and aggregates the Low Rank Adaptation (LoRA) incremental matrix of each client into a global update matrix through an aggregation algorithm based on singular value decomposition (SVD). Decomposition by singular value decomposition (SVD) And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation (LoRA) matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r Represents the first r rows and columns of the left singular matrix, S r Represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix;

[0078] Adaptive matrix freezing module;

[0079] The parameter server evaluates the training stability of each global weight increment matrix and decides whether to freeze certain global weight increment matrices based on the freezing threshold δ to reduce the amount of parameters that need to be transmitted. At the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

[0080] This embodiment also provides a large language model training device based on elastic federated low-rank adaptive fine-tuning, including:

[0081] one or more processors;

[0082] A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the large language model training method based on elastic federated low-rank adaptive fine-tuning.

[0083] This embodiment also provides a large language model training product based on elastic federated low-rank adaptive fine-tuning, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the large language model training method based on elastic federated low-rank adaptive fine-tuning.

[0084] The present invention is further described below through specific experiments.

[0085] In the experimental environment, experiments were conducted on a server equipped with 256GB of memory and six NVIDIA GeForce RTX4090 GPUs. The client's upload and download bandwidth was uniformly set to 8Mbps. Computation times were sampled and run on NVIDIA Jetson AGX Orin and Jetson Orin NX. The authors added LoRA modules to the multi-head attention module and feedforward network of each Transformer layer, specifically the four weight matrices: query, key, value, and output projection.

[0086] In terms of model and dataset selection, this paper fine-tunes the BERT-large and RoBERTa-large models on the 20NEWS and AGNEWS datasets and performs text classification tasks, and fine-tunes the DistilBERT model on the SST-2 dataset to perform sentiment analysis tasks.

[0087] In terms of hyperparameter settings, the batch size is set to 4, the number of local training rounds and the learning rate are set to 5 and 0.01 respectively, the sequence lengths of the 20NEWS, AGNEWS, and SST-2 datasets are set to 256, 64, and 64 respectively, and the initial rank is set to 32. In terms of thresholds, the rank assignment threshold θ and the freezing threshold δ are set to 0.9 and 0.1 respectively.

[0088] In terms of the selection of baseline methods, this experiment compares FedPETuning, FFA-LoRA and the method of the present invention, and calculates the time taken to reach the target test accuracy. FedPETuning is used as the baseline for comparison to calculate the speedup ratio. The relevant experimental results are shown in the following table. In addition, this experiment further compares the network traffic and memory usage consumed by these three methods during training. The results are shown in Tables 1 and 2. Figure 2 、 3 As shown in Table 1, for the 20NEWS dataset and BERT-large model, the time consumption of these three methods is 40.78h, 74.72h, and 10.67h respectively, and the acceleration ratio is 1.00×, 0.55×, and 3.82× respectively. On other datasets and models, E 2 LoRA can also achieve a speedup ratio of 2.54-4.38×, which is better than the existing baseline method. This shows that the method of the present invention significantly reduces the communication and computational overhead of the FedLoRA system during fine-tuning by eliminating aggregation errors, dynamically adjusting the rank of low-rank decomposition, and periodically freezing the weight matrix level, thereby accelerating the system training process.

[0089] In addition, the experiment also counted the communication and memory overhead of the system. Figure 2As shown, compared with FedPETuning, E 2 The communication volume of LoRA is reduced by 87.4% to 97.7%; Figure 3 As shown, compared with FedPETuning, E 2 LoRA's memory consumption on the 20NEWS dataset and BERT-large model is reduced to 35.5%-37.7%.

[0090] Table 1

[0091]

[0092]

[0093] This paper proposes an efficient and flexible federated low-rank adaptation fine-tuning framework E for large models. 2 LoRA solves the problem of low-rank adaptation fine-tuning through a federated aggregation algorithm based on SVD, a dynamic rank assignment algorithm based on the importance of singular values, and an adaptive matrix freezing mechanism.

[0094] It should be understood that the embodiments described above are only some of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention may be arbitrarily combined with each other to form a feasible technical solution. Such combination is not restricted by the order of steps and / or structural composition mode, but must be based on the ability of ordinary technicians in this field to implement it. When the combination of technical solutions is mutually inconsistent or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0095] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of protection of the patent of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A large language model training method based on elastic federated low-rank adaptive fine-tuning, wherein the participating entities include a parameter server and a client; characterized in that: The following steps are involved: Step 1: Client selection and global model parameter distribution; At the beginning of the tth round of training, the parameter server randomly selects a part of the clients And the global model parameters Distributed to these clients; where the client set They are respectively The result after low-rank decomposition; Step 2: Local low-rank adaptation fine-tuning; After receiving the global model parameters, the selected client performs E-step stochastic gradient descent using local data to generate a locally updated low-rank adaptation increment matrix And send it back to the parameter server; Step 3: Federated aggregation and dynamic rank assignment based on singular value decomposition; The parameter server receives the low-rank adaptive incremental matrix of all clients, and aggregates the low-rank adaptive incremental matrix of each client into a global weighted incremental matrix through an aggregation algorithm based on singular value decomposition. Decomposition via singular value decomposition And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r Represents the first r rows and columns of the left singular matrix, S r Represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix; Step 4: Adaptive matrix freezing; The parameter server evaluates the training stability of each global weight increment matrix and decides whether to freeze certain global weight increment matrices based on the freezing threshold δ to reduce the amount of parameters that need to be transmitted. At the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

2. The large language model training method based on flexible federated low-rank adaptive fine-tuning according to claim 1 is characterized by: In step 3, the aggregation algorithm based on singular value decomposition, for each selected client i, the parameter server is based on its local low-rank adaptation matrix module Calculate the corresponding local low-rank adaptation increment matrix The parameter server performs weighted averaging on the local low-rank adaptation incremental matrices of all clients to obtain the aggregated global weight incremental matrix. And use singular value decomposition to decompose the matrix to get U, S, V T ; Among them, the singular values ​​in S are arranged in descending order; according to the rank assignment algorithm, each singular value σ in S is traversed sequentially to find the minimum rank Make the front The ratio of the sum of squares of singular values ​​to the total sum of squares is not less than the rank assignment threshold θ; After that, through U, S, V T Perform truncation and multiplication to generate a new global low-rank adaptation matrix.

3. The large language model training method based on flexible federated low-rank adaptive fine-tuning according to claim 1 is characterized by: In step 4, first, the training stability of each global weight increment matrix is ​​calculated by calculating its update score To evaluate, where M is the window size of the historical round number, η is the learning rate; the global weight increment matrix after aggregation is i represents the client, j represents the index of the Transformer layer, and k represents the type of the weight increment matrix; if the update score of a global weight increment matrix If the value is lower than the set threshold δ, the global weight increment matrix is ​​determined to be stable, and a freeze operation is performed to exclude it from subsequent updates. At the same time, the freeze period is dynamically adjusted. If the global weight increment matrix continues to be stable, its freezing period is extended; otherwise, the freezing period is shortened to allow it to re-participate in the update and prevent the model from falling into a suboptimal state.

4. A large language model training system based on elastic federated low-rank adaptive fine-tuning, the participating entities include a parameter server and a client; characterized in that: Includes the following modules: Client selection and global model parameter distribution module; Used to randomly select a part of clients at the beginning of the tth round of training. And the global model parameters Distributed to these clients; where the client set They are respectively The result after low-rank decomposition; Local low-rank adaptation fine-tuning module; After the selected client receives the global model parameters, it uses the local data to perform E-step stochastic gradient descent to generate a locally updated low-rank adaptation increment matrix And send it back to the parameter server; Federation aggregation and dynamic rank allocation module based on singular value decomposition; The parameter server receives the low-rank adaptive incremental matrix of all clients, and aggregates the low-rank adaptive incremental matrix of each client into a global weighted incremental matrix through an aggregation algorithm based on singular value decomposition. Decomposition via singular value decomposition And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r Represents the first r rows and columns of the left singular matrix, S r Represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix; Adaptive matrix freezing module; The parameter server evaluates the training stability of each global weight increment matrix and decides whether to freeze certain global weight increment matrices based on the freezing threshold δ to reduce the amount of parameters that need to be transmitted. At the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

5. The large language model training system based on flexible federated low-rank adaptive fine-tuning according to claim 4 is characterized by: In the federated aggregation and dynamic rank allocation module based on singular value decomposition, the aggregation algorithm based on singular value decomposition, for each selected client i, the parameter server is based on its local low-rank adaptation matrix module Calculate the corresponding local low-rank adaptation increment matrix The parameter server performs weighted averaging on the local low-rank adaptation incremental matrices of all clients to obtain the aggregated global weight incremental matrix. And use singular value decomposition to decompose the matrix to get U, S, V T ; Among them, the singular values ​​in S are arranged in descending order; according to the rank assignment algorithm, each singular value σ in S is traversed sequentially to find the minimum rank Make the front The ratio of the sum of squares of singular values ​​to the total sum of squares is not less than the rank assignment threshold θ; After that, through U, S, V T Perform truncation and multiplication to generate a new global low-rank adaptation matrix.

6. The large language model training system based on flexible federated low-rank adaptive fine-tuning according to claim 4, characterized in that: In the adaptive matrix freezing module, first, the training stability of each global weight increment matrix is ​​calculated by calculating its update score To evaluate, where M is the window size of the historical round number, η is the learning rate; the global weight increment matrix after aggregation is i represents the client, j represents the index of the Transformer layer, and k represents the type of the weight increment matrix; if the update score of a global weight increment matrix If the value is lower than the set threshold δ, the global weight increment matrix is ​​determined to be stable, and a freeze operation is performed to exclude it from subsequent updates. At the same time, the freeze period is dynamically adjusted. If the global weight increment matrix continues to be stable, its freezing period is extended; otherwise, the freezing period is shortened to allow it to re-participate in the update and prevent the model from falling into a suboptimal state.

7. A large language model training device based on elastic federated low-rank adaptive fine-tuning, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the large language model training method based on elastic federated low-rank adaptive fine-tuning as described in any one of claims 1 to 3.

8. A large language model training product based on elastic federated low-rank adaptive fine-tuning, comprising computer program instructions, characterized in that: When the computer program instructions are executed on a computer, the computer is caused to execute the large language model training method based on elastic federated low-rank adaptive fine-tuning according to any one of claims 1 to 3.

Images