Large language model training method and system based on elastic federal low-rank adaptation fine tuning

Abstract

The invention discloses a large language model training method and system based on elastic federal low-rank adaptive fine tuning, which adopts federal aggregation based on singular value decomposition (SVD) to reconstruct a local LoRA module of a client into a local increment matrix, then aggregates the local increment matrix into a global increment matrix, and generates a LoRA module of the next round through SVD, thereby effectively eliminating aggregation errors of traditional FedLoRA. According to the method, the LoRA rank is dynamically distributed for each weight matrix, and the balance between statistics and hardware efficiency is realized. In order to further reduce communication overhead and maintain model precision, a self-adaptive matrix freezing mechanism is introduced, and whether freezing is carried out or not is judged according to training stability of a weight matrix. According to the method, challenges of traditional FedLoRA in the aspects of training efficiency, communication overhead, model precision and the like are solved.

Description

Technical Field

[0001] The present invention relates to the fields of computer networks and artificial intelligence, and to a large language model training method and system, and in particular to a large language model training method and system based on low-rank adaptation (LoRA) fine-tuning, cross-device federated learning (Cross-device Federated Learning), singular value decomposition (SVD) and parameter freezing (parameter freezing) technology. Background Art

[0002] Transformer-based large language models, such as GPT and LLaMA, have revolutionized the field of natural language processing, demonstrating unprecedented capabilities in a wide range of tasks such as text generation, sentiment analysis, and chatbots. As the number of parameters of these large models expands to billions or even trillions, full parameter fine-tuning is difficult to deploy in practice due to its huge storage and computing resource requirements. Currently, low-rank adaptive fine-tuning technology (LoRA) is usually used to approximate the high-dimensional structure of large models through low-dimensional structures to reduce model complexity. Although LoRA has obvious advantages, its fine-tuning process usually requires collecting a large amount of data from the client, which may contain sensitive information such as personal images or medical records, which may lead to serious privacy leakage. Cross-client federated learning enables multiple clients to collaboratively train a global model without sharing local data, providing a new solution for data privacy protection. However, although the existing federated low-rank adaptive fine-tuning technology (FedLoRA) achieves efficient parameter fine-tuning, it often faces the problems of performance degradation and excessive communication overhead during model aggregation, which seriously affects training efficiency. Specifically, first, FedLoRA expresses the updated value of the weight matrix in the model as the product of low-rank matrices A and B, and independently aggregates the A and B matrices through methods such as FedAvg. However, this independent aggregation method will produce aggregation errors because it ignores the intrinsic correlation between the A and B matrices during the model update process. Secondly, the fixed LoRA rank is difficult to balance statistical efficiency and hardware efficiency when processing different weight matrices. Too high a rank value may cause the model to overfit, while too low a rank value may slow down the model convergence speed. In addition, due to the huge number of parameters in large models, FedLoRA often generates high communication overhead in resource-constrained client environments, further reducing training efficiency. Finally, existing model freezing methods are usually performed at fine granularity (such as parameters, vectors) or coarse granularity (Transformer layer), without fully considering the structural characteristics of large Transformer-based models, and cannot effectively balance the computational requirements and model performance of large models. Summary of the invention

[0003] The present invention aims to solve the problems of aggregation error, training efficiency caused by fixed rank, high communication overhead, etc. existing in the existing FedLoRA system, and provides a large language model training method and system based on parameter-efficient low-rank adaptation (LoRA) fine-tuning technology, singular value decomposition (SVD), and parameter freezing technology.

[0004] The technical solution adopted by the method of the present invention is: a large language model training method based on elastic federated low-rank adaptive fine-tuning, the participating entities include a parameter server and a client; and the following steps are included:

[0005] Step 1: Client selection and global model parameter distribution;

[0006] At the beginning of the tth round of training, the parameter server randomly selects a part of the clients And the global model parameters Distributed to these clients; among them, the client set Respectively The result after low-rank decomposition;

[0007] Step 2: Local low-rank adaptation fine-tuning;

[0008] After receiving the global model parameters, the selected client performs E-step stochastic gradient descent using local data to generate a low-rank adaptation increment matrix for local updates and sends it back to the parameter server;

[0009] Step 3: Federated aggregation and dynamic rank assignment based on singular value decomposition;

[0010] The parameter server receives the low-rank adaptation incremental matrix of all clients, and aggregates the low-rank adaptation incremental matrix of each client into a global weighted incremental matrix through an aggregation algorithm based on singular value decomposition. Decomposition via singular value decomposition And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r represents the first r rows and columns of the left singular matrix, S r represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix;

[0011] Step 4: Adaptive matrix freezing;

[0012] The parameter server evaluates the training stability of each global weight increment matrix, and decides whether to freeze some global weight increment matrices according to the freezing threshold δ to reduce the amount of parameters that need to be transmitted; at the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

[0013] Preferably, in step 3, the aggregation algorithm based on singular value decomposition, for each selected client i, the parameter server is based on its local low-rank adaptation matrix module Calculate the corresponding local low-rank adaptation increment matrix The parameter server performs a weighted average of the local low-rank adaptation increment matrices of all clients to obtain the aggregated global low-rank adaptation increment matrix. And use singular value decomposition to decompose the matrix to get U, S, V T ; Among them, the singular values ​​in S are arranged in descending order; according to the rank assignment algorithm, the singular values ​​σ in S are sequentially traversed to find the smallest rank Make the front The ratio of the sum of squares of singular values ​​to the total sum of squares is not less than the rank assignment threshold θ; After that, by U, S, V T Truncation and multiplication are performed to generate a new global low-rank adaptation matrix.

[0014] Preferably, in step 4, first, the training stability of each global weight increment matrix is ​​calculated by calculating its update score To evaluate, where M is the window size of the historical round number, η is the learning rate; the global weight increment matrix after aggregation i represents the client, j represents the index of the Transformer layer, and k represents the type of the weight increment matrix; if the update score of a global weight increment matrix If the value is lower than the set threshold δ, the global weight increment matrix is ​​determined to be stable, and a freezing operation is performed to exclude it from subsequent updates. At the same time, the freezing period is dynamically adjusted. If the global weight increment matrix continues to be stable, extend its freezing period; otherwise, shorten the freezing period so that it can re-participate in the update and prevent the model from falling into a suboptimal state.

[0015] The technical solution adopted by the system of the present invention is: a large language model training system based on elastic federated low-rank adaptive fine-tuning, the participating entities include a parameter server and a client; and the following modules are included:

[0016] Client selection and global model parameter distribution module;

[0017] Used to randomly select a part of clients at the beginning of the tth round of training. And the global model parameters Distributed to these clients; among them, the client set Respectively The result after low-rank decomposition;

[0018] Local low-rank adaptation fine-tuning module;

[0019] After the selected client receives the global model parameters, it uses the local data to perform E-step stochastic gradient descent to generate a low-rank adaptation increment matrix for local updates and sends it back to the parameter server;

[0020] Federated aggregation and dynamic rank allocation module based on singular value decomposition;

[0021] The parameter server receives the low-rank adaptation incremental matrix of all clients, and aggregates the low-rank adaptation incremental matrix of each client into a global weighted incremental matrix through an aggregation algorithm based on singular value decomposition. Decomposition via singular value decomposition And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r represents the first r rows and columns of the left singular matrix, S r represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix;

[0022] Adaptive matrix freezing module;

[0023] The parameter server evaluates the training stability of each global weight increment matrix, and decides whether to freeze some global weight increment matrices according to the freezing threshold δ to reduce the amount of parameters that need to be transmitted; at the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

[0024] Preferably, in the singular value decomposition-based federated aggregation and dynamic rank allocation module, the singular value decomposition-based aggregation algorithm, for each selected client i, the parameter server is based on its local low-rank adaptation matrix module Calculate the corresponding local low-rank adaptation increment matrix The parameter server performs a weighted average of the local low-rank adaptation increment matrices of all clients to obtain the aggregated global low-rank adaptation increment matrix. And use singular value decomposition to decompose the matrix to get U, S, VT ; Among them, the singular values ​​in S are arranged in descending order; according to the rank assignment algorithm, the singular values ​​σ in S are sequentially traversed to find the smallest rank Make the front The ratio of the sum of squares of singular values ​​to the total sum of squares is not less than the rank assignment threshold θ; After that, by U, S, V T Truncation and multiplication are performed to generate a new global low-rank adaptation matrix.

[0025] Preferably, in the adaptive matrix freezing module, first, the training stability of each global weight increment matrix is ​​calculated by calculating its update score To evaluate, where M is the window size of the historical round number, η is the learning rate; the global weight increment matrix after aggregation i represents the client, j represents the index of the Transformer layer, and k represents the type of the weight increment matrix; if the update score of a global weight increment matrix If the value is lower than the set threshold δ, the global weight increment matrix is ​​determined to be stable, and a freezing operation is performed to exclude it from subsequent updates. At the same time, the freezing period is dynamically adjusted. If the global weight increment matrix continues to be stable, extend its freezing period; otherwise, shorten the freezing period so that it can re-participate in the update and prevent the model from falling into a suboptimal state.

[0026] The technical solution adopted by the device of the present invention is: a large language model training device based on elastic federated low-rank adaptive fine-tuning, comprising:

[0027] one or more processors;

[0028] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the large language model training method based on elastic federated low-rank adaptive fine-tuning.

[0029] The technical solution adopted by the product of the present invention is: a large language model training product based on elastic federated low-rank adaptive fine-tuning, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the large language model training method based on elastic federated low-rank adaptive fine-tuning.

[0030] Compared with the prior art, the beneficial effects of the present invention include:

[0031] (1) This paper proposes an efficient and flexible federated fine-tuning framework E for large models. 2LoRA, this is the first attempt to combine singular value decomposition (SVD) to optimize model aggregation and LoRA low-rank adaptation in federated learning. It can complete efficient global model updates by eliminating aggregation errors, reduce communication overhead by dynamically adjusting the rank of the weight matrix, and adopt an adaptive matrix freezing mechanism to balance training efficiency and model accuracy, solving the challenges of traditional FedLoRA in training efficiency, communication overhead, model accuracy, etc.

[0032] (2) E of the present invention 2 LoRA aggregates the client's local LoRA incremental matrix into a global incremental matrix through the SVD optimized aggregation process, and generates the corresponding low-rank matrix through SVD decomposition, effectively eliminating the errors introduced by traditional FedLoRA and its variants in the model aggregation process, ensuring the consistency and efficiency of the global model.

[0033] (3) E of the present invention 2 LoRA has designed a rank assignment strategy based on the importance of singular values, which can dynamically assign appropriate LoRA ranks to each weight matrix. By selecting update directions with larger singular values, statistical efficiency and hardware efficiency are effectively balanced, avoiding overfitting or slow convergence problems caused by fixed rank values, thereby improving the overall performance and training speed of the model.

[0034] (4) E of the present invention 2 LoRA implements an adaptive matrix freezing mechanism that evaluates the training stability of the weight matrix and dynamically freezes those weight matrices that have stabilized, significantly reducing communication overhead and data transmission while maintaining the accuracy and performance of the model. This mechanism realizes parameter freezing of appropriate granularity in the federated LoRA framework for the first time, improving training efficiency.

[0035] (5) The present invention improves the training efficiency and reduces the communication overhead in the process of federated fine-tuning by comprehensively applying SVD optimized aggregation, rank allocation based on the importance of singular values, and an adaptive matrix freezing mechanism, thereby significantly improving the overall efficiency of federated large model fine-tuning. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The technical solution of the present invention is further described below using embodiments and specific implementation methods. In addition, in the process of describing the technical solution, some drawings are also used. For those skilled in the art, other drawings and the intention of the present invention can also be obtained based on these drawings without paying creative work.

[0037] Figure 1 A schematic diagram of a method according to an embodiment of the present invention;

[0038] Figure 2A schematic diagram of the experimental results of comparing system communication overhead in an embodiment of the present invention;

[0039] Figure 3 Schematic diagram of system memory overhead comparison experiment results in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0041] First, the formal definition of the t-th round symbol in this embodiment is given:

[0042] Singular value decomposition function SVD(·);

[0043] Client Collection

[0044] Client parameters

[0045] LoRA incremental matrix for client

[0046] Aggregate weight of the client

[0047] The selected client set

[0048] LoRA updates received by parameter server

[0049] Global LoRA updates aggregated by parameter servers

[0050] Rank assignment threshold θ;

[0051] Matrix freezing threshold δ;

[0052] Please see Figure 1 , this implementation provides a large language model training method based on elastic federated low-rank adaptive fine-tuning, the participating entities include a parameter server and a client; and includes the following steps:

[0053] Step 1: Client selection and global model parameter distribution;

[0054] At the beginning of the tth round of training, the parameter server randomly selects a part of the clients And the global model parameters Distributed to these clients; among them, the client set Respectively The result after low-rank decomposition;

[0055] Step 2: Local Low Rank Adaptation (LoRA) fine-tuning;

[0056] After receiving the global model parameters, the selected client performs E-step stochastic gradient descent (SGD) using local data to generate a locally updated low-rank adaptation (LoRA) increment matrix and sends it back to the parameter server;

[0057] Step 3: federated aggregation and dynamic rank assignment based on singular value decomposition (SVD);

[0058] The parameter server receives the low-rank adaptation (LoRA) update incremental matrix of all clients, and aggregates the low-rank adaptation (LoRA) incremental matrix of each client into a global weight incremental matrix through an aggregation algorithm based on singular value decomposition (SVD) Decomposition by singular value decomposition (SVD) And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation (LoRA) matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r represents the first r rows and columns of the left singular matrix, S r represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix;

[0059] Step 4: Adaptive matrix freezing;

[0060] The parameter server evaluates the training stability of each global weight increment matrix, and decides whether to freeze some global weight increment matrices according to the freezing threshold δ to reduce the amount of parameters that need to be transmitted; at the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

[0061]

[0062]

[0063] In Algorithm 1, represents the round in which the weight matrix k in the jth layer was most recently frozen, Indicates the corresponding freezing period; The value of the low-rank matrix representing the weight matrix k in layer j when it was most recently frozen.

[0064] The traditional FedLoRA system treats the LoRA modules B and A as independent low-rank matrices, and aggregates the local LoRA modules from different clients separately, but when the client updates the LoRA module locally, it jointly optimizes B and A. The difference between local optimization and global aggregation strategies will introduce additional aggregation errors. According to theoretical derivation, this error will increase with the increase of factors such as the number of clients participating in training, the number of local training rounds, and the heterogeneity of client data, which greatly affects the training efficiency of the FedLoRA system. In order to eliminate this aggregation error, the SVD-based federated aggregation scheme first reconstructs the corresponding incremental matrix through the LoRA module updated locally by the client, and then performs weighted aggregation on the incremental matrices of all local clients. Then, SVD is used to decompose the aggregated incremental matrix and restore the LoRA modules B and A as the trainable parameters for the next round, which effectively eliminates the additional errors introduced by the traditional FedLoRA system during aggregation.

[0065] Fixed LoRA rank often makes it difficult to balance statistical efficiency and hardware efficiency. Specifically, a larger rank can reduce the number of rounds required for model convergence, but it will lead to overfitting and a larger amount of communication and local computation on the client; a smaller rank can reduce the amount of communication, the amount of local computation on the client, and avoid overfitting, but it will increase the number of training rounds and affect model performance.

[0066] To this end, in one embodiment, E 2 LoRA has designed a dynamic rank assignment strategy based on the importance of singular values. Since larger singular values ​​represent more important directions in model updates, retaining updates in these important directions while ignoring updates in unimportant directions can help the model find the optimal point more efficiently. Specifically, we select several most important model update directions by the size of the singular values, and the number of these singular values ​​is the dynamically adjusted LoRA rank.

[0067] Since the above two processes are highly coupled, they are put together in Algorithm 2. First, for each selected client i, the server uses its local LoRA module Calculate the corresponding local weight increment matrix Subsequently, the server performs a weighted average of the local weight increment matrices of all clients to obtain the aggregated global weight increment matrix. And use SVD to decompose the matrix to get U, S, V R It is worth noting that the singular values ​​in S are arranged in descending order. According to the rank assignment algorithm, the sequential traversal For each singular value σ in, find the minimum rank Make the front The ratio of the sum of squares of singular values ​​to the total sum of squares is not less than θ. After that, by U, S, V R The new LoRA module is restored by truncation and multiplication.

[0068]

[0069]

[0070] In one implementation, to further reduce communication overhead, E 2 LoRA introduces an adaptive matrix freezing mechanism. First, the training stability of each global weight increment matrix is ​​calculated by calculating its update score To evaluate, where M is the window size of the historical round number, η is the learning rate, and the aggregated global weight increment matrix i represents the client, j represents the index of the Transformer layer, and k represents the type of the weight matrix, which can be Q / K / V. If the update score of a global weight matrix If the value is lower than the set threshold δ, the global weight increment matrix is ​​determined to be stable, and a freeze operation is performed to exclude it from subsequent updates. At the same time, a window mechanism similar to TCP congestion control is used to dynamically adjust the freeze period. If the global weight increment matrix continues to be stable, its freezing period is extended; otherwise, the freezing period is shortened to allow it to re-participate in the update and prevent the model from falling into a suboptimal state. 2 LoRA can reduce communication volume and computational overhead while maintaining the accuracy and stability of the model.

[0071] This embodiment also provides a large language model training device based on elastic federated low-rank adaptive fine-tuning, wherein the participating entities include a parameter server and a client; and the device includes the following modules:

[0072] Client selection and global model parameter distribution module;

[0073] Used to randomly select a part of clients at the beginning of the tth round of training. And the global model parameters Distributed to these clients; among them, the client set Respectively The result after low-rank decomposition;

[0074] Local Low Rank Adaptation (LoRA) fine-tuning module;

[0075] After the selected client receives the global model parameters, it uses local data to perform E-step stochastic gradient descent (SGD) to generate a local updated low-rank adaptation (LoRA) increment matrix and sends it back to the parameter server;

[0076] A federated aggregation and dynamic rank assignment module based on singular value decomposition (SVD);

[0077] The parameter server receives the low-rank adaptation (LoRA) incremental matrix of all clients, and aggregates the low-rank adaptation (LoRA) incremental matrix of each client into a global update matrix through an aggregation algorithm based on singular value decomposition (SVD). Decomposition by singular value decomposition (SVD) And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation (LoRA) matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r represents the first r rows and columns of the left singular matrix, S r represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix;

[0078] Adaptive matrix freezing module;

[0079] The parameter server evaluates the training stability of each global weight increment matrix, and decides whether to freeze some global weight increment matrices according to the freezing threshold δ to reduce the amount of parameters that need to be transmitted; at the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

[0080] This embodiment also provides a large language model training device based on elastic federated low-rank adaptive fine-tuning, including:

[0081] one or more processors;

[0082] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the large language model training method based on elastic federated low-rank adaptive fine-tuning.

[0083] This embodiment also provides a large language model training product based on elastic federated low-rank adaptive fine-tuning, including computer program instructions. When the computer program instructions are executed on a computer, the computer executes the large language model training method based on elastic federated low-rank adaptive fine-tuning.

[0084] The present invention is further described below through specific experiments.

[0085] In the experimental environment setting, the present invention conducted experiments on a server equipped with 256GB memory and six NVIDIA GeForce RTX4090 GPUs. The upload and download bandwidth of the client is uniformly set to 8Mbps. The calculation time is obtained by running and sampling on NVIDIA Jetson AGX Orin and Jetson Orin NX. The present invention adds LoRA modules to the multi-head attention module and feedforward network of each Transformer layer, namely the four weight matrices of query, key, value and output projection.

[0086] In terms of the selection of models and datasets, the present invention fine-tunes the BERT-large and RoBERTa-large models on the 20NEWS and AGNEWS datasets and performs text classification tasks, and fine-tunes the DistilBERT model on the SST-2 dataset to perform sentiment analysis tasks.

[0087] In terms of hyperparameter settings, the present invention sets the batch size to 4, the number of local training rounds and the learning rate to 5 and 0.01 respectively, the sequence lengths of the three datasets 20NEWS, AGNEWS, and SST-2 are set to 256, 64, and 64 respectively, and the initial rank is set to 32. In terms of thresholds, the rank allocation threshold θ and the freezing threshold δ are set to 0.9 and 0.1 respectively.

[0088] In terms of the selection of baseline methods, this experiment compares FedPETuning, FFA-LoRA and the method of the present invention, and counts the time taken to reach the target test accuracy. FedPETuning is used as the baseline for comparison to calculate the speedup ratio. The relevant experimental results are shown in the following table. In addition, this experiment further compares the network traffic and memory usage consumed by these three methods during the training process. The results are shown in Tables 1 and 2. Figure 2 , 3 As shown in Table 1, for the 20NEWS dataset and BERT-large model, the time consumption of these three methods is 40.78h, 74.72h, and 10.67h respectively, and the speedup ratio is 1.00×, 0.55×, and 3.82× respectively. 2 LoRA can also achieve a speedup ratio of 2.54-4.38×, which is better than the existing baseline method. It can be seen that the method of the present invention significantly reduces the communication and computing overhead of the FedLoRA system during fine-tuning by eliminating aggregation errors, dynamically adjusting the rank of low-rank decomposition, and periodic freezing of the weight matrix level, thereby accelerating the training process of the system.

[0089] In addition, the experiment also counted the communication and memory overhead of the system. Figure 2As shown, compared with FedPETuning, E 2 The communication volume of LoRA is reduced by 87.4% to 97.7%; Figure 3 As shown, compared with FedPETuning, E 2 The memory consumption of LoRA on the 20NEWS dataset and BERT-large model is reduced to 35.5%-37.7%.

[0090] Table 1

[0091]

[0092]

[0093] This paper proposes an efficient and flexible federated low-rank adaptation fine-tuning framework E for large models. 2 LoRA,solves the problem of low-rank adaptation fine-tuning through a federated aggregation algorithm based on SVD, a dynamic rank assignment algorithm based on the importance of singular values, and an adaptive matrix freezing mechanism.

[0094] It should be understood that the embodiments described above are part of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0095] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A large language model training method based on elastic federated low-rank adaptive fine-tuning, the participating entities include a parameter server and a client; characterized in that: The following steps are involved: Step 1: Client selection and global model parameter distribution; At the beginning of the tth round of training, the parameter server randomly selects a part of the clients And the global model parameters Distributed to these clients; among them, the client set Respectively The result after low-rank decomposition; Step 2: Local low-rank adaptation fine-tuning; After receiving the global model parameters, the selected client performs E-step stochastic gradient descent using local data to generate a low-rank adaptation increment matrix for local updates and sends it back to the parameter server; Step 3: Federated aggregation and dynamic rank assignment based on singular value decomposition; The parameter server receives the low-rank adaptation incremental matrix of all clients, and aggregates the low-rank adaptation incremental matrix of each client into a global weighted incremental matrix through an aggregation algorithm based on singular value decomposition. Decomposition via singular value decomposition And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r represents the first r rows and columns of the left singular matrix, S r represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix; Step 4: Adaptive matrix freezing; The parameter server evaluates the training stability of each global weight increment matrix, and decides whether to freeze some global weight increment matrices according to the freezing threshold δ to reduce the amount of parameters that need to be transmitted; at the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

2. The large language model training method based on flexible federated low-rank adaptive fine-tuning according to claim 1 is characterized by: In step 3, the aggregation algorithm based on singular value decomposition, for each selected client i, the parameter server is based on its local low-rank adaptation matrix module Calculate the corresponding local low-rank adaptation increment matrix The parameter server performs weighted averaging on the local low-rank adaptation increment matrices of all clients to obtain the aggregated global weight increment matrix. And use singular value decomposition to decompose the matrix to get U, S, V T ; Among them, the singular values ​​in S are arranged in descending order; according to the rank assignment algorithm, the singular values ​​σ in S are sequentially traversed to find the smallest rank Make the front The ratio of the sum of squares of singular values ​​to the total sum of squares is not less than the rank assignment threshold θ; After that, by U, S, V T Truncation and multiplication are performed to generate a new global low-rank adaptation matrix.

3. The large language model training method based on flexible federated low-rank adaptive fine-tuning according to claim 1 is characterized by: In step 4, first, the training stability of each global weight increment matrix is ​​calculated by calculating its update score To evaluate, where M is the window size of the historical round number, η is the learning rate; the global weight increment matrix after aggregation i represents the client, j represents the index of the Transformer layer, and k represents the type of the weight increment matrix; if the update score of a global weight increment matrix If the value is lower than the set threshold δ, the global weight increment matrix is ​​determined to be stable, and a freezing operation is performed to exclude it from subsequent updates. At the same time, the freezing period is dynamically adjusted. If the global weight increment matrix continues to be stable, extend its freezing period; otherwise, shorten the freezing period so that it can re-participate in the update and prevent the model from falling into a suboptimal state.

4. A large language model training system based on elastic federated low-rank adaptive fine-tuning, the participating entities include a parameter server and a client; characterized in that: Includes the following modules: Client selection and global model parameter distribution module; Used to randomly select a part of clients at the beginning of the tth round of training. And the global model parameters Distributed to these clients; among them, the client set Respectively The result after low-rank decomposition; Local low-rank adaptation fine-tuning module; After the selected client receives the global model parameters, it uses the local data to perform E-step stochastic gradient descent to generate a low-rank adaptation increment matrix for local updates and sends it back to the parameter server; Federated aggregation and dynamic rank allocation module based on singular value decomposition; The parameter server receives the low-rank adaptation incremental matrix of all clients, and aggregates the low-rank adaptation incremental matrix of each client into a global weighted incremental matrix through an aggregation algorithm based on singular value decomposition. Decomposition via singular value decomposition And assign appropriate ranks according to the importance of singular values ​​to generate a new global low-rank adaptation matrix and Among them, U represents the left singular matrix, S represents the singular value vector, V represents the right singular matrix, U r represents the first r rows and columns of the left singular matrix, S r represents the first r elements of the singular value vector, V r represents the first r rows and first r columns of the right singular matrix; Adaptive matrix freezing module; The parameter server evaluates the training stability of each global weight increment matrix, and decides whether to freeze some global weight increment matrices according to the freezing threshold δ to reduce the amount of parameters that need to be transmitted; at the same time, it dynamically adjusts the freezing period and continues to train the unfrozen global weight increment matrices.

5. The large language model training system based on flexible federated low-rank adaptive fine-tuning according to claim 4 is characterized by: In the singular value decomposition-based federated aggregation and dynamic rank allocation module, the singular value decomposition-based aggregation algorithm, for each selected client i, the parameter server is based on its local low-rank adaptation matrix module Calculate the corresponding local low-rank adaptation increment matrix The parameter server performs weighted averaging on the local low-rank adaptation increment matrices of all clients to obtain the aggregated global weight increment matrix. And use singular value decomposition to decompose the matrix to get U, S, V T ; Among them, the singular values ​​in S are arranged in descending order; according to the rank assignment algorithm, the singular values ​​σ in S are sequentially traversed to find the smallest rank Make the front The ratio of the sum of squares of singular values ​​to the total sum of squares is not less than the rank assignment threshold θ; After that, by U, S, V T Truncation and multiplication are performed to generate a new global low-rank adaptation matrix.

6. The large language model training system based on flexible federated low-rank adaptive fine-tuning according to claim 4 is characterized by: In the adaptive matrix freezing module, first, the training stability of each global weight increment matrix is ​​calculated by calculating its update score To evaluate, where M is the window size of the historical round number, η is the learning rate; the global weight increment matrix after aggregation i represents the client, j represents the index of the Transformer layer, and k represents the type of the weight increment matrix; if the update score of a global weight increment matrix If the value is lower than the set threshold δ, the global weight increment matrix is ​​determined to be stable, and a freezing operation is performed to exclude it from subsequent updates. At the same time, the freezing period is dynamically adjusted. If the global weight increment matrix continues to be stable, extend its freezing period; otherwise, shorten the freezing period so that it can re-participate in the update and prevent the model from falling into a suboptimal state.

7. A large language model training device based on elastic federated low-rank adaptive fine-tuning, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the large language model training method based on elastic federated low-rank adaptive fine-tuning as described in any one of claims 1 to 3.

8. A large language model training product based on elastic federated low-rank adaptive fine-tuning, including computer program instructions, characterized in that: When the computer program instructions are executed on a computer, the computer is enabled to execute the large language model training method based on elastic federated low-rank adaptive fine-tuning as described in any one of claims 1 to 3.

Images