Large language model federated fine-tuning method and apparatus based on gradient compression

Abstract

Disclosed in the present invention are a large language model federated fine-tuning method and apparatus based on gradient compression. The method comprises the following steps: constructing, on the basis of a gradient tensor generated during fine tuning of a large language model, a raw data set having a time series relationship, performing inference by means of an autoencoder to obtain a reconstructed gradient data set, and constructing a reconstruction loss function to optimize the autoencoder; and initializing a base model of the large language model as a global model at a server end, the server end updating the global model to a client, using a pre-trained encoder to obtain a compressed gradient at the client, and at the server end, using a pre-trained decoder to decode and aggregate the compressed gradient, and then updating the global model. The present invention can improve the fine-tuning efficiency of the large language model and reduce computing resource requirements while ensuring data privacy protection, and is suitable for application scenarios such as communication optimization improvement and privacy protection enhancement in the process of scientific computing-oriented large model fine-tuning and training.

Description

Gradient compression-based large language model federated fine-tuning method and device TECHNICAL FIELD

[0001] The present application belongs to the technical field of large language model fine-tuning, and particularly relates to a gradient compression-based large language model federated fine-tuning method and device. BACKGROUND

[0002] Since the advent of models such as ChatGPT, the field of large language models (LLM) has experienced rapid development. LLMs, with their complexity in deep learning and wide applicability, have shown amazing performance in multiple fields, such as mathematical reasoning, natural language understanding, and common sense reasoning.

[0003] For training strategies, in order to apply general LLMs to specific fields, model fine-tuning is a crucial step. Model fine-tuning is a process of adapting a pre-trained general model to a specific task or domain. In this process, the parameters of the model need to be finely adjusted so that it can better handle specific types of data and tasks. However, due to the extremely large number of parameters contained in these LLMs, even with a standard and relatively economical GPU such as RTX 4090, it is difficult to fully manage all parameters. Even for a model as large as LLaMA-7B, the large parameter size not only increases the complexity of computation, but also increases the difficulty of tuning.

[0004] Therefore, it is necessary to find an efficient fine-tuning method to achieve the best results under limited computing resources. The adaptive low-rank decomposition strategy is one of the effective means. By performing low-rank decomposition on the parameters, the high-dimensional parameter space is reduced to a low-dimensional subspace, thereby reducing the number of parameters that need to be tuned. In this way, only a selected subset of parameters needs to be tuned, and similar results to full tuning can be achieved. This method not only improves the efficiency of tuning, but also reduces the demand for computing resources.

[0005] However, the above-mentioned efficient fine-tuning method is not omnipotent, and a large amount of high-quality data resources are still needed for effective fine-tuning of LLM in terms of training data. However, with the increasing strictness of data privacy regulations, the collection, storage and use of personal data are restricted, making data acquisition and cross-platform data sharing more and more difficult. In this case, federated learning (FL) as a new distributed machine learning framework provides a new solution. Federated learning allows multiple participants to jointly train a model without sharing the original data. Each participant trains the model on their own local data and sends the updates of the model to the central server for aggregation, so that a high-quality model can be trained by multiple participants while protecting data privacy.

[0006] For the fine-tuning of LLM, federated learning also has important application value. It is necessary to construct an efficient federated fine-tuning strategy specifically for LLM to fully utilize the data of multiple participants to jointly optimize a general LLM to adapt to specific tasks or fields, while ensuring data privacy protection, improving the efficiency of federated fine-tuning and reducing the demand for computing resources.

[0007] SUMMARY

[0008] In view of the above, the purpose of the present application is to provide a gradient compression-based large language model federated fine-tuning method and device, which can improve the efficiency of large language model federated fine-tuning and reduce the demand for computing resources while ensuring data privacy protection, and is suitable for application scenarios such as communication optimization improvement and privacy protection enhancement in the fine-tuning and training process of large language models for scientific computing.

[0009] To achieve the above-mentioned purpose of the application, the technical solutions provided by the present application are as follows:

[0010] In a first aspect, the gradient compression-based large language model federated fine-tuning method provided by the embodiments of the present application comprises the following steps:

[0011] Based on the gradient tensor generated by the large language model during the fine-tuning process, an original data set with a time series relationship is constructed, the original data set is input into a self-encoder to obtain a reconstructed gradient data set, a reconstruction loss function is constructed based on the original data set and the reconstructed gradient data set to optimize the self-encoder, and the optimized self-encoder is split into a pre-training encoder and a pre-training decoder;

[0012] The base model of the large language model is initialized as a global model on the server side, the global model is updated to at least one selected client, the pre-trained encoder is used to obtain compressed gradients in the local training stage of the global model on the client side, and the server side is transmitted, the pre-trained decoder is used to decode and aggregate the compressed gradients in the global aggregation stage of the global model on the server side, the aggregated gradients are used to update the global model, and thus the global model after federated fine-tuning is obtained as the final application large language model deployed on the server side and updated to the client side.

[0013] Preferably, the gradient tensor generated by the large language model during the fine-tuning process is constructed as an original data set with a time series relationship, including:

[0014] The gradient tensor generated by the large language model based on the gradient descent algorithm during the fine-tuning process is obtained, and the gradient tensor is constructed as a gradient tensor set with a time series relationship according to the time stamp, as an original data set.

[0015] Preferably, the original data set is input into the autoencoder to obtain the reconstructed gradient data set, including:

[0016] The original data set is input into the autoencoder, the original data set is encoded by the encoder in the autoencoder to obtain a compressed gradient tensor low-dimensional feature representation, and the compressed gradient tensor low-dimensional feature representation is decoded by the decoder in the autoencoder to obtain a reconstructed gradient data set.

[0017] Preferably, the encoder and the decoder in the autoencoder are of a symmetrical structure, the encoder includes a multi-layer convolutional neural network and a plurality of residual blocks, and the decoder includes a plurality of residual blocks and a multi-layer deconvolutional neural network.

[0018] Preferably, the reconstructed loss function is constructed based on the original data set and the reconstructed gradient data set to optimize the autoencoder, including:

[0019] The difference between the corresponding gradient tensors in the original data set and the reconstructed gradient data set is calculated to construct a reconstruction loss function L, which is represented as follows:

[0020] Wherein, F i represents the i-th gradient tensor in the original data set, G i represents the i-th gradient tensor in the reconstructed gradient data set, and ||·||2 represents the two-norm;

[0021] The autoencoder is iteratively optimized by minimizing the reconstruction loss function L.

[0022] Preferably, the pre-trained encoder is used to obtain compressed gradients in the local training stage of the global model on the client side, and the server side is transmitted, including:

[0023] The global model is trained locally on each client based on low-rank decomposition, and the gradient low-rank decomposition result containing multiple low-dimensional matrix products is obtained.

[0024] On each client, a pre-trained encoder is used to encode the low-rank gradient decomposition result to obtain the compressed gradient.

[0025] Each client sends the compressed gradient to the server.

[0026] Preferably, the step of decoding and aggregating the compressed gradients using a pre-trained decoder in the global model aggregation stage on the server side, and updating the global model using the aggregated gradients, includes:

[0027] The server collects compressed gradients from at least one selected client.

[0028] The server decodes the compressed gradients using a pre-trained encoder.

[0029] The server performs average aggregation of the decoded gradients on the subspace of the low-rank decomposition.

[0030] The server updates the global model parameters using gradient descent based on the aggregated gradients.

[0031] Secondly, in order to achieve the above-mentioned objectives, the present invention also provides a gradient compression-based large language model federated fine-tuning device, comprising: a gradient compression autoencoder pre-training module and a gradient compression large language model federated fine-tuning module.

[0032] The gradient compression autoencoder pre-training module is used to construct an original dataset with time-series relationship based on the gradient tensor generated during the fine-tuning of the large language model. The original dataset is input into the autoencoder for inference to obtain the reconstructed gradient dataset. Based on the original dataset and the reconstructed gradient dataset, a reconstruction loss function is constructed to optimize the autoencoder. The optimized autoencoder is then split into a pre-trained encoder and a pre-trained decoder.

[0033] The gradient compression large language model federated fine-tuning module is used to initialize the base model of the large language model as a global model on the server side. The server side updates the global model to at least one selected client. During the local training phase of the global model on the client side, the compressed gradient is obtained by using a pre-trained encoder and sent to the server side. During the global aggregation phase of the global model on the server side, the compressed gradient is decoded and aggregated by a pre-trained decoder. The aggregated gradient is used to update the global model, thereby obtaining the federated fine-tuned global model as the final application large language model deployed on the server side and updated to the client side.

[0034] Thirdly, to achieve the above-mentioned objectives, embodiments of the present invention also provide a large language model federated fine-tuning device based on gradient compression, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-mentioned large language model federated fine-tuning method based on gradient compression when the computer program is executed.

[0035] Fourthly, to achieve the above-mentioned objectives, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-mentioned large language model federated fine-tuning method based on gradient compression.

[0036] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0037] (1) This invention adopts a gradient compression strategy. By optimizing the autoencoder in advance, the autoencoder performance can be effectively trained, enabling efficient gradient reconstruction during federated learning. No further training of the autoencoder is required during federated learning. Then, in the federated fine-tuning stage, the autoencoder is split into an encoder and a decoder and deployed to the client and server respectively. Based on the gradient compression, transmission, and decompression strategies of the client and server, federated fine-tuning of the large language model is carried out, thereby achieving high compression of gradient information in the upstream process of federated learning and reducing the amount of communication during data transmission, thus improving the hardware processing speed.

[0038] (2) The present invention adopts a federated learning fine-tuning method, which can overcome the limitations brought about by data privacy regulations and make full use of the data resources of multiple participants to improve the performance of the model, and realize efficient federated fine-tuning of large language models. It is of great significance for communication optimization and privacy protection in the fine-tuning of large language models with extremely large gradient parameters.

[0039] (3) This invention uses gradient information to train the autoencoder. The gradient information contains features with time-series relationships, which directly reflect the performance of the model on a specific task. Therefore, the training process can have better adaptability, convergence and stability, and is suitable for efficient training of large-scale datasets, so that the autoencoder can be better applied to the federated fine-tuning of large language models. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 is a flowchart illustrating the federated fine-tuning method for large language models based on gradient compression provided in an embodiment of the present invention.

[0042] Figure 2 is a schematic diagram of the autoencoder model structure in stage one provided by an embodiment of the present invention;

[0043] Figure 3 is a schematic diagram of federated fine-tuning under gradient compression in stage two provided by an embodiment of the present invention;

[0044] Figure 4 is a general framework diagram of the large language model federated fine-tuning method based on gradient compression provided in the embodiments of the present invention;

[0045] Figure 5 is a schematic diagram of the structure of the large language model federated fine-tuning device based on gradient compression provided in an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0047] The inventive concept of this invention is as follows: Addressing the limitations of data privacy protection and the low efficiency and high communication costs of federated fine-tuning of large language models in existing technologies, this invention provides a method and apparatus for federated fine-tuning of large language models based on gradient compression. It employs an auto-encoder architecture, as shown in Figure 2. This auto-encoder architecture includes an encoder and a decoder. The client uses the encoder to compress the gradient information generated during the large language model fine-tuning process into a compact feature representation, while the server uses the decoder to reconstruct these features into accurate gradients. The main challenge lies in training the encoder and decoder to accurately reconstruct gradients without compromising privacy. To solve this problem, this invention divides the training process into two stages: time-aware gradient compression auto-encoder pre-training (stage one) and large language model federated fine-tuning under gradient compression involving the autoencoder (stage two). Typically, to ensure gradient compression accuracy, a direct method is to train the autoencoder and the large language model together. However, this method incurs significant additional training costs. Furthermore, the dynamic nature of the target large language model's gradient changes during training complicates the fitting process. Therefore, this invention does not employ the strategy of training the autoencoder and the large language model together. Instead, it innovatively trains the autoencoder in Stage 1 by identifying the characteristics of the target large language model's gradients and designing a temporal ensemble pre-training scheme. Specifically, by recording the gradients of the target large language model at different training stages, a consistent and homogeneous dataset can be created to effectively train the autoencoder. This dataset can be compiled through collaboration among clients before federated learning, i.e., aggregating the local gradients of each client. By pre-training the autoencoder with this gradient dataset before federated learning in Stage 1, the optimized autoencoder can efficiently reconstruct gradients during federated learning in Stage 2 without further training. In Stage 2, as shown in Figure 3, the gradient information is encoded and compressed by encoders deployed on multiple clients and transmitted to a unified server. The decoder deployed on the server decodes and decompresses the compressed gradients before gradient aggregation for federated fine-tuning of the large language model. This significantly reduces the communication volume between the client and server, thereby improving the efficiency of federated fine-tuning of the large language model and reducing computational resource requirements.

[0048] Figure 1 is a flowchart illustrating the federated fine-tuning method for large language models based on gradient compression provided in an embodiment of the present invention. As shown in Figure 1, the embodiment provides a federated fine-tuning method for large language models based on gradient compression, including the following steps:

[0049] S1, based on the gradient tensor generated during the fine-tuning of the large language model, constructs an original dataset with time-series relationships. The original dataset is input into the autoencoder for inference to obtain a reconstructed gradient dataset. Based on the original dataset and the reconstructed gradient dataset, a reconstruction loss function is constructed to optimize the autoencoder. The optimized autoencoder is then split into a pre-trained encoder and a pre-trained decoder.

[0050] Specifically, as shown in Figure 4, Phase 1: Pre-training of a time-aware gradient compression autoencoder.

[0051] Input: Gradient tensors with time-series relationships generated during the fine-tuning of the large language model.

[0052] Output: Optimized autoencoder, which includes a pre-trained encoder and a pre-trained decoder.

[0053] S1.1, adopting a knowledge-based question-answering large language model for natural language processing tasks, obtains the gradient tensors generated by the gradient descent algorithm during the fine-tuning process of the large language model, and constructs a set of gradient tensors with time-series relationships [G1, G2, ..., G...] according to their generation timestamps. M ] is the original dataset, where M is the total number of gradient tensors.

[0054] S1.2, the original dataset is input into the autoencoder. The encoder encodes the original dataset to obtain a compressed low-dimensional feature representation of the gradient tensor. Then, the decoder decodes and reconstructs the compressed low-dimensional feature representation of the gradient tensor to obtain a set of reconstructed gradient tensors [F1, F2, ..., F]. M [This serves as a dataset for reconstructing gradients.]

[0055] In this embodiment, as shown in Figure 2, the encoder and decoder in the autoencoder have a symmetrical structure. The encoder includes a multi-layer convolutional neural network and multiple residual blocks, and the decoder includes multiple residual blocks and a multi-layer deconvolutional neural network. The specific network structure information is shown in Table 1.

[0056] Table 1. Autoencoder Model Structure

[0057] S1.3, the difference between the corresponding gradient tensors in the original dataset and the reconstructed gradient dataset is calculated to obtain the squared L2 norm of the difference, which is the constructed reconstruction loss function L, expressed as follows:

[0058] Among them, F i Let G represent the i-th gradient tensor in the original dataset. i Let ||·||2 represent the i-th gradient tensor in the reconstructed gradient dataset, and let ||·||2 represent the L2 norm.

[0059] S1.4, the autoencoder is iteratively optimized by minimizing the reconstruction loss function L.

[0060] S1.5, end the dynamic iteration, output the gradient that meets the stopping condition, which is the reconstruction of the original gradient, and finally obtain the optimized autoencoder;

[0061] S1.6, save the resulting optimized autoencoder, which includes the pre-trained encoder and the pre-trained decoder.

[0062] S2, on the server side, the base model of the large language model is initialized as the global model. The server side updates the global model to at least one selected client. In the local training phase of the global model on the client side, the compressed gradient is obtained by using the pre-trained encoder and sent to the server side. In the global aggregation phase of the global model on the server side, the compressed gradient is decoded and aggregated by the pre-trained decoder. The aggregated gradient is used to update the global model, thereby obtaining the federated fine-tuned global model as the final application of the large language model, which is deployed on the server side and updated to the client side.

[0063] Specifically, as shown in Figure 4, Phase 2: Federated fine-tuning of a large language model under gradient compression with the participation of an autoencoder.

[0064] Input: W, the base model of the large language model base After obtaining the pre-trained encoder and pre-trained decoder in Phase 1, fine-tune the total number of training rounds T, the client random sampling ratio a, and the total number of clients N.

[0065] Output: Federated fine-tuning of the large language model W T .

[0066] The base models of large language models are typically pre-trained on large-scale data, possessing powerful language understanding and generation capabilities. However, these models are often designed for broad tasks and may not be accurate enough for direct applications in specific domains or tasks. Therefore, fine-tuning the base models according to specific needs is a common practice to improve the model's performance on specific tasks. First, we need to clarify the specific tasks we want the large language model to perform, such as sentiment analysis, question answering systems, text generation, named entity recognition, etc. In this example, the fine-tuning goal is to obtain a knowledge-based question-answering large language model for natural language processing tasks. The specific fine-tuning process is as follows.

[0067] S2.1, On the server side, perform global model W. global The initialization is performed on the input base model, i.e., W. global =W base .

[0068] S2.2, for training rounds t = 0, ..., T-1, perform the following iterations:

[0069] S2.2.1, The server randomly samples the clients, with a random sampling ratio of a, and the number of clients selected in each round is N×a;

[0070] S2.2.2, the server will use the global model W global Update to at least one of the selected clients;

[0071] S2.2.3, During the local training phase of the global model on the client side, for the selected client, the following iterations are performed:

[0072] S2.2.3.1, perform local training of the global model based on low-rank decomposition on each client, and obtain the gradient low-rank decomposition result containing multiple low-dimensional matrix products. In the example, the low-rank decomposition result of the i-th gradient is represented as G. i =B i A i The gradient matrix G i Decomposed into two smaller low-rank matrices B i and A i ;

[0073] S2.2.3.2, on each client, the low-rank gradient decomposition result is encoded using a pre-trained encoder to obtain the compressed gradient;

[0074] S2.2.3.3, each client transmits the compressed gradient to the server;

[0075] S2.2.4, During the global aggregation phase of the server-side global model, the following steps are performed:

[0076] S2.2.4.1, The server collects compressed gradients from at least one selected client;

[0077] S2.2.4.2, the server decodes the compressed gradient using a pre-trained encoder;

[0078] S2.2.4.3, The server performs average aggregation of the decoded gradients on the subspace of the low-rank decomposition;

[0079] S2.2.4.4, the server updates the global model parameters using gradient descent based on the aggregated gradients. In this embodiment, the formula for updating the global model parameters using gradient descent is: W new =W old +ηBA

[0080] Among them, W old W represents the global model parameters before the update.new Here are the updated global model parameters, η is the learning rate, and the low-rank matrix B = ∑ i B i The low-rank matrix A = ∑ i A i .

[0081] S2.3, after reaching the convergence condition or the predetermined number of iterations, stop the iteration and save the updated global model W. T This completes the federated fine-tuning. Finally, the federated fine-tuned global model W is... T The large language model, as the final application, is deployed on the server side and updated to the client. The federated and finely tuned large language model can be deployed to real-world application scenarios, such as integration into apps, website backend services, chatbots, etc., to provide specific language processing functions.

[0082] Furthermore, comparative experiments were conducted to evaluate the performance of the gradient compression-based federated fine-tuning method for large language models provided in this invention. The experimental design is as follows: the first three sets of experiments used LlaMa-7B, Aplaca-7B, and ChatGLM-6B as base models, respectively, and fine-tuned them on the Databricks-dolly-15k dataset; the fourth set of experiments showed the fine-tuning results of the LlaMa-7B model on the C-eval dataset. Different fine-tuning strategies were employed in each set of experiments, including centered fine-tuning (Cent), the gradient compression federated fine-tuning (Compress-FT) innovatively proposed in this invention, low-rank decomposition federated fine-tuning (LoRA-FT), and the performance of the un-fine-tuned base model (Base) as a control.

[0083] The experimental results are shown in Table 2, which covers the three base models LlaMa-7B, Aplaca-7B, and ChatGLM-6B, as well as two datasets for fine-tuning: Databricks-dolly-15k and C-eval. The evaluation method used was a multiple-choice question-and-answer format, where the question was fed as input to the trained large language model, which responded with one correct answer from four options, and the model's output was scored. These experiments aimed to evaluate the model's question-and-answer performance across six subject areas: natural sciences, social sciences, humanities, and other disciplines. The overall average score and the average score for difficult questions were recorded. The average score for difficult questions involved eight high-difficulty test items, such as advanced data processing, discrete mathematics, probability and statistics, university-level chemistry and physics, and high school mathematics, chemistry, and physics.

[0084] Table 2 Comparison of experimental results

[0085] Comprehensive analysis shows that our proposed method demonstrates consistent and excellent performance regardless of the base model or fine-tuned dataset, highlighting its universality and effectiveness in improving multi-domain question answering capabilities.

[0086] In summary, a federated fine-tuning method for large language models based on gradient compression achieves high compression of gradient information and reduction of communication volume in the upstream process of federated learning through the aforementioned gradient encoding and decoding process. This is of great significance for communication optimization and privacy protection in the fine-tuning of large language models with extremely large gradient parameters, enabling efficient, low-communication, and highly noise-robust fine-tuning of large language models in federated learning.

[0087] Based on the same inventive concept, as shown in Figure 5, this embodiment of the invention also provides a large language model federated fine-tuning device 500 based on gradient compression, including: a gradient compression autoencoder pre-training module 510 and a gradient compression large language model federated fine-tuning module 520.

[0088] Among them, the gradient compression autoencoder pre-training module 510 is used to construct an original dataset with time-series relationship based on the gradient tensor generated during the fine-tuning process of the large language model. The original dataset is input into the autoencoder for inference to obtain the reconstructed gradient dataset. Based on the original dataset and the reconstructed gradient dataset, a reconstruction loss function is constructed to optimize the autoencoder. The optimized autoencoder is then split into a pre-trained encoder and a pre-trained decoder.

[0089] The gradient compression large language model federated fine-tuning module 520 is used to initialize the base model of the large language model as a global model on the server side, update the global model to at least one selected client on the server side, obtain the compressed gradient using the pre-trained encoder during the local training phase of the global model on the client side and send it to the server side, decode and aggregate the compressed gradient using the pre-trained decoder during the global aggregation phase of the global model on the server side, update the global model using the aggregated gradient, and thus obtain the federated fine-tuned global model as the final application large language model deployed on the server side and updated to the client side.

[0090] Based on the same inventive concept, embodiments of the present invention also provide a large language model federated fine-tuning device based on gradient compression, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described large language model federated fine-tuning method based on gradient compression when executing the computer program.

[0091] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described gradient compression-based federated fine-tuning method for large language models.

[0092] It should be noted that the gradient compression-based large language model federated fine-tuning device, gradient compression-based large language model federated fine-tuning equipment, and computer-readable storage medium provided in the embodiments of the present invention all belong to the same inventive concept as the gradient compression-based large language model federated fine-tuning method. For details of their specific implementation process, please refer to the embodiments of the gradient compression-based large language model federated fine-tuning method, which will not be repeated here.

[0093] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A federated fine-tuning method for large language models based on gradient compression, characterized in that, Includes the following steps: The gradient tensor generated during the fine-tuning of the large language model is used to construct an original dataset with time-series relationships. The original dataset is then input into the autoencoder for inference to obtain a reconstructed gradient dataset. A reconstruction loss function is constructed based on the original dataset and the reconstructed gradient dataset to optimize the autoencoder. The optimized autoencoder is then split into a pre-trained encoder and a pre-trained decoder. On the server side, the base model of the large language model is initialized as the global model. The server side updates the global model to at least one selected client. During the local training phase of the global model on the client side, the compressed gradient is obtained by using the pre-trained encoder and sent to the server side. During the global aggregation phase of the global model on the server side, the compressed gradient is decoded and aggregated by the pre-trained decoder. The aggregated gradient is used to update the global model, thereby obtaining the federated fine-tuned global model as the final application of the large language model, which is deployed on the server side and updated to the client side.

2. The federated fine-tuning method for large language models based on gradient compression according to claim 1, characterized in that, The gradient tensor generated during the fine-tuning of the large language model is used to construct an original dataset with time-series relationships, including: Obtain the gradient tensors generated by the gradient descent algorithm during the fine-tuning process of the large language model, and construct a set of gradient tensors with time-series relationships according to the generation timestamps, which serves as the original dataset.

3. The federated fine-tuning method for large language models based on gradient compression according to claim 1, characterized in that, The process of inputting the original dataset into the autoencoder inference to obtain the reconstructed gradient dataset includes: The original dataset is input into the autoencoder, and the encoder within it processes the original dataset. The compressed gradient tensor low-dimensional feature representation is obtained by encoding, and then the decoder is used to decode and reconstruct the compressed gradient tensor low-dimensional feature representation to obtain the reconstructed gradient dataset.

4. The federated fine-tuning method for large language models based on gradient compression according to claim 1 or 3, characterized in that, The encoder and decoder in an autoencoder have a symmetrical structure. The encoder includes a multi-layer convolutional neural network and multiple residual blocks, while the decoder includes multiple residual blocks and a multi-layer deconvolutional neural network.

5. The federated fine-tuning method for large language models based on gradient compression according to claim 1, characterized in that, The optimization of the autoencoder by constructing a reconstruction loss function based on the original dataset and the reconstructed gradient dataset includes: The reconstruction loss function L is constructed by taking the difference between the corresponding gradient tensors in the original dataset and the reconstructed gradient dataset, as follows: Among them, F i Let G represent the i-th gradient tensor in the original dataset. i Let represent the i-th gradient tensor in the reconstructed gradient dataset, and ||·|2 represent the L2 norm; The autoencoder is iteratively optimized by minimizing the reconstruction loss function L.

6. The federated fine-tuning method for large language models based on gradient compression according to claim 1, characterized in that, The step of obtaining compressed gradients using a pre-trained encoder and transmitting them to the server during the local training phase of the global model on the client side includes: The global model is trained locally on each client based on low-rank decomposition, and the gradient low-rank decomposition result containing multiple low-dimensional matrix products is obtained. On each client, a pre-trained encoder is used to encode the low-rank gradient decomposition result to obtain the compressed gradient. Each client sends the compressed gradient to the server.

7. The federated fine-tuning method for large language models based on gradient compression according to claim 1, characterized in that, The step of using a pre-trained decoder to decode and aggregate the compressed gradients during the global aggregation phase on the server side, and then using the aggregated gradients to update the global model, includes: The server collects compressed gradients from at least one selected client. The server decodes the compressed gradients using a pre-trained encoder. The server performs average aggregation of the decoded gradients on the subspace of the low-rank decomposition. The server updates the global model parameters using gradient descent based on the aggregated gradients.

8. A federated fine-tuning device for a large language model based on gradient compression, characterized in that, include: Gradient compression autoencoder pre-training module and gradient compression large language model federated fine-tuning module; The gradient compression autoencoder pre-training module is used to construct an original dataset with time-series relationship based on the gradient tensor generated during the fine-tuning of the large language model. The original dataset is input into the autoencoder for inference to obtain the reconstructed gradient dataset. Based on the original dataset and the reconstructed gradient dataset, a reconstruction loss function is constructed to optimize the autoencoder. The optimized autoencoder is then split into a pre-trained encoder and a pre-trained decoder. The gradient compression large language model federated fine-tuning module is used to initialize the base model of the large language model as a global model on the server side. The server side updates the global model to at least one selected client. During the local training phase of the global model on the client side, the compressed gradient is obtained by using a pre-trained encoder and sent to the server side. During the global aggregation phase of the global model on the server side, the compressed gradient is decoded and aggregated by a pre-trained decoder. The aggregated gradient is used to update the global model, thereby obtaining the federated fine-tuned global model as the final application large language model deployed on the server side and updated to the client side.

9. A large language model federated fine-tuning device based on gradient compression, comprising a memory and one or more processors, wherein the memory is used to store computer programs, characterized in that, The processor is configured to implement the gradient compression-based large language model federated fine-tuning method according to any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a computer, it implements the large language model federated fine-tuning method based on gradient compression as described in any one of claims 1-7.

Images