Large language model federated fine-tuning method based on structural sparsification and related device

Abstract

The application belongs to the technical field of model training, and provides a large language model federated fine-tuning method based on structural sparsification and related equipment. Embodiments of the application train a large language model and an adapter model through local sample data, obtain model parameters of the adapter model, the model parameters including a low-rank first parameter matrix and a second parameter matrix; perform structural sparsification processing on the first parameter matrix and the second parameter matrix respectively, and then perform element-level compression processing on a sparse adaptation matrix obtained by multiplying the first sparsified matrix and the second sparsified matrix, to obtain compressed matrix information; update the next round of model parameters of the large language model based on a target compressed matrix that is aggregated based on compressed matrix information of multiple clients and sent by a server, and repeatedly perform the model training step until a trained large language model is obtained. The application can reduce compression errors in a federated LoRA fine-tuning scenario and reduce the negative impact of fine-tuning on the LLM.

Description

Technical Field

[0001] This application belongs to the field of model training technology, and in particular relates to a federated fine-tuning method and related equipment for large language models based on structural sparsity. Background Technology

[0002] Large Language Models (LLMs), as one of the core technologies in the field of artificial intelligence, have profoundly changed the way information is processed, language understanding, and knowledge extraction are performed, driving a revolution in next-generation information technology. During the training process of LLMs, algorithms such as Low-Rank Adaptation (LoRA) are generally used to transform the training of LLMs into fine-tuning the parameters of low-rank matrices, thereby reducing the number of training parameters. Furthermore, to protect the privacy of training data from each client, methods such as Federated Learning (FL) can be used to achieve collaborative training among multiple clients.

[0003] Although LoRA fine-tuning can significantly reduce the number of trainable parameters in an LLM, it still introduces significant communication overhead due to the large size of the LLM in federated learning scenarios. Furthermore, the communication efficiency of model training is further constrained by the limited uplink bandwidth in federated learning systems.

[0004] In related technologies, model gradient compression methods, such as quantization and sparsification, can be used to address the communication overhead during the fine-tuning process. Sparsification, in particular, has attracted significant attention due to its ability to provide higher compression ratios. However, existing sparsification methods are not suitable for federated LoRA fine-tuning scenarios, as they easily introduce large compression errors, negatively impacting LLM fine-tuning. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method and related equipment for federated fine-tuning of large language models based on structural sparsity, which can reduce compression errors in federated LoRA fine-tuning scenarios and reduce the negative impact on LLM fine-tuning.

[0006] A first aspect of this application provides a federated fine-tuning method for large language models based on structural sparsity, applied to a client, the method comprising:

[0007] Obtain pre-trained large language models and adapter models;

[0008] The large language model and the adapter model are trained using local sample data to obtain the model parameters of the adapter model after this round of training; the model parameters include a low-rank first parameter matrix and a second parameter matrix;

[0009] The first parameter matrix and the second parameter matrix are subjected to structural sparsification to obtain a first sparsification matrix and a second sparsification matrix; the structural sparsification is used to indicate the row vectors or column vectors with key information in the first parameter matrix and the second parameter matrix to be retained; the first sparsification matrix and the second sparsification matrix are submatrices of the first parameter matrix and the second parameter matrix, respectively.

[0010] The sparse adaptation matrix is ​​compressed element-wise to obtain the compressed matrix information of the adapter model after this round of training; the sparse adaptation matrix is ​​obtained by multiplying the first sparse matrix and the second sparse matrix.

[0011] The compression matrix information is sent to the server, which aggregates the compression matrix information sent by multiple clients and returns the aggregated target compression matrix to each client.

[0012] The next round of model parameters for the large language model is updated based on the target compression matrix, and the model training steps are repeated until the trained large language model is obtained.

[0013] In some embodiments, performing structural sparsification on the first parameter matrix and the second parameter matrix respectively to obtain a first sparsified matrix and a second sparsified matrix includes:

[0014] Obtain structural sparse parameters; the structural sparse parameters are used to characterize the ratio of row vectors and column vectors determined from the first parameter matrix and the second parameter matrix, respectively;

[0015] Based on the structural sparsity parameters, the first parameter matrix and the second parameter matrix are subjected to structural sparsification processing to obtain the first sparsification matrix and the second sparsification matrix.

[0016] In some embodiments, the structural sparsity parameter includes a first sparsity parameter and a second sparsity parameter, wherein the first sparsity parameter is used to characterize the ratio of the filtered row vectors and the second sparsity parameter is used to characterize the ratio of the filtered column vectors.

[0017] The step of performing structural sparsification on the first parameter matrix and the second parameter matrix based on the structural sparsity parameters to obtain the first sparsified matrix and the second sparsified matrix includes:

[0018] Calculate the square norm of each row vector in the first parameter matrix, and calculate the square norm of each column vector in the second parameter matrix;

[0019] The target row vectors corresponding to the first sparse parameters are determined according to the order of the square norm of each row vector in the first parameter matrix from largest to smallest, and the first sparsification matrix is ​​constructed based on the target row vectors.

[0020] The target column vector corresponding to the second sparse parameter is determined according to the order of the square norm of each column vector in the second parameter matrix from largest to smallest, and the second sparsification matrix is ​​constructed based on the target column vector.

[0021] In some embodiments, the structural sparsity parameter is determined by the following steps:

[0022] Determine the selection metric for the structured sparsity parameters;

[0023] For each row vector in the first parameter matrix and each column vector in the second parameter matrix, modeling is performed under the selected metric to construct a first metric distribution function for the first parameter matrix and a second metric distribution function for the second parameter matrix;

[0024] Based on the selection metric, the first metric distribution function, and the second metric distribution function, a compression error function for the structural sparsity processing is defined, wherein the compression error function is a function related to the structural sparsity parameters;

[0025] Under the constraint of the upper limit of available resources, the compression error function is minimized to calculate the value of the structural sparsity parameter.

[0026] In some embodiments, the element-wise compression of the sparse adaptation matrix to obtain the compressed matrix of the adapter model after this round of training includes:

[0027] The sparse adaptation matrix is ​​obtained by performing a multiplication operation based on the first sparsification matrix and the second sparsification matrix;

[0028] Obtain element sparsity parameters, which are calculated based on the communication bandwidth constraints of the client;

[0029] Based on the sparse parameters of the elements, the sparse adaptation matrix is ​​compressed element-wise to obtain the compressed matrix information of the adapter model after this round of training.

[0030] In some embodiments, the element sparsity parameter includes a preset number of elements;

[0031] The step of performing element-wise compression on the sparse adaptation matrix based on the element-wise sparse parameters to obtain the compressed matrix information of the adapter model after this round of training includes:

[0032] Target matrix elements that satisfy the preset number of elements are determined from the sparse adaptive matrix in descending order of the absolute values ​​of the matrix elements.

[0033] Obtain the position index of the target matrix element, and use the target matrix element and the corresponding position index as the compression matrix information of the adapter model after this round of training.

[0034] In some embodiments, updating the next round of model parameters of the large language model based on the target compression matrix and repeating the model training steps until the trained large language model is obtained includes:

[0035] The current model parameters of the large language model and the target compression matrix are used as the next model parameters of the large language model.

[0036] The model parameters of the adapter model are randomly initialized, and the steps of training the large language model and the adapter model with local sample data are returned until the trained large language model is obtained.

[0037] In some embodiments, during model training, the model parameters of the large language model are frozen, and the model parameters of the adapter model are updated.

[0038] A second aspect of this application provides a federated fine-tuning device for a large language model based on structural sparsity, applied to a client, the device comprising:

[0039] The acquisition module is used to acquire pre-trained large language models and adapter models;

[0040] The training module is used to train the large language model and the adapter model using local sample data to obtain the model parameters of the adapter model after this round of training; the model parameters include a low-rank first parameter matrix and a second parameter matrix;

[0041] A structured sparsity module is used to perform structured sparsity processing on the first parameter matrix and the second parameter matrix respectively to obtain a first sparsity matrix and a second sparsity matrix; the structured sparsity processing is used to indicate the retention of row vectors or column vectors with key information in the first parameter matrix and the second parameter matrix; the first sparsity matrix and the second sparsity matrix are submatrices of the first parameter matrix and the second parameter matrix respectively;

[0042] The element-sparse module is used to perform element-level compression processing on the sparse adaptation matrix to obtain the compressed matrix information of the adapter model after this round of training; the sparse adaptation matrix is ​​obtained by multiplying the first sparse matrix and the second sparse matrix.

[0043] The sending module is used to send the compression matrix information to the server. The server aggregates the compression matrix information sent by multiple clients and returns the aggregated target compression matrix to each client.

[0044] The update module is used to update the next round of model parameters of the large language model based on the target compression matrix, and repeat the model training steps until the trained large language model is obtained.

[0045] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the large language model federated fine-tuning method based on structural sparsity provided in the first aspect.

[0046] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the large language model federated fine-tuning method based on structural sparsity provided in the first aspect.

[0047] The fifth aspect of this application provides a computer program product that, when run on a processor, enables the processor to implement the steps of the large language model federated fine-tuning method based on structural sparsity provided in the first aspect.

[0048] The first aspect of this application provides a federated fine-tuning method for a large language model based on structural sparsity. This method involves obtaining a pre-trained large language model and an adapter model; training the large language model and adapter model using local sample data to obtain the model parameters of the adapter model after this training round; the model parameters include a low-rank first parameter matrix and a second parameter matrix; and performing structural sparsity processing on the first parameter matrix and the second parameter matrix respectively to obtain a first sparsified matrix and a second sparsified matrix; the structural sparsity processing is used to indicate the row vectors or column vectors that retain key information in the first parameter matrix and the second parameter matrix; the first... The sparse matrix and the second sparse matrix are submatrices of the first parameter matrix and the second parameter matrix, respectively. Element-wise compression is performed on the sparse adaptation matrix to obtain the compressed matrix information of the adapter model after this round of training. The sparse adaptation matrix is ​​obtained by multiplying the first sparse matrix and the second sparse matrix. The compressed matrix information is sent to the server, which aggregates the compressed matrix information sent by multiple clients and returns the aggregated target compressed matrix to each client. The next round of model parameters of the large language model is updated based on the target compressed matrix, and the model training steps are repeated until the trained large language model is obtained.

[0049] Therefore, this application provides a novel federated fine-tuning framework called Structured Sparsity-Based Federated LoRA Fine-tuning (FLR-SS). By retaining the row or column vectors with key information in the first and second parameter matrices respectively, structural sparsity processing is performed, ensuring that the resulting first and second sparsified matrices possess structured sparsity, i.e., containing entire rows and columns of zero values. Thus, in subsequent matrix multiplication calculations, these zero-valued parts can be directly skipped, thereby reducing both communication and computational overhead. Furthermore, since the first and second sparsified matrices retain the row or column vectors with key information from the original matrices, the structure of matrix multiplication of two low-rank matrices in the adapter model and the dependencies between rows and columns in matrix multiplication are considered. This reduces the gap between the matrix multiplication result obtained through structural sparsification and the result of multiplying the original low-rank matrices, thereby reducing compression errors in federated LoRA fine-tuning scenarios, reducing the negative impact on LLM fine-tuning, and improving the model's convergence rate.

[0050] Furthermore, after completing the structural sparsity processing, element-level compression is applied to the sparse adaptation matrix obtained by matrix multiplication to meet the strict bandwidth limitations of client uploads, enabling collaborative sparsity compression of low-rank matrices on resource-constrained clients. Additionally, the server then aggregates these compressed matrix information, which possess both structured and element-level sparsity, avoiding the inconsistencies inherent in directly aggregating low-rank matrices.

[0051] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating a federated fine-tuning method for a large language model based on structural sparsity, provided in an embodiment of this application.

[0054] Figure 2 This is a partial flowchart of another federated fine-tuning method for large language models based on structural sparsity provided in an embodiment of this application;

[0055] Figure 3This is a schematic diagram of a sparse compression process provided in an embodiment of this application;

[0056] Figure 4 This is a partial flowchart of another federated fine-tuning method for large language models based on structural sparsity provided in an embodiment of this application;

[0057] Figure 5 This is a partial flowchart of another federated fine-tuning method for large language models based on structural sparsity provided in an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of the structure of a federated fine-tuning device for a large language model based on structural sparsity, provided in an embodiment of this application.

[0059] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0060] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0061] It should be understood that, when used in this application, the term "comprising" indicates the presence of the described feature, integral, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0062] It should also be understood that the term “and / or” as used in this application refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes such combinations.

[0063] As used in this application, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0064] Furthermore, in the description of this application, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0065] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0066] First, some terms used in the embodiments of this application will be explained to facilitate understanding by those skilled in the art.

[0067] Large Language Models (LLMs) are deep learning-based artificial intelligence models capable of understanding, generating, and reasoning about natural language. The core characteristic of LLMs is their massive number of parameters (e.g., typically billions to trillions), and they are trained on vast amounts of text data, thus possessing extensive language processing capabilities.

[0068] Parameter-Efficient Fine-Tuning (PEFT) is a transfer learning technique that adapts a pre-trained language model to various downstream tasks by updating only a very small portion of the parameters in the large language model while keeping most of the original parameters unchanged.

[0069] Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning algorithm that has attracted widespread attention due to its simplicity and efficiency. LoRA introduces a pair of trainable low-rank matrices into the training layers of an LLM (Limited Model), while freezing the parameters of the original model, thus transforming the training of the LLM into fine-tuning the parameters of the low-rank matrices. This design significantly reduces the number of trainable parameters, enabling efficient fine-tuning of LLMs even in resource-constrained environments.

[0070] Federated Learning (FL) is a machine learning framework that uses data for learning and training while ensuring privacy and data security. As a distributed machine learning paradigm, Federated Learning can effectively solve the data silo problem, allowing participants to jointly model data without sharing it, thus technically breaking down data silos and enabling collaborative training.

[0071] Stochastic Gradient Descent (SGD) is an iterative algorithm for optimizing machine learning models. It updates model parameters by using the gradient of only one sample (or mini-batch of samples) at a time, thereby accelerating convergence and reducing computational costs.

[0072] Generally, the large number of parameters in Large Language Models (LLMs) means that processing input data requires significant computational resources, making it impossible to train LLMs from scratch. To address this issue, the Low-Rank Adaptation (LoRA) algorithm is employed, transforming LLM training into fine-tuning the parameters of a low-rank matrix. Furthermore, in real-world scenarios, training data for LLMs is typically distributed across multiple geographically dispersed clients, and due to privacy concerns, the server training the LLM cannot centralize individual data from clients, making direct fine-tuning of the LLM difficult. Federated Learning (FL) solves this problem through decentralized training, where clients train the model locally and periodically send model gradients to a Parameter Server (PS) for aggregation, enabling collaborative training across multiple clients. By combining FL with LoRA, collaborative LLM fine-tuning based on distributed private datasets is achieved.

[0073] Although LoRA fine-tuning significantly reduces the number of trainable parameters in an LLM, significant communication overhead still occurs in federated learning scenarios due to the sheer size of the LLM. For example, fine-tuning a GPT-3 model with 175 billion parameters still requires transmitting approximately 1 billion parameters per round. Furthermore, the communication efficiency of model training is further constrained by the limited uplink bandwidth in federated learning systems, especially in wireless or edge environments.

[0074] In related technologies, model gradient compression methods, such as quantization and sparsification, can be used to address the communication overhead during the fine-tuning process. Sparsification, in particular, has attracted significant attention due to its ability to provide higher compression ratios. However, existing sparsification methods are not suitable for federated LoRA fine-tuning scenarios, as they easily introduce large compression errors, negatively impacting LLM fine-tuning.

[0075] To address the aforementioned technical issues, this application provides a federated fine-tuning method for large language models based on structural sparsity. This method involves acquiring a pre-trained large language model and an adapter model; training the large language model and adapter model using local sample data to obtain the model parameters of the adapter model after this training round; the model parameters include a low-rank first parameter matrix and a second parameter matrix; and performing structural sparsity processing on the first parameter matrix and the second parameter matrix respectively to obtain a first sparsified matrix and a second sparsified matrix; the structural sparsity processing is used to indicate which row vectors or column vectors with key information in the first parameter matrix and the second parameter matrix are retained. The first and second sparsification matrices are submatrices of the first and second parameter matrices, respectively. Element-wise compression is performed on the sparse adaptation matrix to obtain the compressed matrix information of the adapter model after this round of training. The sparse adaptation matrix is ​​obtained by multiplying the first and second sparsification matrices. The compressed matrix information is sent to the server, which aggregates the compressed matrix information sent by multiple clients and returns the aggregated target compressed matrix to each client. The next round of model parameters of the large language model is updated based on the target compressed matrix, and the model training steps are repeated until the trained large language model is obtained.

[0076] Therefore, this application provides a novel federated fine-tuning framework called Structured Sparsity-Based Federated LoRA Fine-tuning (FLR-SS). By retaining the row or column vectors with key information in the first and second parameter matrices respectively, structural sparsity processing is performed, ensuring that the resulting first and second sparsified matrices possess structured sparsity, i.e., containing entire rows and columns of zero values. Thus, in subsequent matrix multiplication calculations, these zero-valued parts can be directly skipped, thereby reducing both communication and computational overhead. Furthermore, since the first and second sparsified matrices retain the row or column vectors with key information from the original matrices, the structure of matrix multiplication of two low-rank matrices in the adapter model and the dependencies between rows and columns in matrix multiplication are considered. This reduces the gap between the matrix multiplication result obtained through structural sparsification and the result of multiplying the original low-rank matrices, thereby reducing compression errors in federated LoRA fine-tuning scenarios, reducing the negative impact on LLM fine-tuning, and improving the model's convergence rate.

[0077] Furthermore, after completing the structural sparsity processing, element-level compression is applied to the sparse adaptation matrix obtained by matrix multiplication to meet the strict bandwidth limitations of client uploads, enabling collaborative sparsity compression of low-rank matrices on resource-constrained clients. Additionally, the server then aggregates these compressed matrix information, which possess both structured and element-level sparsity, avoiding the inconsistencies inherent in directly aggregating low-rank matrices.

[0078] like Figure 1 As shown in the embodiment of this application, a federated fine-tuning method for a large language model based on structural sparsity is provided and applied to a client, including the following steps S101 to S106:

[0079] Step S101: Obtain the pre-trained large language model and adapter model.

[0080] Among them, the pre-trained large language model refers to the pre-trained base model, which may include, but is not limited to, models such as ChatGPT and LLaMA.

[0081] The adapter model can be a fine-tuned model that employs parameter-efficient fine-tuning techniques. Examples include LoRA (Lower-Rank Adaptation) and SFT (Supervised Fine-Tuning). In this embodiment, LoRA is used as an example for illustration.

[0082] Optionally, the server determines which clients need to participate in federated learning and distributes a pre-trained large language model and its corresponding adapter model to the clients. Alternatively, the clients can directly obtain the pre-trained large language model and its corresponding adapter model from the server.

[0083] Step S102: Train the large language model and the adapter model using local sample data to obtain the model parameters of the adapter model after this round of training; the model parameters include a low-rank first parameter matrix and a second parameter matrix.

[0084] Local sample data can refer to a client-owned local dataset. The data type of the local sample data is related to the downstream tasks that the large language model needs to apply. For example, if the downstream task includes image recognition, the local sample data can include, but is not limited to, at least one type of data such as image sample data and video sample data. If the downstream task includes video-text understanding, the local sample data can include, but is not limited to, multimodal data such as video sample data and text sample data.

[0085] Optionally, the client first acquires local sample data, and then uses this local sample data to train the large language model and the adapter model in each training round. During model training, the model parameters of the large language model are frozen, and the model parameters of the adapter model are updated to obtain the model parameters of the adapter model after this round of training. The model parameters include a low-rank first parameter matrix. Second parameter matrix .

[0086] This section uses the LoRA adapter model as an example to briefly describe the fine-tuning training process involved in this application. The core idea of ​​LoRA is to not update the weight matrix of the complete large language model. Instead, it uses the model matrix of a pre-trained large language model. Freeze, and update only the two inserted low-rank matrices: and ,in The updated weights are represented as ,in LoRA enables LLM fine-tuning across distributed, resource-constrained clients in FL, thereby expanding the deployment scope of LLM.

[0087] In federated learning, LoRA provides a practical solution for fine-tuning LLMs on resource-constrained distributed clients, assuming a given set of clients. A federated learning system with distributed clients, each client... Holding local datasets The global model consists of a fixed pre-trained weight matrix. and two low-rank matrices trained by the client. and Components. Client The adaptation model is represented as Its local loss function is defined as ,in This represents the loss evaluation function for a specific task. Indicates from dataset The samples are randomly sampled from the middle. The goal of federated LoRA fine-tuning is to update only and share the low-rank matrix. and To minimize the global loss function:

[0088]

[0089] Federated LoRA fine-tuning can involve multiple global iterative processes, with all clients pre-deploying the same large language model. To ensure the consistency of the initial parameters, in the first... During the round of iteration, the server will set the global LoRA parameters. and Distributed to all clients, clients Initialize the local LoRA weights to and And perform using stochastic gradient descent (SGD) The next local iteration, specifically the steps are as follows:

[0090]

[0091]

[0092] in, , Indicates the learning rate. Indicates the first The first round of global iteration During the next local iteration, from the local dataset A sample randomly selected from the data.

[0093] Specifically, on the client side Conduct the first Before the start of the global training round, the initial model parameters of the adapter model are low-rank matrices. and During the training process, the SGD algorithm can be used to train low-rank matrices. and Fine-tuning can be performed locally on the client side. After the next local iteration, the updated low-rank matrix is ​​finally obtained, which is the first parameter matrix of the low-rank matrix. Second parameter matrix .

[0094] Step S103: Perform structural sparsification on the first parameter matrix and the second parameter matrix respectively to obtain a first sparsification matrix and a second sparsification matrix; the structural sparsification is used to indicate the row vectors or column vectors with key information in the first parameter matrix and the second parameter matrix to be retained; the first sparsification matrix and the second sparsification matrix are submatrices of the first parameter matrix and the second parameter matrix respectively.

[0095] Because fine-tuning training uses an adapter model, only two inserted low-rank matrices need to be updated. Therefore, in each global iteration of the federated LoRA training process, the number of parameters trained and communicated by the client is only 0.1% to 1% of the total LLM model parameters. Resource-constrained clients can participate in LLM fine-tuning, and the communication efficiency of FL is significantly improved. However, due to the large language model's weight matrix... The dimensionality is extremely high; even if only the first and second parameter matrices of the adapter model are transmitted, i.e., the low-rank matrix... and The communication cost of LoRA can also be unacceptable. To further reduce communication overhead, in low-rank matrices... and Before data is transmitted from the client to the server, related technologies primarily employ algorithms such as TopK sparsity to compress it. While these compression algorithms achieve some success, they are not actually suitable for federated LoRA fine-tuning of LLMs.

[0096] The inventors of this application discovered that existing compression algorithms primarily perform independent compression of two low-rank matrices based on matrix elements, without considering the inherent structural characteristics of matrix multiplication in federated LoRA fine-tuning for LLMs. Specifically, in LoRA, two low-rank matrices require multiplication before being applied to the input data. If the low-rank matrices are compressed separately... and Applying an independent TopK compression algorithm will significantly reduce the approximate value of their product, i.e. In other words, this compression algorithm ignores the structure of matrix multiplication and the dependencies between rows and columns in matrix multiplication. In this case, a smaller element in a low-rank matrix may be amplified after multiplying with a larger element in another low-rank matrix, resulting in the erroneous discarding of important information and thus easily introducing large compression errors.

[0097] To address the error issues arising from applying the aforementioned compression algorithm to federated LoRA fine-tuning of LLM, this embodiment of the application performs matrix multiplication locally on the client side, generating the matrix before subsequent processing. However, directly performing matrix multiplication will produce different results. and The computational and memory overhead of matrix multiplication makes it difficult to implement in high-dimensional LLM scenarios. To address this issue, this application proposes a novel Federated LoRA fine-tuning algorithm, called Federated LoRA with Structural Sparsification (FLR-SS). FLR-SS employs structural sparsification techniques to reduce the first parameter matrix while preserving key information. Second parameter matrix The dimension is reduced to decrease the resources required for subsequent matrix multiplication.

[0098] Optionally, the client obtains a first sparse matrix by performing structural sparsification on the first parameter matrix and a second sparse matrix by performing structural sparsification on the second parameter matrix. This structural sparsification process indicates the retention of row or column vectors containing key information in both the first and second parameter matrices. Key information refers to the most informative components in both the first and second parameter matrices. The first sparse matrix is ​​a submatrix of the first parameter matrix, and the second sparse matrix is ​​a submatrix of the second parameter matrix. That is, the first sparse matrix consists of a portion of the row vectors from the first parameter matrix, and the second sparse matrix consists of a portion of the column vectors from the second parameter matrix.

[0099] In some embodiments, such as Figure 2 As shown, the step of performing structural sparsification on the first parameter matrix and the second parameter matrix respectively to obtain a first sparsified matrix and a second sparsified matrix includes:

[0100] Step S201: Obtain structural sparse parameters; the structural sparse parameters are used to characterize the ratio of row vectors and column vectors determined from the first parameter matrix and the second parameter matrix, respectively;

[0101] Step S202: Based on the structural sparsity parameters, perform structural sparsification processing on the first parameter matrix and the second parameter matrix respectively to obtain the first sparsification matrix and the second sparsification matrix.

[0102] Optionally, after obtaining the pre-calculated structural sparsity parameters, the client can select the most informative row and column vectors to be retained from the first parameter matrix and the second parameter matrix, respectively, based on the ratio of row and column vectors indicated by the structural sparsity parameters. This allows for the performance of structural sparsity processing, forming the first and second sparsity matrices from the retained row and column vectors. For example, the selection of row or column vectors can be based on the magnitude of the row vectors in the first parameter matrix or the column vectors in the second parameter matrix. The vector magnitude can be selected using at least one metric, such as the square norm or the L1 norm. The square norm describes the Euclidean distance between vectors or the sum of squares of a matrix. The L1 norm describes the sum of the absolute values ​​of the vector elements.

[0103] In some embodiments, the structural sparsity parameters include a first sparsity parameter and a second sparsity parameter, wherein the first sparsity parameter is used to characterize the ratio of the selected row vectors (e.g., ...). The second sparse parameter is used to characterize the ratio of the selected column vectors (e.g., The first sparse parameter reflects the ratio of the number of selected row vectors to the total number of corresponding row vectors. The second sparse parameter reflects the ratio of the number of selected column vectors to the total number of corresponding column vectors. For example, if the total number of row vectors in the first parameter matrix is ​​10, If the value is 0.2, then there are 2 row vectors determined from the first parameter matrix. If the total number of row vectors in the second parameter matrix is ​​20, If the value is 0.3, then there are 6 column vectors determined from the second parameter matrix.

[0104] Specifically, such as Figure 3 As shown, after local fine-tuning, based on the properties of matrix multiplication, by starting from the low-rank first parameter matrix... Selected from Row, from the second parameter matrix Selected from The column vectors and all other unselected row and column vectors are set to zero. The two matrices are structurally sparsified, and the resulting first and second sparsified matrices are denoted as follows: and This structural sparsification introduces entire rows and columns of zero elements into the matrix multiplication result. These elements can be ignored and have no impact on the final result. Therefore, the sparse adaptation matrix obtained by the final matrix multiplication... Size from Reduced to This significantly reduces the computational and memory overhead required for matrix multiplication. For example, when each matrix retains only 10% of its elements (i.e., ... When multiplying the matrices, the computational and memory costs required for the final matrix multiplication are only 1% of the original values.

[0105] In some embodiments, such as Figure 4 As shown, the step of performing structural sparsification on the first parameter matrix and the second parameter matrix based on the structural sparsity parameters to obtain the first sparsified matrix and the second sparsified matrix includes:

[0106] Step S401: Calculate the square norm of each row vector in the first parameter matrix and calculate the square norm of each column vector in the second parameter matrix;

[0107] Step S402: Determine the target row vector corresponding to the first sparse parameter according to the order of the square norm of each row vector in the first parameter matrix from largest to smallest, and construct the first sparsification matrix based on the target row vector;

[0108] Step S403: Determine the target column vector corresponding to the second sparse parameter according to the order of the square norm of each column vector in the second parameter matrix from largest to smallest, and construct the second sparsification matrix based on the target column vector.

[0109] In this embodiment, taking the square norm as the selection metric, the client calculates the square norm of each row vector in the first parameter matrix and the square norm of each column vector in the second parameter matrix. The terminal determines the target row vector corresponding to the first sparse parameter from the first parameter matrix in descending order of the square norm of each row vector, sets the other row vectors to zero, and combines the target row vectors to obtain the first sparsification matrix. Similarly, it determines the target column vector corresponding to the second sparse parameter in descending order of the square norm of each column vector in the second parameter matrix, sets the other column vectors to zero, and combines the target column vectors to obtain the second sparsification matrix.

[0110] In some embodiments, the structural sparsity parameter is determined by the following steps:

[0111] Determine the selection metric for the structured sparsity parameters;

[0112] For each row vector in the first parameter matrix and each column vector in the second parameter matrix, modeling is performed under the selected metric to construct a first metric distribution function for the first parameter matrix and a second metric distribution function for the second parameter matrix;

[0113] Based on the selection metric, the first metric distribution function, and the second metric distribution function, a compression error function for the structural sparsity processing is defined, wherein the compression error function is a function related to the structural sparsity parameters;

[0114] Under the constraint of the upper limit of available resources, the compression error function is minimized to calculate the value of the structural sparsity parameter.

[0115] In this embodiment, the compression error introduced by the structural sparsity of low-rank matrices is analyzed to explore the optimal vector selection strategy under resource constraints, i.e., to determine the structural sparsity parameters. The process of determining the structural sparsity parameters is described in detail below.

[0116] In FLR-SS, the training process of the global model is represented by the following formula: Client The local update rule for upper-low-rank matrices is defined as follows:

[0117]

[0118]

[0119] in The transformation in the first equation is because LoRA sets up a low-rank matrix. It is initialized to a zero matrix. To analyze the convergence properties of the FLR-SS algorithm, some general assumptions need to be made about the model being analyzed, including the assumption that the training loss function is... Smooth, stochastic gradient It is unbiased, and the upper bound of its square norm is . In addition, relevant positive constants are defined. This makes the following inequality true:

[0120]

[0121]

[0122]

[0123] Related research has proven the validity of the above assumptions. This application extends this research by analyzing the first parameter matrix in low-rank conditions. Second parameter matrix The convergence of LLM under co-compression and aggregation conditions is investigated, and the influence of compression error is also explored.

[0124] Both structural sparsification and TopK sparsification in FLR-SS introduce compression errors that may affect LLM convergence. Therefore, the compression error in FLR-SS is defined here as: Among them, compression error It is a variable that restricts the compressed result relative to the original matrix product. The upper limit of error.

[0125] Both structural sparsity and TopK sparsity in FLR-SS are derived from the original matrix. Select the most important element and set the remaining elements to zero. The resulting final compression error satisfies... This satisfies the necessary condition for model convergence.

[0126] FLR-SS passed In each global iteration, federated LoRA fine-tuning of the LLM is performed. During each global iteration, the client collaboratively compresses and aggregates the two low-rank matrices through structural sparsity. Based on the above assumptions and the definition of compression error, the learning rate is set to... The convergence rate of the model trained by FLR-SS is derived as follows:

[0127]

[0128] in, and , Represents the training loss function. This represents the loss function value corresponding to the globally optimal model. The smoothing constant represents the smoothness constant of the training loss function that satisfies smoothness. This represents the upper bound of the square norm of the stochastic gradient of the training loss function.

[0129] Based on the above analysis, this work concludes that FLR-SS achieves the same convergence rate as uncompressed federated LoRA fine-tuning. However, FLR-SS accelerates communication through compression technology, introducing additional compression errors. This error affects the convergence of the model; therefore, we will next explore how to minimize the compression error under given computational and communication constraints. .

[0130] Based on the above analysis, the compression error originates from two aspects: the structural sparsity of the low-rank matrix and the TopK sparsity of the resulting matrix. Since TopK minimizes the compression error under given communication constraints, no additional optimization is needed. The focus here shifts to choosing the low-rank first parameter matrix. Second parameter matrix The row and column vectors are used to minimize the compression error caused by structural sparsity.

[0131] Here, structural sparsity is defined as: from the first parameter matrix Selected from Row, from the second parameter matrix Selected from Columns are selected to reduce the computational and memory overhead of matrix multiplication operations. The submatrix corresponding to each row is used as the first sparsified matrix after structural sparsification. Selected The submatrix corresponding to each column serves as the second sparsified matrix after structural sparsification. And constraints were defined. ,in This represents the upper limit of available resources. Furthermore, it allows... and They respectively represent the first parameter matrix The set of rows selected from the second parameter matrix The set of columns selected in the above definition. Based on the above definition, the following optimization problem is designed:

[0132]

[0133]

[0134]

[0135]

[0136] in, and Let each represent the first parameter matrix of the low rank. Second parameter matrix The row and number List.

[0137] However, optimization problem This presents a certain challenge because the optimization variables in this problem involve two optimization sets. and The optimization of sets significantly expands and complicates the search space, making it more difficult to solve the optimization problem. First, a selection metric (i.e., a standard) is defined for selecting row and column vectors. Then, the numerical value of the structural sparsity parameter, i.e., the first sparsity parameter, is calculated based on this selection metric. With the second sparse parameter The value of .

[0138] Next, we first analyze the objective function for optimization, and we can obtain the following results:

[0139]

[0140]

[0141] Based on the above analysis of the loss function, we can conclude that the upper bound of the optimization objective function is influenced by the square norm of the row and column vectors in the matrix. Furthermore, rows and columns with larger norms can contain more information related to the matrix, i.e., they contain more information. Therefore, in the FLR-SS algorithm, structural sparsity uses the square norm of each vector as a selection metric. Specifically, this algorithm starts from the first parameter matrix... Select the one with the largest norm Each row vector, from the second parameter matrix Select the one with the largest norm Column vectors. By defining the square norm as the selection metric, the result of structural sparsification is then expressed as:

[0142]

[0143] in, and Let each represent the first parameter matrix. Sort by square norm Large row vectors, and the second parameter matrix Sort by square norm Large column vectors.

[0144] To analyze the compression error, the first parameter matrix... Each row vector and the second parameter matrix For each column vector, modeling is performed under a chosen metric, constructing a first metric distribution function for the first parameter matrix and a second metric distribution function for the second parameter matrix. Specifically, a statistical distribution can be used to model the squared norm of each row vector and each column vector respectively. Based on experimental results, the following conclusion is proved: During the federated LoRA fine-tuning process of LLM, the first parameter matrix... Row vectors and second parameter matrix The square norms of the column vectors in the middle, when arranged in descending order, follow a power-law distribution, that is: ,in, It is the first metric distribution function of the first parameter matrix. It is the second metric distribution function of the second parameter matrix, and the parameters are... It is the decay exponent, which controls the decay rate of the distribution, while It is a constant greater than 0.

[0145] Based on the selection metric of structural sparsity and the first and second metric distribution functions described above under this selection metric, a compression error function for structural sparsity processing is defined. The compression error function is related to the structural sparsity parameter. and The relevant functions are then optimization problems. The objective function can be re-derived as follows:

[0146]

[0147]

[0148]

[0149]

[0150] Based on the above analysis, the optimization problem After adjustments, the following optimization problem is obtained: determining the optimal structural sparsity parameters. and :

[0151]

[0152]

[0153] when When fixed, compression error function right The second derivative is ,therefore, It is about A convex function. Similarly, when When fixed, It is about A convex function. Given a pair and Under linear constraints, This is a biconvex optimization problem, which can be solved by using the alternating convex search (ACS) method with different initial values. To achieve an efficient solution, the compression error function is minimized under the constraint of the upper limit of available resources, thereby calculating the structural sparsity parameters. and The value.

[0154] After determining the structural sparsity parameters and Based on the square norm of the row and column vectors in the first and second parameter matrices, row and column vectors can be selected to generate structurally sparsified submatrices, namely the first sparsification matrix and the second sparsification matrix. By determining the structural sparsification parameters, the compression error introduced by FLR-SS can be minimized, the communication efficiency can be optimized in the federated LoRA fine-tuning of LLM, and the convergence of LLM can be accelerated.

[0155] The above embodiments analyze the convergence of the designed FLR-SS algorithm in theory and model the row / column vector selection problem in structural sparsification as a biconvex optimization problem based on the analysis results. This allows the problem to be solved efficiently, performing optimal structural sparsification on two low-rank matrices, reducing the difficulty and computational cost of determining the structural sparsification parameters, and improving model performance and convergence rate.

[0156] Step S104: Perform element-level compression on the sparse adaptation matrix to obtain the compressed matrix information of the adapter model after this round of training; the sparse adaptation matrix is ​​obtained by multiplying the first sparse matrix and the second sparse matrix.

[0157] Element-level compression refers to compressing each element of the matrix individually. Optionally, element-level compression may include, for example, Topk element sparsification.

[0158] Specifically, the client can first obtain the sparse adaptation matrix obtained by multiplying the first and second sparse matrices, and then perform element-wise compression on the matrix elements in the sparse adaptation matrix to obtain the compressed matrix information of the adapter model after this round of training. Optionally, the compressed matrix information may include the compressed matrix after element-wise compression.

[0159] In some embodiments, such as Figure 5 As shown, the element-wise compression of the sparse adaptation matrix to obtain the compressed matrix of the adapter model after this round of training includes:

[0160] Step S501: Perform a multiplication operation based on the first sparsification matrix and the second sparsification matrix to obtain the sparsity adaptation matrix;

[0161] Step S502: Obtain element sparsity parameters, which are calculated based on the communication bandwidth constraints of the client.

[0162] Step S503: Based on the element sparsity parameters, perform element-level compression processing on the sparse adaptation matrix to obtain the compressed matrix information of the adapter model after this round of training.

[0163] Specifically, after obtaining the first and second sparsification matrices, the client can perform local multiplication on these two sparsification matrices to obtain the sparsification adaptation matrix. Then, it obtains the element-wise sparsity parameters and, based on the parameter values ​​indicated by these parameters, performs element-wise compression processing on the sparsity adaptation matrix, i.e., further sparsification processing, to obtain the compressed matrix information of the adapter model after this round of training.

[0164] In some embodiments, the element sparsity parameter includes a preset number of elements. The step of performing element-wise compression processing on the sparse adaptation matrix based on the element sparsity parameter to obtain the compressed matrix information of the adapter model after this round of training includes:

[0165] Target matrix elements that satisfy the preset number of elements are determined from the sparse adaptive matrix in descending order of the absolute values ​​of the matrix elements.

[0166] Obtain the position index of the target matrix element, and use the target matrix element and the corresponding position index as the compression matrix information of the adapter model after this round of training.

[0167] Specifically, following the order of absolute values ​​of matrix elements in the sparse adaptation matrix from largest to smallest, the client determines the target matrix elements that satisfy the preset number of elements from the sparse adaptation matrix and sets the remaining matrix elements to zero, i.e., performing Topk sparsification processing on the sparse adaptation matrix. The client obtains the position index of the target matrix element and uses the encoding information and the corresponding position index of the target matrix element as the compression matrix information of the adapter model after this round of training.

[0168] In practical applications, continue as follows Figure 3 As shown, in the first sparse matrix Second sparsification matrix After performing the multiplication operation, the client Obtain the sparse adaptation matrix * Its dimension is significantly lower than that of the original matrix. Under the constraint of upload traffic, the client based on The absolute value of each element is used to select the target matrix element from the sparse adaptation matrix using TopK sparsity reduction, i.e., element-level compression is performed to obtain... TopK sparsity can minimize compression error while satisfying communication bandwidth constraints. Because... The dimensionality is reduced, and the number of bits required to represent the position index is reduced from... Reduce to .

[0169] This application introduces a structural sparsity strategy, obtaining a first sparsified matrix and a second sparsified matrix by selecting row and column vectors from the first parameter matrix and the second parameter matrix, thereby reducing the compression error between the compressed matrix and the original matrix. Furthermore, element-level compression processing is combined to meet the constraints of communication bandwidth.

[0170] To illustrate the compression error caused by the structural sparsity strategy introduced in this application, assume that two low-rank matrices exist on a client:

[0171]

[0172] Suppose that element-wise Top2 sparsity is applied to each matrix, retaining only the matrix... and The two elements with the largest absolute values ​​in the matrix are retained. Elements 5 and 4 in the middle, and the reserved Taking elements 4 and 5 from the matrix and setting the other matrix elements to zero, considering the matrix multiplication structure in LoRA, the compressed matrix product is:

[0173]

[0174] This means that the compressed low-rank matrix does not carry any effective information, and the compression error of directly using element-level sparsification is... .

[0175] In contrast, the structural sparsity method proposed in this application starts from a low-rank matrix. Select the row vectors with the most information (i.e., rows 1 and 4), from... Select the column vectors with the most information (i.e., columns 1 and 2), and generate a submatrix based on the selection. and Then calculate the product of the submatrices. And applying Top2 to this product yields:

[0176]

[0177] The compression error obtained based on the compression algorithm of this application is: This error value is much smaller than that of directly performing the element-level sparsity compression described above. This example demonstrates that, compared to element-level TopK compression, the structural sparsity strategy of this application better preserves the semantic content of the update matrix, thereby improving communication efficiency while maintaining accuracy.

[0178] The above embodiments significantly reduce the computational and memory overhead required for multiplying the low-rank first and second parameter matrices by utilizing structural sparsification techniques. Subsequently, TopK sparsification is directly applied to the sparse adaptation matrix of the resulting product. This avoids the additional errors introduced by directly sparsifying and aggregating the first and second parameter matrices, achieving high resource utilization while ensuring model accuracy, making it particularly suitable for bandwidth-constrained federated learning scenarios.

[0179] Step S105: The compression matrix information is sent to the server. The server aggregates the compression matrix information sent by multiple clients and returns the aggregated target compression matrix to each client.

[0180] Specifically, the client sends the compression matrix information to the server. The server receives the compression matrix information uploaded by N clients, aggregates it using a weighted average, and then outputs the aggregated target compression matrix. Send it to each client.

[0181] Step S106: Update the next round of model parameters of the large language model based on the target compression matrix, and repeat the model training steps until the trained large language model is obtained.

[0182] Optionally, after obtaining the target compression matrix, the client can use singular value decomposition (SVD) to compress the target compression matrix. Decompose into a new low-rank matrix and Then, based on the new low-rank matrix, the next round of global iteration is initiated, repeating multiple model training steps until the model convergence condition is met, resulting in a trained large language model. It should be noted that the target compression matrix... The decomposition step can be performed on the server. After the server performs the matrix decomposition calculation, it can output the new low-rank matrix. and Distribute to each client.

[0183] In some embodiments, updating the next round of model parameters of the large language model based on the target compression matrix and repeating the model training steps until the trained large language model is obtained includes:

[0184] The current model parameters of the large language model and the target compression matrix are used as the next model parameters of the large language model.

[0185] The model parameters of the adapter model are randomly initialized, and the steps of training the large language model and the adapter model with local sample data are returned until the trained large language model is obtained.

[0186] Specifically, the client obtains the target compression matrix. Update the frozen weight matrix according to the LoRA formula to obtain the target compression matrix. Replace the current model parameters of the local adapter model and combine them with the current model parameters of the large language model. Determine the next round of model parameters for the large language model. Before the next global iteration, each client can use the same random seed to randomly initialize the fitter model and regenerate the model parameters for the next round of fitter model. and Then, the next round of global iteration is started, that is, the step of training the large language model and the adapter model with local sample data is returned until the training convergence condition of the model is reached and the trained large language model is obtained.

[0187] In the above embodiments, due to the high-dimensionality of LLM, the compressed and aggregated target matrix is ​​directly distributed to the client. This allows the target compressed matrix to be directly applied to the frozen weight matrix, and the model parameters of the adapter model to be reinitialized without needing to perform matrix decomposition calculations on the target compressed matrix. This eliminates the time-consuming singular value decomposition operation on the target compressed matrix, reduces the computational overhead introduced by the decomposition process, and simplifies the model update process, thereby improving the fine-tuning efficiency of LLM.

[0188] The federated fine-tuning method for large language models based on structural sparsity provided in this application offers a novel federated fine-tuning framework, called Federated LoRA Fine-tuning Based on Structural Sparsity (FLR-SS). By retaining row or column vectors with key information in the first and second parameter matrices respectively for structural sparsity processing, it ensures that the resulting first and second sparsified matrices possess structured sparsity, i.e., containing entire rows and columns of zero values. Thus, in subsequent matrix multiplication calculations, these zero-value parts can be directly skipped, thereby reducing both communication and computational overhead. Furthermore, since the first and second sparsified matrices retain row or column vectors with key information from the original matrices, the structure of matrix multiplication of two low-rank matrices in the adapter model and the dependencies between rows and columns in matrix multiplication are considered. This reduces the gap between the matrix multiplication result obtained after structural sparsity processing and the result of multiplication of the original low-rank matrices, thereby reducing compression errors in federated LoRA fine-tuning scenarios, reducing the negative impact on LLM fine-tuning, and improving the model's convergence rate.

[0189] Furthermore, after completing the structural sparsity processing, element-level compression is applied to the sparse adaptation matrix obtained by matrix multiplication to meet the strict bandwidth limitations of client uploads, enabling collaborative sparsity compression of low-rank matrices on resource-constrained clients. Additionally, the server then aggregates these compressed matrix information, which possess both structured and element-level sparsity, avoiding the inconsistencies inherent in directly aggregating low-rank matrices.

[0190] Furthermore, the FLR-SS algorithm is suitable for federated LoRA fine-tuning compression scenarios. This algorithm uses structural sparsity techniques to achieve cooperative compression between low-rank matrices and theoretically guarantees the convergence of the LLM in FLR-SS fine-tuning. A corresponding optimization problem is constructed to calculate the relevant parameters of structural sparsity to mitigate the impact of compression errors. Extensive experiments were conducted on public NLP datasets. Experimental results show that FLR-SS outperforms existing algorithms in both model accuracy and communication efficiency, improving accuracy by 1.66% to 8.63% and reducing communication overhead by 21.42% to 53.33%. This application designs a corresponding sparsity strategy specifically for LoRA characteristics, avoiding errors introduced by independent compression and aggregation, and providing a new research direction for LoRA fine-tuning in resource-constrained federated environments.

[0191] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0192] Corresponding to the federated fine-tuning method for large language models based on structural sparsity described in the above embodiments, Figure 6 This diagram illustrates the structure of a federated fine-tuning device for large language models based on structural sparsity, as provided in an embodiment of this application. For ease of explanation, only the parts relevant to the embodiments of this application are shown. Figure 6 The large language model federated fine-tuning device 600 based on structural sparsity provided in this application embodiment is applied to a client. The device 600 includes:

[0193] Module 601 is used to acquire pre-trained large language models and adapter models;

[0194] Training module 602 is used to train the large language model and the adapter model using local sample data to obtain the model parameters of the adapter model after this round of training; the model parameters include a low-rank first parameter matrix and a second parameter matrix;

[0195] The structured sparsity module 603 is used to perform structured sparsity processing on the first parameter matrix and the second parameter matrix respectively to obtain a first sparsity matrix and a second sparsity matrix; the structured sparsity processing is used to indicate the retention of row vectors or column vectors with key information in the first parameter matrix and the second parameter matrix; the first sparsity matrix and the second sparsity matrix are submatrices of the first parameter matrix and the second parameter matrix respectively;

[0196] The element-sparse module 604 is used to perform element-level compression processing on the sparse adaptation matrix to obtain the compressed matrix information of the adapter model after this round of training; the sparse adaptation matrix is ​​obtained by multiplying the first sparse matrix and the second sparse matrix.

[0197] The sending module 605 is used to send the compression matrix information to the server, the server aggregates the compression matrix information sent by multiple clients, and returns the aggregated target compression matrix to each client;

[0198] The update module 606 is used to update the next round of model parameters of the large language model based on the target compression matrix, and repeat the model training steps until the trained large language model is obtained.

[0199] In some embodiments, the structured sparse module 603 includes:

[0200] The first parameter acquisition unit is used to acquire structural sparse parameters; the structural sparse parameters are used to characterize the ratio of row vectors and column vectors determined from the first parameter matrix and the second parameter matrix, respectively.

[0201] The structured sparsity unit is used to perform structured sparsity processing on the first parameter matrix and the second parameter matrix based on the structured sparsity parameters, respectively, to obtain the first sparsified matrix and the second sparsified matrix.

[0202] In some embodiments, the structural sparsity parameter includes a first sparsity parameter and a second sparsity parameter, wherein the first sparsity parameter characterizes the ratio of the selected row vectors, and the second sparsity parameter characterizes the ratio of the selected column vectors; the structural sparsity unit is further used for:

[0203] Calculate the square norm of each row vector in the first parameter matrix, and calculate the square norm of each column vector in the second parameter matrix;

[0204] The target row vectors corresponding to the first sparse parameters are determined according to the order of the square norm of each row vector in the first parameter matrix from largest to smallest, and the first sparsification matrix is ​​constructed based on the target row vectors.

[0205] The target column vector corresponding to the second sparse parameter is determined according to the order of the square norm of each column vector in the second parameter matrix from largest to smallest, and the second sparsification matrix is ​​constructed based on the target column vector.

[0206] In some embodiments, the first parameter acquisition unit is further configured to:

[0207] Determine the selection metric for the structured sparsity parameters;

[0208] For each row vector in the first parameter matrix and each column vector in the second parameter matrix, modeling is performed under the selected metric to construct a first metric distribution function for the first parameter matrix and a second metric distribution function for the second parameter matrix;

[0209] Based on the selection metric, the first metric distribution function, and the second metric distribution function, a compression error function for the structural sparsity processing is defined, wherein the compression error function is a function related to the structural sparsity parameters;

[0210] Under the constraint of the upper limit of available resources, the compression error function is minimized to calculate the value of the structural sparsity parameter.

[0211] In some embodiments, the element sparsity module 604 includes:

[0212] A matrix processing unit is used to perform multiplication operations based on the first sparsified matrix and the second sparsified matrix to obtain the sparsity adaptation matrix;

[0213] The second parameter acquisition unit is used to acquire element sparsity parameters, which are calculated based on the communication bandwidth constraints of the client.

[0214] The element-sparse unit is used to perform element-level compression processing on the sparse adaptation matrix based on the element-sparse parameters to obtain the compressed matrix information of the adapter model after this round of training.

[0215] In some embodiments, the element sparsity parameter includes a preset number of elements; the element sparsity unit is further configured to:

[0216] Target matrix elements that satisfy the preset number of elements are determined from the sparse adaptive matrix in descending order of the absolute values ​​of the matrix elements.

[0217] Obtain the position index of the target matrix element, and use the target matrix element and the corresponding position index as the compression matrix information of the adapter model after this round of training.

[0218] In some embodiments, the update module 606 is further configured to:

[0219] The current model parameters of the large language model and the target compression matrix are used as the next model parameters of the large language model.

[0220] The model parameters of the adapter model are randomly initialized, and the steps of training the large language model and the adapter model with local sample data are returned until the trained large language model is obtained.

[0221] In some embodiments, during model training, the model parameters of the large language model are frozen, and the model parameters of the adapter model are updated.

[0222] In applications, the modules in the above-mentioned large language model federated fine-tuning device based on structural sparsity can be software program modules, or they can be implemented by different logic circuits integrated in the processor, or they can be implemented by multiple distributed processors.

[0223] like Figure 7 As shown, this application embodiment also provides an electronic device 700, including: at least one processor 701 ( Figure 7 The diagram shows only one processor, memory 702, and computer program 703 stored in memory 702 and executable on at least one processor 701. When processor 701 executes computer program 703, it implements the steps in any of the above method embodiments.

[0224] In applications, electronic device 700 can be a computing device such as a desktop computer, laptop, handheld computer, and cloud server. The electronic device may include, but is not limited to, processors and memory. Those skilled in the art will understand that... Figure 7 The examples shown are merely examples of electronic devices and do not constitute a limitation on electronic devices. They may include more or fewer components than shown, or combinations of certain components, or different components. For example, they may also include area capture devices, input / output devices, network access devices, etc.

[0225] In applications, the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0226] In applications, the memory may be an internal storage unit, such as a hard drive or RAM, in some embodiments. In other embodiments, the memory may be an external storage device for a bionic robot, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory may include both internal and external storage units. The memory is used to store operating systems, applications, boot loaders, data, and other programs, such as program code for computer programs. The memory can also be used to temporarily store data that has been output or will be output.

[0227] It should be noted that the information interaction and execution process between the above-mentioned devices / modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0228] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The functional modules in the embodiments can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules can be implemented in hardware or as software functional modules. Furthermore, the specific names of the functional modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the modules in the above-described device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0229] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps described in the various method embodiments above.

[0230] This application provides a computer program product that, when run on a processor, enables the processor to execute the steps described in the various method embodiments above.

[0231] If an integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / bionic robot, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0232] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0233] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0234] In the embodiments provided in this application, it should be understood that the disclosed apparatus / device and method can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or modules may be electrical, mechanical, or other forms.

[0235] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0236] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A federated fine-tuning method for large language models based on structural sparsification, characterized in that, Applied to a client, the method includes: Obtain pre-trained large language models and adapter models; The large language model and the adapter model are trained using local sample data to obtain the model parameters of the adapter model after this round of training; the model parameters include a low-rank first parameter matrix and a second parameter matrix; The first parameter matrix and the second parameter matrix are subjected to structural sparsification to obtain a first sparsification matrix and a second sparsification matrix; the structural sparsification is used to indicate the row vectors or column vectors with key information in the first parameter matrix and the second parameter matrix to be retained; the first sparsification matrix and the second sparsification matrix are submatrices of the first parameter matrix and the second parameter matrix, respectively. The sparse adaptation matrix is ​​compressed element-wise to obtain the compressed matrix information of the adapter model after this round of training; the sparse adaptation matrix is ​​obtained by multiplying the first sparse matrix and the second sparse matrix. The compression matrix information is sent to the server, which aggregates the compression matrix information sent by multiple clients and returns the aggregated target compression matrix to each client. The next round of model parameters of the large language model are updated based on the target compression matrix, and the model training steps are repeated until the trained large language model is obtained. The first sparsification matrix is ​​constructed based on the target row vectors in the first parameter matrix, and the target row vectors are determined in descending order of the square norm of each row vector in the first parameter matrix; the second sparsification matrix is ​​constructed based on the target column vectors in the second parameter matrix, and the target column vectors are determined in descending order of the square norm of each column vector in the second parameter matrix.

2. The structural sparsification-based large language model federated fine-tuning method of claim 1, wherein, The step of performing structural sparsification on the first parameter matrix and the second parameter matrix respectively to obtain a first sparsified matrix and a second sparsified matrix includes: Obtain structural sparse parameters; the structural sparse parameters are used to characterize the ratio of row vectors and column vectors determined from the first parameter matrix and the second parameter matrix, respectively; Based on the structural sparsity parameters, the first parameter matrix and the second parameter matrix are subjected to structural sparsification processing to obtain the first sparsification matrix and the second sparsification matrix.

3. The structural sparsification-based large language model federated fine-tuning method of claim 2, wherein, The structural sparsity parameters include a first sparsity parameter and a second sparsity parameter. The first sparsity parameter is used to characterize the ratio of the filtered row vectors, and the second sparsity parameter is used to characterize the ratio of the filtered column vectors. The step of performing structural sparsification on the first parameter matrix and the second parameter matrix based on the structural sparsity parameters to obtain the first sparsified matrix and the second sparsified matrix includes: Calculate the square norm of each row vector in the first parameter matrix, and calculate the square norm of each column vector in the second parameter matrix; The target row vectors corresponding to the first sparse parameters are determined according to the order of the square norm of each row vector in the first parameter matrix from largest to smallest, and the first sparsification matrix is ​​constructed based on the target row vectors. The target column vector corresponding to the second sparse parameter is determined according to the order of the square norm of each column vector in the second parameter matrix from largest to smallest, and the second sparsification matrix is ​​constructed based on the target column vector.

4. The structural sparsification-based large language model federated fine-tuning method of claim 2, wherein, The structural sparsity parameters are determined through the following steps: Determine the selection metric for the structural sparsity parameter; For each row vector in the first parameter matrix and each column vector in the second parameter matrix, modeling is performed under the selected metric to construct a first metric distribution function for the first parameter matrix and a second metric distribution function for the second parameter matrix; Based on the selection metric, the first metric distribution function, and the second metric distribution function, a compression error function for the structural sparsity processing is defined, wherein the compression error function is a function related to the structural sparsity parameters; Under the constraint of the upper limit of available resources, the compression error function is minimized to calculate the value of the structural sparsity parameter.

5. The federated fine-tuning method of large language model based on structural sparsification according to any one of claims 1-4, wherein, The element-wise compression of the sparse adaptation matrix to obtain the compressed matrix of the adapter model after this round of training includes: The sparse adaptation matrix is ​​obtained by performing a multiplication operation based on the first sparsification matrix and the second sparsification matrix; Obtain element sparsity parameters, which are calculated based on the communication bandwidth constraints of the client; Based on the sparse parameters of the elements, the sparse adaptation matrix is ​​compressed element-wise to obtain the compressed matrix information of the adapter model after this round of training.

6. The structural sparsification-based large language model federated fine-tuning method according to claim 5, wherein, The element sparsity parameter includes a preset number of elements; The step of performing element-wise compression on the sparse adaptation matrix based on the element-wise sparse parameters to obtain the compressed matrix information of the adapter model after this round of training includes: Target matrix elements that satisfy the preset number of elements are determined from the sparse adaptive matrix in descending order of the absolute values ​​of the matrix elements. Obtain the position index of the target matrix element, and use the target matrix element and the corresponding position index as the compression matrix information of the adapter model after this round of training.

7. The federated fine-tuning method for large language models based on structural sparsity as described in any one of claims 1-4, characterized in that, The step of updating the next round of model parameters of the large language model based on the target compression matrix and repeating the model training steps until the trained large language model is obtained includes: The current model parameters of the large language model and the target compression matrix are used as the next model parameters of the large language model. The model parameters of the adapter model are randomly initialized, and the steps of training the large language model and the adapter model with local sample data are returned until the trained large language model is obtained.

8. The federated fine-tuning method for large language models based on structural sparsity as described in any one of claims 1-4, characterized in that, During model training, the model parameters of the large language model are frozen, and the model parameters of the adapter model are updated.

9. A federated fine-tuning device for a large language model based on structural sparsity, characterized in that, Applied to a client, the device includes: The acquisition module is used to acquire pre-trained large language models and adapter models; The training module is used to train the large language model and the adapter model using local sample data to obtain the model parameters of the adapter model after this round of training; the model parameters include a low-rank first parameter matrix and a second parameter matrix; A structured sparsity module is used to perform structured sparsity processing on the first parameter matrix and the second parameter matrix respectively to obtain a first sparsity matrix and a second sparsity matrix; the structured sparsity processing is used to indicate the retention of row vectors or column vectors with key information in the first parameter matrix and the second parameter matrix; the first sparsity matrix and the second sparsity matrix are submatrices of the first parameter matrix and the second parameter matrix respectively; The element-sparse module is used to perform element-level compression processing on the sparse adaptation matrix to obtain the compressed matrix information of the adapter model after this round of training; the sparse adaptation matrix is ​​obtained by multiplying the first sparse matrix and the second sparse matrix. The sending module is used to send the compression matrix information to the server. The server aggregates the compression matrix information sent by multiple clients and returns the aggregated target compression matrix to each client. The update module is used to update the next round of model parameters of the large language model based on the target compression matrix, and repeat the model training steps until the trained large language model is obtained. The first sparsification matrix is ​​constructed based on the target row vectors in the first parameter matrix, and the target row vectors are determined in descending order of the square norm of each row vector in the first parameter matrix; the second sparsification matrix is ​​constructed based on the target column vectors in the second parameter matrix, and the target column vectors are determined in descending order of the square norm of each column vector in the second parameter matrix.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the federated fine-tuning method for large language models based on structural sparsity as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the large language model federated fine-tuning method based on structural sparsity as described in any one of claims 1 to 8.

Images