A gpu high-performance fine-tuning training method and system based on a large language model

Abstract

The application discloses a GPU high-performance fine-tuning training method and system based on a large language model, and the method comprises the following steps: S1, a large language model and initial training data are acquired, the large language model comprises a plurality of work nodes and a plurality of parameter matrices, and the large language model allocates the corresponding parameter matrices to each work node; S2, training of the large language model is divided into a model inference stage and a gradient communication stage; S3, in the model inference stage, the parameter matrices are divided into low-dimensional submatrices based on a low-rank decomposition strategy, and a gradient matrix is obtained; S4, in the gradient communication stage, the gradient matrix is compressed based on double compression, and the parameter matrices are replaced and updated; and S5, training is performed in the corresponding work nodes based on the replaced and updated parameter matrices. The GPU high-performance fine-tuning training method and system based on the large language model have the characteristics of fast calculation speed and small memory occupation.

Description

Technical Field

[0001] This invention relates to the field of distributed deep learning, specifically to a high-performance GPU fine-tuning training method and system based on a large language model. Background Technology

[0002] In recent years, with the development of GPU hardware, the wave of deep learning based on neural networks has triggered a new round of artificial intelligence revolution. Among them, large language models, as a culmination of popular technologies such as natural language processing, computer vision, and affective computing, are a milestone in the development of artificial intelligence. The earliest application of large language models can be traced back to Google's T5 large language model developed in 2019. Through the annotation of massive amounts of data and the use of expert feedback mechanisms, large language models began to possess near-human cognitive and comprehension abilities. At the end of 2022, OpenAI developed the latest version of the large language model - Chatgpt, which generated a strong response in the industry and academia of artificial intelligence. At the same time, a variety of large language models emerged both domestically and internationally in 2023. However, for developers, how to refine the large language model according to their needs by providing training data corpora based on general large language models, and design industry-specific large language models that meet their work requirements, has become a hot topic.

[0003] Because of the massive data and model size of large language models, their training scenarios require multi-level, multi-GPU mechanisms. Therefore, training large language models often involves distributed deep learning. The structure of a large language model consists of multi-layered deep neural networks. However, these deep neural network layers generate a large amount of tensor data during inference operations, consuming not only GPU memory but also significantly slowing down the inference speed of the large language model. Furthermore, since deep neural network models use parallel methods such as data parallelism, tensor parallelism, and pipelined parallelism to partition the training model and data, periodic data aggregation is necessary to offset errors introduced by the training data. During the communication between GPU nodes during data aggregation, the large number of parameters or gradient matrices also leads to severe communication load problems.

[0004] Therefore, the training task of large language models based on distributed deep learning mainly suffers from the following two problems. On the one hand, in the model inference module, the calculation and updating of high-dimensional and large-scale parameter matrices slows down the model's computation time and consumes more GPU memory. On the other hand, in the model communication module, due to the limited GPU communication bandwidth, high-dimensional and large-scale parameters or gradient matrices can cause severe network communication congestion, ultimately leading to an extension of the training time for large language models. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a high-performance GPU fine-tuning training method and system based on a large language model that features fast computation speed and low memory consumption.

[0006] Technical solution: The GPU high-performance fine-tuning training method based on a large language model described in this invention includes the following steps:

[0007] S1. Obtain a large language model and initial training data. The large language model includes several working nodes and several parameter matrices. The parameter matrices are obtained by preprocessing the initial training data and inputting it into the large language model. The large language model assigns a corresponding parameter matrix to each working node.

[0008] S2. The large language model is used to train sequentially in the working nodes according to the parameter matrix, and the training of the large language model is divided into a model inference stage and a gradient communication stage.

[0009] S3. In the model inference stage, the parameter matrix is ​​divided into low-dimensional sub-matrices based on the low-rank decomposition strategy, and the gradient of the low-dimensional sub-matrices is calculated to obtain the gradient matrix.

[0010] S4. In the gradient communication stage, the gradient matrix is ​​compressed based on dual compression to generate a compression result, and the parameter matrix is ​​replaced and updated using the compression result.

[0011] S5. Based on the replaced and updated parameter matrix, train the large language model in the corresponding working node, and the training is complete.

[0012] Furthermore, in step S3, the gradient matrix includes several initial elements, which represent gradient values.

[0013] Furthermore, the gradient communication phase includes a gradient uplink phase and a gradient downlink phase.

[0014] Furthermore, the dual compression includes gradient sparsification and unbiased quantization operations.

[0015] Furthermore, the gradient sparsity operation includes the following steps:

[0016] S411. Take the absolute value of the gradient value of the initial element, and sort the initial elements according to the size of the absolute value, with one initial element corresponding to one sorting number;

[0017] S412. Set the compression ratio, and select a portion of the initial elements according to the compression ratio in sorted order, and assign the gradient value of the other portion of the initial elements that were not selected to zero;

[0018] S413. Save the sort number and gradient value of the selected initial element.

[0019] Furthermore, in the gradient uploading stage, the sorting number of the selected initial element and the gradient value are aggregated into a subgradient matrix; the subgradient matrix includes several non-zero elements, and the non-zero elements represent gradient values.

[0020] Furthermore, the unbiased quantization operation includes the following steps:

[0021] S421. Calculate the second normal form of the subgradient matrix;

[0022] S422. Take the absolute value of the gradient value of the non-zero element, and divide the absolute value of all non-zero elements by the second normal form in turn to obtain the probability of the non-zero element.

[0023] S423. Based on the probability, divide the quantization levels and reduce the non-zero elements to the corresponding quantization levels to generate the compression result.

[0024] Furthermore, during the gradient descent phase, the compression result is used to replace and update the corresponding parameter matrix.

[0025] Furthermore, the gradient sparsification operation is implemented based on the Top-k algorithm, and the unbiased quantization operation is implemented based on the Qsgd algorithm.

[0026] Technical solution: The GPU high-performance fine-tuning training system based on a large language model described in this invention includes:

[0027] The model acquisition module is used to acquire a large language model and initial training data. The large language model includes several working nodes. The initial training data is preprocessed to obtain several parameter matrices. The large language model assigns a corresponding parameter matrix to each working node.

[0028] The training partitioning module is used to train the large language model sequentially in the working nodes according to the parameter matrix, thereby dividing the training of the large language model into a model inference stage and a gradient communication stage.

[0029] The model inference module is used to divide the parameter matrix into low-dimensional sub-matrices based on a low-rank decomposition strategy during the model inference stage, and to perform gradient calculation on the low-dimensional sub-matrices to obtain the gradient matrix.

[0030] A gradient communication module is used to compress the gradient matrix based on dual compression during the gradient communication stage to generate a compression result; and to replace and update the parameter matrix using the compression result.

[0031] The model training module is used to train the large language model in the corresponding working node based on the replaced and updated parameter matrix, and the training is completed.

[0032] Beneficial Effects: This invention has the following significant effects: 1. Fast computation speed during training: For the model inference training stage of large language models, low-rank decomposition technology is adopted. By decomposing the parameter matrix of the deep neural network model into feature matrices, smaller low-dimensional sub-matrices are obtained. Parameter updates are performed on these low-dimensional sub-matrices using stochastic gradient descent, thus saving time. Gradient compression technology is employed, combining the lossy gradient sparsity technique Top-k algorithm and the statistically unbiased numerical quantization strategy Qsgd to compress gradients, thereby saving training time and achieving fast computation speed. 2. Low memory consumption during training: By optimizing the model inference and gradient communication training stages of the distributed large model training architecture, model training time is saved while reducing model load loss. Simultaneously, the large language model training framework reduces model performance degradation. Gradient tensor sparsity is achieved by sorting and selecting elements with larger absolute values ​​in the gradient matrix for transmission, while gradient quantization uses lower bits to represent the gradient matrix that is represented by higher bits. Here, an error compensation mechanism is mainly used to reduce errors. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0034] Figure 2 This is a schematic diagram of the dual compression algorithm based on the model gradient communication stage proposed in this invention;

[0035] Figure 3 This is a flowchart of the Top-k sparsity algorithm used in this invention;

[0036] Figure 4 This is a flowchart of the Qsgd quantization algorithm used in this invention. Detailed Implementation

[0037] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0038] Please see Figure 1 As shown, this invention discloses a high-performance GPU fine-tuning training method and system based on a large language model.

[0039] The GPU high-performance fine-tuning training method includes the following steps:

[0040] S1. Obtain a large language model and initial training data. The large language model includes several working nodes and several parameter matrices. The parameter matrices are obtained by preprocessing the initial training data and inputting it into the large language model. The large language model assigns a corresponding parameter matrix to each working node.

[0041] S2. The large language model is used to train sequentially in the working nodes according to the parameter matrix, and the training of the large language model is divided into a model inference stage and a gradient communication stage.

[0042] S3. In the model inference stage, the parameter matrix is ​​divided into low-dimensional sub-matrices based on the low-rank decomposition strategy, and the gradient of the low-dimensional sub-matrices is calculated to obtain the gradient matrix.

[0043] S4. In the gradient communication stage, the gradient matrix is ​​compressed based on dual compression to generate a compression result, and the parameter matrix is ​​replaced and updated using the compression result.

[0044] S5. Based on the replaced and updated parameter matrix, train the large language model in the corresponding working node, and the training is complete.

[0045] This high-performance GPU fine-tuning training system includes:

[0046] The model acquisition module is used to acquire a large language model and initial training data. The large language model includes several working nodes, and the several parameter matrices are obtained by preprocessing the initial training data and inputting it into the large language model. The large language model assigns a corresponding parameter matrix to each working node.

[0047] The training partitioning module is used to train the large language model sequentially in the working nodes according to the parameter matrix, thereby dividing the training of the large language model into a model inference stage and a gradient communication stage.

[0048] The model inference module is used to divide the parameter matrix into low-dimensional sub-matrices based on a low-rank decomposition strategy during the model inference stage, and to perform gradient calculation on the low-dimensional sub-matrices to obtain the gradient matrix.

[0049] A gradient communication module is used to compress the gradient matrix based on dual compression during the gradient communication stage to generate a compression result; and to replace and update the parameter matrix using the compression result.

[0050] The model training module is used to train the large language model in the corresponding working node based on the replaced and updated parameter matrix, and the training is completed.

[0051] The GPU high-performance fine-tuning training method and system of this invention mainly utilizes model compression and the sparsification and quantization of gradient parameter tensors to compress and quantize parameters and gradient matrices for both the model inference and model communication stages of large language models applied in distributed deep learning scenarios. The GPU high-performance fine-tuning training method and system of this invention will be further explained below.

[0052] In step S3, the model inference module decomposes the parameter matrix into low-dimensional submatrices based on a low-rank decomposition strategy. The gradient matrix obtained from the low-dimensional submatrices includes several initial elements, which represent gradient values. In a distributed deep learning-based training framework, using low-dimensional matrices for matrix parameter updates and gradient calculations effectively saves the time computation overhead caused by high-dimensional, large-scale parameter matrices, thereby effectively reducing memory requirements and saving time for calculating gradient tensors of the parameter matrices. Through computational optimization in the model inference stage, the overall model training time is reduced.

[0053] Low-rank decomposition strategies typically employ singular value decomposition (SVD) to represent high-dimensional, large-scale matrices using low-dimensional matrices. Generally, to address the issue of excessively large parameter matrices in deep neural network models impacting inference speed and GPU memory usage, a simple approach is to utilize matrix multiplication, using multiple low-dimensional submatrices to represent the high-dimensional matrix through multiplication. As a typical example of matrix decomposition techniques, this invention employs low-rank decomposition to reduce the dimensionality of high-dimensional matrices. The specific steps are as follows:

[0054] Suppose a parameter matrix A is an m×n dimensional matrix, A T Let A be the transpose of matrix A. Calculate the symmetric matrix AA. T Let U be the characteristic matrix of , where U is an m×m dimensional matrix. Calculate A. T The eigenvalue matrix A is V, and its eigenvalue matrix P is P, where V is an n×n matrix. Performing a square root operation on each element of the eigenvalue matrix P yields the corresponding matrix Q. According to singular value decomposition, for the parameter matrix A, A = UQV. T Let UQ be the left matrix in the low-rank submatrix, and V T This is the right matrix in the low-rank decomposition submatrix. At this point, the low-rank decomposition based on the parameter matrix A is complete. Taking a two-dimensional matrix S as an example, the low-rank decomposition technique can decompose the matrix into low-dimensional submatrices U and V, where S = U. T V. Where the two-dimensional matrix S, the low-dimensional submatrix U, and V have the following forms:

[0055]

[0056] U T =[u11 ,u 21 ,u 31 ,...,u m1 ],V=[v 11 ,v 12 ,v 13 ,...,v 1n ].

[0057] In step S4, the gradient communication module includes a gradient uplink phase and a gradient downlink phase. The dual compression includes gradient sparsification and unbiased quantization. In this embodiment, the gradient sparsification operation, the gradient uplink phase operation, the unbiased quantization operation, and the gradient downlink phase operation are performed sequentially. The overall process is as follows:

[0058] 1) The gradient sparsity operation includes the following steps:

[0059] S411. Take the absolute value of the gradient value of the initial element, and sort the initial elements according to the size of the absolute value, with one initial element corresponding to one sorting number.

[0060] S412. Set the compression ratio, and select a portion of the initial elements according to the compression ratio in sorted order, and assign the gradient value of the other unselected initial elements to zero.

[0061] S413. Save the sort number and gradient value of the selected initial element.

[0062] 2) In the gradient uploading stage, the sorting number of the selected initial element and the gradient value are aggregated into a subgradient matrix; the subgradient matrix includes several non-zero elements, and the non-zero elements represent gradient values.

[0063] 3) Unbiased quantization includes the following steps:

[0064] S421. Calculate the second normal form of the subgradient matrix.

[0065] S422. Take the absolute value of the gradient of the non-zero element, and divide the absolute value of all non-zero elements by the second normal form in turn to obtain the probability of the non-zero element.

[0066] S423. Based on the probability, divide the quantization levels and reduce the non-zero elements to the corresponding quantization levels to generate the compression result.

[0067] 4) During the gradient descent phase, the compression result is used to replace and update the corresponding parameter matrix. The compression result is transmitted during the gradient descent phase, and the corresponding node restores and updates the corresponding parameter matrix.

[0068] In this embodiment, the gradient sparsity operation is implemented based on the Top-k algorithm, and the unbiased quantization operation is implemented based on the Qsgd algorithm. A dual compression technique, consisting of a lossy gradient sparsity technique (Top-k algorithm) and an unbiased numerical quantization algorithm (Qsgd algorithm), is used to compress the communication gradient of the model multiple times, reducing the size of the communication gradient matrix and thus saving the model's communication time, thereby accelerating the training speed of the overall large language model.

[0069] The Top-k algorithm first sorts the original gradient matrix based on the absolute value of the gradients, selects the matrix elements with the largest absolute values, and sets the values ​​of the remaining matrix elements to 0. Setting the k value (i.e., the compression ratio) to 0.5 means selecting half of the elements in the matrix with the largest absolute values. In gradient communication applied to distributed deep learning, the system uses Top-k sorting to transmit part of the gradient values ​​and their corresponding sort numbers, saving training time by reducing the size of the transmitted gradient matrix under conditions of limited network communication.

[0070] The core idea of ​​the QSGD algorithm is to represent the gradient matrix using higher bit depths with lower bit depths. To achieve expected unbiased compression in a mathematically statistical sense, its main method is to first calculate the second normal form of the entire gradient matrix, then divide the absolute value of a specific element by the second normal form value to obtain the corresponding probability, and finally reduce it to the corresponding quantization level. The probability of element V3 is 0.61, and its probability of being reduced to 16 bits (corresponding to the 0.5 level) is also 0.61, while its probability of being reduced to 24 bits (corresponding to the 0.75 level) is 0.29. Therefore, element V3 is reduced to 16 bits. The QSGD algorithm unbiasedly reduces the original gradient to the corresponding number of bits for quantization and records the sequence number and numerical sign of the corresponding elements, transmitting them together during the gradient descent phase. After the parameter matrix is ​​top-k sparsified locally, blanks are used to represent the initial elements set to zero. Then, the gradient matrix is ​​uploaded and aggregated, and the aggregated gradient matrix is ​​compressed and quantized based on the QSGD quantization method. Here, the gradient elements are represented using low bits, and then it is returned to each worker node to update the parameter matrix.

[0071] Meanwhile, since gradient compression inevitably introduces errors, leading to numerical loss in the gradient matrix, this invention employs an error compensation mechanism for each working node to offset the numerical shift caused by these errors. This error compensation mechanism reduces the loss in model accuracy due to errors in data compression. Existing techniques, such as error compensation mechanisms, typically employ a strategy of accumulating the compression error of the current round to the next round for compensation, and this has been proven both theoretically and practically. Thus, by using bidirectional compression for both the uplink and downlink stages during the gradient matrix transmission phase, training time can be saved while maintaining model performance as much as possible.

Claims

1. A high-performance GPU fine-tuning training method based on a large language model, characterized in that, The method includes the following steps: S1. Obtain a large language model and initial training data. The large language model includes several working nodes and several parameter matrices. The parameter matrices are obtained by preprocessing the initial training data and inputting it into the large language model. The large language model assigns a corresponding parameter matrix to each working node. S2. The large language model is used to train sequentially in the working nodes according to the parameter matrix, and the training of the large language model is divided into a model inference stage and a gradient communication stage. S3. In the model inference stage, the parameter matrix is ​​divided into low-dimensional sub-matrices based on the low-rank decomposition strategy, and the gradient of the low-dimensional sub-matrices is calculated to obtain the gradient matrix. S4. In the gradient communication stage, the gradient matrix is ​​compressed based on dual compression to generate a compression result, and the parameter matrix is ​​replaced and updated using the compression result. S5. Based on the replaced and updated parameter matrix, train the large language model in the corresponding working node, and the training is complete.

2. The GPU high-performance fine-tuning training method based on a large language model according to claim 1, characterized in that, In step S3, the gradient matrix includes several initial elements, which represent gradient values.

3. The GPU high-performance fine-tuning training method based on a large language model according to claim 2, characterized in that, The gradient communication phase includes a gradient uplink phase and a gradient downlink phase.

4. The GPU high-performance fine-tuning training method based on a large language model according to claim 3, characterized in that, The dual compression includes gradient sparsification and unbiased quantization.

5. The GPU high-performance fine-tuning training method based on a large language model according to claim 4, characterized in that, The gradient sparsification operation includes the following steps: S411. Take the absolute value of the gradient value of the initial element, and sort the initial elements according to the size of the absolute value, with one initial element corresponding to one sorting number; S412. Set the compression ratio, and select a portion of the initial elements according to the compression ratio in sorted order, and assign the gradient value of the other portion of the initial elements that were not selected to zero; S413. Save the sort number and gradient value of the selected initial element.

6. The GPU high-performance fine-tuning training method based on a large language model according to claim 5, characterized in that, The gradient uploading phase aggregates the sorting number of the selected initial elements and the gradient values ​​into a subgradient matrix; the subgradient matrix includes several non-zero elements, and the non-zero elements represent gradient values.

7. The GPU high-performance fine-tuning training method based on a large language model according to claim 6, characterized in that, The unbiased quantization operation includes the following steps: S421. Calculate the second normal form of the subgradient matrix; S422. Take the absolute value of the gradient value of the non-zero element, and divide the absolute value of all non-zero elements by the second normal form in turn to obtain the probability of the non-zero element. S423. Based on the probability, divide the quantization levels and reduce the non-zero elements to the corresponding quantization levels to generate the compression result.

8. The GPU high-performance fine-tuning training method based on a large language model according to claim 3, characterized in that, During the gradient descent phase, the compression result is used to replace and update the corresponding parameter matrix.

9. The GPU high-performance fine-tuning training method based on a large language model according to claim 4, characterized in that, The gradient sparsification operation is implemented based on the Top-k algorithm, and the unbiased quantization operation is implemented based on the Qsgd algorithm.

10. A GPU-based high-performance fine-tuning training system for large language models, characterized in that, include: The model acquisition module is used to acquire a large language model and initial training data. The large language model includes several working nodes. The initial training data is preprocessed to obtain several parameter matrices. The large language model assigns a corresponding parameter matrix to each working node. The training partitioning module is used to train the large language model sequentially in the working nodes according to the parameter matrix, thereby dividing the training of the large language model into a model inference stage and a gradient communication stage. The model inference module is used to divide the parameter matrix into low-dimensional sub-matrices based on a low-rank decomposition strategy during the model inference stage, and to perform gradient calculation on the low-dimensional sub-matrices to obtain the gradient matrix. A gradient communication module is used to compress the gradient matrix based on dual compression during the gradient communication stage to generate a compression result. The parameter matrix is ​​replaced and updated using the compression result; The model training module is used to train the large language model in the corresponding working node based on the replaced and updated parameter matrix, and the training is completed.

Images