Large language model fine-tuning method and device

Abstract

This invention discloses a method and apparatus for fine-tuning a large language model, comprising: acquiring a fine-tuning text dataset; preprocessing the fine-tuning text dataset to output a fine-tuning text validation set and multiple fine-tuning weight increment matrices; folding each fine-tuning weight increment matrix layer by layer to determine an initial third-order tensor; applying a pre-set tensor singular value thresholding algorithm to impose a rank constraint on the initial third-order tensor to determine a target third-order tensor; determining a second-order low-rank global fine-tuning weight increment matrix based on the target third-order tensor; and using the second-order low-rank global fine-tuning weight increment matrix and the fine-tuning text validation set to fine-tune the weights of a pre-set initial large language model to determine the target large language model. This invention solves the technical problem of poor performance of large language models caused by existing large language model fine-tuning methods.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence natural language processing technology, and in particular to a method and apparatus for fine-tuning a large language model. Background Technology

[0002] Large Language Models (LLMs) are models that can process large amounts of text data and generate natural language text. They play an important role in the field of natural language processing and are widely used in machine translation, text generation, dialogue systems, and other areas.

[0003] There are many types of large language models, the most famous being OpenAI's (Open Artificial Intelligence) GPT (Generative Pre-trained Transformer) series and Google's BERT (Bidirectional Encoder Representations from Transformers) series. These models are based on deep neural networks, have billions or even tens of billions of parameters, can process massive amounts of natural language text, and perform exceptionally well in tasks such as predicting missing parts of text and generating text. Fine-tuning of large language models refers to further performing a small amount of supervised learning on an already trained large language model to adapt it to specific natural language processing tasks.

[0004] Most existing fine-tuning methods for large language models use low-rank adaptation (LoRA) to approximate full parameter updates by updating the low-rank matrix. However, selecting a suitable rank for the low-rank matrix is ​​very difficult. A rank that is too large or too small will make the rank constraint of the fine-tuning increment matrix inaccurate, resulting in poor performance of the large language model. Summary of the Invention

[0005] This invention provides a method and apparatus for fine-tuning large language models, which solves the technical problem that existing large language model fine-tuning methods result in poor performance of large language models.

[0006] The first aspect of this invention provides a method for fine-tuning a large language model, comprising:

[0007] Obtain the fine-tuned text dataset, preprocess the fine-tuned text dataset, and output the fine-tuned text validation set and multiple fine-tuned weight increment matrices;

[0008] The fine-tuning weight increment matrices are folded layer by layer to determine the initial third-order tensor.

[0009] The initial third-order tensor is subjected to rank constraint using a preset tensor singular value threshold algorithm to determine the target third-order tensor.

[0010] Based on the target third-order tensor, determine the second-order low-rank global fine-tuning weight increment matrix;

[0011] The second-order low-rank global fine-tuning weight increment matrix and the fine-tuning text validation set are used to fine-tune the weights of the preset initial large language model to determine the target large language model.

[0012] Further, the step of preprocessing the fine-tuned text dataset to output a fine-tuned text validation set and multiple fine-tuned weight increment matrices includes:

[0013] The fine-tuned text dataset is divided into a fine-tuned text validation set and a fine-tuned text training set;

[0014] The fine-tuned text training set is segmented to determine multiple fine-tuned text training subsets;

[0015] Distributed training is performed on each of the fine-tuned text training subsets, and the fine-tuning weight increment matrix corresponding to each fine-tuned text training subset is output.

[0016] Further, the step of applying a preset tensor singular value threshold algorithm to the initial third-order tensor to impose a rank constraint and determine the target third-order tensor includes:

[0017] Perform a discrete Fourier transform on the third dimension of the initial third-order tensor to determine the transformed third-order tensor;

[0018] The third dimension of the transformed third-order tensor is expanded layer by layer to determine multiple Fourier domain fine-tuning increment matrices;

[0019] Singular value decomposition is performed on each of the Fourier domain fine-tuning increment matrices to output the singular value matrix corresponding to each of the Fourier domain fine-tuning increment matrices.

[0020] The target third-order tensor is determined based on the singular value matrices and the preset element thresholds.

[0021] Further, the step of determining the target third-order tensor based on each of the singular value matrices and a preset element threshold includes:

[0022] Each singular value matrix element is compared with a preset element threshold.

[0023] Any matrix element less than or equal to the preset element threshold is removed and set to zero to determine multiple new core singular value matrices;

[0024] The corresponding Fourier domain fine-tuning increment matrix is ​​updated using the new core singular value matrix, thereby determining multiple new Fourier domain fine-tuning increment matrices.

[0025] The target third-order tensor is determined by performing layer-by-layer folding and discrete Fourier inverse transform on each of the new Fourier domain fine-tuning increment matrices.

[0026] Further, the step of determining the second-order low-rank global fine-tuning weight increment matrix based on the target third-order tensor includes:

[0027] The target third-order tensor is expanded layer by layer to determine multiple target fine-tuning weight increment matrices;

[0028] The mean of each target fine-tuning weight increment matrix is ​​calculated to determine the second-order low-rank global fine-tuning weight increment matrix.

[0029] Further, the step of using the second-order low-rank global fine-tuning weight increment matrix and the fine-tuning text validation set to fine-tune the weights of the preset initial large language model and determine the target large language model includes:

[0030] The weights of the preset initial large language model are fine-tuned using the second-order low-rank global fine-tuning weight increment matrix to determine the intermediate large language model.

[0031] The fine-tuned text validation set is used as input to the intermediate large language model, and the text accuracy is output.

[0032] Compare the accuracy of the text with a preset accuracy threshold;

[0033] If the text accuracy is greater than or equal to the preset accuracy threshold, then the intermediate large language model is used as the target large language model.

[0034] A second aspect of the present invention provides a large language model fine-tuning device, comprising:

[0035] The acquisition module is used to acquire the fine-tuned text dataset, preprocess the fine-tuned text dataset, and output the fine-tuned text validation set and multiple fine-tuned weight increment matrices.

[0036] The determination module is used to fold each of the fine-tuning weight increment matrices layer by layer to determine the initial third-order tensor.

[0037] A module is used to apply a preset tensor singular value thresholding algorithm to the rank constraint of the initial third-order tensor to determine the target third-order tensor;

[0038] The second-order module is used to determine the second-order low-rank global fine-tuning weight increment matrix based on the target third-order tensor.

[0039] The target module is used to fine-tune the weights of the preset initial large language model using the second-order low-rank global fine-tuning weight increment matrix and the fine-tuning text validation set, so as to determine the target large language model.

[0040] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the large language model fine-tuning method as described in any of the preceding claims.

[0041] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, wherein the computer program / instructions, when executed by a processor, implement the steps of the large language model fine-tuning method as described in any of the preceding claims.

[0042] The fifth aspect of the present invention provides a computer program product comprising a computer program / instructions, wherein when the computer program / instructions are executed by a processor, the steps of the large language model fine-tuning method as described in any of the preceding claims are implemented.

[0043] As can be seen from the above technical solutions, the present invention has the following advantages:

[0044] The above-mentioned technical solution of the present invention provides a method for fine-tuning a large language model. First, a fine-tuning text dataset is acquired and preprocessed to output a fine-tuning text validation set and multiple fine-tuning weight increment matrices. Next, each fine-tuning weight increment matrix is ​​folded layer by layer to determine an initial third-order tensor. A pre-set tensor singular value thresholding algorithm is used to constrain the rank of the initial third-order tensor to determine a target third-order tensor. Based on the target third-order tensor, a second-order low-rank global fine-tuning weight increment matrix is ​​determined. Finally, the second-order low-rank global fine-tuning weight increment matrix and the fine-tuning text validation set are used to fine-tune the weights of a pre-set initial large language model to determine the target large language model. Based on the above scheme, the process of folding each fine-tuning weight increment matrix layer by layer into an initial third-order tensor, and determining the target large language model based on the pre-set tensor singular value thresholding algorithm and the initial third-order tensor, can integrate all fine-tuning information into a single tensor while preserving the low-rank structure of the fine-tuning weights. This achieves more accurate rank constraints on the fine-tuning increment matrix, further improving the performance of the large language model. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating the steps of a large language model fine-tuning method provided in this embodiment of the invention;

[0047] Figure 2 A framework diagram of tensor singular value decomposition provided in an embodiment of the present invention;

[0048] Figure 3 A flowchart illustrating the steps of another large language model fine-tuning method provided in this embodiment of the invention;

[0049] Figure 4 A flowchart illustrating the large language model fine-tuning method provided in an embodiment of the present invention;

[0050] Figure 5 This is a structural block diagram of a large language model fine-tuning device provided in an embodiment of the present invention. Detailed Implementation

[0051] This invention provides a method and apparatus for fine-tuning a large language model, which addresses the technical problem that existing large language model fine-tuning methods result in poor performance of the large language model.

[0052] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0053] Please see Figure 1 , Figure 1 A flowchart illustrating the steps of a large language model fine-tuning method provided in an embodiment of the present invention.

[0054] This invention provides a method for fine-tuning a large language model, comprising:

[0055] Step 101: Obtain the fine-tuned text dataset, preprocess the fine-tuned text dataset, and output the fine-tuned text validation set and multiple fine-tuned weight increment matrices.

[0056] The fine-tuned text dataset includes data of multiple text types.

[0057] It should be noted that the preprocessing of the fine-tuned text dataset is as follows: the fine-tuned text dataset is divided into two parts according to the proportion of the fine-tuned text dataset: the fine-tuned text validation set and the fine-tuned text training set.

[0058] Furthermore, the fine-tuning text training set is segmented to determine multiple fine-tuning text training subsets. Specifically, the fine-tuning text training set D is first divided into n equal parts (fine-tuning text training subsets), which can be represented as D = {D1, ..., D2}. n}, positive natural numbers The condition n ≤ |D| is satisfied; then the fine-tuned texts after segmentation are used to train subsets D1, ..., D2. n The data is loaded onto the GPUs for distributed training, and the fine-tuning weight increment matrices ΔW1,...,ΔW corresponding to each fine-tuned text training subset are output. n The fine-tuning weight increment matrix is ​​obtained by multiplying two low-rank matrices, and is represented as ΔW. i =A i B i , where A i For the i-th fine-tuning weight increment matrix ΔW i The lower projection factor matrix, n is the total number of fine-tuning weight increment matrices, r i For the i-th fine-tuning weight increment matrix ΔW i The rank of , d is the rank of the i-th fine-tuning weight increment matrix ΔW i The number of rows; B i For the i-th fine-tuning weight increment matrix ΔW i The upper projection factor matrix, k is the i-th fine-tuning weight increment matrix ΔW i The number of columns.

[0059] Furthermore, the calculation of each fine-tuned weight increment matrix is ​​independent and obtained through distributed training in parallel on different graphics cards. By adopting a multi-graphics card parallel distributed training approach, n fine-tuned weight increment matrices can be obtained quickly, and each fine-tuned weight increment matrix can learn the features of the fine-tuned text training set in different feature dimensions, which is beneficial to improving the generalization ability of the model after fine-tuning.

[0060] In this embodiment, a fine-tuned text dataset is obtained, and the fine-tuned text dataset is preprocessed to output a fine-tuned text validation set and multiple fine-tuned weight increment matrices.

[0061] Step 102: Fold each fine-tuning weight increment matrix layer by layer to determine the initial third-order tensor.

[0062] It should be noted that the initial third-order tensor ΔW = fold(ΔW1; ...; ΔW) is obtained by folding all the fine-tuning weight increment matrices layer by layer. nThe operator fold folds all the fine-tuning weight increment matrices into an initial third-order tensor, which can integrate the fine-tuning information contained in the n fine-tuning weight increment matrices into the initial third-order tensor while preserving the low-rank structure of the fine-tuning weights. ΔW is the initial third-order tensor. fold(·) is a layer-by-layer folding operation, n is the total number of fine-tuning weight increment matrices, and d is the i-th fine-tuning weight increment matrix ΔW. i The row number, k is the i-th fine-tuning weight increment matrix ΔW i The number of columns.

[0063] In this embodiment, each fine-tuning weight increment matrix is ​​folded layer by layer to determine the initial third-order tensor.

[0064] Step 103: Use a preset tensor singular value threshold algorithm to perform rank constraints on the initial third-order tensor to determine the target third-order tensor.

[0065] Specifically, a Discrete Fourier Transform is performed on the third dimension of the initial third-order tensor to determine the transformed third-order tensor, i.e., the Discrete Fourier Transform of the initial third-order tensor along the third dimension is calculated. This process can be represented as follows: in, To transform a third-order tensor, DFT3(·) is the Discrete Fourier Transform along the third dimension.

[0066] Furthermore, the third dimension of the transformed third-order tensor is expanded layer by layer to determine multiple Fourier domain fine-tuning increment matrices. That is, the transformed third-order tensor obtained by the discrete Fourier transform in the third dimension is expanded layer by layer along the third dimension to obtain multiple Fourier domain fine-tuning increment matrices. This is the fine-tuning increment matrix for the i-th Fourier domain.

[0067] Furthermore, singular value decomposition is performed on each Fourier domain fine-tuning increment matrix, outputting the singular value matrix corresponding to each Fourier domain fine-tuning increment matrix. Based on each singular value matrix and a preset element threshold, the target third-order tensor is determined. The multiple singular value matrices include multiple first singular value matrices, multiple second singular value matrices, and multiple core singular value matrices. The singular value decomposition process can be represented as:

[0068]

[0069] in, U is the fine-tuning increment matrix for the i-th Fourier domain; i S is the first singular value matrix corresponding to the fine-tuning increment matrix of the i-th Fourier domain; i The core singular value matrix corresponding to the fine-tuning increment matrix of the i-th Fourier domain; It is the transpose of the second singular value matrix corresponding to the fine-tuning increment matrix of the i-th Fourier domain.

[0070] Please see Figure 2 For example, based on the tensor singular value decomposition algorithm, the tensor is first subjected to discrete Fourier transform to obtain the transformed tensor. Then based on the transformation tensor Perform singular value decomposition layer by layer, first factor tensor Composed of multiple first singular value matrices folded together, second factor tensor The core factor tensor is composed of multiple second singular value matrices folded together. It is formed by folding multiple core singular value matrices; where n represents that each factor tensor is formed by folding n singular value matrices, d is the number of rows and columns of the first singular value matrix, and k is the number of rows and columns of the second singular value matrix.

[0071] Furthermore, the multiple first singular value matrices, multiple second singular value matrices, and multiple core singular value matrices obtained through the above steps can be folded one by one and then subjected to an inverse discrete Fourier transform to obtain three factor tensors: the first factor tensor corresponding to the multiple first singular value matrices, the second factor tensor corresponding to the multiple second singular value matrices, and the core factor tensor corresponding to the multiple core singular value matrices. The core factor tensor can be expressed as... in, Let IDFT3(·) be the core factor tensor, and IDFT3(·) be the discrete inverse Fourier transform along the third dimension. The step principles for the first factor tensor and the second factor tensor are the same as those for the core factor tensor, and will not be elaborated further here.

[0072] Furthermore, the process of determining the target third-order tensor based on each singular value matrix and preset element thresholds is essentially establishing a tensor nuclear norm optimization model for the core factor tensor. After applying rank constraints, the core factor tensor is updated, meaning the core factor tensor is updated in the Fourier domain by truncating the core singular value matrix S. i To implement rank constraints, the process involves comparing each element in each core singular value matrix with a preset element threshold. Any element less than or equal to the preset element threshold is removed and set to zero, while any element greater than the preset element threshold is retained. This process determines multiple new core singular value matrices. The preset element threshold is set to a value greater than zero. This process can be represented as:

[0073]

[0074] in, S is the new core singular value matrix for the i-th block; i S is the core singular value matrix of the i-th block; α is the preset element threshold; (S i -α) + For Si All matrix elements that are greater than the preset element threshold.

[0075] Furthermore, the increment matrix is ​​fine-tuned for each Fourier domain. The process involves updating each new core singular value matrix sequentially, that is, updating the corresponding Fourier domain fine-tuning increment matrix using each new core singular value matrix to determine multiple new Fourier domain fine-tuning increment matrices. This process can be represented as:

[0076]

[0077] in, For the i-th new Fourier domain fine-tuning increment matrix; U is the new core singular value matrix for the i-th block; i Let be the first singular value matrix corresponding to the fine-tuning increment matrix of the i-th Fourier domain; It is the transpose of the second singular value matrix corresponding to the fine-tuning increment matrix of the i-th Fourier domain.

[0078] Furthermore, after folding each new Fourier domain fine-tuning increment matrix layer by layer, an inverse discrete Fourier transform is performed to determine the target third-order tensor, which can be expressed as: Let U be the target third-order tensor, and U be the new first-factor tensor. For the new core factor tensor, V T This is the transpose of the new second factor tensor. * represents the tensor T-product.

[0079] In this embodiment, a preset tensor singular value threshold algorithm is used to impose a rank constraint on the initial third-order tensor to determine the target third-order tensor.

[0080] Step 104: Based on the target third-order tensor, determine the second-order low-rank global fine-tuning weight increment matrix.

[0081] Specifically, after updating the initial third-order tensor to the target third-order tensor through the above steps, the target third-order tensor is expanded layer by layer and the summation and mean value are obtained to obtain the second-order low-rank global fine-tuning weight increment matrix. That is, the target third-order tensor is expanded layer by layer to determine multiple target fine-tuning weight increment matrices. This process can be expressed as:

[0082]

[0083] Wherein, ΔW1 * Fine-tune the weight increment matrix for the first objective; Fine-tune the weight increment matrix for the nth objective; The target is a third-order tensor; unfold(·) is the layer-by-layer expansion operation.

[0084] Furthermore, the mean of each target fine-tuning weight increment matrix is ​​calculated to determine the second-order low-rank global fine-tuning weight increment matrix ΔW. g Unfolding is the inverse operation of folding, and unfolding is described by the operator unfold.

[0085] In this embodiment, a second-order low-rank global fine-tuning weight increment matrix is ​​determined based on the target third-order tensor.

[0086] Step 105: Use a second-order low-rank global fine-tuning weight increment matrix and a fine-tuning text validation set to fine-tune the weights of the pre-set initial large language model and determine the target large language model.

[0087] Specifically, a second-order low-rank global fine-tuning weight increment matrix is ​​used to fine-tune the weights of the preset initial large language model to determine the intermediate large language model. The fine-tuned text validation set is used as the input of the intermediate large language model, and the text accuracy is output. The text accuracy is compared with a preset accuracy threshold. If the text accuracy is greater than or equal to the preset accuracy threshold, the intermediate large language model is used as the target large language model. If the text accuracy is less than the preset accuracy threshold, the preset element threshold is reduced to obtain a new preset element threshold. Then, the process jumps to the step of comparing all matrix elements in each core singular value matrix with the preset element threshold one by one until the text accuracy is greater than or equal to the preset accuracy threshold. The intermediate large language model with the text accuracy greater than or equal to the preset accuracy threshold is used as the target large language model.

[0088] In this embodiment, a second-order low-rank global fine-tuning weight increment matrix and a fine-tuning text validation set are used to fine-tune the weights of a pre-set initial large language model to determine the target large language model.

[0089] In this embodiment of the invention, a method for fine-tuning a large language model is provided. First, a fine-tuning text dataset is acquired and preprocessed to output a fine-tuning text validation set and multiple fine-tuning weight increment matrices. Next, each fine-tuning weight increment matrix is ​​folded layer by layer to determine an initial third-order tensor. A pre-set tensor singular value thresholding algorithm is used to impose rank constraints on the initial third-order tensor to determine a target third-order tensor. Based on the target third-order tensor, a second-order low-rank global fine-tuning weight increment matrix is ​​determined. Finally, the second-order low-rank global fine-tuning weight increment matrix and the fine-tuning text validation set are used to fine-tune the weights of a pre-set initial large language model to determine the target large language model. Based on the above scheme, the process of folding each fine-tuning weight increment matrix layer by layer into an initial third-order tensor and determining the target large language model based on the pre-set tensor singular value thresholding algorithm and the initial third-order tensor can integrate all fine-tuning information into a single tensor while preserving the low-rank structure of the fine-tuning weights. This achieves more accurate rank constraints on the fine-tuning increment matrix, further improving the performance of the large language model.

[0090] Please see Figure 3 , Figure 3 A flowchart illustrating the steps of another large language model fine-tuning method provided in an embodiment of the present invention.

[0091] This invention provides a method for fine-tuning a large language model, comprising:

[0092] Step 301: Obtain the fine-tuned text dataset and divide it into a fine-tuned text validation set and a fine-tuned text training set.

[0093] In this embodiment, a fine-tuned text dataset is obtained and divided into a fine-tuned text validation set and a fine-tuned text training set.

[0094] Step 302: Divide the fine-tuned text training set into multiple fine-tuned text training subsets.

[0095] In this embodiment, the fine-tuned text training set is segmented to determine multiple fine-tuned text training subsets.

[0096] Step 303: Perform distributed training on each fine-tuned text training subset and output the fine-tuning weight increment matrix corresponding to each fine-tuned text training subset.

[0097] In this embodiment, distributed training is performed on each fine-tuned text training subset, and the fine-tuning weight increment matrix corresponding to each fine-tuned text training subset is output.

[0098] Step 304: Fold each fine-tuning weight increment matrix layer by layer to determine the initial third-order tensor.

[0099] In this embodiment, each fine-tuning weight increment matrix is ​​folded layer by layer to determine the initial third-order tensor.

[0100] Step 305: Use a preset tensor singular value threshold algorithm to perform rank constraints on the initial third-order tensor to determine the target third-order tensor.

[0101] Furthermore, step 305 may include the following sub-steps:

[0102] S51. Perform a discrete Fourier transform on the third dimension of the initial third-order tensor to determine the transformed third-order tensor.

[0103] S52. Expand the third dimension of the transformed third-order tensor layer by layer to determine multiple Fourier domain fine-tuning increment matrices.

[0104] S53. Perform singular value decomposition on each Fourier domain fine-tuning increment matrix and output the singular value matrix corresponding to each Fourier domain fine-tuning increment matrix.

[0105] S54. Determine the target third-order tensor based on each singular value matrix and the preset element threshold.

[0106] Furthermore, S54 may include the following sub-steps:

[0107] S541. Compare the matrix elements in the core singular value matrix of each singular value matrix with the preset element threshold;

[0108] S542. Remove and set to zero any matrix element that is less than or equal to a preset element threshold to determine multiple new core singular value matrices.

[0109] S543. The corresponding Fourier domain fine-tuning increment matrix is ​​updated using each new core singular value matrix to determine multiple new Fourier domain fine-tuning increment matrices.

[0110] S544. Perform layer-by-layer folding and discrete Fourier inverse transform on each new Fourier domain fine-tuning increment matrix to determine the target third-order tensor.

[0111] It should be noted that a third-order tensor nuclear norm optimization model is established, and the nuclear norm of the third-order tensor is constrained based on the tensor singular value thresholding algorithm. After applying rank constraints to the third-order tensor nuclear norm optimization model, the core factor tensor is updated. Based on the updated core factor tensor, the target third-order tensor is obtained.

[0112] In this embodiment, a preset tensor singular value threshold algorithm is used to impose a rank constraint on the initial third-order tensor to determine the target third-order tensor.

[0113] Step 306: Based on the target third-order tensor, determine the second-order low-rank global fine-tuning weight increment matrix.

[0114] Furthermore, step 306 may include the following sub-steps:

[0115] S61. Expand the target third-order tensor layer by layer to determine multiple target fine-tuning weight increment matrices;

[0116] S62. Perform mean calculation on the fine-tuning weight increment matrix of each target to determine the second-order low-rank global fine-tuning weight increment matrix.

[0117] It should be noted that the calculation of the third-order tensor after nuclear norm constraint, i.e. the target third-order tensor, is obtained by expanding the target third-order tensor layer by layer and summing the results to obtain the second-order low-rank global fine-tuning weight increment matrix.

[0118] In this embodiment, a second-order low-rank global fine-tuning weight increment matrix is ​​determined based on the target third-order tensor.

[0119] Step 307: Use a second-order low-rank global fine-tuning weight increment matrix and a fine-tuning text validation set to fine-tune the weights of the pre-set initial large language model and determine the target large language model.

[0120] Furthermore, step 307 may include the following sub-steps:

[0121] S71. Use a second-order low-rank global fine-tuning weight increment matrix to fine-tune the weights of the preset initial large language model and determine the intermediate large language model.

[0122] S72. Use the fine-tuned text validation set as input to the intermediate large language model and output the text accuracy.

[0123] S73. Compare the text accuracy with the preset accuracy threshold;

[0124] S74. If the text accuracy is greater than or equal to the preset accuracy threshold, then the intermediate large language model will be used as the target large language model.

[0125] It should be noted that the second-order low-rank global fine-tuning weight increment matrix that satisfies the accuracy threshold condition of the fine-tuned text validation set is saved. If the accuracy threshold condition is not met, the kernel norm optimization model is returned for further constraints until the accuracy threshold condition of the fine-tuned text validation set is met. Specifically, the weights of the preset initial large language model are fine-tuned using the second-order low-rank global fine-tuning weight increment matrix to determine the intermediate large language model. The fine-tuned text validation set is used as the input of the intermediate large language model, and the text accuracy is output. The text accuracy is compared with the preset accuracy threshold. If the text accuracy is greater than or equal to the preset accuracy threshold, the intermediate large language model is used as the target large language model, and the second-order low-rank global fine-tuning weight increment matrix ΔW is adjusted. g Perform singular value decomposition to obtain Let A g =Ug , Thus, the low-rank matrix A is preserved. g With B g This allows you to save the second-order low-rank global fine-tuning weight increment matrix ΔW. g , among which, U g S is a first- or second-order low-rank factor matrix. g The core is the second-order low-rank factor tensor. A is the transpose of the second-order low-rank factor matrix. g Let B be the first low-rank matrix. g It is the second low-rank matrix.

[0126] Furthermore, if the text accuracy is less than the preset accuracy threshold, the preset element threshold is reduced to obtain a new preset element threshold, and the process jumps to execute S541 until the text accuracy is greater than or equal to the preset accuracy threshold. The intermediate large language model when the text accuracy is greater than or equal to the preset accuracy threshold is used as the target large language model.

[0127] Please see Figure 4 For example, suppose the accuracy of the output fine-tuned text validation set is Val acc The validation set accuracy threshold (preset accuracy threshold) is T. acc First, the fine-tuned text dataset is preprocessed to obtain a fine-tuned text validation set and multiple fine-tuned weight increment matrices. Then, a tensor nuclear norm optimization model is established, including the objective function and constraints of the nuclear norm optimization model, described as follows: st、Val acc ≥T acc The tensor nuclear norm optimization model minimizes the nuclear norm of the core factor tensors while satisfying the validation set accuracy threshold. The tensor singular value thresholding algorithm is used to optimize the tensor nuclear norm. Update the core factor tensor respectively With the third-order tensor ΔW and The solution process for the tensor nuclear norm optimization model includes calculating the core factor tensors of the nuclear norm optimization model using the tensor singular value decomposition algorithm. The initial values ​​are used to optimize the core factor tensor of the nuclear norm optimization model using the tensor singular value thresholding algorithm, resulting in a new core factor tensor. Calculate the target third-order tensor after nuclear norm constraints. And calculate the second-order low-rank global fine-tuning weight increment matrix ΔW g The validation set accuracy threshold condition is determined based on the second-order low-rank global fine-tuning weight increment matrix. If the condition is not met, the threshold is reduced. If the condition is met, the target language model is output while maintaining the second-order low-rank global fine-tuning weight increment matrix.

[0128] In this embodiment, a second-order low-rank global fine-tuning weight increment matrix and a fine-tuning text validation set are used to fine-tune the weights of a pre-set initial large language model to determine the target large language model.

[0129] For comparison of technical effectiveness, existing technologies can be referenced. Supervised Fine-Tuning (SFT) is one of the most direct and effective methods, but the massive number of parameters in today's large language models brings enormous computational and memory pressure. Therefore, Parameter-Efficient Fine-Tuning (PEFT) methods have been proposed to improve fine-tuning performance. Taking Low-Rank Adaptation (LoRA) as an example, it approximates the full parameter update by updating the low-rank matrix, significantly reducing computation and memory usage while maintaining good performance of the original model. However, the performance of LoRA for fine-tuning large language models largely depends on the selection of the rank of the low-rank matrix; excessively large or small rank will degrade model performance.

[0130] To address the aforementioned issues, this invention proposes a large language model fine-tuning method. This method loads domain-specific segmented datasets onto different GPUs for distributed training, effectively improving the fine-tuning training speed of the large language model. Simultaneously, it learns the features of the domain-specific fine-tuning training set across different feature dimensions, enhancing the generalization ability of the fine-tuned large language model for that domain. Furthermore, this application proposes folding all fine-tuning weight increment matrices into third-order tensors. This integrates all fine-tuning information into a single tensor while preserving the low-rank structure of the fine-tuning weights, storing fine-tuning information from different feature dimensions in a compact form. This facilitates the discovery of common low-rank structures in the fine-tuning information across different feature dimensions, thereby achieving more accurate rank constraints on the fine-tuning increment matrix. Finally, this application establishes a tensor kernel norm optimization model, which more accurately constrains the rank of the fine-tuning weight increment matrix. By minimizing the kernel norm of the core factor tensor while satisfying the validation set accuracy threshold, it ensures the performance of the fine-tuned model while compressing the rank of the fine-tuning weight increment matrix. In application scenarios, this reduces the memory footprint of the fine-tuning weights, improving the performance of the fine-tuned large language model in a specific domain while maintaining parameter efficiency.

[0131] In summary, this invention addresses the problem of inaccurate rank constraints on the fine-tuning increment matrix in existing methods, enabling efficient fine-tuning of large language models in multi-GPU environments. It can be applied to all parameters of pre-trained large language models, including but not limited to attention layers and fully connected layers. By imposing more reasonable and accurate rank constraints on the fine-tuning increment matrix, it improves the performance of large language models in specific domains.

[0132] In this embodiment of the invention, a method for fine-tuning a large language model is provided. First, a fine-tuning text dataset is acquired and preprocessed to output a fine-tuning text validation set and multiple fine-tuning weight increment matrices. Next, each fine-tuning weight increment matrix is ​​folded layer by layer to determine an initial third-order tensor. A pre-set tensor singular value thresholding algorithm is used to impose rank constraints on the initial third-order tensor to determine a target third-order tensor. Based on the target third-order tensor, a second-order low-rank global fine-tuning weight increment matrix is ​​determined. Finally, the second-order low-rank global fine-tuning weight increment matrix and the fine-tuning text validation set are used to fine-tune the weights of a pre-set initial large language model to determine the target large language model. Based on the above scheme, the process of folding each fine-tuning weight increment matrix layer by layer into an initial third-order tensor and determining the target large language model based on the pre-set tensor singular value thresholding algorithm and the initial third-order tensor can integrate all fine-tuning information into a single tensor while preserving the low-rank structure of the fine-tuning weights. This achieves more accurate rank constraints on the fine-tuning increment matrix, further improving the performance of the large language model.

[0133] Please see Figure 5 , Figure 5 This is a structural block diagram of a large language model fine-tuning device provided in an embodiment of the present invention.

[0134] This invention provides a large language model fine-tuning device, comprising:

[0135] The acquisition module 501 is used to acquire the fine-tuned text dataset, preprocess the fine-tuned text dataset, and output the fine-tuned text validation set and multiple fine-tuned weight increment matrices.

[0136] Module 502 is used to fold each fine-tuning weight increment matrix layer by layer to determine the initial third-order tensor.

[0137] Module 503 is used to apply a preset tensor singular value thresholding algorithm to the rank constraint of the initial third-order tensor to determine the target third-order tensor.

[0138] Second-order module 504 is used to determine the second-order low-rank global fine-tuning weight increment matrix based on the target third-order tensor.

[0139] The target module 505 is used to fine-tune the weights of a pre-set initial large language model using a second-order low-rank global fine-tuning weight increment matrix and a fine-tuning text validation set, thereby determining the target large language model.

[0140] Furthermore, module 501 is specifically used for:

[0141] The fine-tuned text dataset is divided into a fine-tuned text validation set and a fine-tuned text training set.

[0142] The fine-tuned text training set is segmented to determine multiple fine-tuned text training subsets;

[0143] Distributed training is performed on each fine-tuned text training subset, and the fine-tuning weight increment matrix corresponding to each fine-tuned text training subset is output.

[0144] Further, module 502 is defined as including:

[0145] The first submodule is used to perform a discrete Fourier transform on the third dimension of the initial third-order tensor to determine the transformed third-order tensor.

[0146] The second submodule is used to expand the third dimension of the transformed third-order tensor layer by layer to determine multiple Fourier domain fine-tuning increment matrices.

[0147] The third submodule is used to perform singular value decomposition on each Fourier domain fine-tuning increment matrix and output the singular value matrix corresponding to each Fourier domain fine-tuning increment matrix.

[0148] The fourth submodule is used to determine the target third-order tensor based on each singular value matrix and a preset element threshold.

[0149] Furthermore, module 503 is employed, specifically for:

[0150] Each singular value matrix element is compared with a preset element threshold.

[0151] Remove and set to zero any matrix element that is less than or equal to a preset element threshold to determine multiple new core singular value matrices;

[0152] The corresponding Fourier domain fine-tuning increment matrix is ​​updated using each new core singular value matrix to determine multiple new Fourier domain fine-tuning increment matrices.

[0153] The target third-order tensor is determined by performing layer-by-layer folding and discrete Fourier inverse transform on each new Fourier domain fine-tuning increment matrix.

[0154] Furthermore, the second-order module 504 is specifically used for:

[0155] The target third-order tensor is expanded layer by layer to determine multiple target fine-tuning weight increment matrices;

[0156] The mean of the fine-tuning weight increment matrices for each objective is calculated to determine the second-order low-rank global fine-tuning weight increment matrix.

[0157] Furthermore, target module 505 is specifically used for:

[0158] The weights of the preset initial large language model are fine-tuned using a second-order low-rank global fine-tuning weight increment matrix to determine the intermediate large language model.

[0159] The fine-tuned text validation set is used as input to the intermediate large language model, and the output is the text accuracy.

[0160] Compare the text accuracy with the preset accuracy threshold;

[0161] If the text accuracy is greater than or equal to the preset accuracy threshold, the intermediate large language model will be used as the target large language model.

[0162] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, modules, and sub-modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0163] This invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the large language model fine-tuning method as described in any of the above embodiments.

[0164] This invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the large language model fine-tuning method as described in any of the above embodiments.

[0165] This invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the large language model fine-tuning method as described in any of the above embodiments.

[0166] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0167] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for fine-tuning a large language model, characterized in that, include: Obtain the fine-tuned text dataset, preprocess the fine-tuned text dataset, and output the fine-tuned text validation set and multiple fine-tuning weight increment matrices, including: The fine-tuned text dataset is divided into a fine-tuned text validation set and a fine-tuned text training set; The fine-tuned text training set is segmented to determine multiple fine-tuned text training subsets; Each of the fine-tuned text training subsets is loaded onto a graphics card for distributed training, and the fine-tuned weight increment matrix corresponding to each of the fine-tuned text training subsets is output. The fine-tuning weight increment matrices are folded layer by layer to determine the initial third-order tensor. The initial third-order tensor is subjected to rank constraint using a preset tensor singular value threshold algorithm to determine the target third-order tensor. Based on the target third-order tensor, determine the second-order low-rank global fine-tuning weight increment matrix; The second-order low-rank global fine-tuning weight increment matrix and the fine-tuning text validation set are used to fine-tune the weights of the preset initial large language model to determine the target large language model.

2. The method for fine-tuning a large language model according to claim 1, characterized in that, The step of applying a preset tensor singular value threshold algorithm to the rank constraint of the initial third-order tensor to determine the target third-order tensor includes: Perform a discrete Fourier transform on the third dimension of the initial third-order tensor to determine the transformed third-order tensor; The third dimension of the transformed third-order tensor is expanded layer by layer to determine multiple Fourier domain fine-tuning increment matrices; Singular value decomposition is performed on each of the Fourier domain fine-tuning increment matrices to output the singular value matrix corresponding to each of the Fourier domain fine-tuning increment matrices. The target third-order tensor is determined based on the singular value matrices and the preset element thresholds.

3. The method for fine-tuning a large language model according to claim 2, characterized in that, The step of determining the target third-order tensor based on each singular value matrix and a preset element threshold includes: Each singular value matrix element is compared with a preset element threshold. Any matrix element less than or equal to the preset element threshold is removed and set to zero to determine multiple new core singular value matrices; The corresponding Fourier domain fine-tuning increment matrix is ​​updated using the new core singular value matrix, thereby determining multiple new Fourier domain fine-tuning increment matrices. The target third-order tensor is determined by performing layer-by-layer folding and discrete Fourier inverse transform on each of the new Fourier domain fine-tuning increment matrices.

4. The method for fine-tuning a large language model according to claim 1, characterized in that, The step of determining the second-order low-rank global fine-tuning weight increment matrix based on the target third-order tensor includes: The target third-order tensor is expanded layer by layer to determine multiple target fine-tuning weight increment matrices; The mean of each target fine-tuning weight increment matrix is ​​calculated to determine the second-order low-rank global fine-tuning weight increment matrix.

5. The method for fine-tuning a large language model according to claim 1, characterized in that, The step of using the second-order low-rank global fine-tuning weight increment matrix and the fine-tuning text validation set to fine-tune the weights of the pre-set initial large language model and determine the target large language model includes: The weights of the preset initial large language model are fine-tuned using the second-order low-rank global fine-tuning weight increment matrix to determine the intermediate large language model. The fine-tuned text validation set is used as input to the intermediate large language model, and the text accuracy is output. Compare the accuracy of the text with a preset accuracy threshold; If the text accuracy is greater than or equal to the preset accuracy threshold, then the intermediate large language model is used as the target large language model.

6. A large language model fine-tuning device, characterized in that, include: The acquisition module is used to acquire the fine-tuned text dataset, preprocess the fine-tuned text dataset, and output the fine-tuned text validation set and multiple fine-tuned weight increment matrices. The determination module is used to fold each of the fine-tuning weight increment matrices layer by layer to determine the initial third-order tensor. A module is used to apply a preset tensor singular value thresholding algorithm to the rank constraint of the initial third-order tensor to determine the target third-order tensor; The second-order module is used to determine the second-order low-rank global fine-tuning weight increment matrix based on the target third-order tensor. The target module is used to fine-tune the weights of the preset initial large language model using the second-order low-rank global fine-tuning weight increment matrix and the fine-tuning text validation set, and to determine the target large language model. The acquisition module is specifically used for: The fine-tuned text dataset is divided into a fine-tuned text validation set and a fine-tuned text training set; The fine-tuned text training set is segmented to determine multiple fine-tuned text training subsets; Each of the fine-tuned text training subsets is loaded onto a graphics card for distributed training, and the fine-tuned weight increment matrix corresponding to each fine-tuned text training subset is output.

7. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the large language model fine-tuning method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the large language model fine-tuning method as described in any one of claims 1-5.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the large language model fine-tuning method as described in any one of claims 1-5.

Images