A method and system for fine-tuning precise parameters of a pre-trained language model

Abstract

The present invention discloses a method and system for fine-tuning precise parameters of a pre-trained language model. The method comprises: constructing a robust adapter; connecting a plurality of constructed robust adapters in series to form a robust adapter chain and determining a maximum chain node; preparing the weight of the pre-trained language model and a data set labeled with a corresponding task; firstly performing preliminary training on each robust adapter in the robust adapter chain to optimize and approximately fine-tune the parameters of the robust adapter; after approximately fine-tuning the parameters of the robust adapter, merging the robust adapter with the weight of the pre-frozen pre-trained language model to complete the calculation of the current robust adapter chain node, and generating residual information for the next iteration, and updating the parameters of the current robust adapter at the same time; performing iterative training, updating the residual information, until the number of iterations is equal to the maximum node of the robust adapter chain, obtaining the optimal weight of the pre-trained language model and the optimal parameters of the robust adapter. The present invention can provide more stable convergence and simpler hyperparameter adjustment.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and in particular to a method and system for fine-tuning precise parameters of a pre-trained language model. Background Art

[0002] Large Language Models (LLMs), as the hottest research area in the field of deep learning in recent years, have completely changed the field of natural language processing and have shown excellent performance in a wide range of language tasks. However, while it has brought technological progress, it has also brought huge computing costs, memory costs, and data costs, especially for those who train LLMs from scratch. In this case, using limited data to fine-tune the parameters of pre-trained LLMs to improve performance for specific tasks is currently the most efficient and economical method, and the ability of the model fine-tuning method determines the performance of the model within the task.

[0003] Traditional fine-tuning methods include using a pre-trained model as an encoder to generate contextual representations, then adding a task-specific classification layer, and then training the combined network end-to-end to minimize the loss of the specific task. This approach usually has obvious limitations: LLMs are large and complex, which leads to a lack of understandability in the modifications during fine-tuning; when LLMs are extended to a large number of tasks, the entire model needs to be retrained for different tasks; the computing resources and computing time required for end-to-end fine-tuning methods are also unacceptable; and the risk of overfitting increases when fine-tuning on smaller task datasets.

[0004] The currently popular fine-tuning method of LLMs is to optimize on a limited set of parameters, which requires only a small part of the computational and memory cost of the overall parameter fine-tuning method. Low-rank adaptive methods and their derivatives are usually used to train low-rank adapter layers to select model layers. In fact, this method cannot restore the accuracy of model parameters for complex fine-tuning tasks, that is, when the complexity of the target task is high, the possibility of parameter accuracy deviation is higher.

[0005] In the prior art, the invention patent with the patent publication number CN118296549A discloses a water field analysis method and system based on a dynamic adapter and an adaptive adjustment mechanism. The efficient parameter adjustment strategy in the article: the DALM-Tuning algorithm achieves efficient adjustment of large language model parameters by introducing low-rank adapters and sparse adapters. However, this patent only analyzes data in the water field, uses a single low-rank adapter and a sparse adapter, and uses a global constraint method to train sparse adapters. Sparse mask calculations are not performed, and the amount of calculation is large. Summary of the invention

[0006] The technical problem to be solved by the present invention is: when effectively fine-tuning the parameters of LLMs, ensuring that the precision of the fine-tuned parameters is consistent with the complete fine-tuning of all parameters.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] A method for fine-tuning precise parameters of a pre-trained language model, comprising:

[0009] S100, building robust adapters for fine-tuning parameters of pre-trained language models, including low-rank adapters and sparse adapters;

[0010] S200, connecting multiple constructed robust adapters in series to form a robust adapter chain, and determining the maximum node of the robust adapter chain;

[0011] S300, preparing the weights of the pre-trained language model and the dataset labeled for the corresponding task;

[0012] S400, select a subset of data from the data set, and first perform preliminary training optimization on each robust adapter in the robust adapter chain. At this time, freeze the weight of the pre-trained language model so that the weight of the pre-trained language model only participates in the calculation but not in the update, and approximately fine-tune the parameters of the robust adapter;

[0013] S500, after approximately fine-tuning the parameters of the robust adapter, merge it with the weights of the pre-trained language model frozen in the previous order, complete the calculation of the current robust adapter chain node, generate residual information for the next iteration, and simultaneously update the parameters of the current robust adapter;

[0014] S600, taking the remaining data subset in the data set as input, performing iterative training, updating the residual information, until the number of iterations is equal to the maximum node of the robust adapter chain, and obtaining the optimal weight of the pre-trained language model and the optimal parameters of the robust adapter.

[0015] Beneficial effect: The robust adapter chain built by the robust adapter further reduces the gap between the partial parameter fine-tuning results of LLMs and the full parameter fine-tuning results of LLMs without adding additional computational overhead and memory overhead.

[0016] In one embodiment of the present invention, before the robust adapter performs preliminary training optimization, the sparse adapter first generates a sparse adaptive mask for the pre-trained language model, including the following steps:

[0017] S41, randomly select a subset of data from the dataset as a mask training set;

[0018] S42, initialize gradient distribution is 0; represents the number of iterations to generate the sparse adaptive mask;

[0019] S43, traverse the mask training set, according to the weight of the pre-trained language model The loss function calculates the gradient of each sample in the mask training set using the gradient accumulation index The gradient distribution of the current sample is obtained by weighted accumulation ,

[0020] S44, select gradient distribution The front The maximum gradient value is used as the mask preselection ;

[0021] S45, pre-select all the masks obtained , integrated into a vector and then selected the pre-trained language model weights All parameters before The gradient value is used as the generated mask .

[0022] In one embodiment of the present invention, the gradient distribution , obtained by the following formula:

[0023] ;

[0024] In the formula, , Represents the current sample The gradient of Represents the data in the current sample, Expressed as gradient calculation, Expressed as a loss function;

[0025] Mask preselection , obtained by the following formula:

[0026] ;

[0027] in, Expressed as sparse adapter density, Represented as the pre-trained language model weights The number of parameters, Expressed as a ranking, Represents a sparse mask generation operation.

[0028] In one embodiment of the present invention, when calculating and generating the mask, the gradient of the weight matrix of a layer of pre-trained language model is saved in the GPU each time, and the weight matrix of the pre-trained language model that is not currently involved in the training is transferred to the CPU.

[0029] In one embodiment of the present invention, the robust adapter performs preliminary training optimization, including:

[0030] S410, initializing a low-rank adapter and a sparse adapter;

[0031] S420, freeze the weights of the pre-trained language model ;

[0032] S430, select a subset of data from the data set as the adapter training set. During training, first use forward propagation and then use backward propagation to adjust the parameters of the low-rank adapter. , , parameters of sparse adapter Perform joint training until the optimization goal of the robust adapter is achieved;

[0033] in, Represented as the number of weight layers in the pre-trained language model, .

[0034] In one embodiment of the present invention, the optimization target of the robust adapter is the following formula:

[0035] ;

[0036] ;

[0037] In the formula, Denoted as the adapter training set, Denote as the rank of the low-rank adapter, Expressed as calculating the rank of the matrix, Represented as constraint rules, Indicates the calculation of zero-norm regularization, Expressed as sparse adapter density, Expressed as Layer weights The weight parameter matrix, It is expressed as a loss function.

[0038] In one embodiment of the present invention, when used, the sparse adapter is stored in a compressed sparse row format, using three lists to represent The sparse matrix : The first list contains the nonzero elements in the matrix; the second list contains numerical values, each of which represents the column index position of the first non-zero element in each row of the matrix in the first list; the third list contains the column index positions of all elements in the first list; where, is represented as the size of the weight matrix.

[0039] In one embodiment of the present invention, in S500, residual information for the next iteration is generated, and the parameters of the current robust adapter are updated at the same time, through the following formula:

[0040] ;

[0041] ;

[0042] In the formula, is the number of iterations when updating the robust adapter chain, , Denote as two approximately fine-tuned parameters of the low-rank adapter of the current iteration, represents the parameters of the approximate fine-tuning of the sparse adapter for the current iteration, represents the adapter update weight for the current iteration, Represented as constraint rules, Indicates the calculation of zero-norm regularization, Expressed as sparse adapter density, Expressed as The dimensions of the layer weight matrices, Expressed as the loss function, Expressed as calculating the rank of the matrix, Represented as the remaining data subset in the dataset, Expressed as The weights of the pre-trained language model trained in iterations, Denote as the rank of the low-rank adapter, Represented as the number of layers of weights in the pre-trained language model.

[0043] In one embodiment of the present invention, the optimal weights of the pre-trained language model and the optimal parameters of the robust adapter are obtained by the following formula:

[0044]

[0045] In the formula, Denotes the optimal weight of the pre-trained language model, , , Expressed as the three optimal parameters of the robust adapter, Denoted as equal to the maximum node of the robust adapter chain The number of iterations.

[0046] The present invention also provides a system for fine-tuning precise parameters of a pre-trained language model, which uses the method for fine-tuning precise parameters of a pre-trained language model described above, including:

[0047] The adapter module is used to build robust adapters for fine-tuning parameters of pre-trained language models, including low-rank adapters and sparse adapters;

[0048] An adapter chain module, used to connect multiple built robust adapters in series to form a robust adapter chain, and determine the maximum node of the robust adapter chain;

[0049] The data module is used to prepare the weights of the pre-trained language model and the dataset labeled for the corresponding task;

[0050] The adapter approximate fine-tuning module is used to select a subset of data from the data set, and first perform preliminary training optimization on each robust adapter in the robust adapter chain. At this time, the weight of the pre-trained language model is frozen so that the weight of the pre-trained language model only participates in the calculation but not in the update, and the parameters of the robust adapter are approximately fine-tuned;

[0051] The weight joint fine-tuning module is used to approximately fine-tune the parameters of the robust adapter and then merge them with the weights of the pre-trained language model frozen in the previous order to complete the calculation of the current robust adapter chain node, generate residual information for the next iteration, and update the parameters of the current robust adapter at the same time;

[0052] The fine-tuning result module is used to take the remaining data subset in the dataset as input, perform iterative training, and update the residual information until the number of iterations is equal to the maximum node of the robust adapter chain, so as to obtain the optimal weights of the pre-trained language model and the optimal parameters of the robust adapter.

[0053] Compared with the prior art, the beneficial effects of the present invention are: reducing the computational and memory costs required for fine-tuning LLMs, effectively fine-tuning the parameters of LLMs on a limited parameter set, while ensuring that the accuracy of the fine-tuned parameters is consistent with the complete fine-tuning of all parameters, providing more stable convergence and simpler hyperparameter adjustment.

[0054] The present invention combines low-rank adapters and sparse adapters to efficiently fine-tune the parameters of frozen pre-trained LLMs weights, and finally obtains a fine-tuning result similar to the fine-tuning of all parameters of LLMs, reducing the computational cost and memory cost in the process of fine-tuning the parameters of LLMs. The robust adapter design of the present invention consists of a standard low-rank adapter and a sparse adapter, which mainly defines a high-performance sparse adaptive mask, provides a collaborative training mechanism that can converge stably, and provides efficient sparse back-propagation system support.

[0055] The present invention uses a robust adapter chain built by a robust adapter to further reduce the gap between the LLMs partial parameter fine-tuning results and the LLMs full parameter fine-tuning results without adding additional computational overhead and memory overhead. It bridges the gap between the adapter method and the LLMs full parameter fine-tuning and controls computational and memory costs.

[0056] The present invention designs a sparse adaptive mask method, which performs mask generation calculation through partial samples to ensure that in sparse fine-tuning, parameters not involved in fine-tuning are frozen, so that the sparse adaptive mask can achieve a sparsity level of over 99%, further reducing computational and memory overheads.

[0057] The present invention first initializes the parameters of the robust adapter, initializes the sparse adapter parameter values ​​therein by parameter preheating, then uses the robust adapter to formally fine-tune the LLMs parameters, and links with the new robust adapter after updating the model parameters, so as to achieve accurate fine-tuning of the LLMs parameters of the robust adapter chain. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of a method for fine-tuning precise parameters of a pre-trained language model according to an embodiment of the present invention.

[0059] Figure 2 A diagram of a fine-tuning framework for precise parameters of a pre-trained language model according to an embodiment of the present invention.

[0060] Figure 3 A block diagram of a system for fine-tuning precise parameters of a pre-trained language model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to facilitate those skilled in the art to understand the technical solution of the present invention, the technical solution of the present invention is further described in conjunction with the accompanying drawings of the specification.

[0062] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0063] See also Figure 1 and Figure 2 As shown, the present invention discloses a method for fine-tuning precise parameters of a pre-trained language model, comprising:

[0064] S100, builds robust adapters for fine-tuning parameters of pre-trained language models, including low-rank adapters and sparse adapters.

[0065] In one embodiment of the present invention, based on the deficiency that the low-rank adaptation method (LoRA) alone cannot effectively perform parameter fine-tuning for complex tasks, and the sparse adapter alone can only fine-tune some important parameters, combining the characteristics of the two, the present invention adopts a combination of a low-rank adapter and a sparse adapter to form a robust adapter to approximate the fine-tuning parameters.

[0066] In one embodiment of the present invention, the symbol definitions used in the process of fine-tuning the parameters of the pre-trained language model are first given. represents the weights of all layers in the pre-trained language model, where , Indicates the number of layers of the pre-trained language model. Parameter fine-tuning of the pre-trained language model is to fine-tune some parameters of the pre-trained language model for a specific task under a given dataset and loss function. And, Expressed as a real number, Expressed as Dimensions of the layer weight matrix.

[0067] S200, multiple constructed robust adapters are connected in series to form a robust adapter chain, and the maximum node of the robust adapter chain is determined.

[0068] S300, prepare the weights of the pre-trained language model and the dataset labeled for the corresponding task.

[0069] S400, select a partial data subset from the data set, first perform preliminary training optimization on each robust adapter in the robust adapter chain, at this time, freeze the weight of the pre-trained language model so that the weight of the pre-trained language model only participates in the calculation and does not participate in the update, and approximately fine-tune the parameters of the robust adapter.

[0070] In one embodiment of the present invention, before the robust adapter performs preliminary training optimization, the sparse adapter first generates a sparse adaptive mask for the pre-trained language model, including the following steps:

[0071] S41, randomly select a subset of data from the dataset as a mask training set.

[0072] S42, initialize gradient distribution is 0; Represents the number of iterations to generate the sparse adaptive mask.

[0073] S43, traverse the mask training set, according to the weight of the pre-trained language model The loss function calculates the gradient of each sample in the mask training set using the gradient accumulation index The gradient distribution of the current sample is obtained by weighted accumulation .

[0074] In this embodiment, the gradient distribution , obtained by the following formula:

[0075] ;

[0076] In the formula, , Represents the current sample The gradient of Represents the data in the current sample, Expressed as gradient calculation, Expressed as a loss function;

[0077] S44, select gradient distribution The front The maximum gradient value is used as the mask preselection .

[0078] ;

[0079] in, Expressed as sparse adapter density, Represented as the pre-trained language model weights The number of parameters, Expressed as a ranking, Represents a sparse mask generation operation.

[0080] S45, pre-select all the masks obtained , integrated into a vector and then selected the pre-trained language model weights All parameters before The gradient value is used as the generated mask .

[0081] In this embodiment, the present invention adopts a learnable sparse mask generation method, accumulates the gradient sum of the training sample set during the training process, and selects the highest weight in each layer of accumulated gradient information, that is, the one with the largest gradient The gradient value is used as the corresponding generated mask, here The final result shows that there are many rows or columns with empty values ​​in the generated mask. By sorting, we do not provide calculations for parameters that do not need to be optimized to reduce the computational overhead.

[0082] In this embodiment, all parameters of the pre-trained language model need to be calculated during the mask calculation process. In order to reduce memory requirements, the present invention is implemented by saving the gradient of the weight matrix of a layer of pre-trained language model in the GPU each time, and transferring the weight matrix of the pre-trained language model that is not currently involved in the training to the CPU, which can greatly reduce memory overhead.

[0083] In one embodiment of the present invention, the robust adapter performs preliminary training optimization, including:

[0084] S410, initialize the low-rank adapter and the sparse adapter.

[0085] In this embodiment, the low-rank adapter initialization mainly performs the parameters of the low-rank adapter. , The initial assignment for the matrix Initialized with random Gaussian distribution, the matrix Assigning a value of 0 ensures that the low-rank adapter parameters are initialized 0. And, initialize the parameters of the sparse adapter Assign a value of 0. The sparse adapter generates a sparse adaptive mask for the pre-trained language model, just initialize the parameters of the sparse adapter When carried out.

[0086] S420, freeze the weights of the pre-trained language model .

[0087] In this embodiment, the weight It participates in the computation when generating the sparse adaptive mask, but does not participate in the update when training the parameters of the robust adapter.

[0088] S430, select a subset of data from the data set as the adapter training set. During training, first use forward propagation and then use backward propagation to adjust the parameters of the low-rank adapter. , , parameters of sparse adapter The joint training is performed until the optimization target of the robust adapter is reached.

[0089] In one embodiment of the present invention, for the pre-trained weight matrix The low-rank adapter is trained in a traditional way, by using a low-rank decomposition In the training process, is frozen and does not receive gradient updates, while and Contains trainable parameters. In the specific calculation process, and All are multiplied by the same input data and then summed at the corresponding coordinates in the output vector.

[0090] In one embodiment of the present invention, the weights not involved in the operation of the sparse adapter are frozen to achieve high sparsity of the overall parameters. , obtaining a sparsity level of more than 99%. Parameters for training the sparse adapter When the sparse adapter uses a fixed mask , mask In addition, the present invention designs an index list that sorts the non-zero elements in the matrix row for kernel startup and load balancing during fine-tuning. Specifically, the sparse adapter is stored in a compressed sparse row format and is represented by three lists: The sparse matrix : The first list contains the nonzero elements in the matrix; the second list contains numerical values, each of which represents the column index position of the first non-zero element in each row of the matrix in the first list; the third list contains the column index positions of all elements in the first list.

[0091] In one embodiment of the present invention, it is further explained that when fine-tuning parameters, forward propagation and then back propagation training can achieve the working principle of efficient propagation. During forward propagation training: the fine-tuning sample is used to perform forward propagation calculations of the network model. Considering the similarities of each layer of the model, only the forward propagation process of a fully connected layer is introduced here. Specifically, assuming that the adapter weight matrix calculated by this layer is Ignoring the bias parameter, assuming that a batch of fine-tuning samples , size is , then the output of this layer can be expressed as:

[0092] ;

[0093] calculate It is necessary to consider the addition and multiplication of dense matrices and sparse matrices, which takes a long time to calculate. The sparse matrix multiplication kernel is used to perform operations on dense matrices and sparse matrices. The thread library is distributed according to the rows of the sparse matrix, and then all non-zero values ​​in the sparse matrix are traversed and added to the corresponding positions of the dense matrix, which has high calculation efficiency. It represents the multiplication of two low-rank matrices and is very fast. The combination of the two realizes the efficient forward propagation operation of the robust adapter.

[0094] In one embodiment of the present invention, during back propagation training: for a given output gradient , that is, calculate the gradient of its parameters and input for back propagation, the calculation method is as follows:

[0095] ;

[0096] ;

[0097] ;

[0098] ;

[0099] In the formula, It is represented as the matrix transpose. , and The calculation strategy is consistent with the forward propagation method, and the results can be calculated efficiently. An improved dense-dense matrix multiplication algorithm is used in the algorithm. Its main function is to multiply two dense matrices when only specific elements need to be output, which can be understood as outputting only the output value of a specific index. The specific calculation method is to start the number of threads with non-zero rows and columns in the matrix, first sort the rows according to the number of non-zero values, and then calculate the additional required computing cores without increasing the amount of additional calculations, which helps to achieve efficient fine-tuning.

[0100] In one embodiment of the present invention, the optimization target of the robust adapter is the following formula:

[0101] ;

[0102] ;

[0103] In the formula, Denoted as the adapter training set, Denote as the rank of the low-rank adapter, Expressed as calculating the rank of the matrix, Represented as constraint rules, Indicates the calculation of zero-norm regularization, Expressed as sparse adapter density, Expressed as Layer weights The weight parameter matrix, It is expressed as a loss function.

[0104] S500, after approximately fine-tuning the parameters of the robust adapter, merge it with the weights of the pre-trained language model frozen in the previous order, complete the calculation of the current robust adapter chain node, generate residual information for the next iteration, and simultaneously update the parameters of the current robust adapter.

[0105] In one embodiment of the present invention, the weight update of the pre-trained language model during fine-tuning is expressed as , the ideal optimal weight is expressed as , each robust adapter is fine-tuned in the above way. The entire robust adapter chain follows an iterative residual learning algorithm, and the fine-tuning of each robust adapter Can be used as learning The residual of simplifies the optimization problem.

[0106] In this embodiment, specifically, residual information for the next iteration is generated, and the parameters of the current robust adapter are updated at the same time, through the following parameters:

[0107] ;

[0108] ;

[0109] In the formula, is the number of iterations when updating the robust adapter chain, -1 indicates the number of previous iterations of the current iteration. , Denote as two approximately fine-tuned parameters of the low-rank adapter of the current iteration, represents the parameters of the approximate fine-tuning of the sparse adapter for the current iteration, represents the adapter update weight for the current iteration, Represented as constraint rules, Indicates the calculation of zero-norm regularization, Expressed as sparse adapter density, Expressed as The dimensions of the layer weight matrices, Expressed as the loss function, Expressed as calculating the rank of the matrix, Represented as the remaining data subset in the dataset, Expressed as The weights of the pre-trained language model trained in iterations, Denote as the rank of the low-rank adapter, Represented as the number of layers of weights in the pre-trained language model. Merging the robust adapter into the frozen pre-trained language model weights can keep the memory overhead of the robust adapter chain consistent with training a single robust adapter during fine-tuning.

[0110] S600, taking the remaining data subset in the data set as input, performing iterative training, updating the residual information, until the number of iterations is equal to the maximum node of the robust adapter chain, and obtaining the optimal weight of the pre-trained language model and the optimal parameters of the robust adapter.

[0111] In one embodiment of the present invention, after forming the chain node, the robust adapter is reinitialized to extend the chain so that the pre-trained language model can adapt to the residual weight update required by the specific task. Specifically, the optimal weight of the pre-trained language model and the optimal parameters of the robust adapter are obtained by the following formula:

[0112]

[0113] In the formula, Denotes the optimal weight of the pre-trained language model, , , Expressed as the three optimal parameters of the robust adapter, Denoted as equal to the maximum node of the robust adapter chain The number of iterations.

[0114] In one embodiment of the present invention, in order to verify the rationality of the technical solution of the present invention, the parameters of the public pre-trained large models OPT-1.3B and Llama2-7B were fine-tuned, and 7 public data sets were selected, including CoLA, SST-2, MRPC, STS-B, QQP, RTE, and WNLI, and compared with low-rank adapters and robust adapters. This experiment uses AdamW as the basic optimizer, and a total of 20 iterations of training are performed. The low-rank adapter parameters is set to 16, the sparse adapter parameter is set to 2%, and the initial learning rate of the sparse mask training process is set to 10 -3 , the termination learning rate is set to 10 -5 , set the number of robust adapter chain nodes is 16; the results are shown in Tables 1 and 2.

[0115] Table 1 Various experimental results on OPT-1.3B

[0116]

[0117] Table 1 shows that the application of robust adapters has increased the number of parameters and memory usage compared to low-rank adapters, but it brings better parameter fine-tuning effects. The application of the robust adapter chain method does not bring more parameters and memory usage, but the fine-tuning effect is improved to a certain extent, indicating that the present invention has a better application effect in the fine-tuning of pre-trained language model parameters.

[0118] Table 2 Various experimental results in Llama2-7B

[0119]

[0120] Table 2 shows parameter fine-tuning based on another pre-trained language model. The experimental results have consistent performance with those in Table 1, which also illustrates that the present invention has a better application effect in the work of pre-trained language model parameter fine-tuning.

[0121] See also Figures 1 to 3 As shown, the present invention also provides a system for fine-tuning precise parameters of a pre-trained language model, which uses the method for fine-tuning precise parameters of a pre-trained language model described above, including:

[0122] The adapter module is used to build robust adapters for fine-tuning pre-trained language model parameters, including low-rank adapters and sparse adapters.

[0123] The adapter chain module is used to connect multiple constructed robust adapters in series to form a robust adapter chain and determine the maximum node of the robust adapter chain.

[0124] The data module is used to prepare the weights of the pre-trained language model and the dataset labeled for the corresponding task.

[0125] The adapter approximate fine-tuning module is used to select a subset of data from the dataset and first perform preliminary training optimization on each robust adapter in the robust adapter chain. At this time, the weights of the pre-trained language model are frozen so that the weights of the pre-trained language model only participate in the calculation but not in the update, and the parameters of the robust adapter are approximately fine-tuned.

[0126] The weight joint fine-tuning module is used to approximately fine-tune the parameters of the robust adapter and then merge them with the weights of the pre-trained language model frozen in the previous order to complete the calculation of the current robust adapter chain node, generate residual information for the next iteration, and update the parameters of the current robust adapter at the same time.

[0127] The fine-tuning result module is used to take the remaining data subset in the dataset as input, perform iterative training, and update the residual information until the number of iterations is equal to the maximum node of the robust adapter chain, so as to obtain the optimal weights of the pre-trained language model and the optimal parameters of the robust adapter.

[0128] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting from any point of view, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0129] The above-described embodiments merely represent implementation methods of the invention. The protection scope of the present invention is not limited to the above-described embodiments. For those skilled in the art, several modifications and improvements may be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for fine-tuning precise parameters of a pre-trained language model, characterized in that: include: S100, building robust adapters for fine-tuning parameters of pre-trained language models, including low-rank adapters and sparse adapters; S200, connecting multiple constructed robust adapters in series to form a robust adapter chain, and determining the maximum node of the robust adapter chain; S300, preparing the weights of the pre-trained language model and the dataset labeled for the corresponding task; S400, select a subset of data from the data set, and first perform preliminary training optimization on each robust adapter in the robust adapter chain. At this time, freeze the weight of the pre-trained language model so that the weight of the pre-trained language model only participates in the calculation but not in the update, and approximately fine-tune the parameters of the robust adapter; S500, after approximately fine-tuning the parameters of the robust adapter, merge it with the weights of the pre-trained language model frozen in the previous order, complete the calculation of the current robust adapter chain node, generate residual information for the next iteration, and simultaneously update the parameters of the current robust adapter; S600, taking the remaining data subset in the data set as input, performing iterative training, updating the residual information, until the number of iterations is equal to the maximum node of the robust adapter chain, and obtaining the optimal weight of the pre-trained language model and the optimal parameters of the robust adapter.

2. The method for fine-tuning precise parameters of a pre-trained language model according to claim 1, characterized in that: Before the robust adapter performs preliminary training optimization, the sparse adapter generates sparse adaptive masks for the pre-trained language model, including the following steps: S41, randomly select a subset of data from the dataset as a mask training set; S42, initialize gradient distribution is 0; represents the number of iterations to generate the sparse adaptive mask; S43, traverse the mask training set, according to the weight of the pre-trained language model The loss function calculates the gradient of each sample in the mask training set using the gradient accumulation index The gradient distribution of the current sample is obtained by weighted accumulation , S44, select gradient distribution The front The maximum gradient value is used as the mask preselection ; S45, pre-select all the masks obtained , integrated into a vector and then selected the pre-trained language model weights All parameters before The gradient value is used as the generated mask .

3. The method for fine-tuning precise parameters of a pre-trained language model according to claim 2, characterized in that: Gradient distribution , obtained by the following formula: ； In the formula, , Represents the current sample The gradient of Represents the data in the current sample, Expressed as gradient calculation, Expressed as a loss function; Mask preselection , obtained by the following formula: ； in, Expressed as sparse adapter density, Represented as the pre-trained language model weights The number of parameters, Expressed as a ranking, Represents a sparse mask generation operation.

4. The method for fine-tuning precise parameters of a pre-trained language model according to claim 2, characterized in that: When calculating the generated mask, the gradient of the weight matrix of a layer of pre-trained language model is saved in the GPU each time, and the weight matrix of the pre-trained language model that is not currently involved in the training is transferred to the CPU.

5. The method for fine-tuning precise parameters of a pre-trained language model according to claim 2, characterized in that: The robust adapter performs preliminary training optimizations, including: S410, initializing a low-rank adapter and a sparse adapter; S420, freeze the weights of the pre-trained language model ; S430, select a subset of data from the data set as the adapter training set. During training, first use forward propagation and then use backward propagation to adjust the parameters of the low-rank adapter. , , parameters of sparse adapter Perform joint training until the optimization goal of the robust adapter is achieved; in, Represented as the number of weight layers in the pre-trained language model, .

6. The method for fine-tuning precise parameters of a pre-trained language model according to claim 5, characterized in that: The optimization objective of the robust adapter is as follows: ； ； In the formula, Denoted as the adapter training set, Denote as the rank of the low-rank adapter, Expressed as calculating the rank of the matrix, Expressed as constraint rules, Indicates the calculation of zero-norm regularization, Expressed as sparse adapter density, Expressed as Layer weights The weight parameter matrix, It is expressed as a loss function.

7. The method for fine-tuning precise parameters of a pre-trained language model according to claim 5, characterized in that: When used, the sparse adapter is stored in a compressed sparse row format, represented by three lists. The sparse matrix : The first list contains the nonzero elements in the matrix; the second list contains numerical values, each of which represents the column index position of the first non-zero element in each row of the matrix in the first list; the third list contains the column index positions of all elements in the first list; where, is represented as the size of the weight matrix.

8. The method for fine-tuning precise parameters of a pre-trained language model according to claim 1, characterized in that: In S500, the residual information for the next iteration is generated, and the parameters of the current robust adapter are updated at the same time, through the following formula: ； ； In the formula, is the number of iterations when updating the robust adapter chain, , Denote as two approximately fine-tuned parameters of the low-rank adapter of the current iteration, represents the parameters of the approximate fine-tuning of the sparse adapter for the current iteration, represents the adapter update weight for the current iteration, Expressed as constraint rules, Indicates the calculation of zero-norm regularization, Expressed as sparse adapter density, Expressed as The dimensions of the layer weight matrices, Expressed as the loss function, Expressed as calculating the rank of the matrix, Represented as the remaining data subset in the dataset, Expressed as The weights of the pre-trained language model trained in iterations, Denote as the rank of the low-rank adapter, Represented as the number of layers of weights in the pre-trained language model.

9. The method for fine-tuning precise parameters of a pre-trained language model according to claim 1, characterized in that: Get the optimal weights of the pre-trained language model and the optimal parameters of the robust adapter using the following formula: In the formula, Denotes the optimal weight of the pre-trained language model, , , Expressed as the three optimal parameters of the robust adapter, Denoted as equal to the maximum node of the robust adapter chain The number of iterations.

10. A system for fine-tuning precise parameters of a pre-trained language model, characterized in that: A method for fine-tuning precise parameters of a pre-trained language model according to any one of claims 1 to 9, comprising: The adapter module is used to build robust adapters for fine-tuning parameters of pre-trained language models, including low-rank adapters and sparse adapters; An adapter chain module, used to connect multiple built robust adapters in series to form a robust adapter chain, and determine the maximum node of the robust adapter chain; The data module is used to prepare the weights of the pre-trained language model and the dataset labeled for the corresponding task; The adapter approximate fine-tuning module is used to select a subset of data from the data set, and first perform preliminary training optimization on each robust adapter in the robust adapter chain. At this time, the weight of the pre-trained language model is frozen so that the weight of the pre-trained language model only participates in the calculation but not in the update, and the parameters of the robust adapter are approximately fine-tuned; The weight joint fine-tuning module is used to approximately fine-tune the parameters of the robust adapter and then merge them with the weights of the pre-trained language model frozen in the previous order to complete the calculation of the current robust adapter chain node, generate residual information for the next iteration, and update the parameters of the current robust adapter at the same time; The fine-tuning result module is used to take the remaining data subset in the dataset as input, perform iterative training, and update the residual information until the number of iterations is equal to the maximum node of the robust adapter chain, so as to obtain the optimal weights of the pre-trained language model and the optimal parameters of the robust adapter.

Images