Parameter fine tuning method and device of pre-training model, equipment and medium

Abstract

The invention discloses a parameter fine tuning method and device of a pre-training model, equipment and a medium. The parameter fine tuning method is used for performing parameter fine tuning on a pre-training model for natural language processing based on a text data set, the pre-training model comprises a linear layer with an initial weight matrix, and the parameter fine tuning method comprises the following steps: applying orthogonal transformation based on at least one rotation matrix to the linear layer and obtaining an angle parameter matrix according to the orthogonal transformation, constructing an orthogonal fine tuning layer based on the initial weight matrix and the angle parameter matrix, and replacing a linear layer in the pre-training model with the orthogonal fine tuning layer to obtain a to-be-fine-tuned model; only updating the angle parameter matrix through a gradient back propagation algorithm based on the text data set so as to train the to-be-fine-tuned model; and outputting the trained model to be fine-tuned as a parameter fine-tuning result.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence, and in particular relates to a method for efficiently fine-tuning parameters of a pre-training model. Background Art

[0002] Self-supervised pre-trained large models have demonstrated strong representation and generalization capabilities in various downstream tasks. Fine-tuning pre-trained models to adapt to more specialized tasks has become a more mainstream paradigm. As the number of model parameters increases, pre-trained models continue to break through their capabilities. However, this also brings significant challenges to fine-tuning the model.

[0003] Faced with such demands and challenges, how to fine-tune only a small number of parameters to achieve results on downstream tasks similar to those achieved by fine-tuning all parameters of the entire model, the importance of efficient parameter fine-tuning methods has become increasingly prominent.

[0004] LoRA was the mainstream method for efficient parameter fine-tuning in the past, but it failed to strike a good balance between retaining the capabilities of the initial pre-trained model and adapting to downstream tasks. Summary of the invention

[0005] The embodiments of the present disclosure provide a parameter fine-tuning method and a parameter fine-tuning device for a pre-trained model, so that by updating only a very small number of parameters, performance similar to or better than full parameter fine-tuning can be obtained in downstream tasks, while effectively reducing storage and computing overheads, and achieving more flexible and efficient downstream adaptation while maintaining the initial representation capability of the pre-trained model.

[0006] An embodiment of the present disclosure provides a parameter fine-tuning method for a pre-trained model, wherein the parameter fine-tuning method performs parameter fine-tuning on a pre-trained model for natural language processing based on a text dataset, wherein the pre-trained model includes a linear layer having an initial weight matrix, and the parameter fine-tuning method includes: applying an orthogonal transformation based on at least one rotation matrix to the linear layer and obtaining an angle parameter matrix according to the orthogonal transformation, constructing an orthogonal fine-tuning layer based on the initial weight matrix and the angle parameter matrix, and replacing the linear layer in the pre-trained model with the orthogonal fine-tuning layer to obtain a model to be fine-tuned;

[0007] Based on the text data set, only the angle parameter matrix is ​​updated by a gradient back-propagation algorithm to implement training of the model to be fine-tuned; and

[0008] The trained model to be fine-tuned is output as a result of parameter fine-tuning.

[0009] Optionally, the angle parameter matrix may be the product of all rotation matrices in the at least one rotation matrix.

[0010] Optionally, the orthogonal fine-tuning layer is the initial weight matrix multiplied by the angle parameter matrix.

[0011] Optionally, the orthogonal fine-tuning layer is expressed as: W Ort =W 0 R, where W Orth is the orthogonal fine-tuning layer, W 0 is the initial weight matrix, R is the angle parameter matrix, R = R 1 ·R 2 ·…·R n , R 1 ,R 2 ,…,R n is a rotation matrix in at least one rotation matrix, and each rotation matrix R in at least one rotation matrix i It is expressed as:

[0012]

[0013] Where d is the dimension of the rotation matrix, and d = 2 a , B F (k) is the k×k dimensional rotation subspace, k=2 b ,b∈[1,a], a and b are positive integers, diag(·) represents the function of diagonalizing the vector, and the rotation subspace B F (k) is expressed as:

[0014]

[0015] d / k means that there are d / k rotation subspaces B in the d-dimensional matrix F (k) to fill,

[0016]

[0017] θ 1 ,θ 2 , ...θ k / 2 is the learnable angle parameter.

[0018] Optionally, the parameter fine-tuning method further comprises a step of initializing the angle parameter matrix, wherein one of the at least one rotation matrix is ​​assigned an initialization matrix R 0 , and at least one of the rotation matrices is initialized to the identity matrix, R 0 It is expressed as:

[0019]

[0020] Among them, W 0 is the initial weight matrix of the pre-trained model, and U is obtained by The left singular vector matrix obtained by singular value decomposition, the gradient Represents the loss function L on the initial weight matrix W 0 The partial derivative, U [r] is the first r columns of the left singular vector matrix U, W 0[r] is the initial weight matrix W 0 The first r columns, I [n-r] represents the identity matrix of (nr)×(nr), where n is the weight matrix W 0 Dimension.

[0021] Optionally, the pre-trained model includes one of a multi-layer perceptron, a convolution-based neural network, and a Transformer-based neural network, and the linear layer is at least one of a feedforward network, a convolution layer, and a Transformer attention layer corresponding to the pre-trained model.

[0022] An embodiment of the present disclosure provides a parameter fine-tuning device for a pre-trained model. The parameter fine-tuning device fine-tunes the parameters of a pre-trained model for natural language processing based on a text data set, the pre-trained model includes a linear layer with an initial weight matrix, and the parameter fine-tuning device includes: an initialization module, configured to apply an orthogonal transformation based on at least one rotation matrix to the linear layer and obtain an angle parameter matrix according to the orthogonal transformation, construct an orthogonal fine-tuning layer based on the initial weight matrix and the angle parameter matrix, replace the linear layer in the pre-trained model with the orthogonal fine-tuning layer to obtain a model to be fine-tuned; a training module, configured to update only the angle parameter matrix based on the text data set through a gradient back-propagation algorithm to achieve training of the model to be fine-tuned; and an output module, configured to output the trained model to be fine-tuned as a result of parameter fine-tuning.

[0023] An embodiment of the present disclosure provides an electronic device, which includes a storage medium, a processor, and instructions or codes stored on the storage medium and executable on the processor, and when the processor executes the program, the parameter fine-tuning method of the pre-trained model as described above is implemented.

[0024] An embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the parameter fine-tuning method of the pre-trained model as described above is implemented.

[0025] According to one or more embodiments of the embodiments of the present disclosure, the parameter fine-tuning method and parameter fine-tuning device of the pre-trained model of the present disclosure obtain an orthogonal fine-tuning layer by introducing a rotation matrix in a specific linear layer of the pre-trained model, and only update the learnable angle parameters of these rotation matrices to achieve parameter fine-tuning of the pre-trained model, so that the parameter fine-tuning process is decomposed into finding a suitable normal vector and adjusting the direction of the initial weight matrix by rotating these vectors. The only parameter that needs to be updated is the angle parameter matrix in the model to be fine-tuned. Compared with the number of all parameters of the pre-trained model, the proportion of parameters required for fine-tuning is very small. At the same time, a dense orthogonal matrix is ​​formed by the product of multiple rotation matrices, which ensures the adaptability to downstream tasks. In general, the parameters efficiently complete the fine-tuning task of the pre-trained model on downstream tasks, achieving a balance between retaining the initial pre-trained model capabilities and adapting to downstream tasks, and ensuring that there is no additional storage and reasoning overhead compared to the initial pre-trained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments of the present invention are briefly introduced below. Obviously, the described embodiments are only part of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Figure 1 Schematic diagram of a pre-trained model to be fine-tuned according to an embodiment of the present invention.

[0028] Figure 2 The present invention is a flowchart of a method for fine-tuning parameters of a pre-trained model according to an embodiment of the present invention.

[0029] Figure 3 Schematic diagram of a model to be fine-tuned according to a parameter fine-tuning method of a pre-trained model according to an embodiment of the present invention.

[0030] Figure 4 Schematic diagram of a rotation matrix of a parameter fine-tuning method for a pre-trained model according to an embodiment of the present invention.

[0031] Figure 5 4 is a structural block diagram of a parameter fine-tuning device for a pre-training model according to an embodiment of the present invention.

[0032] Figure 6 is a block diagram illustrating a computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0034] The present invention discloses a method for fine-tuning parameters of a pre-trained model on the one hand. The method fine-tunes parameters of a pre-trained model for natural language processing based on a text dataset.

[0035] The parameter fine-tuning method is applicable to many application fields and model fine-tuning scenarios in multiple data forms where pre-trained models can be applied: for example, in the field of natural language processing, the basic model can be pre-trained on a large amount of unlabeled general text corpus (such as news reports, social media posts, encyclopedia articles), and then a relatively small-scale but carefully labeled text dataset can be used for parameter fine-tuning for specific tasks (such as legal text classification, medical report abstracts, financial text analysis), thereby achieving customized and high-performance model applications while significantly reducing training overhead.

[0036] According to an exemplary embodiment of the present disclosure, the pre-trained model applicable to the parameter fine-tuning method includes one of a multi-layer perceptron (MLP), a convolutional neural network (CNN), and a Transformer-based neural network (Transformer-based models), and can be selected according to the type of target task. For example, for prediction or classification tasks based on structured numerical features, multi-layer perceptrons are more efficient in pre-training and fine-tuning; and for complex tasks with text data or multimodal information as the main input (such as machine translation, text summarization, dialogue understanding, speech recognition, multimodal retrieval), Transformer-based neural networks build powerful feature representation capabilities through multi-head attention mechanisms and large-scale pre-training corpora, and then perform parameter fine-tuning based on the labeled data of the target task, thereby achieving accurate and efficient downstream task adaptation.

[0037] In addition, the linear layer of the pre-trained model is at least one of a feedforward network, a convolutional layer, and an attention layer of a Transformer corresponding to the pre-trained model.

[0038] Taking the decoder of the Transformer-based pre-trained model as an example, a typical decoder based on the Transformer architecture uses a multi-layer attention mechanism and a feedforward network as the core to process the input sequence and generate predictions for the target sequence.

[0039] Figure 1 Schematic diagram of a pre-trained model to be fine-tuned according to an embodiment of the present invention.

[0040] The following will refer to Figure 1 , the Transformer pre-trained model 10 is used as an example for description.

[0041] Reference Figure 1 , the Transformer-based pre-training model 10 includes: an input layer 100, a decoder 200, and an output layer 300.

[0042] The input layer 100 includes: a mask input module 110, a vector embedding module 120 and a position encoding module 130.

[0043] First, the mask input module 110 receives a tag sequence (such as text, words, etc.) input from the outside to generate a masked tag sequence. The mask input module 110 is used to mask specific tags in the sequence to learn context information or prevent future information leakage during training.

[0044] The masked token sequence output from the mask input module 110 is passed to the vector embedding module 120. The vector embedding module 120 maps the discrete tokens (e.g., word IDs) of the masked token sequence into a continuous high-dimensional vector representation. This step is completed by the embedding layer, which converts the token sequence into a vector form that can be used for model calculation.

[0045] The high-dimensional vector output by the vector embedding module 120 is further input to the position encoding module 130, which adds position information to each high-dimensional vector (usually through a fixed position encoding function or a trainable position embedding). In this way, the model can capture the position information of each token in the sequence.

[0046] Finally, the output vectors of the vector embedding module 120 and the position encoding module 130 are fused (usually by addition) to obtain a vector embedding sequence containing position information. This sequence is then used as the input of the decoder 200.

[0047] The decoder 200 may be formed by stacking a plurality of layers, each layer in the decoder 200 includes: a masked self-attention layer 210 , a Layer Norm regularization layer 220 , and a feedforward neural network 230 .

[0048] The masked self-attention layer 210 receives the vector embedding with position information output by the input layer 100. The masked self-attention layer 210 uses the input vector to calculate the query, key and value, and restricts each tag to only pay attention to itself and the previous tags through the mask mechanism. The output of the masked self-attention layer passes through the multi-head attention mechanism to generate a feature representation rich in context information. Figure 1 The structure of the masked self-attention layer 210 on the right is described in detail below.

[0049] The output of the masked self-attention layer 210 needs to be normalized by the Layer Norm regularization layer 220 to stabilize the training and accelerate convergence. Figure 1 A Layer Norm regularization layer 220 is inserted between the medium mask self-attention layer 210 and the feedforward neural network 230 to ensure that the data distribution output after the residual connection is stable.

[0050] After the normalization operation, the result flows into the feedforward neural network 230. The feedforward neural network 230 may include a linear layer (ie, a weight matrix) and a nonlinear activation function. The feedforward neural network 230 is used to further transform and extract features of the attention output. The processed result is again passed through a residual connection (not shown) and another Layer Norm regular layer 220 to generate the output of the current layer of the decoder 200, and the output is used as the input of the next layer of the decoder 200. At the last layer (ie, the top layer) of the stacked layer of the decoder 200, the output result continues to flow to the output layer 300.

[0051] The output layer 300 includes: a linear layer 310 and a Softmax layer 320. The feature vector output by the last layer of the decoder is mapped to the dimension size of the output space, such as the size of a set of categories or a set of possible targets, by the linear layer 310. The vector output by the linear layer 310 contains the original score of each possible target. In some embodiments, the linear layer 310 can be a fully connected layer. Subsequently, these scores are input to the Softmax layer 320, which converts these scores into a probability distribution used to represent the predicted probability of each target. Ultimately, the model selects the most likely target as the prediction result based on the probability distribution.

[0052] Figure 1 The internal structure of the masked self-attention layer 210 is shown on the right side. The masked self-attention layer 210 includes: a query vector (Query vector), a key vector (Key vector), a value vector (Value vector), an attention matrix W q , W k , W v, multi-head attention mechanism 211, multi-head fusion matrix 212 and output.

[0053] Each input vector (e.g., the vector embedding with position information output by the input layer 100 or the previous masked self-attention layer) will be transformed through three independent linear transformations (linear layers) to obtain the corresponding Query vector, Key vector, and Value vector. The weight matrix corresponding to these three linear transformations is the attention matrix W q , W k , W v Specifically, the matrix W q Represents the linear transformation weight matrix that maps the input vector to the Query space. Matrix W k Represents the linear transformation weight matrix that maps the input vector to the Key space. Matrix W b Represents the linear transformation weight matrix that maps the input vector to the Value space. If the input vector is X, it can be expressed as: Q = XW q ,K=XW k ,V=XW b .

[0054] Next, the multi-head attention mechanism 211 divides these Q, K, and V vectors into several subspaces (corresponding to multiple heads), and calculates the attention distribution inside each head. For each attention head (Head), a dot product operation is performed on the Q and K corresponding to the head to calculate the attention score, and scaling and masking are performed. The mask here ensures the autoregressive property, that is, in sequence prediction, each position can only "pay attention" to itself and the previous position, thereby avoiding future information leakage. After completing the mask, the scaled attention score is normalized by Softmax to obtain the attention weight distribution, and then the Value vector is weighted and summed with this weight to generate the attention output of the head.

[0055] Since the multi-head attention mechanism 211 performs multiple different attention calculations at the same time (i.e., there are multiple heads), each head performs attention calculations independently in a lower-dimensional subspace, thereby being able to capture the features and relationships of different subspaces in the input sequence. After the calculation is completed, the attention output results of all heads are concatenated along the feature dimension. The concatenated results are then linearly transformed through the multi-head fusion matrix 212 as a linear layer to fuse the different feature information extracted by multiple heads to form a richer and more expressive representation of the input sequence.

[0056] Finally, the result of the fused multi-head attention will be used as output for subsequent processing steps (such as residual connection with the input, layer normalization, further nonlinear transformation through the feed-forward neural network layer, etc.), thereby completing the processing flow of the masked self-attention layer 210.

[0057] Figure 2 The present invention is a flowchart of a method for fine-tuning parameters of a pre-trained model according to an embodiment of the present invention. Figure 3 Schematic diagram of a model to be fine-tuned according to a parameter fine-tuning method of a pre-trained model according to an embodiment of the present invention. Figure 4 Schematic diagram of a rotation matrix of a parameter fine-tuning method for a pre-trained model according to an embodiment of the present invention.

[0058] The parameter fine-tuning method according to the embodiment of the present invention can be used for Figure 1 The pre-trained model for natural language processing described above is fine-tuned. The pre-trained model includes a linear layer with an initial weight matrix, as shown above. Figure 1 As described, the matrix W q , W k , W v , the multi-head fusion matrix 212, the linear layer in the feedforward neural network 230, and the linear layer 310 of the output layer 300 can all be selected as the linear layer of the parameter fine-tuning method of the pre-trained model according to an embodiment of the present invention.

[0059] Reference Figure 2 In step S1, an orthogonal transformation based on at least one rotation matrix is ​​applied to the linear layer and an angle parameter matrix is ​​obtained according to the orthogonal transformation, an orthogonal fine-tuning layer is constructed based on the initial weight matrix and the angle parameter matrix of the linear layer, and the linear layer in the pre-trained model is replaced with the orthogonal fine-tuning layer to obtain a model to be fine-tuned.

[0060] Reference Figure 3 In order to better achieve a balance between retaining the initial pre-training model capabilities and adapting to downstream tasks, the orthogonal fine-tuning layer according to an embodiment of the present invention is the initial weight matrix multiplied by the angle parameter matrix. The orthogonal fine-tuning layer is represented by: W Orth =W 0 R, where W Orth is the orthogonal fine-tuning layer, W 0 is the initial weight matrix, and R is the angle parameter matrix.

[0061] The angle parameter matrix R is the product of all rotation matrices in at least one rotation matrix. The angle parameter matrix R is expressed as R = R 1 ·R 2 ·…·R n , where R 1 ,R 2,…,R n is a rotation matrix in at least one rotation matrix, each rotation matrix R i It is expressed as:

[0062]

[0063] Where d is the dimension of the rotation matrix, and d = 2 a , B F (k) is the k×k dimensional rotation subspace, k=2 b ,b∈[1,a], a and b are positive integers, diag(·) represents the function of diagonalizing the vector, and the rotation subspace B F (k) is expressed as:

[0064]

[0065] d / k means that there are d / k rotation subspaces B in the d-dimensional matrix F (k) to fill,

[0066]

[0067] θ 1 ,θ 2 , ...θ k / 2 is the learnable angle parameter.

[0068] Rotation matrix R i As a sparse rotation matrix, it can introduce flexible transformations at a low parameter cost. Only a small subspace is adjusted each time, reducing the interference to the entire matrix. The angle parameter matrix R is composed of the sparse rotation matrix R i The resulting dense orthogonal matrix is ​​able to represent more complex global transformations.

[0069] The following will Figure 4 The specific steps of obtaining the rotation matrix are described by the example shown in FIG.

[0070] like Figure 4 As shown, the rotation matrix decomposes the n×n matrix into n / k k×k dimensional rotation subspaces, each of which is filled with a rotation subspace B F (k), whose angle introduces learnable parameters, and different decomposition methods form different rotation matrices. The operation of applying an orthogonal transformation based on several rotation matrices to the selected linear layer to obtain an orthogonal fine-tuning layer can also be understood as replacing the selected linear layer with the model to be fine-tuned obtained above, thereby obtaining a pre-trained model to be fine-tuned with the linear layer replaced.

[0071] Example A1: When d = 8, k = 2, d / k = 4. Matrix B F (2) is a 2×2 elementary rotation matrix:

[0072]

[0073] Therefore, the matrix R i Can be composed of four B F (2) Composition:

[0074]

[0075] Similarly, in Example A2, when d=8, k=4, the matrix B F (4) is:

[0076]

[0077] Therefore, the matrix R i Can be composed of two B F (4) Composition:

[0078]

[0079] Similarly, in Example A3, when d=8, k=8, the matrix B F (8) is:

[0080]

[0081] At this time, therefore, the matrix R i That is B F (8) Itself.

[0082] The above three examples show how to construct the rotation matrix R according to different values ​​of d and k. i . The structure of each matrix is ​​composed of the rotation subspace B F (k) are composed of different blocks and filled into a d-dimensional matrix by diagonalization. It should be noted that the above examples are only given for ease of understanding, and the values ​​of d and k can be further determined according to the dimension of the selected linear layer and the hyperparameters of the pre-trained model.

[0083] For the rotation matrix R i The initialization can be achieved by randomly initializing the angle of the rotation matrix.

[0084] In another embodiment, the rotation matrix R i The initialization step can also be determined by the following derivation: where W is the target fine-tuning weight matrix, W 0 is the initial weight matrix, R is the angle parameter matrix of the orthogonal fine-tuning layer, and one of the at least one rotation matrix is ​​assigned to the initialization matrix R 0 , and the remaining rotation matrices in the at least one rotation matrix are initialized to the identity matrix. As an example, the rotation matrix R1 ,R 2 ,…,R n R 2 ,…,R n are initialized to the unit matrix, so the initialization process of the angle parameter matrix R can be simplified to the rotation matrix R 1 Initialize.

[0085] In an embodiment, R 0 is the initialization matrix to be determined, and the inverse This is to ensure that the starting point of fine-tuning can satisfy W = W before iterative optimization. 0 To preserve the initial capabilities of the pre-trained model.

[0086] The derivation process of initialization is as follows:

[0087]

[0088] Find the partial derivative of the rotation matrix R and perform gradient propagation,

[0089]

[0090] Fine-tune through the method of this application, η represents the learning rate, and the optimized W is obtained opt :

[0091]

[0092] When fine-tuning using the full parameter fine-tuning method, the gradient propagation process is:

[0093]

[0094] By minimizing the norm ||W′ opt -W opt ||Align the gradient propagation process between the proposed method and the full parameter fine-tuning, and substitute W′ opt and W opt You can get:

[0095]

[0096] Solving this optimization goal, we can get the initialization solution,

[0097]

[0098] In order to obtain R 0 Initialization of variables in the scheme needs to be based on the 0 Singular value decomposition of partial derivatives,

[0099]

[0100] Among them, the gradient Represents the loss function L on the initial weight matrix W 0 , U and V are orthogonal matrices, their column vectors represent the left singular vector and right singular vector obtained by singular value decomposition, Σ is a non-negative real diagonal matrix whose diagonal elements are singular values, r is the rank of Σ, and W 0[r] They are matrix U and matrix W respectively 0 The first r columns. [n-r] represents the identity matrix of (nr)×(nt), where n is the weight matrix W 0 Dimension.

[0101] Return to reference Figure 2 In step S2, only the angle parameter matrix is ​​updated through the gradient back-propagation algorithm based on the text dataset to realize the training of the fine-tuning model.

[0102] In an embodiment, the loss function adopts a cross entropy loss function:

[0103] Loss(P,Q)=∑E x～P (logQ(x))

[0104] The error back propagation algorithm is used to update the parameters of the model to be fine-tuned, and the optimizer uses the AdamW optimizer.

[0105] In step S3, the trained model to be fine-tuned is output as a result of parameter fine-tuning.

[0106] In some embodiments, the initial weight matrix of the linear layer of the pre-trained model is multiplied by the angle parameter matrix of the orthogonal fine-tuning layer of the trained model to be fine-tuned to obtain the fine-tuned weight matrix of the linear layer, and the initial weight matrix of the linear layer of the pre-trained model is replaced by the fine-tuned weight matrix to obtain the result of parameter fine-tuning. This process can be called matrix fusion. After matrix fusion, the model to be fine-tuned including the orthogonal fine-tuning layer can be restored to the initial structure of the pre-trained model to obtain a target model adapted to the target task (i.e., the target task corresponding to the text data set used for training) for subsequent reasoning.

[0107] In the embodiment, through step S2, the angle parameters of the rotation matrix used to construct the orthogonal fine-tuning layer are fully learned. On the other hand, through step S3, the orthogonal fine-tuning layer of the model to be fine-tuned is replaced with the result obtained in the matrix fusion step, so that the fine-tuned pre-trained model is consistent with the original model. Figure 2The pre-trained models shown have a consistent structure and are better able to adapt to the target model of the target task. Compared to the original pre-trained model without fine-tuning parameters, the target model does not introduce any additional storage and inference overhead because the fine-tuned pre-trained model has exactly the same structure.

[0108] Specifically, Table 1 shows the quantitative comparative experimental results of the parameter fine-tuning method of the pre-trained model according to an embodiment of the present invention on the GLUE evaluation task. The pre-trained model is DeBERTaV3-base, and the linear layer selects the self-attention layer and the linear layer of the multi-layer perceptron. The control group uses LoRA for optimization, and Example 1 uses the parameter fine-tuning method according to the embodiment of the present invention. This task includes 8 subtasks. Experiments show that the parameter fine-tuning method according to the embodiment of the present invention can achieve better fine-tuning effects than LoRA with fewer parameters. Experiments have proved that the parameter fine-tuning method according to the embodiment of the present invention is a very practical pre-training model fine-tuning method.

[0109] Table 1

[0110] method Average score Parameter quantity accounts for the ratio of all parameters Embodiment 1 89.25290423 0.1123％ Comparative Example (LoRA) 88.53589671 0.2401％

[0111] Another aspect of the present invention discloses a device for fine-tuning parameters of a pre-training model. Figure 5 4 is a structural block diagram of a parameter fine-tuning device for a pre-training model according to an embodiment of the present invention.

[0112] Reference Figure 5 The parameter fine-tuning device of the pre-training model includes: an initialization module 501, a training module 502 and a fusion module 503. The pre-training model can be any of the pre-training models described above, and repeated descriptions are omitted here.

[0113] The initialization module 501 is configured to apply an orthogonal transformation based on at least one rotation matrix to the linear layer and obtain an angle parameter matrix according to the orthogonal transformation, construct an orthogonal fine-tuning layer based on the initial weight matrix and the angle parameter matrix, and replace the linear layer in the pre-trained model with the orthogonal fine-tuning layer to obtain a model to be fine-tuned. The initialization module 501 is configured to perform a reference Figure 2 The step S1 described above will not be described repeatedly.

[0114] The training module 502 is configured to update only the angle parameter matrix based on the text data set through the gradient back propagation algorithm to implement the training of the fine-tuning model. The training module 502 is configured to perform reference Figure 2 The step S2 described above will be omitted here for repetitive description.

[0115] The output module 503 is configured to output the trained model to be fine-tuned as the result of parameter fine-tuning. Figure 2The step S3 described above will be omitted here for repetitive description.

[0116] Figure 6 is a block diagram illustrating a computing device according to an embodiment of the present disclosure.

[0117] Reference Figure 6 According to an embodiment of the present disclosure, the computing device 600 may include a processor 610 and a memory 620. The processor 610 may include (but not limited to) a central processing unit (CPU), a digital signal processor (DSP), a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a microprocessor, an application-specific integrated circuit (ASIC), etc. The memory 620 may store a computer program to be executed by the processor 610. The memory 620 includes a high-speed random access memory and / or a non-volatile computer-readable storage medium. When the processor 610 executes the computer program stored in the memory 620, the parameter fine-tuning method of the pre-trained model as described above may be implemented.

[0118] The parameter fine-tuning method of the pre-trained model according to an embodiment of the present disclosure can be written as a computer program and stored on a computer-readable storage medium. When the computer program is executed by a processor, the parameter fine-tuning method of the pre-trained model as described above can be implemented. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. In one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0119] Although some embodiments of the present disclosure have been shown and described, it will be appreciated by those skilled in the art that modifications may be made to the embodiments without departing from the principles and spirit of the present disclosure, the scope of which is defined by the claims and their equivalents.

[0120] It should be noted that the functions or steps that can be implemented by the above-mentioned storage medium or electronic device can refer to the relevant description in the aforementioned method embodiment, and will not be repeated here.

[0121] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several deformations and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the patent of this application shall be based on the attached claims.

Claims

1. A method for fine-tuning parameters of a pre-trained model, characterized in that: The parameter fine-tuning method performs parameter fine-tuning on a pre-trained model for natural language processing based on a text dataset, wherein the pre-trained model includes a linear layer having an initial weight matrix, and the parameter fine-tuning method includes: Applying an orthogonal transformation based on at least one rotation matrix to the linear layer and obtaining an angle parameter matrix according to the orthogonal transformation, constructing an orthogonal fine-tuning layer based on the initial weight matrix and the angle parameter matrix, and replacing the linear layer in the pre-trained model with the orthogonal fine-tuning layer to obtain a model to be fine-tuned; Based on the text data set, only the angle parameter matrix is ​​updated by a gradient back-propagation algorithm to implement training of the model to be fine-tuned; and The trained model to be fine-tuned is output as a result of parameter fine-tuning.

2. The parameter fine-tuning method according to claim 1, characterized in that: The angle parameter matrix is ​​the product of all rotation matrices in the at least one rotation matrix.

3. The parameter fine-tuning method according to claim 2, characterized in that: The orthogonal fine-tuning layer is the initial weight matrix multiplied by the angle parameter matrix.

4. The parameter fine-tuning method according to claim 3, characterized in that: The orthogonal fine-tuning layer is represented as: W Orth =W0·R, Among them, W Orth is the orthogonal fine-tuning layer, W0 is the initial weight matrix, R is the angle parameter matrix, R = R1·R2·…·R n , R1,R2,…,R n is a rotation matrix in the at least one rotation matrix, Each rotation matrix R in the at least one rotation matrix i It is expressed as: Where d is the dimension of the rotation matrix, and d = 2 a , B F (k) is the k×k dimensional rotation subspace, k=2 b ,b∈[1,a], a and b are positive integers, diag(·) represents the function of diagonalizing the vector, and the rotation subspace B F (k) is expressed as: d / k means that there are d / k rotation subspaces B in the d-dimensional matrix F (k) to fill, θ1,θ2,……θ k / 2 is the learnable angle parameter.

5. The parameter fine-tuning method according to claim 4, characterized in that: The parameter fine-tuning method further comprises the step of initializing the angle parameter matrix. wherein one of the at least one rotation matrix is ​​assigned an initialization matrix R0, and the remaining rotation matrices of the at least one rotation matrix are initialized to unit matrices, R0 is expressed as: Where W0 is the initial weight matrix of the pre-trained model, U is obtained by The left singular vector matrix obtained by singular value decomposition, the gradient represents the partial derivative of the loss function L with respect to the initial weight matrix W0, U [r] is the first r columns of the left singular vector matrix U, W 0[r] are the first r columns of the initial weight matrix W0, I [n-r] represents the (nr)×(nr) identity matrix, where n is the dimension of the weight matrix W0.

6. The parameter fine-tuning method according to claim 1, characterized in that: Outputting the trained model to be fine-tuned as a result of parameter fine-tuning includes: The initial weight matrix of the linear layer is multiplied by the angle parameter matrix to obtain the fine-tuned weight matrix of the linear layer, and the initial weight matrix of the linear layer of the pre-trained model is replaced by the fine-tuned weight matrix to obtain the result of the parameter fine-tuning.

7. The parameter fine-tuning method according to claim 1, characterized in that: The pre-trained model includes one of a multi-layer perceptron, a convolution-based neural network and a Transformer-based neural network, and the linear layer is at least one of a feedforward network, a convolution layer and a Transformer attention layer corresponding to the pre-trained model.

8. A parameter fine-tuning device for a pre-trained model, characterized in that: The parameter fine-tuning device fine-tunes parameters of a pre-trained model for natural language processing based on a text data set, wherein the pre-trained model includes a linear layer having an initial weight matrix, and the parameter fine-tuning device includes: An initialization module is configured to apply an orthogonal transformation based on at least one rotation matrix to the linear layer and obtain an angle parameter matrix according to the orthogonal transformation, construct an orthogonal fine-tuning layer based on the initial weight matrix and the angle parameter matrix, and replace the linear layer in the pre-trained model with the orthogonal fine-tuning layer to obtain a model to be fine-tuned; A training module, configured to update only the angle parameter matrix based on the text data set through a gradient back propagation algorithm to implement training of the model to be fine-tuned; and The output module is configured to output the trained model to be fine-tuned as a result of parameter fine-tuning.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the parameter fine-tuning method of the pre-training model according to any one of claims 1 to 7 is implemented.

10. A computing device, characterized in that: The computing device comprises: Processor; and A memory storing a computer program, which, when executed by a processor, implements a parameter fine-tuning method for a pre-trained model according to any one of claims 1 to 7.

Images