Methods, devices, equipment and media for fine-tuning parameters of pre-trained models

Abstract

A method, apparatus, device, and medium for fine-tuning parameters of a pre-trained model are disclosed. The parameter fine-tuning method fine-tunes the parameters of a pre-trained model for natural language processing based on a text dataset. The pre-trained model includes a linear layer with an initial weight matrix. The parameter fine-tuning method includes: applying an orthogonal transformation based on at least one rotation matrix to the linear layer and obtaining an angle parameter matrix based on the orthogonal transformation; constructing an orthogonal fine-tuning layer based on the initial weight matrix and the angle parameter matrix; replacing the linear layer in the pre-trained model with the orthogonal fine-tuning layer to obtain the model to be fine-tuned; updating only the angle parameter matrix using a gradient backpropagation algorithm based on the text dataset to train the model to be fine-tuned; and outputting the trained model to be fine-tuned as the result of parameter fine-tuning.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, and in particular relates to an efficient method for fine-tuning the parameters of a pre-trained model. Background Technology

[0002] Self-supervised pre-trained large-scale models have demonstrated powerful representation and generalization capabilities across various downstream tasks. Fine-tuning pre-trained models to adapt to more specialized tasks has become a more mainstream paradigm. As the number of model parameters increases, pre-trained models continuously push their limits. However, this also presents significant challenges to model fine-tuning.

[0003] Faced with such demands and challenges, the importance of efficient parameter fine-tuning methods becomes increasingly prominent, as it is possible to achieve results on downstream tasks that are close to those achieved by fine-tuning all parameters on the entire model with only a small number of parameters.

[0004] LoRA has been a mainstream method for efficient parameter fine-tuning in the past, but it cannot effectively balance the preservation of the capabilities of the initial pre-trained model and its adaptation to downstream tasks. Summary of the Invention

[0005] The embodiments of this disclosure provide a parameter fine-tuning method and apparatus for a pre-trained model, which enables performance similar to or better than full parameter fine-tuning in downstream tasks with only a very small number of parameters to be updated, while effectively reducing storage and computational overhead, and achieving more flexible and efficient downstream adaptation while maintaining the initial representational capability of the pre-trained model.

[0006] Embodiments of this disclosure provide a parameter fine-tuning method for a pre-trained model. The parameter fine-tuning method fine-tunes the parameters of a pre-trained model for natural language processing based on a text dataset. The pre-trained model includes a linear layer with an initial weight matrix. The parameter fine-tuning method includes: applying an orthogonal transformation based on at least one rotation matrix to the linear layer and obtaining an angle parameter matrix according to the orthogonal transformation; constructing an orthogonal fine-tuning layer based on the initial weight matrix and the angle parameter matrix; and replacing the linear layer in the pre-trained model with the orthogonal fine-tuning layer to obtain the model to be fine-tuned.

[0007] Based on the text dataset, the gradient backpropagation algorithm is used to update only the angle parameter matrix to train the model to be fine-tuned; and

[0008] The trained model to be fine-tuned is output as the result of parameter fine-tuning.

[0009] Optionally, the angle parameter matrix can be the product of all rotation matrices in at least one rotation matrix.

[0010] Optionally, the orthogonal fine-tuning layer is the initial weight matrix multiplied by the angle parameter matrix.

[0011] Optionally, the orthogonal fine-tuning layer is represented as: W Ort =W0·R, where W Orth For orthogonal fine-tuning layers, W0 is the initial weight matrix, R is the angle parameter matrix, and R = R1·R2·…·R n R1,R2,…,R n R is a rotation matrix in at least one rotation matrix, and each rotation matrix in at least one rotation matrix is ​​R i Represented as:

[0012]

[0013] Where d is the dimension of the rotation matrix, and d = 2 a B F (k) is a k×k dimensional rotation subspace, where k = 2. b b∈[1, a], where a and b are positive integers, diag(·) denotes the function that diagonalizes the vector, and the rotation subspace B F (k) is represented as:

[0014]

[0015] d / k represents the number of rotation subspaces B in a d-dimensional matrix consisting of d / k rotations. F (k) to fill

[0016]

[0017] θ1, θ2, ... θ k / 2 These are learnable angle parameters.

[0018] Optionally, the parameter fine-tuning method further includes a step of initializing the angle parameter matrix, wherein one of the rotation matrices in at least one rotation matrix is ​​assigned the value of the initialization matrix R0, and the remaining rotation matrices in at least one rotation matrix are initialized as identity matrices, where R0 is denoted as:

[0019]

[0020] Where W0 is the initial weight matrix of the pre-trained model, and U is the weight matrix obtained by applying gradients. The gradient of the left singular vector matrix obtained by performing singular value decomposition. U represents the partial derivative of the loss function L with respect to the initial weight matrix W0. [r] W is the first r columns of the left singular vector matrix U. 0[r] The first r columns of the initial weight matrix W0, I [n-r]Let W0 be the identity matrix of (nr)×(nr), where n is the dimension of the weight matrix W0.

[0021] Optionally, the pre-trained model includes one of a multilayer perceptron, a convolutional neural network, and a Transformer-based neural network, wherein the linear layer is at least one of a feedforward network, a convolutional layer, and a Transformer attention layer corresponding to the pre-trained model.

[0022] Embodiments of this disclosure provide a parameter fine-tuning apparatus for a pre-trained model. The parameter fine-tuning apparatus fine-tunes the parameters of a pre-trained model for natural language processing based on a text dataset. The pre-trained model includes a linear layer with an initial weight matrix. The parameter fine-tuning apparatus includes: an initialization module configured to apply an orthogonal transformation based on at least one rotation matrix to the linear layer and obtain an angle parameter matrix according to the orthogonal transformation; construct an orthogonal fine-tuning layer based on the initial weight matrix and the angle parameter matrix; and replace the linear layer in the pre-trained model with the orthogonal fine-tuning layer to obtain the model to be fine-tuned; a training module configured to update only the angle parameter matrix based on the text dataset using a gradient backpropagation algorithm to train the model to be fine-tuned; and an output module configured to output the trained model to be fine-tuned as the result of parameter fine-tuning.

[0023] Embodiments of this disclosure provide an electronic device, which includes a storage medium, a processor, and instructions or code stored on the storage medium and executable on the processor. When the processor executes the program, it implements the parameter fine-tuning method of the pre-trained model as described above.

[0024] Embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the parameter fine-tuning method for a pre-trained model as described above.

[0025] According to one or more embodiments of the present disclosure, the parameter fine-tuning method and apparatus for the pre-trained model obtain orthogonal fine-tuning layers by introducing rotation matrices into specific linear layers of the pre-trained model, and only update the learnable angle parameters of these rotation matrices to achieve parameter fine-tuning of the pre-trained model. This decomposes the parameter fine-tuning process into finding suitable normal vectors and adjusting the orientation of the initial weight matrix by rotating these vectors. The only parameters that need to be updated are the angle parameter matrices in the model to be fine-tuned, which is a very small percentage compared to the total number of parameters in the pre-trained model. Simultaneously, a dense orthogonal matrix is ​​formed by multiplying multiple rotation matrices, ensuring adaptability to downstream tasks. Overall, the parameters efficiently complete the fine-tuning task of the pre-trained model on downstream tasks, achieving a balance between preserving the capabilities of the initial pre-trained model and adapting to downstream tasks, and ensuring no additional storage or inference overhead compared to the initial pre-trained model. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the described embodiments are only a part of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Figure 1 This is a schematic diagram of a pre-trained model to be fine-tuned according to an embodiment of the present invention.

[0028] Figure 2 This is a flowchart of a parameter fine-tuning method for a pre-trained model according to an embodiment of the present invention.

[0029] Figure 3 This is a schematic diagram of the model to be fine-tuned in the parameter fine-tuning method of the pre-trained model according to an embodiment of the present invention.

[0030] Figure 4 This is a schematic diagram of the rotation matrix for a parameter fine-tuning method of a pre-trained model according to an embodiment of the present invention.

[0031] Figure 5 This is a structural block diagram of a parameter fine-tuning device for a pre-trained model according to an embodiment of the present invention.

[0032] Figure 6 This is a block diagram illustrating a computing device according to an embodiment of the present disclosure. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] This invention discloses a method for fine-tuning the parameters of a pre-trained model. This method fine-tunes the parameters of a pre-trained model used for natural language processing based on a text dataset.

[0035] The parameter fine-tuning method described above is applicable to a wide range of application areas and model fine-tuning scenarios with diverse data formats for which pre-trained models can be applied. For example, in the field of natural language processing, a basic model can be pre-trained on massive unlabeled general text corpora (such as news reports, social media posts, and encyclopedia articles), and then the parameters can be fine-tuned using a relatively small but carefully labeled text dataset for specific tasks (such as legal text classification, medical report summarization, and financial text analysis). This significantly reduces training overhead while enabling customized and high-performance model applications.

[0036] According to exemplary embodiments of this disclosure, pre-trained models to which parameter fine-tuning methods can be applied include one of multilayer perceptrons (MLP), convolutional neural networks (CNN), and Transformer-based models, and can be selected according to the type of the target task. For example, for prediction or classification tasks that primarily use structured numerical features, MLPs are more efficient in pre-training and fine-tuning; while for complex tasks that primarily use text data or multimodal information as input (such as machine translation, text summarization, dialogue understanding, speech recognition, and multimodal retrieval), Transformer-based neural networks construct powerful feature representation capabilities through multi-head attention mechanisms and large-scale pre-training corpora, and then perform parameter fine-tuning based on labeled data of the target task, thereby achieving accurate and efficient downstream task adaptation.

[0037] Furthermore, the linear layers of the pre-trained model are at least one of the feedforward network, convolutional layer, and attention layer of Transformer corresponding to the pre-trained model.

[0038] Taking the decoder of a Transformer-based pre-trained model as an example, a typical Transformer-based decoder uses a multi-layer attention mechanism and a feedforward network as its core to process the input sequence and generate and predict the target sequence.

[0039] Figure 1 This is a schematic diagram of a pre-trained model to be fine-tuned according to an embodiment of the present invention.

[0040] The following will refer to Figure 1 The following description uses the pre-trained Transformer model 10 as an example.

[0041] Reference Figure 1 The Transformer-based pre-trained model 10 includes: an input layer 100, a decoder 200, and an output layer 300.

[0042] The input layer 100 includes a mask input module 110, a vector embedding module 120, and a position encoding module 130.

[0043] First, the masking input module 110 accepts a sequence of tags (such as text, words, etc.) from an external source to generate a masked sequence of tags. The masking input module 110 is used to mask specific tags in the sequence in order to learn contextual information during training or to prevent the leakage of future information.

[0044] The masked token sequence output from the masking input module 110 is passed to the vector embedding module 120. The vector embedding module 120 maps the discrete tokens (e.g., word IDs) of the masked token sequence to continuous high-dimensional vector representations. This step is accomplished through an embedding layer, which converts the token sequence into a vector form that can be used for model computation.

[0045] The high-dimensional vector output by the vector embedding module 120 is further input to the position encoding module 130, which adds positional information to each high-dimensional vector (usually through a fixed positional encoding function or a trainable positional embedding). In this way, the model can capture the positional information of each marker in the sequence.

[0046] Finally, the output vectors of the vector embedding module 120 and the position encoding module 130 are fused (usually by addition) to obtain a vector embedding sequence containing position information. This sequence is then used as the input to the decoder 200.

[0047] The decoder 200 can be composed of multiple stacked layers, each of which includes: a masked self-attention layer 210, a Layer Norm regularization layer 220, and a feedforward neural network 230.

[0048] The masked self-attention layer 210 receives the positional vector embedding output from the input layer 100. It uses the input vector to calculate the Query, Key, and Value, and employs a masking mechanism to restrict each tag to only focus on itself and the tags preceding it. The output of the masked self-attention layer is processed by a multi-head attention mechanism to generate a context-rich feature representation. (Refer to...) Figure 1 The structure of the mask self-attention layer 210 on the right side is described in detail below.

[0049] The output of the mask from the attention layer 210 needs to be normalized by the Layer Norm regularization layer 220 to stabilize training and accelerate convergence. Figure 1 A Layer Norm regularization layer 220 is inserted between the attention layer 210 and the feedforward neural network 230 in the middle mask to ensure that the data distribution of the output after the residual connection is stable.

[0050] After normalization, the result flows into a feedforward neural network 230. The feedforward neural network 230 may include linear layers (i.e., weight matrices) and non-linear activation functions. The feedforward neural network 230 is used to further transform and extract features from the attention output. The processed result is then passed through a residual connection (not shown) and another Layer Norm regularization layer 220 to generate the output of the current layer of the decoder 200, which will be used as the input to the next layer of the decoder 200. In the last layer (i.e., the top layer) of the stacked layers of the decoder 200, the output continues to flow to the output layer 300.

[0051] Output layer 300 includes a linear layer 310 and a softmax layer 320. The feature vector output from the last layer of the decoder is mapped to the dimension of the output space, such as the size of the class set or the set of possible targets, through the linear layer 310. The vector output by the linear layer 310 contains the raw score for each possible target. In some embodiments, the linear layer 310 may be a fully connected layer. These scores are then fed into the softmax layer 320, which transforms them into a probability distribution representing the predicted probability of each target. Finally, the model selects the most probable target as the prediction based on the probability distribution.

[0052] Figure 1 The right side shows the internal structure of the masked self-attention layer 210. The masked self-attention layer 210 includes: a query vector, a key vector, a value vector, and an attention matrix W. q W k W vMulti-head attention mechanism 211, multi-head fusion matrix 212 and output.

[0053] Each input vector (e.g., a position-informed vector embedding from the input layer 100 or the previous mask output from the attention layer) undergoes three independent linear transformations (linear layers) to obtain the corresponding Query vector, Key vector, and Value vector. The weight matrix corresponding to these three linear transformations is the attention matrix W. q W k W v Specifically, matrix W q This represents the linear transformation weight matrix that maps the input vector to the query space. Matrix W k This represents the linear transformation weight matrix that maps the input vector to the key space. Matrix W b This represents the linear transformation weight matrix that maps the input vector to the Value space. If the input vector is X, then it can be represented as: Q = XW q K = XW k V = XW b .

[0054] Next, the multi-head attention mechanism 211 divides these Q, K, and V vectors into several subspaces (corresponding to multiple heads), and calculates the attention distribution within each head. For each attention head, a dot product operation is performed on the Q and K corresponding to that head to calculate the attention score, followed by scaling and masking. The masking here ensures the autoregressive property, meaning that in sequence prediction, each position can only "focus" on itself and its preceding positions, thus avoiding future information leakage. After masking, the scaled attention score is normalized using Softmax to obtain the attention weight distribution, and then the Value vector is weighted and summed using these weights to generate the attention output for that head.

[0055] Since the multi-head attention mechanism 211 performs multiple different attention calculations simultaneously (i.e., multiple heads), each head independently performs attention calculations in a lower-dimensional subspace, thereby capturing the features and relationships of different subspaces in the input sequence. The attention outputs of all heads are concatenated along the feature dimension after computation. The concatenated result is then linearly transformed by the multi-head fusion matrix 212, which serves as a linear layer, to fuse the different feature information extracted by multiple heads, forming a richer and more expressive representation of the input sequence.

[0056] Finally, the result of the fused multi-head attention will be used as the output for subsequent processing steps (such as making residual connections with the input, performing layer normalization, and performing further nonlinear transformations through the feed-forward network layer), thereby completing the processing flow of the mask self-attention layer 210.

[0057] Figure 2 This is a flowchart of a parameter fine-tuning method for a pre-trained model according to an embodiment of the present invention. Figure 3 This is a schematic diagram of the model to be fine-tuned in the parameter fine-tuning method of the pre-trained model according to an embodiment of the present invention. Figure 4 This is a schematic diagram of the rotation matrix for a parameter fine-tuning method of a pre-trained model according to an embodiment of the present invention.

[0058] The parameter fine-tuning method according to embodiments of the present invention can be used for the above reference. Figure 1 The description describes a pre-trained model for natural language processing that undergoes parameter fine-tuning. The pre-trained model consists of linear layers with an initial weight matrix, as shown above. Figure 1 The matrix W described q W k W v The linear layers in the multi-head fusion matrix 212, the feedforward neural network 230, and the linear layer 310 of the output layer 300 can all be selected to act on the linear layers of the parameter fine-tuning method of the pre-trained model according to the embodiment of the present invention.

[0059] Reference Figure 2 In step S1, an orthogonal transformation based on at least one rotation matrix is ​​applied to the linear layer and the angle parameter matrix is ​​obtained according to the orthogonal transformation. An orthogonal fine-tuning layer is constructed based on the initial weight matrix and angle parameter matrix of the linear layer. The linear layer in the pre-trained model is replaced with the orthogonal fine-tuning layer to obtain the model to be fine-tuned.

[0060] Reference Figure 3 To better balance preserving the capabilities of the initial pre-trained model and adapting to downstream tasks, the orthogonal fine-tuning layer according to an embodiment of the present invention is an initial weight matrix multiplied by an angle parameter matrix. The orthogonal fine-tuning layer is represented as: W Orth =W0·R, where W Orth For the orthogonal fine-tuning layer, W0 is the initial weight matrix, and R is the angle parameter matrix.

[0061] The angle parameter matrix R is the product of all rotation matrices in at least one rotation matrix, and is expressed as R = R1·R2·…·R n Where R1, R2, ..., R n For at least one rotation matrix, each rotation matrix R is a rotation matrix. i Represented as:

[0062]

[0063] Where d is the dimension of the rotation matrix, and d = 2 a B F (k) is a k×k dimensional rotation subspace, where k = 2. b b∈[1, a], where a and b are positive integers, diag(·) denotes the function that diagonalizes the vector, and the rotation subspace B F (k) is represented as:

[0064]

[0065] d / k represents the number of rotation subspaces B in a d-dimensional matrix consisting of d / k rotations. F (k) to fill

[0066]

[0067] θ1, θ2, ... θ k / 2 These are learnable angle parameters.

[0068] Rotation matrix R i As a sparse rotation matrix, it allows for flexible transformations with low parameter cost. Adjustments are made only to small subspaces at a time, reducing interference with the entire matrix. The angle parameter matrix R is derived from the sparse rotation matrix R. i The generated dense orthogonal matrix can represent more complex global transformations.

[0069] The following will be based on Figure 4 The example shown illustrates the specific steps to obtain the rotation matrix.

[0070] like Figure 4 As shown, the rotation matrix is ​​constructed by decomposing an n×n matrix into n / k k×k dimensional rotation subspaces, with each subspace filled by a rotation subspace B. F (k), whose angle introduces learnable parameters, and different decomposition methods form different rotation matrices. Applying an orthogonal transformation based on several rotation matrices to the selected linear layer to obtain an orthogonal fine-tuning layer can also be understood as replacing the selected linear layer with the model to be fine-tuned obtained above, thereby obtaining the pre-trained model to be fine-tuned with the linear layer replaced.

[0071] Example A1: When d = 8 and k = 2, d / k = 4. Matrix B F (2) is a 2×2 elementary rotation matrix:

[0072]

[0073] Therefore, matrix R i It can be made up of four Bs F(2) Composition:

[0074]

[0075] Similarly, in example A2, when d = 8 and k = 4, matrix B F (4) is:

[0076]

[0077] Therefore, matrix R i It can be made by two Bs F (4) Composition:

[0078]

[0079] Similarly, in example A3, when d = 8 and k = 8, matrix B F (8) is:

[0080]

[0081] Therefore, matrix R i That is, B F (8) It itself.

[0082] The three examples above demonstrate how to construct the rotation matrix R based on different values ​​of d and k. i The structure of each matrix is ​​determined by the rotation subspace B. F The different blocks of (k) are filled into the d-dimensional matrix by diagonalization. It should be noted that the above is only an example given for ease of understanding, and the values ​​of d and k can be further determined according to the dimension of the selected linear layer and the hyperparameters of the pre-trained model.

[0083] For rotation matrix R i Initialization can be achieved by randomly initializing the angle of the rotation matrix.

[0084] In another embodiment, the rotation matrix R i The initialization steps can also be determined through the following derivation: where W is the target fine-tuning weight matrix, W0 is the initial weight matrix, R is the angle parameter matrix of the orthogonal fine-tuning layer, and one of the at least one rotation matrix is ​​assigned the initialization matrix R0, while the remaining rotation matrices are initialized as identity matrices. As an example, the rotation matrices R1, R2, ..., R... n R2,…,R n Both are initialized to identity matrices. Therefore, the initialization process of the angle parameter matrix R can be simplified to initializing the rotation matrix R1.

[0085] In this embodiment, R0 is the initialization matrix to be determined, and its inverse is taken. This is to ensure that the starting point of fine-tuning can satisfy W=W0 before iterative optimization, so as to preserve the initial capabilities of the pre-trained model.

[0086] The derivation process for the initialization is as follows:

[0087]

[0088] Take the partial derivative with respect to the rotation matrix R and perform gradient propagation.

[0089]

[0090] Fine-tuning is performed using the method described in this application, where η represents the learning rate, to obtain the optimized W. opt :

[0091]

[0092] When fine-tuning using the full-parameter fine-tuning method, the gradient propagation process is as follows:

[0093]

[0094] By minimizing the norm ||W′ opt -W opt ||Align the gradient propagation process between the method in this application and the full parameter fine-tuning, and substitute W′ opt and W opt You will then receive:

[0095]

[0096] Solving this optimization objective yields the initialization scheme.

[0097]

[0098] To obtain the variables in the R0 initialization scheme, we need to rely on the singular value decomposition of the partial derivatives with respect to W0.

[0099]

[0100] Where, gradient Let L represent the partial derivative of the loss function L with respect to the initial weight matrix W0. U and V are orthogonal matrices, and their column vectors represent the left and right singular vectors obtained from the singular value decomposition, respectively. Σ is a non-negative real diagonal matrix whose diagonal elements are singular values, and r is the rank of Σ. and W 0[r] These are the first r columns of matrix U and matrix W0, respectively. [n-r] Let W0 be the identity matrix of (nr)×(nt), where n is the dimension of the weight matrix W0.

[0101] Return to reference Figure 2 In step S2, the angle parameter matrix is ​​updated only through the gradient backpropagation algorithm based on the text dataset to train the model to be fine-tuned.

[0102] In this embodiment, the loss function used is the cross-entropy loss function:

[0103] Loss(P,Q)=∑E x～P (logQ(x))

[0104] Furthermore, the parameters of the model to be fine-tuned are updated using the backpropagation algorithm, and the optimizer used is the AdamW optimizer.

[0105] In step S3, the trained model to be fine-tuned is output as the result of parameter fine-tuning.

[0106] In some embodiments, the initial weight matrix of the linear layer of the pre-trained model is multiplied by the angle parameter matrix of the orthogonal fine-tuning layer of the trained model to be fine-tuned to obtain the fine-tuned weight matrix of the linear layer, and the initial weight matrix of the linear layer of the pre-trained model is replaced with the fine-tuned weight matrix to obtain the result of parameter fine-tuning. This process can be called matrix fusion. After matrix fusion, the model to be fine-tuned, including the orthogonal fine-tuning layer, can be restored to the initial structure of the pre-trained model to obtain a target model adapted to the target task (i.e., the target task corresponding to the text dataset used for training), so as to facilitate subsequent inference.

[0107] In this embodiment, via step S2, the angle parameters of the rotation matrix used to construct the orthogonal fine-tuning layer are sufficiently learned. On the other hand, via step S3, the orthogonal fine-tuning layer of the model to be fine-tuned is replaced with the result obtained from the matrix fusion step. Thus, the fine-tuned pre-trained model and... Figure 2 The pre-trained models shown have a consistent structure and are better adapted to the target model for the target task. Compared to the original pre-trained model without parameter fine-tuning, the fine-tuned pre-trained model introduces no additional storage and inference overhead because it has the exact same structure.

[0108] Specifically, Table 1 presents the quantitative comparative experimental results of the parameter fine-tuning method of the pre-trained model according to embodiments of the present invention on the GLUE evaluation task. The pre-trained model is DeBERTaV3-base, and the linear layers selected are self-attention layers and linear layers of multilayer perceptrons. LoRA was used for optimization in the comparative model, and the parameter fine-tuning method according to embodiments of the present invention was adopted in Example 1. This task includes 8 sub-tasks. Experiments show that the parameter fine-tuning method according to embodiments of the present invention can achieve better fine-tuning results than LoRA with fewer parameters. Experiments demonstrate that the parameter fine-tuning method according to embodiments of the present invention is a very practical method for fine-tuning pre-trained models.

[0109] Table 1

[0110] method average score Parameters as a percentage of total parameters Example 1 89.25290423 0.1123％ Comparative Example (LoRA) 88.53589671 0.2401％

[0111] Another aspect of the present invention discloses a parameter fine-tuning device for a pre-trained model. Figure 5 This is a structural block diagram of a parameter fine-tuning device for a pre-trained model according to an embodiment of the present invention.

[0112] Reference Figure 5 The parameter fine-tuning device for the pre-trained model includes: an initialization module 501, a training module 502, and a fusion module 503. The pre-trained model can be any of the pre-trained models described above, and repeated descriptions are omitted here.

[0113] The initialization module 501 is configured to apply an orthogonal transformation based on at least one rotation matrix to the linear layer and obtain an angle parameter matrix according to the orthogonal transformation. Based on the initial weight matrix and the angle parameter matrix, an orthogonal fine-tuning layer is constructed, replacing the linear layers in the pre-trained model with the orthogonal fine-tuning layer to obtain the model to be fine-tuned. The initialization module 501 is configured to perform reference... Figure 2 Step S1, which is described in detail, is omitted here for brevity.

[0114] Training module 502 is configured to update only the angle parameter matrix based on the text dataset using the gradient backpropagation algorithm to train the model to be fine-tuned. Training module 502 is configured to perform reference... Figure 2 Step S2, which is described in detail, is omitted here for brevity.

[0115] Output module 503 is configured to output the trained model to be fine-tuned as the result of parameter fine-tuning. Output module 503 is configured to perform reference... Figure 2 Step S3, which is described in detail, is omitted here for brevity.

[0116] Figure 6 This is a block diagram illustrating a computing device according to an embodiment of the present disclosure.

[0117] Reference Figure 6The computing device 600 according to embodiments of the present disclosure may include a processor 610 and a memory 620. The processor 610 may include (but is not limited to) a central processing unit (CPU), a digital signal processor (DSP), a microcomputer, a field-programmable gate array (FPGA), a system-on-a-chip (SoC), a microprocessor, an application-specific integrated circuit (ASIC), etc. The memory 620 may store computer programs to be executed by the processor 610. The memory 620 includes high-speed random access memory and / or a non-volatile computer-readable storage medium. When the processor 610 executes the computer program stored in the memory 620, the parameter fine-tuning method of the pre-trained model as described above can be implemented.

[0118] The parameter fine-tuning method for a pre-trained model according to embodiments of the present disclosure can be written as a computer program and stored on a computer-readable storage medium. When the computer program is executed by a processor, the parameter fine-tuning method for the pre-trained model as described above can be implemented. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. In one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0119] While some embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.

[0120] It should be noted that the functions or steps that the storage medium or electronic device can achieve are described in the relevant descriptions in the foregoing method embodiments, and will not be repeated here.

[0121] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for fine-tuning the parameters of a pre-trained model, characterized in that, The parameter fine-tuning method fine-tunes the parameters of a pre-trained model for natural language processing based on a text dataset. The pre-trained model includes linear layers with initial weight matrices. The parameter fine-tuning method includes: An orthogonal transformation based on at least one rotation matrix is ​​applied to the linear layer and an angle parameter matrix is ​​obtained according to the orthogonal transformation. An orthogonal fine-tuning layer is constructed based on the initial weight matrix and the angle parameter matrix. The linear layer in the pre-trained model is replaced with the orthogonal fine-tuning layer to obtain the model to be fine-tuned. Based on the text dataset, the gradient backpropagation algorithm is used to update only the angle parameter matrix to train the model to be fine-tuned; and The trained model to be fine-tuned is output as the result of parameter fine-tuning. Wherein, the angle parameter matrix is ​​the product of all rotation matrices in the at least one rotation matrix. Wherein, the orthogonal fine-tuning layer is the initial weight matrix multiplied by the angle parameter matrix, and The orthogonal fine-tuning layer is represented as follows: , in, For the orthogonal fine-tuning layer, The initial weight matrix is... The angle parameter matrix, , For the rotation matrix in the at least one rotation matrix, Each of the at least one rotation matrix Represented as: ， in, It is the dimension of the rotation matrix, and , It is a k×k dimensional rotation subspace. a and b are positive integers. Represents a function that diagonalizes a vector, rotating the subspace. Represented as: ， Indicates in In a dimensional matrix, by a rotating subspace To fill, ， , , ... These are learnable angle parameters.

2. The parameter fine-tuning method according to claim 1, characterized in that, The parameter fine-tuning method further includes the step of initializing the angle parameter matrix. Wherein, one of the at least one rotation matrix is ​​assigned the value of the initialization matrix. Furthermore, the remaining rotation matrices in the at least one rotation matrix are initialized as identity matrices. Represented as: in, This is the initial weight matrix of the pre-trained model. By gradient The gradient of the left singular vector matrix obtained by performing singular value decomposition. The loss function L represents the loss function against the initial weight matrix. The partial derivative, It is a left singular vector matrix The first r columns, It is the initial weight matrix The first r columns, express The identity matrix, where n is the weight matrix. Dimensions.

3. The parameter fine-tuning method according to claim 1, characterized in that, The step of outputting the trained model to be fine-tuned as the result of parameter fine-tuning includes: The initial weight matrix of the linear layer is multiplied by the angle parameter matrix to obtain the fine-tuned weight matrix of the linear layer, and the initial weight matrix of the linear layer of the pre-trained model is replaced with the fine-tuned weight matrix to obtain the result of the parameter fine-tuning.

4. The parameter fine-tuning method according to claim 1, characterized in that, The pre-trained model includes one of a multilayer perceptron, a convolutional neural network, and a Transformer-based neural network, wherein the linear layer is at least one of a feedforward network, a convolutional layer, and a Transformer attention layer corresponding to the pre-trained model.

5. A parameter fine-tuning device for a pre-trained model, characterized in that, The parameter fine-tuning device fine-tunes the parameters of a pre-trained model for natural language processing based on a text dataset. The pre-trained model includes linear layers with an initial weight matrix. The parameter fine-tuning device includes: An initialization module is configured to apply an orthogonal transformation based on at least one rotation matrix to the linear layer and obtain an angle parameter matrix according to the orthogonal transformation; construct an orthogonal fine-tuning layer based on the initial weight matrix and the angle parameter matrix; and replace the linear layer in the pre-trained model with the orthogonal fine-tuning layer to obtain the model to be fine-tuned. The training module is configured to train the model to be fine-tuned by updating only the angle parameter matrix based on the text dataset using a gradient backpropagation algorithm; and The output module is configured to output the trained model to be fine-tuned as the result of parameter fine-tuning. Wherein, the angle parameter matrix is ​​the product of all rotation matrices in the at least one rotation matrix. Wherein, the orthogonal fine-tuning layer is the initial weight matrix multiplied by the angle parameter matrix, and The orthogonal fine-tuning layer is represented as follows: , in, For the orthogonal fine-tuning layer, The initial weight matrix is... The angle parameter matrix, , For the rotation matrix in the at least one rotation matrix, Each of the at least one rotation matrix Represented as: ， in, It is the dimension of the rotation matrix, and , It is a k×k dimensional rotation subspace. a and b are positive integers. Represents a function that diagonalizes a vector, rotating the subspace. Represented as: ， Indicates in In a dimensional matrix, by a rotating subspace To fill, ， , , ... These are learnable angle parameters.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the parameter fine-tuning method for the pre-trained model according to any one of claims 1 to 4.

7. A computing device, characterized in that, The computing device includes: processor; and A memory storing a computer program that, when executed by a processor, implements a parameter fine-tuning method for a pre-trained model according to any one of claims 1 to 4.

Images