Model fine-tuning training method based on low-rank tensor adaptation, model application method and related device

Abstract

The invention relates to the technical field of computers, in particular to a model fine-tuning training method based on low-rank tensor adaptation, a model application method and a related device. The model fine-tuning training method based on low-rank tensor adaptation in one embodiment comprises the following steps: acquiring a pre-training model; adding fine tuning parameters for model parameters of the pre-training model to obtain a plurality of fine tuning parameter matrixes, combining the plurality of fine tuning parameter matrixes into a fine tuning parameter tensor, and decomposing the fine tuning parameter tensor into a core tensor and two low-rank factor matrixes; freezing the two low-rank factor matrixes and the model parameters to obtain a to-be-trained model; and performing fine tuning training on the to-be-trained model through sample data, and adjusting the core tensor in the training process to obtain a trained target model. According to the embodiment of the invention, the number of parameters needing to be adjusted when model fine adjustment is carried out on the pre-training model can be reduced, and the parameter fine adjustment cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model fine-tuning training method based on low-rank tensor adaptation, a model application method and related devices, wherein the related devices include a model fine-tuning training device based on low-rank tensor adaptation, a model application device, a computer device, a storage medium and a computer program product. Background Art

[0002] With the development of computer technology, artificial intelligence technology related to natural language processing has also developed rapidly. In the field of natural language processing, a mature solution to practical problems is to use LLM (Large Language Model) for fine-tuning to adapt to the data in the target field. However, as the number of LLM parameters grows, traditional fine-tuning methods become more and more expensive in all aspects. In recent years, many technologies have focused on exploring the problem of efficient parameter fine-tuning, that is, minimizing the parameters required for training while keeping the task performance as good as possible. A common idea in these studies is to consider inserting a subset of a new or existing set of parameters to update during the fine-tuning process. However, the current parameter adjustment method still has the problem of too many fine-tuning update parameters and high fine-tuning costs. Summary of the invention

[0003] Based on this, it is necessary to provide a model fine-tuning training method, model application method and related devices based on low-rank tensor adaptation, which can reduce the number of adjustment parameters and reduce the cost of parameter adjustment, in order to address the above-mentioned technical problems. The related devices include a model fine-tuning training device, a model application device, a computer device, a storage medium and a computer program product based on low-rank tensor adaptation.

[0004] In a first aspect, the present application provides a model fine-tuning training method based on low-rank tensor adaptation. The method comprises:

[0005] Get the pre-trained model;

[0006] Add fine-tuning parameters to the model parameters of the pre-trained model to obtain multiple fine-tuning parameter matrices, and merge the multiple fine-tuning parameter matrices into a fine-tuning parameter tensor and decompose them into a core tensor and two low-rank factor matrices;

[0007] Freeze the two low-rank factor matrices and the model parameters to obtain a model to be trained;

[0008] The model to be trained is fine-tuned using sample data, and the core tensor is adjusted during the training process to obtain a trained target model.

[0009] In one embodiment, the pre-trained model includes at least two model layers, and adding fine-tuning parameters to the model parameters of the pre-trained model to obtain a fine-tuning parameter matrix includes:

[0010] Add fine-tuning parameters to the model parameters of each model layer of the pre-trained model to obtain a fine-tuning parameter matrix for each model layer.

[0011] In one embodiment, the pre-trained model includes at least two transformer modules, and adding fine-tuning parameters to the model parameters of the pre-trained model to obtain multiple fine-tuning parameter matrices includes:

[0012] Add fine-tuning parameters to the model parameters of each transformer module of the pre-trained model to obtain fine-tuning parameter matrices of multiple transformer modules.

[0013] In one embodiment, before freezing the two low-rank factor matrices, the method further includes:

[0014] Get tensor rank;

[0015] Initializing the two low-rank factor matrices according to the tensor rank;

[0016] Initialize the core tensor by a zero matrix.

[0017] In one embodiment, initializing the two low-rank factor matrices according to the tensor rank includes:

[0018] According to the tensor rank, the two low-rank factor matrices are initialized by standard normal distribution.

[0019] In one embodiment, fine-tuning the model to be trained using sample data and adjusting the core tensor during the training process includes:

[0020] During the training process, the sample data is processed by the model to be trained to obtain a model processing result;

[0021] Calculate the training loss based on the model processing result and the sample labels of the sample data;

[0022] The core tensor is updated based on the training loss.

[0023] In a second aspect, the present application provides a model application method. The method comprises:

[0024] Get the data to be processed;

[0025] The data to be processed is processed by a target model to obtain a target processing result, and the target model is trained by the method in any of the above embodiments.

[0026] In a third aspect, the present application also provides a model fine-tuning training device based on low-rank tensor adaptation. The device comprises:

[0027] A pre-trained model acquisition module is used to acquire a pre-trained model;

[0028] A parameter processing module, used for adding fine-tuning parameters to the model parameters of the pre-trained model, obtaining a plurality of fine-tuning parameter matrices, and merging the plurality of fine-tuning parameter matrices into a fine-tuning parameter tensor and decomposing the fine-tuning parameter matrices into a core tensor and two low-rank factor matrices;

[0029] A module for acquiring a model to be trained, used for freezing the two low-rank factor matrices and the model parameters to obtain a model to be trained;

[0030] The training module is used to fine-tune the model to be trained through sample data, adjust the core tensor during the training process, and obtain the trained target model.

[0031] In a fourth aspect, the present application also provides a model application device. The device comprises:

[0032] A data acquisition module, used to acquire data to be processed;

[0033] The model processing module is used to process the data to be processed through a target model to obtain a target processing result, and the target model is trained using the method in any of the above embodiments.

[0034] In a fifth aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method in any of the above embodiments are implemented.

[0035] In a sixth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method in any of the above embodiments are implemented.

[0036] In a seventh aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method in any of the above embodiments are implemented.

[0037] The above-mentioned model fine-tuning training method, model application method and related devices based on low-rank tensor adaptation, during the process of model fine-tuning training, after obtaining multiple fine-tuning parameter matrices by adding fine-tuning parameters to the model parameters of the pre-trained model, the multiple fine-tuning parameter matrices are merged into a fine-tuning parameter tensor, and the fine-tuning parameter tensor is further decomposed into a core tensor and two low-rank factor matrices, and the two low-rank factor matrices and the model parameters of the pre-trained model are frozen, so that during the fine-tuning training process, there is no need to adjust the model parameters and the two low-rank factor matrices of the pre-trained model, that is, only the core tensor can be adjusted during the adjustment process, thereby greatly reducing the number of parameters that need to be adjusted when fine-tuning the model for the pre-trained model, and reducing the cost of parameter fine-tuning. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is an application environment diagram of a model fine-tuning training method and a model application method based on low-rank tensor adaptation in one embodiment;

[0039] Figure 2 1 is a flow chart of a model fine-tuning training method based on low-rank tensor adaptation in one embodiment;

[0040] Figure 3 It is a flowchart of a model fine-tuning training method based on low-rank tensor adaptation in another embodiment;

[0041] Figure 4 It is a flowchart of a model fine-tuning training method based on low-rank tensor adaptation in another embodiment;

[0042] Figure 5 is a schematic diagram of a process of adjusting a core tensor in one embodiment;

[0043] Figure 6 A simplified schematic diagram of the principle of model fine-tuning training based on low-rank tensor adaptation in one embodiment;

[0044] Figure 7 A schematic diagram of a flow chart of a model application method in one embodiment;

[0045] Figure 8 is a structural block diagram of a model fine-tuning training device based on low-rank tensor adaptation in one embodiment;

[0046] Fig. 9 is a structural block diagram of a model application device in an embodiment;

[0047] Fig.10 is an internal structure diagram of a computer device in one embodiment;

[0048] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] The following embodiments of the technical solution of the present application are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0053] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0054] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0055] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0056] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0057] It should be noted that the information (including but not limited to pre-trained models, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the acquisition, transmission, storage, use and processing of relevant data are in compliance with the relevant provisions of national laws and regulations. For content pushed to users, users can refuse or conveniently refuse content push, etc. In the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0058] At present, in the field of natural language processing, a mature method to solve practical problems is to use LLM (Large Language Model) for fine-tuning to adapt to the data in the target field. That is, after obtaining the pre-trained large model LLM, the pre-trained large model is further trained with sample data from the target field to fine-tune the model parameters in the pre-trained large model LLM to obtain a model for the target field, so that the obtained model is suitable for natural language processing in the target field.

[0059] However, as the number of LLM parameters grows, traditional fine-tuning methods become increasingly expensive in all aspects. In recent years, many techniques have focused on exploring the problem of efficient parameter fine-tuning, that is, reducing the number of parameters required for training as much as possible while keeping the task performance as good as possible. A common idea in these studies is to consider inserting a new or a subset of existing parameters to be updated during the fine-tuning process. One type of this is Low-Rank Matrix Factorization, which decomposes the model's weight matrix into two smaller matrices and only updates these smaller matrices to adapt to new tasks.

[0060] These processing types are designed to reduce the computational and storage costs during fine-tuning while maintaining or even improving the performance on the target task. However, although these methods have improved parameter efficiency and reduced computational costs to a certain extent, there is still the problem of too many parameters. Taking the low-rank adaptation (LoRA) method based on low-rank matrix decomposition as an example, it performs low-rank decomposition on the weight matrix of a single transformer layer. In practical applications, it is found that training using this method still has the problem of many parameters that need to be adjusted and high fine-tuning costs.

[0061] After research, it was found that although the method based on low-rank matrix decomposition reduces the number of parameters involved in the update through low-rank decomposition, the parameters after low-rank decomposition need to participate in the parameter update process, which makes it difficult to reduce the number of parameters that need to be adjusted by a large amount, which in turn leads to the problem of high fine-tuning costs. Based on this, after research, a method is proposed to further reduce the parameters involved in fine-tuning on the basis of low-rank decomposition, so as to train the model and further reduce the model training cost.

[0062] The model fine-tuning training method based on low-rank tensor adaptation provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 stores a pre-trained model, which can be a pre-trained large model such as LLM. When it is necessary to obtain an application model suitable for a certain target field, after obtaining the pre-trained large model, add fine-tuning parameters to the model parameters of the pre-trained model, obtain multiple fine-tuning parameter matrices, merge the multiple fine-tuning parameter matrices into a fine-tuning parameter tensor, and decompose the fine-tuning parameter tensor into a core tensor and two low-rank factor matrices; freeze the two low-rank factor matrices and the model parameters to obtain the model to be trained; and train the model to be trained through sample data of the target field, adjust the core tensor during the training process, and obtain the trained target model suitable for the target field. Among them, the above process of obtaining the model to be trained and fine-tuning the model to be trained can be completed by the cooperation of the terminal 102, the server 104, or the terminal 102 and the server 104. After the target model is obtained through training, the target model can be deployed to a terminal (which may be the same as or different from the terminal 102 for training) or a server (which may be the same as or different from the server for training) to apply the target model. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0063] In one embodiment, Figure 2 As shown in the figure, a model fine-tuning training method based on low-rank tensor adaptation is provided, which is applied to Figure 1 The terminal 102 or server 104 is used as an example to illustrate, including the following steps.

[0064] Step S201: Obtain a pre-trained model.

[0065] A pre-trained model refers to a model that has been pre-trained. The pre-trained model can be obtained in various possible ways, such as obtaining the required pre-trained model from an open source pre-trained large model.

[0066] The type of pre-trained model is not limited. Taking the natural domain as an example, the pre-trained model may be an LLM (Large Language Model). In the following descriptions of the embodiments, the pre-trained model is taken as an example of an LLM.

[0067] Step S202: adding fine-tuning parameters to the model parameters of the pre-trained model to obtain multiple fine-tuning parameter matrices, and merging the multiple fine-tuning parameter matrices into a fine-tuning parameter tensor and decomposing them into a core tensor and two low-rank factor matrices.

[0068] The pre-trained model is a model that has been trained with a large amount of data in advance. Therefore, the pre-trained model already has pre-trained model parameters. By adding fine-tuning parameters to the model parameters of the pre-trained model, the model parameters include two parts: one part is the model parameters of the obtained pre-trained model itself, and the other part is the added fine-tuning parameters. Since the pre-trained model has many model parameters, a fine-tuning parameter matrix can be obtained based on the added multiple fine-tuning parameters.

[0069] Since a pre-trained model usually has multiple model layers, fine-tuning parameters can be added to each model layer separately, so as to obtain multiple fine-tuning parameter matrices of multiple model layers. The fine-tuning parameters in multiple fine-tuning parameter matrices can be combined into a fine-tuning parameter tensor.

[0070] There is no limit to the way to decompose the fine-tuning parameter tensor, as long as after decomposition, the fine-tuning parameter tensor can be represented by a core tensor and two low-rank factor matrices. The fine-tuning parameter matrix is ​​denoted as , then ,in, The core tensor The s-th lateral slice matrix of , A and B are both low-rank factor matrices.

[0071] Step S203: Freeze the two low-rank factor matrices and the model parameters to obtain the model to be trained.

[0072] Freezing the two low-rank factor matrices means that during the training process of the obtained model to be trained, the two low-rank factor matrices are frozen and will not be adjusted.

[0073] Similarly, freezing the model parameters means that the model parameters of the pre-trained model are frozen, and the model parameters will not be adjusted during the training process of the obtained model to be trained.

[0074] Step S204: fine-tune the model to be trained using sample data, adjust the core tensor during the training process, and obtain a trained target model.

[0075] In the process of fine-tuning the model to be trained, since the model parameters obtained based on the pre-trained model are frozen and the two low-rank factor matrices are frozen, only the core tensor needs to be adjusted during the training process, so as to perform fine-tuning training and obtain the target model after fine-tuning training.

[0076] The above-mentioned model fine-tuning training method based on low-rank tensor adaptation, during the process of model fine-tuning training, after obtaining multiple fine-tuning parameter matrices by adding fine-tuning parameters to the model parameters of the pre-trained model, merges the multiple fine-tuning parameter matrices into a fine-tuning parameter tensor, and further decomposes the fine-tuning parameter tensor into a core tensor and two low-rank factor matrices, and freezes the two low-rank factor matrices and the model parameters of the pre-trained model, so that during the fine-tuning training process, there is no need to adjust the model parameters and the two low-rank factor matrices of the pre-trained model, that is, only the core tensor can be adjusted during the adjustment process, thereby greatly reducing the number of parameters that need to be adjusted when fine-tuning the model for the pre-trained model, and reducing the cost of parameter fine-tuning.

[0077] In some embodiments, the pre-trained model includes at least two model layers. Figure 3 As shown, the above step S202 of adding fine-tuning parameters to the model parameters of the pre-trained model to obtain a fine-tuning parameter matrix may include:

[0078] Step S2020: Add fine-tuning parameters to the model parameters of each model layer of the pre-trained model to obtain a fine-tuning parameter matrix for each model layer, thereby obtaining multiple fine-tuning parameter matrices.

[0079] Based on this embodiment, for the obtained pre-trained model, fine-tuning parameters can be added to the model parameters of each model layer of the pre-trained model, so that the model parameter matrix of each model layer can be obtained, and multiple fine-tuning parameter matrices can be obtained.

[0080] In some embodiments, the model layer of the pre-trained model may be a transformer module, that is, the pre-trained model includes at least two transformer modules. In this case, fine-tuning parameters are added to the model parameters of the pre-trained model to obtain a fine-tuning parameter matrix, including:

[0081] Add fine-tuning parameters to the model parameters of each transformer module of the pre-trained model to obtain a fine-tuning parameter matrix of each transformer module, thereby obtaining multiple fine-tuning parameter matrices.

[0082] In some embodiments, reference Figure 4 As shown, before freezing the two low-rank factor matrices, the method further includes:

[0083] Step S401: Get the tensor rank.

[0084] Tensor rank refers to the rank of the core tensor, which can determine the desired rank of the core tensor. There is no limit to the way to obtain the tensor rank. In some embodiments, the value input by the user can be obtained as the tensor rank, or one of the preset values ​​can be selected as the tensor rank, which can be determined based on the demand for low rank.

[0085] Step S402: Initialize the two low-rank factor matrices according to the tensor rank.

[0086] There is no limitation on the method of initializing the two low-rank factor matrices according to the tensor rank. In some embodiments, the two low-rank factor matrices can be initialized based on the tensor rank through a standard normal distribution, such as through a standard normal distribution. Initialize two low-rank factor matrices.

[0087] Step S403: Initialize the core tensor through a zero matrix.

[0088] The 0 matrix is ​​a matrix of all 0s. Initializing the core tensor with the 0 matrix can enable the obtained model to be trained to learn the model performance of the pre-trained model well, thereby improving the accuracy of the obtained model to be trained.

[0089] In some embodiments, reference Figure 5 As shown, the above step S204 of fine-tuning the model to be trained by using sample data and adjusting the core tensor during the training process may include:

[0090] Step S2041: During the training process, the sample data is processed by the model to be trained to obtain a model processing result.

[0091] The way in which the model to be trained processes the sample data is not limited and can be determined based on the target domain and target scenario. For example, in some embodiments, taking text classification in the natural language field as an example, the processing of the sample data by the model to be trained can be the classification result of text classification of the sample data. Taking speech prediction in the natural language field as an example, the processing of the sample data by the model to be trained can be the predicted text obtained by speech prediction of the sample data. It should be understood that in other technical fields, the processing of the sample data by the model to be trained can also be other types of processing.

[0092] Step S2042: Calculate the training loss based on the model processing result and the sample labels of the sample data.

[0093] The sample label of sample data refers to the actual processing result of the sample data. Taking text classification in the natural language field as an example, the sample label of sample data refers to the actual classification of the sample data. Through the model processing results and the sample labels of the sample data, the difference between the model processing results and the sample data of the sample data can be determined, thereby calculating the training loss of the model to be trained.

[0094] Among them, the method of calculating the training loss is not limited, and the loss function for calculating the training loss can be selected according to actual needs. The embodiment of the present application does not limit the specific type of the loss function.

[0095] Step S2043: Update the core tensor based on the training loss.

[0096] After the training loss is calculated, the model parameters of the model to be trained can be updated based on the training loss. In an embodiment of the present application, after the training loss is calculated, the core tensor can be updated based on the training loss.

[0097] Based on the embodiments described above, detailed description is given below in combination with specific examples.

[0098] The model fine-tuning training method based on low-rank tensor adaptation provided in the embodiment of the present application is to further reduce the required fine-tuning parameters and reduce the fine-tuning cost by freezing two factor matrices initialized by normal distribution while modeling the low rank between multiple weight matrices, so as to further reduce the required fine-tuning parameters on the basis of similar performance, and the fine-tuning cost is lower. Specifically, after the original parameters are characterized by tensor decomposition, the two full-rank factor matrices initialized by normal distribution are regarded as random projection matrices for freezing, and then only the core tensor of the tensor decomposition is updated in the fine-tuning stage, and finally the two factor matrices and the core tensor are reconstructed into corresponding tensors in the parameter update stage and added to the original parameters, thereby completing the fine-tuning of the model parameters.

[0099] The principle of model fine-tuning training based on low-rank tensor adaptation in the embodiment of the present application can be as follows: Figure 6 The method of the embodiment of the present application uses a random projection matrix and a low-rank decomposition (such as a low-rank Tucker2 decomposition) to approximate the weights in the pre-trained large language model to significantly reduce the cost of fine-tuning the large model. Specifically, the approximate weights can be the weight matrix set of all query Q vectors and value V vectors of the transformer module in the large model. and .by For example, for the weight matrix set in the pre-trained large model , the solution of the embodiment of the present application decomposes it into a core tensor and two low-rank factor matrices A and B such that The sth weight matrix in , where A and B are both low-rank factor matrices, for The s-th lateral slice matrix of . For the matrix set In terms of, its decomposition shares the low-rank factor matrices A and B, but for each matrix Each has a unique lateral slice matrix Therefore, considering freezing Based on this, we further freeze A and B initialized by normal distribution (considered as random projection matrices) and only update the matrix during fine-tuning , and finally after the back propagation update is completed, the updated parameters will be added to the original parameters Complete fine-tuning parameter update, Figure 7 In the figure, the gray background represents freezing, that is, the low-rank factor matrices A and B, and the weight matrix based on the pre-trained model Frozen and not updated during parameter adjustment.

[0100] Accordingly, the model training process based on low-rank tensor adaptation in an embodiment of the present application can be described as follows.

[0101] First, get the tensor rank r, the pre-trained model, the number of transformer modules L in the pre-trained model, and the input size of the transformer module and the output size of the transformer module .

[0102] Based on the number of transformer modules L, get the specific vector weight matrix of L transformer modules from the pre-trained model , add fine-tuning parameters to the model parameters of each transformer module of the pre-trained model, obtain the fine-tuning parameter matrix, and decompose the fine-tuning parameter matrix into a core tensor and two low-rank factor matrices.

[0103] The fine-tuning parameter matrix of the s-th transformer module is expressed as , then the current parameter weight of the s-th transformer module is .in, is the weight matrix of the model parameters of the sth transformer module obtained from the pre-trained model. , A and B are both low-rank factor matrices, The core tensor The s-th lateral slice matrix of .

[0104] Through the standard normal distribution Initialize the factor matrix and .

[0105] Initialize the core tensor by a zero matrix .

[0106] The model to be trained is then trained based on the sample data, and the core tensor is adjusted during the training process.

[0107] Assume that the training model is trained for E iterations, then for each iteration e (e=1,2,…,E), for the sample data in each training batch, the following processing can be performed:

[0108] Based on current weight Perform forward propagation of the entire network, where the low-rank correction term ;

[0109] The training loss L of the model is calculated based on the model processing result (i.e., the output of the model) of the sample data processed by the model to be trained and the target label of the sample data.

[0110] For each transformer linear layer s (s=1,2,…,L), the training loss function L is calculated by the back-propagation algorithm relative to the low-rank correction term Gradient , and use the least squares method to obtain Solution , that is, to solve for the optimal Among them, the optimal one is calculated based on the training loss The method is not limited, and any method based on training loss to update model parameters can be used.

[0111] Based on the solution , which can be used to correct the low-rank term For example, if the AdamW optimizer is used for updating, the updated low-rank correction term is .

[0112] End the current round of fine-tuning process and enter the next round of iteration process.

[0113] When the E-th round of iterations is completed, the final low-rank correction term after the E-th iteration is obtained: , so as to obtain the final weight matrix of the model parameters of the model to be trained: , thus obtaining the transformer linear layer parameters after fine-tuning (i.e. after training). That is, the fine-tuned transformer linear layer parameters are ,in . Thus, the final training objective function can be obtained.

[0114] Based on the model fine-tuning training method based on low-rank tensor adaptation as described above, the embodiment of the present application also provides a model application method, such as Figure 7 As shown, the model application method in some embodiments includes:

[0115] Step S701: Obtain data to be processed;

[0116] Step S702: Process the data to be processed by a target model to obtain a target processing result, wherein the target model is trained by the method in any of the above embodiments.

[0117] Among them, it can be understood that based on the different technical fields and model processing targets, the target models finally obtained may also be different, and the processing based on the target model may also be different. Taking the text classification in the natural language field as described above as an example, the target model obtained may be a text classification model, the data to be processed may be text data to be classified, the target model processes the data to be processed, and the target processing result obtained may be the text category obtained by the target model classifying the text data. Taking the speech prediction in the natural language field as an example, the target model obtained may be a speech prediction model, the data to be processed may be text data to be predicted, the target model processes the data to be processed, and the target processing result obtained may be the speech text to be fed back obtained by the target model performing speech prediction on the text data. It can be understood that in other embodiments, the target model may also be other models, the data to be processed may also be other data, and the corresponding target processing results may also be different, and the embodiments of the present application do not specifically limit this.

[0118] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0119] Based on the same inventive concept, the embodiment of the present application also provides a low-rank tensor adaptation-based model fine-tuning training device for implementing the above-mentioned low-rank tensor adaptation-based model fine-tuning training method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more embodiments of the low-rank tensor adaptation-based model fine-tuning training device provided below can refer to the above limitations on the low-rank tensor adaptation-based model fine-tuning training method, which will not be repeated here.

[0120] In one embodiment, Figure 8 As shown, a model fine-tuning training device based on low-rank tensor adaptation is provided, including: a pre-trained model acquisition module 901, a parameter processing module 902, a to-be-trained model acquisition module 903 and a training module 904, wherein:

[0121] A pre-training model acquisition module 901 is used to acquire a pre-training model;

[0122] A parameter processing module 902 is used to add fine-tuning parameters to the model parameters of the pre-trained model, obtain multiple fine-tuning parameter matrices, and merge the multiple fine-tuning parameter matrices into a fine-tuning parameter tensor and decompose them into a core tensor and two low-rank factor matrices;

[0123] A model acquisition module 903 to be trained is used to freeze the two low-rank factor matrices and the model parameters to obtain a model to be trained;

[0124] The training module 904 is used to fine-tune the model to be trained through sample data, adjust the core tensor during the training process, and obtain a trained target model.

[0125] In some embodiments, the pre-trained model includes at least one transformer module, and the parameter processing module 902 is used to add fine-tuning parameters to the model parameters of each transformer module of the pre-trained model to obtain multiple fine-tuning parameter matrices.

[0126] In some embodiments, the parameter processing module 902 is further used to obtain the tensor rank; initialize the two low-rank factor matrices according to the tensor rank; and initialize the core tensor through a 0 matrix.

[0127] In some embodiments, the parameter processing module 902 is further used to initialize the two low-rank factor matrices through standard normal distribution according to the tensor rank.

[0128] In some embodiments, the training module 904 is used to process the sample data through the model to be trained during the training process to obtain a model processing result; calculate the training loss based on the model processing result and the sample label of the sample data; and update the core tensor based on the training loss.

[0129] Each module in the above-mentioned model fine-tuning training device based on low-rank tensor adaptation can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0130] Based on the same inventive concept, the embodiment of the present application also provides a model application device for implementing the model application method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more model application device embodiments provided below can refer to the limitations on the model application method above, and will not be repeated here.

[0131] In one embodiment, Fig. 9 As shown, a model application device is provided, including: a data acquisition module 901 and a model processing module 902, wherein:

[0132] Data acquisition module 901, used to acquire data to be processed;

[0133] The model processing module 902 is used to process the data to be processed through a target model to obtain a target processing result. The target model is trained using the model training method based on low-rank tensor adaptation in any of the embodiments described above.

[0134] Each module in the above model application device can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0135] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store pre-trained models, sample data, target models, etc. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a model fine-tuning training method based on low-rank tensor adaptation and / or a model application method is implemented.

[0136] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a model fine-tuning training method based on low-rank tensor adaptation and / or a model application method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse.

[0137] Those skilled in the art will understand that Fig.10 , 11The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0138] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the model fine-tuning training method or model application method based on low-rank tensor adaptation in any of the embodiments described above are implemented.

[0139] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the model fine-tuning training method or model application method based on low-rank tensor adaptation in any of the embodiments described above are implemented.

[0140] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the model fine-tuning training method or model application method based on low-rank tensor adaptation in any of the embodiments described above.

[0141] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0142] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0143] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A model fine-tuning training method based on low-rank tensor adaptation, wherein: The method comprises: Get the pre-trained model; Add fine-tuning parameters to the model parameters of the pre-trained model to obtain multiple fine-tuning parameter matrices, and merge the multiple fine-tuning parameter matrices into a fine-tuning parameter tensor and decompose them into a core tensor and two low-rank factor matrices; Freeze the two low-rank factor matrices and the model parameters to obtain a model to be trained; The model to be trained is fine-tuned using sample data, and the core tensor is adjusted during the training process to obtain a trained target model.

2. The method according to claim 1, characterized in that The pre-trained model includes at least two transformer modules, and the adding of fine-tuning parameters to the model parameters of the pre-trained model to obtain a plurality of fine-tuning parameter matrices includes: Add fine-tuning parameters to the model parameters of each transformer module of the pre-trained model to obtain multiple fine-tuning parameter matrices.

3. The method according to claim 1 or 2, characterized in that: Before freezing the two low-rank factor matrices, the method further includes: Get tensor rank; Initializing the two low-rank factor matrices according to the tensor rank; Initialize the core tensor by a zero matrix.

4. The method according to claim 3, characterized in that Initializing the two low-rank factor matrices according to the tensor rank includes: According to the tensor rank, the two low-rank factor matrices are initialized by standard normal distribution.

5. The method according to any one of claims 1 to 4, characterized in that: The fine-tuning training of the model to be trained by using sample data and adjusting the core tensor during the training process include: During the training process, the sample data is processed by the model to be trained to obtain a model processing result; Calculate the training loss based on the model processing result and the sample labels of the sample data; The core tensor is updated based on the training loss.

6. A model application method, characterized in that: The method comprises: Get the data to be processed; The data to be processed is processed by a target model to obtain a target processing result, wherein the target model is trained by the method according to any one of claims 1 to 5.

7. A model fine-tuning training device based on low-rank tensor adaptation, characterized in that: The device comprises: A pre-trained model acquisition module is used to acquire a pre-trained model; A parameter processing module, used for adding fine-tuning parameters to the model parameters of the pre-trained model, obtaining a plurality of fine-tuning parameter matrices, and merging the plurality of fine-tuning parameter matrices into a fine-tuning parameter tensor and decomposing the fine-tuning parameter matrices into a core tensor and two low-rank factor matrices; A module for acquiring a model to be trained, used for freezing the two low-rank factor matrices and the model parameters to obtain a model to be trained; The training module is used to fine-tune the model to be trained through sample data, adjust the core tensor during the training process, and obtain the trained target model.

8. A model application device, characterized in that: The device comprises: A data acquisition module, used to acquire data to be processed; A model processing module is used to process the data to be processed through a target model to obtain a target processing result, wherein the target model is trained using the method described in any one of claims 1 to 5.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images