Model fine-tuning training method based on dynamic low-rank tensor adaptation, model application method and related device

Abstract

The invention relates to the technical field of computers, in particular to a model fine-tuning training method based on dynamic low-rank tensor adaptation, a model application method and a related device. The model fine-tuning training method based on dynamic low-rank tensor adaptation in one embodiment comprises the following steps: acquiring a pre-training model; adding fine tuning parameters to model parameters of the pre-training model to obtain fine tuning parameter matrixes of different layers, combining the fine tuning parameter matrixes of multiple layers into a fine tuning parameter tensor and decomposing the fine tuning parameter tensor into a core tensor and two low-rank factor matrixes to obtain to-be-fine-tuned parameters; and performing fine tuning on the pre-training model through the sample data, freezing original parameters of the pre-training model in the fine tuning process, updating the core tensor and the two low-rank factor matrixes in sequence, and performing low-rank adjustment on the slice matrix of the updated core tensor to obtain updated fine tuning parameters. By adopting the scheme of the embodiment, the performance of the obtained model can be further improved while the parameter fine tuning cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model fine-tuning training method based on dynamic low-rank tensor adaptation, a model application method and related devices, wherein the related devices include a model fine-tuning training device based on dynamic low-rank tensor adaptation, a model application device, a computer device, a storage medium and a computer program product. Background Art

[0002] With the development of computer technology, artificial intelligence technology related to natural language processing has also developed rapidly. In the field of natural language processing, a mature solution to practical problems is to use LLM (Large Language Model) for fine-tuning to adapt to the data in the target field. However, as the number of LLM parameters grows, traditional fine-tuning methods become more and more expensive in all aspects. In recent years, many technologies have focused on exploring the problem of efficient parameter fine-tuning, that is, minimizing the parameters required for training while keeping the task performance as good as possible. A common idea in these studies is to consider inserting a subset of a new or existing set of parameters to update during the fine-tuning process. However, the current parameter adjustment method still has the problem of too many fine-tuning update parameters and the performance of the model is difficult to reach the optimal state. Summary of the invention

[0003] Based on this, it is necessary to provide a model fine-tuning training method, model application method and related devices based on dynamic low-rank tensor adaptation, which can reduce the number of adjustment parameters and improve model performance, in response to the above-mentioned technical problems. The related devices include a model fine-tuning training device, a model application device, a computer device, a storage medium and a computer program product based on dynamic low-rank tensor adaptation.

[0004] In a first aspect, the present application provides a model fine-tuning training method based on dynamic low-rank tensor adaptation. The method comprises:

[0005] Get the pre-trained model;

[0006] Add fine-tuning parameters to the model parameters of the pre-trained model to obtain fine-tuning parameter matrices of different layers, and merge the fine-tuning parameter matrices of multiple layers into a fine-tuning parameter tensor and decompose them into a core tensor and two low-rank factor matrices to obtain parameters to be fine-tuned;

[0007] The pre-trained model is fine-tuned using sample data, the original parameters of the pre-trained model are frozen during the fine-tuning process, the core tensor and the two low-rank factor matrices are updated in sequence, and the slice matrix of the updated core tensor is low-rank adjusted to obtain updated fine-tuning parameters.

[0008] In some embodiments, updating the core tensor and the two low-rank factor matrices in sequence, and performing low-rank adjustment on the slice matrix of the updated core tensor to obtain updated fine-tuning parameters includes:

[0009] Calculating a training loss during training, and updating the two low-rank factor matrices and the core tensor according to the training loss;

[0010] Among the diagonal elements of the slice matrix of the updated core tensor, the diagonal elements that are less than a preset threshold are set to 0 to obtain updated fine-tuning parameters.

[0011] In some embodiments, the pre-trained model includes a transformer module, and the fine-tuning parameter matrix includes a set of weight matrices of all query vectors and value vectors of the transformer module.

[0012] In some embodiments, updating the two low-rank factor matrices and the core tensor according to the training loss includes:

[0013] Based on the training loss and the orthogonal penalty of the two low-rank factor matrices, the two low-rank factor matrices and the core tensor are updated.

[0014] In some embodiments, after decomposing the fine-tuning parameter matrix into a core tensor and two low-rank factor matrices, the method further includes:

[0015] Get the initial tensor rank;

[0016] Initializing the two low-rank factor matrices according to the initial tensor rank;

[0017] Initialize the core tensor by a diagonal matrix of zeros.

[0018] In some embodiments, the initializing the two low-rank factor matrices according to the tensor rank includes:

[0019] According to the initial tensor rank, the two low-rank factor matrices are initialized by standard normal distribution.

[0020] In a second aspect, the present application also provides a model application method, the method comprising:

[0021] Get the data to be processed;

[0022] The data to be processed is processed by a target model to obtain a target processing result, and the target model is trained by the model fine-tuning training method based on dynamic low-rank tensor adaptation of any of the embodiments described above.

[0023] In a third aspect, the present application also provides a model fine-tuning training device based on dynamic low-rank tensor adaptation, the device comprising:

[0024] A pre-trained model acquisition module is used to obtain a pre-trained model;

[0025] A module for acquiring a model to be trained, used to add fine-tuning parameters to the model parameters of the pre-trained model, obtain fine-tuning parameter matrices of different layers, and merge the fine-tuning parameter matrices of multiple layers into a fine-tuning parameter tensor and decompose them into a core tensor and two low-rank factor matrices to obtain parameters to be fine-tuned;

[0026] A training module is used to fine-tune the pre-trained model through sample data, freeze the original parameters of the pre-trained model during the fine-tuning process, update the core tensor and the two low-rank factor matrices in sequence, and perform low-rank adjustment on the slice matrix of the updated core tensor to obtain updated fine-tuning parameters.

[0027] In a fourth aspect, the present application also provides a model application device. The device comprises:

[0028] A data acquisition module, used to acquire data to be processed;

[0029] A model processing module is used to process the data to be processed through a target model to obtain a target processing result. The target model is trained using the model training method based on dynamic low-rank tensor adaptation described in any of the above embodiments.

[0030] In a fifth aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method in any of the above embodiments are implemented.

[0031] In a sixth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method in any of the above embodiments are implemented.

[0032] In a seventh aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method in any of the above embodiments are implemented.

[0033] The above-mentioned model fine-tuning training method, model application method and related devices based on dynamic rank tensor adaptation, during the process of model training, after obtaining the fine-tuning parameter matrices of different layers by adding fine-tuning parameters to the model parameters of the pre-trained model, the fine-tuning parameter matrices of multiple layers are merged into a fine-tuning parameter tensor, and the fine-tuning parameter tensor is further decomposed into a core tensor and two low-rank factor matrices. During the training process, the core tensor and the two low-rank factor matrices are updated, and the slice matrix of the updated core tensor is also low-rank adjusted, so that the rank of the tensor can be dynamically adjusted during the model training process, thereby reducing the number of parameters that need to be adjusted when fine-tuning the model for the pre-trained model, reducing the cost of parameter fine-tuning, and dynamically adjusting the rank of the core tensor, so that there is no need to traverse the optimal value multiple times. During the training process, the rank of the core tensor can be adaptively found for training and learning, which can further reduce the cost of parameter fine-tuning and further improve the performance of the obtained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is an application environment diagram of a model fine-tuning training method and a model application method based on dynamic low-rank tensor adaptation in one embodiment;

[0035] FIG2 is a schematic diagram of a flow chart of a model fine-tuning training method based on dynamic low-rank tensor adaptation in one embodiment;

[0036] Figure 3 A schematic diagram of a process of updating a low-rank factor matrix and adjusting a low-rank factor matrix in one embodiment;

[0037] Figure 4 It is a schematic diagram of the process of low-rank factor matrix update and low-rank adjustment in another embodiment;

[0038] Figure 5 A schematic diagram of a flow chart of a model fine-tuning training method based on dynamic low-rank tensor adaptation in another embodiment;

[0039] Figure 6 A simplified schematic diagram of the principle of model fine-tuning training based on dynamic low-rank tensor adaptation in one embodiment

[0040] Figure 7 A schematic diagram of a flow chart of a model application method in one embodiment;

[0041] Figure 8 is a structural block diagram of a model fine-tuning training device based on low-rank tensor adaptation in one embodiment;

[0042] FIG9 is a block diagram of a model application device in one embodiment;

[0043] Fig.10is an internal structure diagram of a computer device in one embodiment;

[0044] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] The following embodiments of the technical solution of the present application are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0049] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0050] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0051] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0052] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0053] It should be noted that the information (including but not limited to pre-trained models, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the acquisition, transmission, storage, use and processing of relevant data are in compliance with the relevant provisions of national laws and regulations. For content pushed to users, users can refuse or conveniently refuse content push, etc. In the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0054] At present, in the field of natural language processing, a mature method to solve practical problems is to use LLM (Large Language Model) for fine-tuning to adapt to the data in the target field. That is, after obtaining the pre-trained large model LLM, the pre-trained large model is further trained with sample data from the target field to fine-tune the model parameters in the pre-trained large model LLM to obtain a model for the target field, so that the obtained model is suitable for natural language processing in the target field.

[0055] However, as the number of LLM parameters grows, traditional fine-tuning methods become increasingly expensive in all aspects. In recent years, many techniques have focused on exploring the problem of efficient parameter fine-tuning, that is, reducing the number of parameters required for training as much as possible while keeping the task performance as good as possible. A common idea in these studies is to consider inserting a new or a subset of existing parameters to be updated during the fine-tuning process. One type of this is Low-Rank Matrix Factorization, which decomposes the model's weight matrix into two smaller matrices and only updates these smaller matrices to adapt to new tasks.

[0056] These processing types are designed to reduce the computational and storage costs during fine-tuning while maintaining or even improving the performance on the target task. However, although these methods have improved parameter efficiency and reduced computational costs to a certain extent, there is still the problem of too many parameters. Taking the low-rank adaptation (LoRA) method based on low-rank matrix decomposition as an example, it performs low-rank decomposition on the weight matrix of a single transformer layer. In practical applications, it is found that training using this method still has the problem of many parameters that need to be adjusted and high fine-tuning costs.

[0057] After research, it was found that although the method based on low-rank matrix decomposition reduces the number of parameters involved in the update through low-rank decomposition, it is necessary to manually determine the rank of the low-rank matrix decomposition in advance. However, the rank confirmed in advance is not necessarily the optimal rank, which will affect the performance of the model obtained by the final training, resulting in poor performance of the final model. In order to improve the performance of the model, it takes a lot of time to test and traverse to find the optimal rank, and then combine the optimal rank to carry out the model training process, which leads to a long training time and easily leads to poor model performance. Therefore, the model performance has the possibility of further improvement. Based on this, after research, a model fine-tuning training method based on dynamic low-rank tensor adaptation is proposed, which can adaptively and dynamically adjust the rank of the core tensor. On the basis of low-rank decomposition, the method of further reducing the parameters involved in fine-tuning is used to train the model, realizing dynamic low-rank decomposition, so as to further reduce the cost of model training and improve the performance of the obtained model.

[0058] The model fine-tuning training method based on dynamic low-rank tensor adaptation provided in the embodiment of the present application can be applied in the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 stores a pre-trained model, which can be a pre-trained large model such as LLM. When it is necessary to obtain an application model suitable for a certain target field, after obtaining the pre-trained large model, fine-tuning parameters are added to the model parameters of the pre-trained model to obtain fine-tuning parameter matrices of different layers, and the fine-tuning parameter matrices of multiple layers are merged into a fine-tuning parameter tensor, and the fine-tuning parameter tensor is decomposed into a core tensor and two low-rank factor matrices; and the pre-trained model is fine-tuned by sample data of the target field, the original parameters of the pre-trained model are frozen during the fine-tuning process, the core tensor and the two low-rank factor matrices are updated in sequence, and the slice matrix of the updated core tensor is low-rank adjusted to obtain the updated fine-tuning parameters, so as to obtain the trained target model suitable for the target field. Among them, the above process of obtaining the model to be trained and fine-tuning the model to be trained can be completed by the cooperation of the terminal 102, the server 104, or the terminal 102 and the server 104. After the target model is obtained by training, the target model can be deployed to a terminal (which can be the same or different from the terminal 102 for training) or a server (which can be the same or different from the server for training) to apply the target model. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0059] In one embodiment, as shown in FIG2 , a model fine-tuning training method based on dynamic low-rank tensor adaptation is provided, which is described by taking the method applied to the terminal 102 or the server 104 in FIG1 as an example, and includes the following steps.

[0060] Step 201: Obtain a pre-trained model.

[0061] A pre-trained model refers to a model that has been pre-trained. The pre-trained model can be obtained in various possible ways, such as obtaining the required pre-trained model from an open source pre-trained large model.

[0062] The type of pre-trained model is not limited. Taking the natural domain as an example, the pre-trained model may be an LLM (Large Language Model). In the following descriptions of the embodiments, the pre-trained model is taken as an example of an LLM.

[0063] Step 202: Add fine-tuning parameters to the model parameters of the pre-trained model to obtain fine-tuning parameter matrices of different layers, and merge the fine-tuning parameter matrices of multiple layers into a fine-tuning parameter tensor and decompose them into a core tensor and two low-rank factor matrices to obtain the parameters to be fine-tuned.

[0064] The pre-trained model is a model that has been trained with a large amount of data in advance. Therefore, the pre-trained model already has pre-trained model parameters. By adding fine-tuning parameters to the model parameters of the pre-trained model, the model parameters include two parts: one part is the model parameters of the obtained pre-trained model itself, and the other part is the added fine-tuning parameters. Since the pre-trained model has many model parameters, a fine-tuning parameter matrix can be obtained based on the added multiple fine-tuning parameters.

[0065] Since the pre-trained model usually has multiple layers, fine-tuning parameters can be added to each layer separately, and each layer can obtain a fine-tuning parameter matrix, so that the fine-tuning parameter matrices of different layers can be obtained. The fine-tuning parameter matrices of multiple layers can be merged to obtain a fine-tuning parameter tensor. The fine-tuning parameter tensor can be decomposed and graded into a core tensor and two low-rank factor matrices to obtain the parameters to be fine-tuned.

[0066] There is no limit to the way to decompose the fine-tuning parameter tensor, as long as after decomposition, the fine-tuning parameter tensor can be represented by a core tensor and two low-rank factor matrices. The fine-tuning parameter matrix is ​​denoted as , then ,in, The core tensor The s-th lateral slice matrix of , A and B are both low-rank factor matrices.

[0067] Step 203: fine-tune the pre-trained model using sample data, freeze the original parameters of the pre-trained model during the fine-tuning process, update the core tensor and the two low-rank factor matrices in sequence, and perform low-rank adjustment on the slice matrix of the updated core tensor to obtain updated fine-tuning parameters.

[0068] In the process of fine-tuning the model to be trained through sample data, the original model parameters based on the pre-trained model can be frozen, and only the fine-tuning parameters, namely the core tensor and two low-rank factor matrices, are updated. After the core tensor is fine-tuned, the slices of the updated core tensor are further low-rank adjusted, so as to perform fine-tuning accordingly and obtain the updated fine-tuning parameters. After multiple rounds of fine-tuning, the trained target model can be obtained.

[0069] Among them, low-rank adjustment of the slice matrix of the updated core tensor refers to adjusting the rank of the core tensor. In an embodiment of the present application, it may mean lowering the rank of the core tensor while satisfying the requirement of reducing the rank of the core tensor, so as to further reduce the number of parameters adjusted during the training process, and the rank of the core tensor can be adaptively adjusted.

[0070] The above-mentioned model fine-tuning training method based on dynamic rank tensor adaptation, during the process of model training, after obtaining the fine-tuning parameter matrices of different layers by adding fine-tuning parameters to the model parameters of the pre-trained model, the fine-tuning parameter matrices of multiple layers are merged into a fine-tuning parameter tensor, and the fine-tuning parameter tensor is further decomposed into a core tensor and two low-rank factor matrices. During the training process, the core tensor and the two low-rank factor matrices are updated, and the slice matrix of the updated core tensor is also low-rank adjusted, so that the rank of the tensor can be dynamically adjusted during the model training process, thereby reducing the number of parameters that need to be adjusted when fine-tuning the model for the pre-trained model, reducing the cost of parameter fine-tuning, and by dynamically adjusting the rank of the core tensor, there is no need to traverse the optimal value multiple times. During the training process, the rank of the core tensor can be adaptively found for training and learning, which can further reduce the cost of parameter fine-tuning and further improve the performance of the obtained model.

[0071] In some embodiments, reference Figure 3 As shown, in the above step S203, the core tensor and the two low-rank factor matrices are updated in sequence, and the slice matrix of the updated core tensor is low-rank adjusted to obtain the updated fine-tuning parameters, which may include:

[0072] Step S2031: Calculate the training loss during the training process, and update the two low-rank factor matrices and the core tensor according to the training loss.

[0073] The method of calculating the training loss during the training process is not limited. For example, during the training process, the sample data is processed by a pre-trained model with the parameters to be fine-tuned added to obtain a model processing result. The training loss is calculated based on the model processing result and the sample label of the sample data.

[0074] There is no limit to the way in which the pre-trained model with the parameters to be fine-tuned is used to process sample data, and it can be determined based on the target field and the target scenario. For example, in some embodiments, taking text classification in the natural language field as an example, the processing of sample data by the pre-trained model with the parameters to be fine-tuned can be the classification result of text classification of the sample data. Taking speech prediction in the natural language field as an example, the processing of sample data by the pre-trained model with the parameters to be fine-tuned can be the predicted text obtained by speech prediction of the sample data. It should be understood that in other technical fields, the processing of sample data by the pre-trained model with the parameters to be fine-tuned can also be other types of processing.

[0075] The sample label of sample data refers to the actual processing result of sample data. Taking text classification in the natural language field as an example, the sample label of sample data refers to the actual classification of sample data. Through the model processing results and the sample labels of sample data, the difference between the model processing results and the sample data of sample data can be determined, thereby calculating the training loss.

[0076] Among them, the method of calculating the training loss is not limited, and the loss function for calculating the training loss can be selected according to actual needs. The embodiment of the present application does not limit the specific type of the loss function.

[0077] After obtaining the training loss, the two low-rank factor matrices and the core tensor can be updated according to the training loss. After the core tensor is updated, the updated core tensor may be the updated core tensor of this iteration, or further updates may be required, such as low-rank adjustment of the slice matrix, which can be determined based on the diagonal elements of each slice matrix of the updated core tensor.

[0078] Step S2032: among the diagonal elements of the slice matrix of the updated core tensor, those that are smaller than a preset threshold are set to 0 to obtain updated fine-tuning parameters.

[0079] The preset threshold can be set based on technical needs and can be determined based on the minimum requirement or target requirement for the rank of the core tensor. Since the slice matrix of the core tensor is a diagonal matrix, it can be regarded as a singular value matrix. Therefore, when the diagonal elements of the slice matrix of the core tensor are greater than the preset threshold, the diagonal elements less than the preset threshold are set to 0, thereby achieving low rank adjustment and achieving adaptive adjustment of the rank.

[0080] In some embodiments, the pre-trained model includes at least one model layer. In this case, adding fine-tuning parameters to the model parameters of the pre-trained model in step S202 to obtain fine-tuning parameter matrices of different layers may include:

[0081] Add fine-tuning parameters to the model parameters of each layer of the pre-trained model to obtain a fine-tuning parameter matrix for each layer.

[0082] Based on this embodiment, for the obtained pre-trained model, fine-tuning parameters may be added to the model parameters of each model layer of the pre-trained model, so that the model parameter matrix of each model layer may be obtained accordingly.

[0083] In some embodiments, the model layer of the pre-trained model may include a transformer module, that is, the pre-trained model includes at least one transformer module. In this case, fine-tuning parameters are added to the model parameters of the pre-trained model to obtain fine-tuning parameter matrices of different layers, including:

[0084] Add fine-tuning parameters to the model parameters of each transformer module of the pre-trained model to obtain a fine-tuning parameter matrix of each transformer module.

[0085] Accordingly, in some embodiments, the pre-trained model includes a transformer module, and the fine-tuning parameter matrix includes a set of weight matrices of all query vectors and value vectors of the transformer module.

[0086] In some embodiments, reference Figure 4 As shown, the above updating of the two low-rank factor matrices and the core tensor according to the training loss includes:

[0087] Step S20310: Based on the training loss and the orthogonal penalty of the two low-rank factor matrices, update the two low-rank factor matrices and the core tensor.

[0088] Among them, the orthogonal penalty of the low-rank factor matrix can be used to calculate the orthogonal loss of two low-rank factor matrices.

[0089] In some embodiments, the orthogonality penalty of the two low-rank factor matrices may be calculated based on the orthogonality penalty coefficient γ, and the two low-rank factor matrices and the core tensor may be updated accordingly.

[0090] In some specific examples, Solve to obtain the updated low-rank factor matrix and core tensor. Based on the training loss Relative to the low-rank correction term Gradient , γ is the orthogonal penalty coefficient.

[0091] In some embodiments, reference Figure 5As shown, after decomposing the fine-tuning parameter matrix into a core tensor and two low-rank factor matrices, the following may also be included:

[0092] Step S501: Obtain the initial tensor rank.

[0093] The initial tensor rank refers to the rank of the initially set core tensor. Since the rank will be adaptively adjusted during the training process (i.e., the fine-tuning process), the obtained initial tensor rank can be a relatively large value without the need for multiple experimental verifications, so that the rank can be adaptively adjusted during later training and the adaptively adjusted rank can be obtained.

[0094] Step S502: Initializing the two low-rank factor matrices according to the initial tensor rank;

[0095] There is no limitation on the method of initializing the two low-rank factor matrices according to the initial tensor rank. In some embodiments, the two low-rank factor matrices can be initialized based on the initial tensor rank through a standard normal distribution, such as through a standard normal distribution. Initialize two low-rank factor matrices.

[0096] Step S503: Initialize the core tensor through a zero diagonal matrix.

[0097] A 0-diagonal matrix is ​​a matrix whose diagonal elements are also 0. By initializing the core tensor with a 0-diagonal matrix, the model performance of the pre-trained model can be well learned, and the accuracy of the model to be trained can be improved accordingly.

[0098] Based on the embodiments described above, detailed description is given below in combination with specific examples.

[0099] The model fine-tuning training method based on dynamic low-rank tensor adaptation provided in the embodiment of the present application is to force the slice matrix of the core tensor to be a diagonal matrix, that is, to be regarded as a singular value matrix, while modeling the low rank between multiple weight matrices, and then set the values ​​of the diagonal elements of the slice matrix that are less than the threshold to 0 to achieve the function of dynamically adjusting the optimal rank, so as to reduce the fine-tuning cost, and there is no need to traverse the optimal rank. The optimal rank can be adaptively found for learning only through model training, which can further reduce the fine-tuning cost.

[0100] The principle of model fine-tuning training based on low-rank tensor adaptation in the embodiment of the present application can be as follows: Figure 6 The method of the embodiment of the present application uses low-rank decomposition (such as low-rank Tucker2 decomposition) combined with truncated singular value decomposition to approximate the weights in the pre-trained large language model to adaptively adjust the tensor rank and significantly reduce the cost of fine-tuning the large model. Specifically, the approximate weights can be the weight matrix set of all query Q vectors and value V vectors of the transformer module in the large model and .by For example, for the weight matrix set in the pre-trained model , the solution of the embodiment of the present application decomposes it into a core tensor and two low-rank factor matrices A and B such that The sth weight matrix in , where A and B are both low-rank factor matrices, The core tensor The s-th lateral slice matrix of . For the matrix set In terms of, its decomposition shares the factor matrices A and B, but for each matrix Each has a unique lateral slice matrix The solution of the embodiment of this application considers the mandatory is a diagonal matrix. In the process of updating parameters, The values ​​of the diagonal elements that are less than the threshold are set to 0, and finally the updated parameters are added to the original parameters after the back propagation update is completed. Complete fine-tuning parameter update. Figure 6 In the figure, the gray background represents freezing, that is, the weight matrix based on the pre-trained model Frozen and not updated during parameter adjustment.

[0101] Accordingly, the model fine-tuning training process based on low-rank tensor adaptation in an embodiment of the present application can be described as follows.

[0102] First, get the preset tensor rank r, the pre-trained model, the number of transformer modules L of the pre-trained model, and the input size of the transformer module and the output size of the transformer module , low rank correction parameter δ, orthogonal penalty coefficient γ, singular value threshold (ie preset threshold) σ.

[0103] Based on the number of transformer modules L, get the specific vector weight matrix of L transformer modules from the pre-trained model , and add fine-tuning parameters to the model parameters of each transformer module of the pre-trained model, obtain the fine-tuning parameter matrices of different layers, merge the fine-tuning parameter matrices of multiple layers into a fine-tuning parameter tensor, and further decompose the fine-tuning parameter tensor into a core tensor and two low-rank factor matrices.

[0104] The fine-tuning parameter matrix of the s-th transformer module is expressed as , then the current parameter weight of the s-th transformer module is .in, is the weight matrix of the model parameters of the sth transformer module obtained from the pre-trained model. , A and B are both low-rank factor matrices, The core tensor The s-th lateral slice matrix of .

[0105] Through the standard normal distribution Initialize the factor matrix and .

[0106] Initialize the core tensor with a zero diagonal matrix The lateral slice matrix .

[0107] The pre-trained model is then fine-tuned based on the sample data. During the fine-tuning process, the original parameters of the pre-trained model are frozen, and the low-rank factor matrices A and B, the core tensor (or core tensor The L lateral slice matrices ) to make adjustments.

[0108] Assume that the training model is trained for E iterations, then for each iteration e (e=1,2,…,E), for the sample data in each training batch, the following processing can be performed:

[0109] Based on current weight Perform forward propagation of the entire network, where the low-rank correction term .

[0110] Calculate the training loss of the model based on the model processing results (i.e. the output of the model) of the sample data and the target label of the sample data. .

[0111] For each transformer linear layer s (s=1,2,…,L), the training loss function is calculated by the back propagation algorithm Relative to the low-rank correction term Gradient . And use the gradient descent method to get Solution , and Among them, the optimal one is calculated based on the training loss , and The method is not limited, and any method based on training loss to update model parameters can be used.

[0112] Will The values ​​of the diagonal elements that are less than the threshold σ are set to 0 to achieve automatic adjustment of the rank.

[0113] Based on the solution and , and the values ​​of diagonal elements less than the threshold σ are set to 0 , which can be used to correct the low-rank term For example, if the AdamW optimizer is used for updating, the updated low-rank correction term is .

[0114] End the current round of fine-tuning process and enter the next round of iteration process.

[0115] When the E-th round of iterations is completed, the final low-rank correction term after the E-th iteration is obtained: , so as to obtain the final weight matrix of the model parameters of the model to be trained: , thus obtaining the transformer linear layer parameters after fine-tuning (i.e. after training). , that is, the fine-tuned transformer linear layer parameters are ,in , so that the final trained target model can be obtained.

[0116] Based on the model fine-tuning training method based on dynamic low-rank tensor adaptation as described above, the embodiment of the present application also provides a model application method, such as Figure 7 As shown, the model application method in some embodiments includes:

[0117] Step S701: Obtain data to be processed;

[0118] Step S702: Process the data to be processed through a target model to obtain a target processing result, wherein the target model is trained using the model fine-tuning training method based on dynamic low-rank tensor adaptation in any of the embodiments described above.

[0119] Among them, it can be understood that based on the different technical fields and model processing targets, the target models finally obtained may also be different, and the processing based on the target model may also be different. Taking the text classification in the natural language field as described above as an example, the target model obtained may be a text classification model, the data to be processed may be text data to be classified, the target model processes the data to be processed, and the target processing result obtained may be the text category obtained by the target model classifying the text data. Taking the speech prediction in the natural language field as an example, the target model obtained may be a speech prediction model, the data to be processed may be text data to be predicted, the target model processes the data to be processed, and the target processing result obtained may be the speech text to be fed back obtained by the target model performing speech prediction on the text data. It can be understood that in other embodiments, the target model may also be other models, the data to be processed may also be other data, and the corresponding target processing results may also be different, and the embodiments of the present application do not specifically limit this.

[0120] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0121] Based on the same inventive concept, the embodiment of the present application also provides a model fine-tuning training device based on dynamic low-rank tensor adaptation for implementing the model fine-tuning training method based on dynamic low-rank tensor adaptation involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the model fine-tuning training device based on dynamic low-rank tensor adaptation provided below can refer to the limitations of the model fine-tuning training method based on dynamic low-rank tensor adaptation in the above text, and will not be repeated here.

[0122] In one embodiment, Figure 8 As shown, a model fine-tuning training device based on dynamic low-rank tensor adaptation is provided, including: a pre-trained model acquisition module 801, a to-be-trained model acquisition module 802 and a training module 803, wherein:

[0123] A pre-training model acquisition module 801 is used to acquire a pre-training model;

[0124] The to-be-trained model acquisition module 802 is used to add fine-tuning parameters to the model parameters of the pre-trained model, obtain fine-tuning parameter matrices of different layers, and merge the fine-tuning parameter matrices of multiple layers into a fine-tuning parameter tensor and decompose them into a core tensor and two low-rank factor matrices to obtain the to-be-fine-tuned parameters;

[0125] The training module 803 is used to fine-tune the pre-trained model through sample data, freeze the original parameters of the pre-trained model during the fine-tuning process, update the core tensor and the two low-rank factor matrices in sequence, and perform low-rank adjustment on the slice matrix of the updated core tensor to obtain updated fine-tuning parameters.

[0126] In some embodiments, the training module 803 is also used to calculate the training loss during the training process, and update the two low-rank factor matrices and the core tensor according to the training loss; the diagonal elements of the slice matrix of the updated core tensor that are less than a preset threshold are set to 0 to obtain updated fine-tuning parameters.

[0127] In some embodiments, the pre-trained model includes a transformer module, and the fine-tuning parameter matrix includes a set of weight matrices of all query vectors and value vectors of the transformer module.

[0128] In some embodiments, the training module 803 is further used to update the two low-rank factor matrices and the core tensor based on the training loss and the orthogonal penalty of the two low-rank factor matrices.

[0129] In some embodiments, the training model acquisition module 802 is also used to obtain an initial tensor rank; initialize the two low-rank factor matrices according to the initial tensor rank; and initialize the core tensor through a 0 diagonal matrix.

[0130] In some embodiments, the module 802 for acquiring the model to be trained is further used to initialize the two low-rank factor matrices through a standard normal distribution according to the initial tensor rank.

[0131] Each module in the above-mentioned model fine-tuning training device based on dynamic low-rank tensor adaptation can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0132] Based on the same inventive concept, the embodiment of the present application also provides a model application device for implementing the model application method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more model application device embodiments provided below can refer to the limitations on the model application method above, and will not be repeated here.

[0133] In one embodiment, as shown in FIG9 , a model application device is provided, including: a data acquisition module 901 and a model processing module 902, wherein:

[0134] The data acquisition module 901 is used to acquire the data to be processed;

[0135] The model processing module 902 is used to process the data to be processed through a target model to obtain a target processing result. The target model is obtained by using the model fine-tuning training method based on dynamic low-rank tensor adaptation in any of the embodiments described above.

[0136] Each module in the above model application device can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0137] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store pre-trained models, sample data, target models, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a model fine-tuning training method based on dynamic low-rank tensor adaptation and / or a model application method is implemented.

[0138] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.11As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a model fine-tuning training method based on dynamic low-rank tensor adaptation and / or a model application method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse.

[0139] Those skilled in the art will understand that Fig.10 , 11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the model fine-tuning training method or model application method based on dynamic low-rank tensor adaptation in any of the embodiments described above are implemented.

[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the model fine-tuning training method or model application method based on dynamic low-rank tensor adaptation in any of the embodiments described above are implemented.

[0142] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the model fine-tuning training method or model application method based on dynamic low-rank tensor adaptation in any of the embodiments described above.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0144] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A model fine-tuning training method based on dynamic low-rank tensor adaptation, wherein: The method comprises: Get the pre-trained model; Add fine-tuning parameters to the model parameters of the pre-trained model to obtain fine-tuning parameter matrices of different layers, and merge the fine-tuning parameter matrices of multiple layers into a fine-tuning parameter tensor and decompose them into a core tensor and two low-rank factor matrices to obtain parameters to be fine-tuned; The pre-trained model is fine-tuned using sample data, the original parameters of the pre-trained model are frozen during the fine-tuning process, the core tensor and the two low-rank factor matrices are updated in sequence, and the slice matrix of the updated core tensor is low-rank adjusted to obtain updated fine-tuning parameters.

2. The method according to claim 1, characterized in that: The updating of the core tensor and the two low-rank factor matrices in sequence, and low-rank adjustment of the slice matrix of the updated core tensor to obtain updated fine-tuning parameters includes: Calculating a training loss during training, and updating the two low-rank factor matrices and the core tensor according to the training loss; Among the diagonal elements of the slice matrix of the updated core tensor, those that are smaller than a preset threshold are set to 0 to obtain updated fine-tuning parameters.

3. The method according to claim 2, characterized in that The pre-trained model includes a transformer module, and the fine-tuning parameter matrix includes a set of weight matrices of all query vectors and value vectors of the transformer module.

4. The method according to any one of claims 1 to 3, characterized in that: The updating of the two low-rank factor matrices and the core tensor according to the training loss includes: Based on the training loss and the orthogonal penalty of the two low-rank factor matrices, the two low-rank factor matrices and the core tensor are updated.

5. The method according to any one of claims 1 to 3, characterized in that: After decomposing the fine-tuning parameter matrix into a core tensor and two low-rank factor matrices, the method further includes: Get the initial tensor rank; Initializing the two low-rank factor matrices according to the initial tensor rank; Initialize the core tensor by a diagonal matrix of zeros.

6. The method according to claim 5, characterized in that Initializing the two low-rank factor matrices according to the tensor rank includes: According to the initial tensor rank, the two low-rank factor matrices are initialized by standard normal distribution.

7. A model application method, characterized in that: The method comprises: Get the data to be processed; The data to be processed is processed by a target model to obtain a target processing result, wherein the target model is trained by the method according to any one of claims 1 to 6.

8. A model fine-tuning training device based on dynamic low-rank tensor adaptation, characterized in that: The device comprises: A pre-trained model acquisition module is used to obtain a pre-trained model; A module for acquiring a model to be trained, used to add fine-tuning parameters to the model parameters of the pre-trained model, obtain fine-tuning parameter matrices of different layers, and merge the fine-tuning parameter matrices of multiple layers into a fine-tuning parameter tensor and decompose them into a core tensor and two low-rank factor matrices to obtain parameters to be fine-tuned; A training module is used to fine-tune the pre-trained model through sample data, freeze the original parameters of the pre-trained model during the fine-tuning process, update the core tensor and the two low-rank factor matrices in sequence, and perform low-rank adjustment on the slice matrix of the updated core tensor to obtain updated fine-tuning parameters.

9. A model application device, characterized in that: The device comprises: A data acquisition module, used to acquire data to be processed; A model processing module is used to process the data to be processed through a target model to obtain a target processing result, wherein the target model is trained using the method described in any one of claims 1 to 7.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images