Model fine-tuning method, device, server and storage medium based on low-rank adaptation

Abstract

The present application relates to the field of artificial intelligence, and provides a model fine-tuning method, device, server and storage medium based on low-rank adaptation. The embodiment of the present application determines the influence score of each diagonal element in the gating matrix of the rank number configured for the pre-training model on the pre-training model after training the pre-training model N0 times, and based on the influence score of each diagonal element, updates the gating matrix to which the diagonal elements with lower influence of the pre-training model belong, so as to adjust the rank number of the low-rank parameter matrix corresponding to the updated gating matrix, so that when the loss value of the pre-training model before adjusting the rank number is greater than the loss value of the pre-training model after adjusting the rank number, the rank number continues to be adjusted and the pre-training model is trained based on the adjusted rank number, otherwise the training is stopped, thereby preventing the pre-training model from overfitting, improving the parameter fine-tuning effect of the pre-training model, and ensuring the accuracy of the pre-training model after fine-tuning.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model fine-tuning method, device, server and storage medium based on low-rank adaptation. Background Art

[0002] As the pre-trained models in the artificial intelligence systems used by enterprises, organizations, or hospitals become increasingly large, it is no longer possible for them to deploy a model for every task. However, parameter fine-tuning methods only require fine-tuning a small number of parameters for different tasks, allowing large pre-trained models to be adapted to the task, thereby saving the required training data and reducing the graphics card memory space occupied by gradients and optimizers. For example, in a medical scenario, a large model backbone needs to support multiple tasks such as physical examination report extraction, case understanding, knowledge question answering, and patient management. Through parameter fine-tuning algorithms, a small number of parameters of the large model can be fine-tuned for tasks such as physical examination report extraction, case understanding, knowledge question answering, and patient management, thereby adapting the large pre-trained model to multiple tasks such as physical examination report extraction, case understanding, knowledge question answering, and patient management.

[0003] Currently, the most commonly used parameter fine-tuning method is the low-rank adaptation method. For the weight matrix W0 of the pre-trained model, the weight change ΔW can be expressed using a low-rank decomposition: W0+ΔW=W0+BA, where B and A are the parameter matrices to be learned, both of low rank. The dimension of B is D*K, and the dimension of A is K*D, where D is the hidden state dimension of the pre-trained model. K is the rank, the core hyperparameter of the low-rank adaptation method. However, the standard low-rank adaptation method sets the same rank for each module of the pre-trained model. Fine-tuning the parameters of the pre-trained model using the same rank can easily lead to overfitting of the pre-trained model, resulting in poor parameter fine-tuning results and an inability to guarantee the accuracy of the fine-tuned pre-trained model. Therefore, how to improve the parameter fine-tuning effect of pre-trained models is an urgent problem that needs to be solved. Summary of the Invention

[0004] The embodiments of the present application provide a model fine-tuning method, device, server and storage medium based on low-rank adaptation, which aims to improve the parameter fine-tuning effect of the pre-trained model to ensure the accuracy of the fine-tuned pre-trained model.

[0005] In a first aspect, an embodiment of the present application provides a model fine-tuning method based on low-rank adaptation, comprising:

[0006] Configure a low-rank parameter matrix for each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix and the number of diagonal elements in the gating matrix is ​​the same as the rank number;

[0007] According to a preset training sample data set, training the pre-trained model N0 times to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2;

[0008] Determine the influence score of each diagonal element in each of the gating matrices on the pre-trained model and train the first model N1 times based on the training sample dataset to update the low-rank parameter matrix N1 times to obtain a second model, and determine a first loss value of the second model based on a preset validation sample dataset, where N1 is an integer greater than or equal to 2;

[0009] Determining a plurality of target diagonal elements according to the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements;

[0010] Updating the gating matrix to which each target diagonal element belongs to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model;

[0011] Training the third model N1 times based on the training sample dataset to update the low-rank parameter matrix N1 times to obtain a fourth model, and determining a second loss value of the fourth model based on the validation sample dataset;

[0012] When the second loss value is less than the first loss value, updating the first model to the fourth model, and returning to the steps of determining the influence score of each diagonal element in each gating matrix on the pre-trained model and training the first model N1 times according to the training sample dataset to update the low-rank parameter matrix N1 times to obtain a second model, and determining the first loss value of the second model according to a preset validation sample dataset;

[0013] When the second loss value is greater than the first loss value, training is stopped, and each current low-rank parameter matrix is ​​reverted to each low-rank parameter matrix in the first model.

[0014] In a second aspect, an embodiment of the present application further provides a model fine-tuning device based on low-rank adaptation, the model fine-tuning device comprising:

[0015] A configuration module, configured to configure a low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix, and the number of diagonal elements in the gating matrix is ​​the same as the rank number;

[0016] A training module, configured to train the pre-trained model N0 times according to a preset training sample data set to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2;

[0017] A determination module is configured to determine the influence score of each diagonal element in each of the gating matrices on the pre-trained model and train the first model N1 times based on the training sample data set to update the low-rank parameter matrix N1 times to obtain a second model, and determine a first loss value of the second model based on a preset validation sample data set, where N1 is an integer greater than or equal to 2;

[0018] The determination module is further configured to determine a plurality of target diagonal elements based on the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements;

[0019] An updating module, configured to update the gating matrix to which each target diagonal element belongs, so as to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model;

[0020] The training module is further configured to train the third model N1 times based on the training sample dataset to update the low-rank parameter matrix N1 times to obtain a fourth model and determine a second loss value of the fourth model based on the validation sample dataset;

[0021] A loop module, configured to update the first model to the fourth model when the second loss value is less than the first loss value, and return to the determination module to determine the influence score of each diagonal element in each gating matrix on the pre-trained model and train the first model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a second model, and determine the first loss value of the second model according to a preset validation sample data set;

[0022] A fallback module is used to stop training when the second loss value is greater than the first loss value, and to fallback each current low-rank parameter matrix to each low-rank parameter matrix in the first model.

[0023] In a third aspect, an embodiment of the present application further provides a server, comprising a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the model fine-tuning method based on low-rank adaptation as described in the first aspect are implemented.

[0024] In a fourth aspect, an embodiment of the present application further provides a storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the model fine-tuning method based on low-rank adaptation as described in the first aspect are implemented.

[0025] An embodiment of the present application provides a model fine-tuning method, device, server and storage medium based on low-rank adaptation. The model fine-tuning method determines the influence score of each diagonal element in the gating matrix of the rank number configured for the pre-training model on the pre-training model after training the pre-training model N0 times, and based on the influence score of each diagonal element, updates the gating matrix to which the diagonal elements with lower influence of the pre-training model belong to adjust the rank number of the low-rank parameter matrix corresponding to the updated gating matrix. In this way, when the loss value of the pre-training model before adjusting the rank number is greater than the loss value of the pre-training model after adjusting the rank number, the rank number continues to be adjusted and the pre-training model is trained based on the adjusted rank number. Otherwise, the training is stopped, thereby preventing the pre-training model from overfitting and improving the parameter fine-tuning effect of the pre-training model to ensure the accuracy of the pre-training model after fine-tuning. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0027] Figure 1 This is a flow chart of a model fine-tuning method based on low-rank adaptation provided in an embodiment of the present application;

[0028] Figure 2 yes Figure 1 Schematic diagram of the sub-step flow of the model fine-tuning method based on low-rank adaptation in;

[0029] Figure 3 This is a schematic block diagram of a model fine-tuning device based on low-rank adaptation provided in an embodiment of the present application;

[0030] Figure 4 yes Figure 3 A schematic block diagram of submodules of a model fine-tuning device based on low-rank adaptation;

[0031] Figure 5 This is a schematic block diagram of the structure of a server provided in an embodiment of the present application.

[0032] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0035] At present, the commonly used parameter fine-tuning method is mainly the low-rank adaptation method. For the weight matrix W0 of the pre-trained model, the weight change ΔW can be represented by low-rank decomposition, that is, W0+ΔW=W0+BA, where B and A are parameter matrices to be learned, and both are low-rank. The dimension of B is D*K, and the dimension of A is K*D. D is the hidden state dimension of the pre-trained model, and K is the rank number, which is the most core hyperparameter of the low-rank adaptation method. However, the standard low-rank adaptation method sets the same rank number for each module of the pre-trained model, and uses the same rank number to fine-tune the parameters of the pre-trained model, which easily causes the pre-trained model to overfit, resulting in poor parameter fine-tuning effect and inability to guarantee the accuracy of the fine-tuned pre-trained model.

[0036] To solve the above problems, the embodiments of the present application provide a model fine-tuning method, device, server and storage medium based on low-rank adaptation. The model fine-tuning method determines the influence score of each diagonal element in the gating matrix of the rank number configured for the pre-training model on the pre-training model after training the pre-training model N0 times, and based on the influence score of each diagonal element, updates the gating matrix to which the diagonal elements with lower influence of the pre-training model belong to adjust the rank number of the low-rank parameter matrix corresponding to the updated gating matrix. In this way, when the loss value of the pre-training model before adjusting the rank number is greater than the loss value of the pre-training model after adjusting the rank number, the rank number continues to be adjusted and the pre-training model is trained based on the adjusted rank number. Otherwise, the training is stopped, thereby preventing the pre-training model from overfitting and improving the parameter fine-tuning effect of the pre-training model to ensure the accuracy of the fine-tuned pre-training model.

[0037] The model fine-tuning method can be applied to servers or terminal devices, including mobile phones, tablets, laptops, desktop computers, personal digital assistants, and wearable devices. The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0038] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0039] See also Figure 1 , Figure 1 This is a flow chart of a model fine-tuning method based on low-rank adaptation provided in an embodiment of the present application.

[0040] like Figure 1 As shown, the model fine-tuning method includes steps S101 to S108.

[0041] Step S101: configure a low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix.

[0042] In this embodiment, the pre-trained model to be fine-tuned can be an artificial intelligence model with more than 100 million model parameters. Among them, the pre-trained models to be fine-tuned include large language models (LLM), large visual models, and multimodal large models. Large language models focus on processing natural language and can understand, generate, and process large-scale text data. They can include GPT models, LLaMA models, ChatGLM-6B models, and Y-34B-Chat models. Large visual models focus on computer vision tasks such as image classification, target detection, and image generation. They can extract information about objects, scenes, and structures from images, such as the Vision Transformer model. Multimodal large models can process multiple different types of data, such as text, images, speech, etc., and establish associations between these data, such as the Pangu large model and the Gemini model.

[0043] In some embodiments, the gating matrix of the rank of the low-rank parameter matrix is ​​a diagonal matrix, and the number of diagonal elements in the gating matrix is ​​the same as the rank of the low-rank parameter matrix. The initial value of each diagonal element in the gating matrix is ​​1. For example, if the rank is 8, the gating matrix is:

[0044]

[0045] In some embodiments, the rank of the low-rank parameter matrix configured for each module in the pre-trained model to be fine-tuned is the same. The rank of the low-rank parameter matrix can be 8, 16 or 32, and each module in the pre-trained model includes each linear layer and / or each convolutional layer in the pre-trained model. This embodiment configures each module in the pre-trained model with a low-rank parameter matrix of the same rank and a gating matrix of the corresponding rank, thereby making it easier to configure the low-rank parameter matrix and gating matrix of each module in the pre-trained model, thereby improving the convenience of configuration.

[0046] In some embodiments, configuring the low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and the gating matrix of the rank number of the low-rank parameter matrix may include: determining the parameter amount of each module in the pre-trained model to be fine-tuned, and determining the target rank number corresponding to each module according to the parameter amount of each module in the pre-trained model to be fine-tuned; configuring the low-rank parameter matrix and the gating matrix corresponding to the target rank number of each module in the pre-trained model. Among them, the target rank numbers corresponding to different modules can be the same or different, and the parameter amount of the module is positively correlated with the target rank number, that is, the larger the parameter amount of the module, the higher the target rank number, and the smaller the parameter amount of the module, the lower the target rank number. This embodiment can further improve the model fine-tuning effect by adaptively configuring the low-rank parameter matrix and the gating matrix corresponding to the rank number of each module according to the parameter amount of each module in the pre-trained model to be fine-tuned, thereby better ensuring the accuracy of the pre-trained model after fine-tuning.

[0047] In some embodiments, instead of configuring the low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and the gating matrix of the rank number of the low-rank parameter matrix, the low-rank parameter matrix of some modules in the pre-trained model to be fine-tuned and the gating matrix of the rank number of the low-rank parameter matrix are configured. For example, the low-rank parameter matrix of each target module in the pre-trained model to be fine-tuned and the gating matrix of the rank number of the low-rank parameter matrix are configured, the rank number of the low-rank parameter matrix configured for each target module is the same or different, and the target module is a module in the pre-trained model whose parameter amount is greater than or equal to the preset parameter amount. This embodiment can reduce the computational complexity and memory usage of model fine-tuning by configuring the low-rank parameter matrix of some modules in the pre-trained model to be fine-tuned and the gating matrix of the rank number of the low-rank parameter matrix.

[0048] Step S102: According to a preset training sample data set, the pre-trained model is trained N0 times to update the low-rank parameter matrix N0 times to obtain a first model.

[0049] In this embodiment, N0 is an integer greater than or equal to 2, for example, N0 = 100 or 200. The preset training sample data set can be set based on actual conditions, and the embodiments of the present application do not specifically limit this. For example, the training sample data set is a knowledge question and answer question bank required for a knowledge question and answer task, including questions and answers. For another example, the training sample data set is a training sample data set required for a text-to-speech task, including sample text and annotated speech.

[0050] In some embodiments, during the training of a pre-trained model, for a module configured with a low-rank parameter matrix, h = W0x + BΛAx is used instead of h = W0x for forward calculation, where A and B are low-rank parameter matrices, W0 is the initial weight matrix, and Λ is the gating matrix. During the updating of the model parameters of the pre-trained model, the initial weight matrix of the pre-trained model is fixed, the gradient is calculated based on the cross-entropy loss, and the low-rank parameter matrices A1 and B2 are updated using the backpropagation algorithm, and the gating matrix is ​​not updated.

[0051] Step S103: determine the influence score of each diagonal element in each gating matrix on the pre-trained model and train the first model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain the second model, and determine the first loss value of the second model according to the preset verification sample data set.

[0052] In this embodiment, the influence score is a score of the influence of the diagonal elements on the performance (accuracy) of the pre-trained model. N1 is an integer greater than or equal to 2. N1 can be set based on actual conditions and is not specifically limited in this embodiment of the application. For example, N1 = 50 or 150. Among them, N0 and N1 can be equal, or N0 is greater than N1, or N0 is less than N1.

[0053] It can be understood that the method used to train the first model N1 times to update the low-rank parameter matrix N1 times is the same as the method used to train the pre-trained model N0 times to update the low-rank parameter matrix N0 times, which will not be repeated here.

[0054] In some embodiments, a preset validation sample data set includes multiple validation sample data, each validation sample data including validation samples and annotation information. Determining the first loss value of the second model based on the preset validation sample data set may include: inputting each validation sample into the second model for processing to obtain multiple prediction information, with one validation sample corresponding to one prediction information; determining multiple loss values ​​of the second model based on each prediction information and the corresponding annotation information; calculating the average of the multiple loss values ​​of the second model, and determining the average of the multiple loss values ​​of the second model as the first loss value of the second model. The cross-entropy loss function may be used to determine the loss value of the second model based on the prediction information and the corresponding annotation information.

[0055] It is understood that the verification sample and annotation information included in the verification sample data can be set based on actual conditions, and the embodiments of this application do not specifically limit this. For example, in the text-to-speech task scenario, the verification sample data includes sample text and annotated speech. For another example, in the knowledge question-answering task scenario, the verification sample data includes sample questions and annotated answers.

[0056] In some embodiments, as Figure 2 As shown, step S103 includes: sub-steps S1031 to S1033.

[0057] Sub-step S1031, for any diagonal element α ij , determine the diagonal element α ij The corresponding first comparison model group and / or second comparison model group include the first comparison model group and the second reference model, and the second comparison model group includes the third reference model and the fourth reference model.

[0058] In this embodiment, the pre-trained model includes N modules, and each module is configured with a low-rank parameter matrix and a gating matrix with a rank of K. Then the number of low-rank parameter matrices and gating matrices is also N, and the total number of diagonal elements is N*K, and the diagonal element α ij It is the j-th diagonal element in the gating matrix configured for the i-th module, i is greater than or equal to 1 and less than or equal to the number of low-rank parameter matrices N, j is greater than or equal to 1 and less than or equal to the rank number K of the low-rank parameter matrix, and both i and j are positive integers.

[0059] In some embodiments, the first reference model, the second reference model, the third reference model, and the fourth reference model are all related to the first model. ij Set to 0, except for the diagonal elements α in the first model ij All diagonal elements except for α remain unchanged to obtain the first reference model; the diagonal element α in the first model is ijSet to 1, except for the diagonal elements α in the first model ij All diagonal elements except α remain unchanged to obtain a second reference model; and / or, the diagonal elements α in the first model are ij Set to 0, and remove the diagonal elements α in the first model ij All diagonal elements except α are set to 0 to obtain the third reference model; the diagonal elements α in the first model are set to ij Set to 1, and remove the diagonal elements α in the first model ij All diagonal elements except are set to 0 to obtain the fourth reference model.

[0060] Sub-step S1032: determining the loss value of the first reference model and the loss value of the second reference model based on the validation sample data set, and / or determining the loss value of the third reference model and the loss value of the fourth reference model based on the validation sample data set.

[0061] In this embodiment, the verification sample data set includes multiple verification sample data, and the verification sample data includes verification samples and annotation information. Determining the loss value of the first reference model based on the verification sample data set may include: inputting each verification sample into the first reference model for processing to obtain multiple prediction information, one verification sample corresponds to one prediction information; determining the cross-entropy loss value of the first reference model based on each prediction information and the corresponding annotation information; calculating the average of the multiple cross-entropy loss values ​​of the first reference model, and determining the average as the loss value of the first reference model.

[0062] It should be noted that the specific method for determining the loss value of the second reference model, the loss value of the third reference model and the loss value of the fourth reference model can refer to the method for determining the loss value of the first reference model mentioned above, and will not be repeated here.

[0063] Sub-step S1033: Determine the diagonal element α according to the first loss value group and / or the second loss value group. ij The influence score on the pre-trained model.

[0064] In this embodiment, the first loss value group includes loss values ​​of the first reference model and loss values ​​of the second reference model, and the second loss value group includes loss values ​​of the third reference model and loss values ​​of the fourth reference model.

[0065] In some embodiments, the diagonal element α is determined based on the first loss value group and / or the second loss value group. ij The influence score of the pre-trained model may include: determining a first difference between the loss value of the first reference model and the loss value of the second reference model; determining a second difference between the loss value of the third reference model and the loss value of the fourth reference model; and determining a diagonal element α based on the first difference and / or the second difference.ij The influence score on the pre-trained model. The first difference is the loss value of the first reference model minus the loss value of the second reference model, and the second difference is the loss value of the third reference model minus the loss value of the fourth reference model.

[0066] In some embodiments, the diagonal element α is determined based on the first difference and / or the second difference. ij The influence score of the pre-trained model may include determining the first difference or the second difference as a diagonal element α ij The influence score of the pre-trained model. Alternatively, the sum of the first difference and the second difference is determined as the diagonal element α ij The influence score on the pre-trained model.

[0067] Step S104: Determine multiple target diagonal elements based on the influence score of each diagonal element in each gating matrix on the pre-trained model.

[0068] In this embodiment, the influence scores of multiple target diagonal elements are all smaller than the influence scores of any diagonal elements other than the multiple target diagonal elements, that is, the influence scores of multiple target diagonal elements are the reciprocals of the influence scores of several diagonal elements in the diagonal element set (composed of the diagonal elements in each gating matrix).

[0069] In some embodiments, determining multiple target diagonal elements based on the influence scores of each diagonal element in each gating matrix on the pre-trained model may include: sorting each diagonal element in each gating matrix according to the order of the influence scores of each diagonal element on the pre-trained model to obtain a diagonal element queue; determining the total number of diagonal elements in the first model and multiplying the total number by a preset percentage to obtain a target number; and selecting the first or last target number of diagonal elements from the diagonal element queue as the target diagonal elements. The preset percentage can be set based on actual conditions and is not specifically limited in this embodiment of the present application. For example, the preset percentage is 5% or 10%.

[0070] In some embodiments, when the influence score of the first diagonal element in the diagonal element queue is greater than the influence score of the second diagonal element (the earlier the diagonal element in the diagonal element queue, the higher the influence score), the last target number of diagonal elements in the diagonal element queue are taken as target diagonal elements; when the influence score of the first diagonal element in the diagonal element queue is less than the influence score of the second diagonal element (the earlier the diagonal element in the diagonal element queue, the lower the influence score), the first target number of diagonal elements in the diagonal element queue are taken as target diagonal elements.

[0071] For example, if the target number is 5 and the influence scores of the diagonal elements closer to the front of the diagonal element queue are higher, the last 5 diagonal elements in the diagonal element queue can be used as the target diagonal elements. For another example, if the target number is 5 and the influence scores of the diagonal elements closer to the front of the diagonal element queue are lower, the first 5 diagonal elements in the diagonal element queue can be used as the target diagonal elements.

[0072] Step S105: Update the gating matrix to which each target diagonal element belongs to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model.

[0073] In this embodiment, updating the gating matrix to which each target diagonal element belongs may include: for each target diagonal element, setting the target diagonal element in the gating matrix to which the target diagonal element belongs to 0. For example, assuming that the pre-trained model includes n modules, and each module is configured with a low-rank parameter matrix of rank k and a corresponding gating matrix Λ, then the number of low-rank parameter matrices and gating matrices is also n, and the total number of diagonal elements is n*k. Therefore, the set of diagonal elements can be expressed as [α 11 , α 12 ,...,α 1k , α 21 , α 22 ,...,α 2k ,...,α n1 , α n2 ,...,α nk-1 , α nk ], specifically, let k = 8, then the diagonal element set can be expressed as [α 11 , α 12 ,...,α 18 , α 21 , α 22 ,...,α 28 ,...,α n1 , α n2 ,...,α n7 , α n8 ]. Let the target diagonal element be α 26 , then the target diagonal element α 26 The gating matrix Λ2 is:

[0074]

[0075] Among them, α 21 , α 22 , α 23 , α 24 , α 25 , α 26 , α 27 and α 28The value of is 1, then the target diagonal element α in the gating matrix Λ2 can be 26 Set to 0, thereby adjusting the rank of the low-rank parameter matrix corresponding to the gating matrix Λ2. The updated gating matrix Λ2 is:

[0076]

[0077] Step S106: Train the third model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain the fourth model and determine the second loss value of the fourth model according to the verification sample data set.

[0078] In this embodiment, the method used to train the third model N1 times to update the low-rank parameter matrix N1 times is the same as the method used to train the pre-trained model N0 times to update the low-rank parameter matrix N0 times. The specific method for determining the second loss value of the fourth model is the same as the method for determining the first loss value of the second model, which will not be repeated here.

[0079] Step S107: When the second loss value is less than the first loss value, update the first model to the fourth model.

[0080] In this embodiment, when the second loss value is less than the first loss value, the first model is updated to the fourth model, and the process returns to step S103, determines the influence score of each diagonal element in each gating matrix on the pre-trained model, and trains the first model N1 times based on the training sample data set to update the low-rank parameter matrix N1 times to obtain the second model, and determines the first loss value of the second model based on the preset verification sample data set.

[0081] Among them, when the second loss value is less than the first loss value, it can be determined that the performance of the model after rank adjustment is better than the performance of the model before rank adjustment. Therefore, the rank of the low-rank parameter matrix can be further adjusted, that is, steps S103-S106 are repeated, so as to continuously adjust the rank of the low-rank parameter matrix until the performance of the model after rank adjustment is worse than the performance of the model before rank adjustment, thereby preventing the pre-trained model from overfitting and improving the parameter fine-tuning effect of the pre-trained model to ensure the accuracy of the fine-tuned pre-trained model.

[0082] Step S108: When the second loss value is greater than the first loss value, stop training and revert each current low-rank parameter matrix to each low-rank parameter matrix in the first model.

[0083] In this embodiment, when the second loss value is greater than the first loss value, it can be determined that the performance of the model after rank adjustment is worse than the performance of the model before rank adjustment, and there is no need to further adjust the rank of the low-rank parameter matrix. Therefore, training is stopped, and each current low-rank parameter matrix is ​​reverted to the low-rank parameter matrix corresponding to the model with better performance. This can prevent the pre-trained model from overfitting and improve the parameter fine-tuning effect of the pre-trained model to ensure the accuracy of the fine-tuned pre-trained model.

[0084] See also Figure 3 , Figure 3 This is a schematic block diagram of a model fine-tuning device based on low-rank adaptation provided in an embodiment of the present application.

[0085] like Figure 3 As shown, the model fine-tuning device 100 based on low-rank adaptation includes:

[0086] A configuration module 110 is configured to configure a low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix, and the number of diagonal elements in the gating matrix is ​​the same as the rank number;

[0087] A training module 120 is configured to train the pre-trained model N0 times based on a preset training sample data set to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2;

[0088] A determination module 130 is configured to determine an influence score of each diagonal element in each of the gating matrices on the pre-trained model and train the first model N1 times based on the training sample dataset to obtain a second model, and determine a first loss value of the second model based on a preset validation sample dataset, where N1 is an integer greater than or equal to 2;

[0089] The determining module 130 is further configured to determine a plurality of target diagonal elements based on the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements;

[0090] An updating module 140 is configured to update the gating matrix to which each target diagonal element belongs, so as to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model;

[0091] The training module 120 is further configured to train the third model N1 times based on the training sample dataset to update the low-rank parameter matrix N1 times to obtain a fourth model and determine a second loss value of the fourth model based on the validation sample dataset;

[0092] A loop module 150 is configured to update the first model to the fourth model when the second loss value is less than the first loss value, and return to the determination module to determine the influence score of each diagonal element in each gating matrix on the pre-trained model and train the first model N1 times based on the training sample dataset to update the low-rank parameter matrix N0 times to obtain a second model, and determine the first loss value of the second model based on a preset validation sample dataset;

[0093] The fallback module 160 is used to stop training when the second loss value is greater than the first loss value, and fall back each current low-rank parameter matrix to each low-rank parameter matrix in the first model.

[0094] In some embodiments, as Figure 4 As shown, the determination module 130 includes:

[0095] The model determination submodule 131 is used for determining the diagonal element α ij , determine the diagonal element α ij The corresponding first comparison model group and / or second comparison model group, the first comparison model group includes a first reference model and a second reference model, the second comparison model group includes a third reference model and a fourth reference model, the first reference model, the second reference model, the third reference model and the fourth reference model are all related to the first model, i is greater than or equal to 1 and less than or equal to the number of the low-rank parameter matrices, j is greater than or equal to 1 and less than or equal to the rank number of the low-rank parameter matrix, and i and j are both positive integers;

[0096] a loss determination submodule 132, configured to determine a loss value of the first reference model and a loss value of the second reference model based on the validation sample dataset, and / or determine a loss value of the third reference model and a loss value of the fourth reference model based on the validation sample dataset;

[0097] Scoring submodule 133, configured to determine the diagonal element α according to the first loss value group and / or the second loss value group. ij For the influence score of the pre-trained model, the first loss value group includes the loss value of the first reference model and the loss value of the second reference model, and the second loss value group includes the loss value of the third reference model and the loss value of the fourth reference model.

[0098] In some embodiments, the model determination submodule 131 is further configured to:

[0099] The diagonal element α in the first model ij Set to 0, except for the diagonal element α in the first model ij All the diagonal elements except remain unchanged, and the first reference model is obtained;

[0100] The diagonal element α in the first model ij Set to 1, the first model except the diagonal element α ij All the diagonal elements except for remain unchanged, and the second reference model is obtained;

[0101] and / or,

[0102] The diagonal element α in the first model ij Set to 0, and remove the diagonal element α from the first model ij All the diagonal elements except are set to 0 to obtain a third reference model;

[0103] The diagonal element α in the first model ij Set to 1, and remove the diagonal element α from the first model ij All the diagonal elements except are set to 0 to obtain the fourth reference model.

[0104] In some embodiments, the scoring submodule 133 is further configured to:

[0105] determining a first difference between the loss value of the first reference model and the loss value of the second reference model;

[0106] determining a second difference between the loss value of the third reference model and the loss value of the fourth reference model;

[0107] Determine the diagonal element α based on the first difference and / or the second difference ij The influence score of the pre-trained model.

[0108] In some embodiments, the determining module 130 is further configured to:

[0109] Sort each of the diagonal elements in each of the gating matrices according to the order of their influence scores on the pre-trained model to obtain a diagonal element queue;

[0110] determining a total number of the diagonal elements in the first model, and multiplying the total number by a preset percentage to obtain a target number;

[0111] The first or last target number of diagonal elements are taken from the diagonal element queue as the target diagonal elements.

[0112] In some embodiments, the verification sample data set includes a plurality of verification sample data, and the verification sample data includes verification samples and annotation information. The determination module 130 is further configured to:

[0113] Inputting each of the verification samples into the second model for processing to obtain a plurality of prediction information, wherein each verification sample corresponds to one piece of prediction information;

[0114] Determining a plurality of loss values ​​of the second model according to each piece of prediction information and the corresponding labeling information;

[0115] An average of the plurality of loss values ​​of the second model is calculated, and the average of the plurality of loss values ​​of the second model is determined as a first loss value of the second model.

[0116] In some embodiments, the ranks of the low-rank parameter matrices configured for each module in the pre-trained model to be fine-tuned are the same.

[0117] It should be noted that those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned model fine-tuning method embodiment, and will not be repeated here.

[0118] The apparatus provided in the above embodiment can be implemented in the form of a computer program. The computer program can be used in Figure 5 Run on the server shown.

[0119] See also Figure 5 , Figure 5 This is a schematic block diagram of the structure of a server provided in an embodiment of the present application.

[0120] like Figure 5 As shown, the server includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a storage medium and an internal memory.

[0121] The storage medium may store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor may execute any one of the model fine-tuning methods.

[0122] The processor is used to provide computing and control capabilities to support the operation of the entire server.

[0123] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the server to which the solution of the present application is applied. The specific server may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0124] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0125] In one embodiment, the processor is configured to execute a computer program stored in the memory to implement the following steps:

[0126] Configure a low-rank parameter matrix for each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix and the number of diagonal elements in the gating matrix is ​​the same as the rank number;

[0127] According to a preset training sample data set, training the pre-trained model N0 times to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2;

[0128] Determine the influence score of each diagonal element in each of the gating matrices on the pre-trained model and train the first model N1 times based on the training sample dataset to update the low-rank parameter matrix N1 times to obtain a second model, and determine a first loss value of the second model based on a preset validation sample dataset, where N1 is an integer greater than or equal to 2;

[0129] Determining a plurality of target diagonal elements according to the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements;

[0130] Updating the gating matrix to which each target diagonal element belongs to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model;

[0131] Training the third model N1 times based on the training sample dataset to update the low-rank parameter matrix N1 times to obtain a fourth model, and determining a second loss value of the fourth model based on the validation sample dataset;

[0132] When the second loss value is less than the first loss value, updating the first model to the fourth model, and returning to the steps of determining the influence score of each diagonal element in each gating matrix on the pre-trained model and training the first model N1 times according to the training sample dataset to update the low-rank parameter matrix N1 times to obtain a second model, and determining the first loss value of the second model according to a preset validation sample dataset;

[0133] When the second loss value is greater than the first loss value, training is stopped, and each current low-rank parameter matrix is ​​reverted to each low-rank parameter matrix in the first model.

[0134] In some embodiments, when determining the influence score of each diagonal element in each gating matrix on the pre-trained model, the processor is configured to implement:

[0135] For any of the diagonal elements α ij , determine the diagonal element α ij The corresponding first comparison model group and / or second comparison model group, the first comparison model group includes a first reference model and a second reference model, the second comparison model group includes a third reference model and a fourth reference model, the first reference model, the second reference model, the third reference model and the fourth reference model are all related to the first model, i is greater than or equal to 1 and less than or equal to the number of the low-rank parameter matrices, j is greater than or equal to 1 and less than or equal to the rank number of the low-rank parameter matrix;

[0136] Determining a loss value of the first reference model and a loss value of the second reference model based on the validation sample dataset, and / or determining a loss value of the third reference model and a loss value of the fourth reference model based on the validation sample dataset;

[0137] Determine the diagonal element α according to the first loss value group and / or the second loss value group ijFor the influence score of the pre-trained model, the first loss value group includes the loss value of the first reference model and the loss value of the second reference model, and the second loss value group includes the loss value of the third reference model and the loss value of the fourth reference model.

[0138] In some embodiments, the processor implements the process of determining the diagonal element α ij The corresponding first comparison model group and / or second comparison model group is used to achieve:

[0139] The diagonal element α in the first model ij Set to 0, except for the diagonal element α in the first model ij All the diagonal elements except remain unchanged, and the first reference model is obtained;

[0140] The diagonal element α in the first model ij Set to 1, the first model except the diagonal element α ij All the diagonal elements except for remain unchanged, and the second reference model is obtained;

[0141] and / or,

[0142] The diagonal element α in the first model ij Set to 0, and remove the diagonal element α from the first model ij All the diagonal elements except are set to 0 to obtain a third reference model;

[0143] The diagonal element α in the first model ij Set to 1, and remove the diagonal element α from the first model ij All the diagonal elements except are set to 0 to obtain the fourth reference model.

[0144] In some embodiments, the processor determines the diagonal element α according to the first loss value group and / or the second loss value group. ij The influence score of the pre-trained model is used to achieve:

[0145] determining a first difference between the loss value of the first reference model and the loss value of the second reference model;

[0146] determining a second difference between the loss value of the third reference model and the loss value of the fourth reference model;

[0147] Determine the diagonal element α based on the first difference and / or the second difference ij The influence score of the pre-trained model.

[0148] In some embodiments, when determining the plurality of target diagonal elements based on the influence score of each diagonal element in each gating matrix on the pre-trained model, the processor is configured to implement:

[0149] Sort each of the diagonal elements in each of the gating matrices according to the order of their influence scores on the pre-trained model to obtain a diagonal element queue;

[0150] determining a total number of the diagonal elements in the first model, and multiplying the total number by a preset percentage to obtain a target number;

[0151] The first or last target number of diagonal elements are taken from the diagonal element queue as the target diagonal elements.

[0152] In some embodiments, the validation sample data set includes multiple validation sample data, and the validation sample data includes validation samples and annotation information. When the processor determines the first loss value of the second model based on the preset validation sample data set, it is configured to implement:

[0153] Inputting each of the verification samples into the second model for processing to obtain a plurality of prediction information, wherein each verification sample corresponds to one piece of prediction information;

[0154] Determining a plurality of loss values ​​of the second model according to each piece of prediction information and the corresponding labeling information;

[0155] An average of the plurality of loss values ​​of the second model is calculated, and the average of the plurality of loss values ​​of the second model is determined as a first loss value of the second model.

[0156] In some embodiments, the ranks of the low-rank parameter matrices configured for each module in the pre-trained model to be fine-tuned are the same.

[0157] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the server described above can refer to the corresponding process in the aforementioned model fine-tuning method embodiment, and will not be repeated here.

[0158] Through the description of the above implementation methods, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a server (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application or certain parts of the embodiments.

[0159] An embodiment of the present application also provides a storage medium for computer-readable storage, wherein a computer program is stored on the storage medium, and the computer program includes program instructions. The method implemented when the program instructions are executed can be referred to the various embodiments of the model fine-tuning method of the present application.

[0160] The storage medium may be volatile or non-volatile. The storage medium may be an internal storage unit of the server described in the aforementioned embodiment, such as a hard disk or memory of the server. The storage medium may also be an external storage device of the server, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the server.

[0161] Furthermore, the storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0162] The blockchain referred to in this application is a new application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0163] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0164] It should also be understood that the term "and / or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0165] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above description is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A model fine-tuning method based on low-rank adaptation, characterized in that: include: Configure a low-rank parameter matrix for each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix and the number of diagonal elements in the gating matrix is ​​the same as the rank number; According to a preset training sample data set, training the pre-trained model N0 times to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2, and the training sample data set includes sample text and annotated speech; Determine the influence score of each diagonal element in each of the gating matrices on the pre-trained model and train the first model N1 times based on the training sample dataset to update the low-rank parameter matrix N1 times to obtain a second model, and determine a first loss value of the second model based on a preset validation sample dataset, where N1 is an integer greater than or equal to 2; Determining a plurality of target diagonal elements according to the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements; Updating the gating matrix to which each target diagonal element belongs to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model; Training the third model N1 times based on the training sample dataset to update the low-rank parameter matrix N1 times to obtain a fourth model, and determining a second loss value of the fourth model based on the validation sample dataset; When the second loss value is less than the first loss value, updating the first model to the fourth model, and returning to the steps of determining the influence score of each diagonal element in each gating matrix on the pre-trained model and training the first model N1 times according to the training sample dataset to update the low-rank parameter matrix N1 times to obtain a second model, and determining the first loss value of the second model according to a preset validation sample dataset; When the second loss value is greater than the first loss value, training is stopped, and each current low-rank parameter matrix is ​​reverted to each low-rank parameter matrix in the first model.

2. The model fine-tuning method according to claim 1, characterized in that: Determining the influence score of each diagonal element in each gating matrix on the pre-trained model includes: For any of the diagonal elements α ij , determine the diagonal element α ij The corresponding first comparison model group and / or second comparison model group, the first comparison model group includes a first reference model and a second reference model, the second comparison model group includes a third reference model and a fourth reference model, the first reference model, the second reference model, the third reference model and the fourth reference model are all related to the first model, i is greater than or equal to 1 and less than or equal to the number of the low-rank parameter matrices, j is greater than or equal to 1 and less than or equal to the rank number of the low-rank parameter matrix, and i and j are both positive integers; Determining a loss value of the first reference model and a loss value of the second reference model based on the validation sample dataset, and / or determining a loss value of the third reference model and a loss value of the fourth reference model based on the validation sample dataset; Determine the diagonal element α according to the first loss value group and / or the second loss value group ij For the influence score of the pre-trained model, the first loss value group includes the loss value of the first reference model and the loss value of the second reference model, and the second loss value group includes the loss value of the third reference model and the loss value of the fourth reference model.

3. The model fine-tuning method according to claim 2, characterized in that: The determination of the diagonal element α ij The corresponding first comparison model group and / or second comparison model group include: The diagonal element α in the first model ij Set to 0, except for the diagonal element α in the first model ij All the diagonal elements except remain unchanged, and the first reference model is obtained; The diagonal element α in the first model ij Set to 1, the first model except the diagonal element α ij All the diagonal elements except for remain unchanged, and the second reference model is obtained; and / or, The diagonal element α in the first model ij Set to 0, and remove the diagonal element α from the first model ij All the diagonal elements except are set to 0 to obtain a third reference model; The diagonal element α in the first model ij Set to 1, and remove the diagonal element α from the first model ij All the diagonal elements except are set to 0 to obtain the fourth reference model.

4. The model fine-tuning method according to claim 2, characterized in that: The diagonal element α is determined according to the first loss value group and / or the second loss value group. ij The influence score of the pre-trained model, including: determining a first difference between the loss value of the first reference model and the loss value of the second reference model; determining a second difference between the loss value of the third reference model and the loss value of the fourth reference model; Determine the diagonal element α based on the first difference and / or the second difference ij The influence score of the pre-trained model.

5. The model fine-tuning method according to claim 1, characterized in that: Determining a plurality of target diagonal elements according to the influence score of each diagonal element in each gating matrix on the pre-trained model includes: Sort each of the diagonal elements in each of the gating matrices according to the order of their influence scores on the pre-trained model to obtain a diagonal element queue; determining a total number of the diagonal elements in the first model, and multiplying the total number by a preset percentage to obtain a target number; The first or last target number of diagonal elements are taken from the diagonal element queue as the target diagonal elements.

6. The model fine-tuning method according to any one of claims 1 to 5, characterized in that: The validation sample data set includes a plurality of validation sample data, the validation sample data includes validation samples and annotation information, and determining the first loss value of the second model according to the preset validation sample data set includes: Inputting each of the verification samples into the second model for processing to obtain a plurality of prediction information, wherein each verification sample corresponds to one piece of prediction information; Determining a plurality of loss values ​​of the second model according to each piece of prediction information and the corresponding labeling information; An average of the plurality of loss values ​​of the second model is calculated, and the average of the plurality of loss values ​​of the second model is determined as a first loss value of the second model.

7. The model fine-tuning method according to any one of claims 1 to 5, characterized in that: The ranks of the low-rank parameter matrices configured for each module in the pre-trained model to be fine-tuned are the same.

8. A model fine-tuning device based on low-rank adaptation, characterized in that: The model fine-tuning device comprises: A configuration module, configured to configure a low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix, and the number of diagonal elements in the gating matrix is ​​the same as the rank number; A training module, configured to train the pre-trained model N0 times based on a preset training sample data set to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2, and the training sample data set includes sample text and annotated speech; a determination module, configured to determine an influence score of each diagonal element in each of the gating matrices on the pre-trained model and train the first model N1 times based on the training sample dataset to obtain a second model, and determine a first loss value of the second model based on a preset validation sample dataset, where N1 is an integer greater than or equal to 2; The determination module is further configured to determine a plurality of target diagonal elements based on the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements; An updating module, configured to update the gating matrix to which each target diagonal element belongs, so as to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model; The training module is further configured to train the third model N1 times based on the training sample dataset to update the low-rank parameter matrix N1 times to obtain a fourth model and determine a second loss value of the fourth model based on the validation sample dataset; A loop module, configured to update the first model to the fourth model when the second loss value is less than the first loss value, and return to the determination module to determine the influence score of each diagonal element in each gating matrix on the pre-trained model and train the first model N1 times according to the training sample data set to update the low-rank parameter matrix N0 times to obtain a second model, and determine the first loss value of the second model according to a preset validation sample data set; A fallback module is used to stop training when the second loss value is greater than the first loss value, and to fallback each current low-rank parameter matrix to each low-rank parameter matrix in the first model.

9. A server, characterized in that: The server includes a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the model fine-tuning method based on low-rank adaptation are implemented as described in any one of claims 1 to 7.

10. A storage medium for computer-readable storage, characterized in that: The storage medium stores a computer program, wherein when the computer program is executed by the processor, the steps of the model fine-tuning method based on low-rank adaptation are implemented as described in any one of claims 1 to 7.

Images