Model fine tuning method and device based on low-rank adaptation, server and storage medium

Abstract

The invention relates to the field of artificial intelligence, and provides a model fine tuning method and device based on low-rank adaptation, a server and a storage medium. According to the embodiment of the invention, after the pre-training model is trained for N0 times, the influence score of each diagonal element in the gating matrix of the rank number configured to the pre-training model on the pre-training model is determined, and based on the influence score of each diagonal element, the gating matrix to which the diagonal element with relatively low influence of the pre-training model belongs is updated, so that the pre-training model is updated. The method comprises the following steps: updating a gating matrix to adjust the rank number of a low-rank parameter matrix corresponding to the updated gating matrix, so that when a loss value of a pre-training model before the rank number is adjusted is greater than the loss value of the pre-training model after the rank number is adjusted, continuing to adjust the rank number and training the pre-training model based on the adjusted rank number, otherwise, stopping training. Therefore, the pre-training model can be prevented from being over-fitted, the parameter fine tuning effect of the pre-training model is improved, and the accuracy of the pre-training model after fine tuning is ensured.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model fine-tuning method, device, server and storage medium based on low-rank adaptation. Background Art

[0002] As the pre-trained models in the artificial intelligence systems used by enterprises, organizations or hospitals become larger and larger, enterprises, organizations or hospitals cannot deploy a model for each task. However, through parameter fine-tuning methods, only a few parameters need to be fine-tuned for different tasks to adapt the large pre-trained model to the task, thereby saving the required training data and reducing the graphics card memory space occupied by gradients and optimizers. For example, in a medical scenario, a large model backbone needs to support multiple tasks such as physical examination report extraction, case understanding, knowledge question and answer, and patient management. Through parameter fine-tuning algorithms, the few parameters of the large model are fine-tuned for tasks such as physical examination report extraction, case understanding, knowledge question and answer, and patient management, so that the large pre-trained model can be adapted to multiple tasks such as physical examination report extraction, case understanding, knowledge question and answer, and patient management.

[0003] At present, the commonly used parameter fine-tuning method is mainly the low-rank adaptation method. For the weight matrix W0 of the pre-trained model, the weight change ΔW can be represented by low-rank decomposition, that is, W0+ΔW=W0+BA, where B and A are parameter matrices to be learned, and both are low-rank. The dimension of B is D*K, and the dimension of A is K*D. D is the hidden state dimension of the pre-trained model. K is the rank number, which is the core hyperparameter of the low-rank adaptation method. However, the standard low-rank adaptation method sets the same rank number for each module of the pre-trained model, and the same rank number is used to fine-tune the parameters of the pre-trained model, which easily causes the pre-trained model to overfit, resulting in poor parameter fine-tuning effect and unable to guarantee the accuracy of the fine-tuned pre-trained model. Therefore, how to improve the parameter fine-tuning effect of the pre-trained model is an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide a model fine-tuning method, device, server and storage medium based on low-rank adaptation, which aims to improve the parameter fine-tuning effect of the pre-trained model to ensure the accuracy of the pre-trained model after fine-tuning.

[0005] In a first aspect, an embodiment of the present application provides a model fine-tuning method based on low-rank adaptation, comprising:

[0006] Configure a low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix, and the number of diagonal elements in the gating matrix is ​​the same as the rank number;

[0007] According to a preset training sample data set, training the pre-trained model N0 times to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2;

[0008] Determine the influence score of each of the diagonal elements in each of the gating matrices on the pre-trained model and train the first model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a second model, and determine a first loss value of the second model according to a preset validation sample data set, where N1 is an integer greater than or equal to 2;

[0009] Determine a plurality of target diagonal elements according to the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements;

[0010] Updating the gating matrix to which each of the target diagonal elements belongs to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model;

[0011] According to the training sample data set, training the third model N1 times to update the low-rank parameter matrix N1 times to obtain a fourth model and determining a second loss value of the fourth model according to the validation sample data set;

[0012] When the second loss value is less than the first loss value, the first model is updated to the fourth model, and the steps of determining the influence score of each diagonal element in each gating matrix on the pre-trained model and training the first model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a second model, and determining the first loss value of the second model according to a preset verification sample data set are returned to execute;

[0013] When the second loss value is greater than the first loss value, training is stopped and each current low-rank parameter matrix is ​​reverted to each low-rank parameter matrix in the first model.

[0014] In a second aspect, an embodiment of the present application further provides a model fine-tuning device based on low-rank adaptation, the model fine-tuning device comprising:

[0015] A configuration module, used to configure a low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix, and the number of diagonal elements in the gating matrix is ​​the same as the rank number;

[0016] A training module, used to train the pre-trained model N0 times according to a preset training sample data set to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2;

[0017] A determination module, used to determine the influence score of each of the diagonal elements in each of the gating matrices on the pre-trained model and train the first model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a second model, and determine a first loss value of the second model according to a preset verification sample data set, where N1 is an integer greater than or equal to 2;

[0018] The determination module is further used to determine a plurality of target diagonal elements according to the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements;

[0019] An updating module, used for updating the gating matrix to which each of the target diagonal elements belongs, so as to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model;

[0020] The training module is further used to train the third model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a fourth model and determine a second loss value of the fourth model according to the verification sample data set;

[0021] A loop module, used for updating the first model to the fourth model when the second loss value is less than the first loss value, and returning to the determination module, for determining the influence score of each diagonal element in each of the gating matrices on the pre-trained model and training the first model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a second model, and determining the first loss value of the second model according to a preset verification sample data set;

[0022] A fallback module is used to stop training when the second loss value is greater than the first loss value, and to fallback each current low-rank parameter matrix to each low-rank parameter matrix in the first model.

[0023] In a third aspect, an embodiment of the present application further provides a server, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the model fine-tuning method based on low-rank adaptation as described in the first aspect are implemented.

[0024] In a fourth aspect, an embodiment of the present application further provides a storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the model fine-tuning method based on low-rank adaptation as described in the first aspect are implemented.

[0025] The embodiments of the present application provide a model fine-tuning method, device, server and storage medium based on low-rank adaptation. The model fine-tuning method determines the influence score of each diagonal element in the gating matrix of the rank number configured for the pre-trained model on the pre-trained model after training the pre-trained model N0 times, and based on the influence score of each diagonal element, updates the gating matrix to which the diagonal elements with lower influence of the pre-trained model belong, so as to adjust the rank number of the low-rank parameter matrix corresponding to the updated gating matrix. In this way, when the loss value of the pre-trained model before adjusting the rank number is greater than the loss value of the pre-trained model after adjusting the rank number, the rank number continues to be adjusted and the pre-trained model is trained based on the adjusted rank number, otherwise the training is stopped, thereby preventing the pre-trained model from overfitting and improving the parameter fine-tuning effect of the pre-trained model to ensure the accuracy of the pre-trained model after fine-tuning. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 It is a flowchart of a model fine-tuning method based on low-rank adaptation provided in an embodiment of the present application;

[0028] Figure 2 yes Figure 1 Schematic diagram of the sub-steps of the low-rank adaptation-based model fine-tuning method in ;

[0029] Figure 3 is a schematic block diagram of a model fine-tuning device based on low-rank adaptation provided in an embodiment of the present application;

[0030] Figure 4 yes Figure 3 A schematic block diagram of submodules of a model fine-tuning device based on low-rank adaptation;

[0031] Figure 5 It is a schematic block diagram of the structure of a server provided in an embodiment of the present application.

[0032] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0035] At present, the commonly used parameter fine-tuning method is mainly the low-rank adaptation method. For the weight matrix W0 of the pre-trained model, the weight change ΔW can be represented by low-rank decomposition, that is, W0+ΔW=W0+BA, where B and A are parameter matrices to be learned, and both are low-rank. The dimension of B is D*K, and the dimension of A is K*D. D is the hidden state dimension of the pre-trained model. K is the rank number, which is the most core hyperparameter of the low-rank adaptation method. However, the standard low-rank adaptation method sets the same rank number for each module of the pre-trained model, and uses the same rank number to fine-tune the parameters of the pre-trained model, which easily causes the pre-trained model to overfit, resulting in poor parameter fine-tuning effect and inability to guarantee the accuracy of the pre-trained model after fine-tuning.

[0036] To solve the above problems, the embodiments of the present application provide a model fine-tuning method, device, server and storage medium based on low-rank adaptation. The model fine-tuning method determines the influence score of each diagonal element in the gating matrix of the rank number configured for the pre-training model on the pre-training model after training the pre-training model N0 times, and based on the influence score of each diagonal element, updates the gating matrix to which the diagonal elements with lower influence of the pre-training model belong to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix, so that when the loss value of the pre-training model before adjusting the rank number is greater than the loss value of the pre-training model after adjusting the rank number, continue to adjust the rank number and train the pre-training model based on the adjusted rank number, otherwise stop training, thereby preventing the pre-training model from overfitting, improving the parameter fine-tuning effect of the pre-training model, and ensuring the accuracy of the pre-training model after fine-tuning.

[0037] The model fine-tuning method can be applied to servers or terminal devices, and the terminal devices can be mobile phones, tablet computers, laptops, desktop computers, personal digital assistants, wearable devices, etc. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0038] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0039] See also Figure 1 , Figure 1 It is a flow chart of a model fine-tuning method based on low-rank adaptation provided in an embodiment of the present application.

[0040] like Figure 1 As shown, the model fine-tuning method includes steps S101 to S108.

[0041] Step S101: configure a low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix.

[0042] In this embodiment, the pre-trained model to be fine-tuned can be an artificial intelligence model with more than 100 million model parameters. Among them, the pre-trained models to be fine-tuned include large language models (LLM), large visual models, and multimodal large models. Large language models focus on processing natural language, can understand, generate and process large-scale text data, and can include GPT models, LLaMA models, ChatGLM-6B models, and Y-34B-Chat models. Large visual models focus on computer vision tasks such as image classification, target detection, image generation, etc. They can extract information about objects, scenes, and structures from images, such as Vision Transformer models. Multimodal large models can process a variety of different types of data, such as text, images, speech, etc., and establish associations between these data, such as Pangu large models and Gemini models.

[0043] In some embodiments, the gating matrix of the rank number of the low-rank parameter matrix is ​​a diagonal matrix, and the number of diagonal elements in the gating matrix is ​​the same as the rank number of the low-rank parameter matrix. Wherein, the initial value of each diagonal element in the gating matrix is ​​1. For example, if the rank number is 8, the gating matrix is:

[0044]

[0045] In some embodiments, the rank number of the low-rank parameter matrix configured for each module in the pre-trained model to be fine-tuned is the same. The rank number of the low-rank parameter matrix can be 8, 16 or 32, and each module in the pre-trained model includes each linear layer and / or each convolutional layer in the pre-trained model. This embodiment configures the low-rank parameter matrix and the gating matrix of each module in the pre-trained model with the same rank number and the corresponding rank number, which makes it easier to configure the low-rank parameter matrix and the gating matrix of each module in the pre-trained model, thereby improving the convenience of configuration.

[0046] In some embodiments, configuring the low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and the gating matrix of the rank number of the low-rank parameter matrix may include: determining the parameter amount of each module in the pre-trained model to be fine-tuned, and determining the target rank number corresponding to each module according to the parameter amount of each module in the pre-trained model to be fine-tuned; configuring the low-rank parameter matrix and the gating matrix corresponding to the target rank number of each module in the pre-trained model. Among them, the target rank numbers corresponding to different modules may be the same or different, and the parameter amount of the module is positively correlated with the target rank number, that is, the larger the parameter amount of the module, the higher the target rank number, and the smaller the parameter amount of the module, the lower the target rank number. This embodiment can further improve the model fine-tuning effect by adaptively configuring the low-rank parameter matrix and the gating matrix of the corresponding rank number of each module through the parameter amount of each module in the pre-trained model to be fine-tuned, thereby better ensuring the accuracy of the pre-trained model after fine-tuning.

[0047] In some embodiments, instead of configuring the low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and the gating matrix of the rank number of the low-rank parameter matrix, the low-rank parameter matrix of some modules in the pre-trained model to be fine-tuned and the gating matrix of the rank number of the low-rank parameter matrix may be configured. For example, the low-rank parameter matrix of each target module in the pre-trained model to be fine-tuned and the gating matrix of the rank number of the low-rank parameter matrix are configured, the rank number of the low-rank parameter matrix configured for each target module is the same or different, and the target module is a module in the pre-trained model whose parameter amount is greater than or equal to the preset parameter amount. This embodiment can reduce the amount of computation and memory usage of model fine-tuning by configuring the low-rank parameter matrix of some modules in the pre-trained model to be fine-tuned and the gating matrix of the rank number of the low-rank parameter matrix.

[0048] Step S102: train the pre-trained model N0 times according to a preset training sample data set to update the low-rank parameter matrix N0 times to obtain a first model.

[0049] In this embodiment, N0 is an integer greater than or equal to 2, for example, N0 = 100 or 200. The preset training sample data set can be set based on actual conditions, and the embodiment of the present application does not specifically limit this. For example, the training sample data set is a knowledge question and answer question bank required for a knowledge question and answer task, including questions and answers. For another example, the training sample data set is a training sample data set required for a text-to-speech task, including sample text and annotated speech.

[0050] In some embodiments, in the process of training the pre-trained model, for the module configured with the low-rank parameter matrix, h=W0x+BΛAx is used instead of h=W0x for forward calculation, A and B are low-rank parameter matrices, W0 is the initial weight matrix, and Λ is the gating matrix. In the process of updating the model parameters of the pre-trained model, the initial weight matrix of the pre-trained model is fixed, the gradient is calculated according to the cross entropy loss, and the low-rank parameter matrices A1 and B2 are updated using the back propagation algorithm, and the gating matrix is ​​not updated.

[0051] Step S103, determine the influence score of each diagonal element in each gating matrix on the pre-trained model and train the first model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain the second model, and determine the first loss value of the second model according to the preset verification sample data set.

[0052] In this embodiment, the influence score is a score of the influence of the diagonal element on the performance (accuracy) of the pre-trained model, N1 is an integer greater than or equal to 2, N1 can be set based on actual conditions, and this embodiment of the application does not specifically limit this. For example, N1 = 50 or 150. Among them, N0 and N1 can be equal, or N0 is greater than N1, or N0 is less than N1.

[0053] It can be understood that the method used to train the first model N1 times to update the low-rank parameter matrix N1 times is the same as the method used to train the pre-trained model N0 times to update the low-rank parameter matrix N0 times, which will not be repeated here.

[0054] In some embodiments, the preset validation sample data set includes multiple validation sample data, the validation sample data includes validation samples and annotation information, and determining the first loss value of the second model according to the preset validation sample data set may include: inputting each validation sample into the second model for processing to obtain multiple prediction information, one validation sample corresponds to one prediction information; determining multiple loss values ​​of the second model according to each prediction information and the corresponding annotation information; calculating the average of the multiple loss values ​​of the second model, and determining the average of the multiple loss values ​​of the second model as the first loss value of the second model. Wherein, the cross entropy loss function can be used to determine the loss value of the second model based on the prediction information and the corresponding annotation information.

[0055] It is understandable that the verification sample and annotation information included in the verification sample data can be set based on actual conditions, and the embodiments of the present application do not specifically limit this. For example, in the text-to-speech task scenario, the verification sample data includes sample text and annotated speech, and in the knowledge question-answering task scenario, the verification sample data includes sample questions and annotated answers.

[0056] In some embodiments, Figure 2 As shown, step S103 includes: sub-steps S1031 to S1033.

[0057] Sub-step S1031, for any diagonal element α ij , determine the diagonal element α ij The corresponding first comparison model group and / or second comparison model group, the first comparison model group includes the first reference model and the second reference model, and the second comparison model group includes the third reference model and the fourth reference model.

[0058] In this embodiment, the pre-trained model includes N modules, and each module is configured with a low-rank parameter matrix and a gating matrix with a rank number of K. Then the number of low-rank parameter matrices and gating matrices is also N, and the total number of diagonal elements is N*K, and the diagonal element α ij It is the j-th diagonal element in the gating matrix configured for the i-th module, i is greater than or equal to 1 and less than or equal to the number of low-rank parameter matrices N, j is greater than or equal to 1 and less than or equal to the rank number K of the low-rank parameter matrix, and both i and j are positive integers.

[0059] In some embodiments, the first reference model, the second reference model, the third reference model, and the fourth reference model are all related to the first model. ij Set to 0, except for the diagonal elements α in the first model ij All diagonal elements except for α remain unchanged to obtain the first reference model; the diagonal element α in the first model ijSet to 1, except for the diagonal elements α in the first model ij All diagonal elements except for α remain unchanged to obtain a second reference model; and / or, the diagonal elements α in the first model are ij Set to 0, and remove all diagonal elements α from the first model. ij All diagonal elements except α are set to 0 to obtain the third reference model; the diagonal elements α in the first model are set to ij Set to 1, and remove the diagonal elements α from the first model ij All diagonal elements except are set to 0 to obtain the fourth reference model.

[0060] Sub-step S1032: determining the loss value of the first reference model and the loss value of the second reference model according to the validation sample data set, and / or determining the loss value of the third reference model and the loss value of the fourth reference model according to the validation sample data set.

[0061] In this embodiment, the verification sample data set includes multiple verification sample data, and the verification sample data includes verification samples and annotation information. According to the verification sample data set, determining the loss value of the first reference model may include: inputting each verification sample into the first reference model for processing to obtain multiple prediction information, one verification sample corresponds to one prediction information; determining the cross entropy loss value of the first reference model according to each prediction information and the corresponding annotation information; calculating the average of the multiple cross entropy loss values ​​of the first reference model, and determining the average as the loss value of the first reference model.

[0062] It should be noted that the specific method for determining the loss value of the second reference model, the loss value of the third reference model and the loss value of the fourth reference model can refer to the method for determining the loss value of the first reference model mentioned above, and will not be repeated here.

[0063] Sub-step S1033: Determine the diagonal element α according to the first loss value group and / or the second loss value group. ij The influence score on the pre-trained model.

[0064] In this embodiment, the first loss value group includes loss values ​​of the first reference model and loss values ​​of the second reference model, and the second loss value group includes loss values ​​of the third reference model and loss values ​​of the fourth reference model.

[0065] In some embodiments, the diagonal element α is determined according to the first loss value group and / or the second loss value group. ij The influence score of the pre-trained model may include: determining a first difference between the loss value of the first reference model and the loss value of the second reference model; determining a second difference between the loss value of the third reference model and the loss value of the fourth reference model; determining a diagonal element α according to the first difference and / or the second differenceij The influence score on the pre-trained model. The first difference is obtained by subtracting the loss value of the second reference model from the loss value of the first reference model, and the second difference is obtained by subtracting the loss value of the fourth reference model from the loss value of the third reference model.

[0066] In some embodiments, the diagonal element α is determined based on the first difference and / or the second difference. ij The influence score of the pre-trained model may include: determining the first difference or the second difference as a diagonal element α ij The influence score on the pre-trained model. Alternatively, the sum of the first difference and the second difference is determined as the diagonal element α ij The influence score on the pre-trained model.

[0067] Step S104: Determine multiple target diagonal elements according to the influence score of each diagonal element in each gating matrix on the pre-trained model.

[0068] In this embodiment, the influence scores of multiple target diagonal elements are all smaller than the influence scores of any diagonal elements other than the multiple target diagonal elements, that is, the influence scores of the multiple target diagonal elements are the inverses of the influence scores of the diagonal elements in the set of diagonal elements (composed of the diagonal elements in each gating matrix).

[0069] In some embodiments, according to the influence scores of each diagonal element in each gating matrix on the pre-trained model, determining multiple target diagonal elements may include: sorting each diagonal element according to the order of the influence scores of each diagonal element in each gating matrix on the pre-trained model to obtain a diagonal element queue; determining the total number of diagonal elements in the first model, multiplying the total number by a preset percentage to obtain a target number; taking the diagonal elements before or after the target number from the diagonal element queue as the target diagonal elements. Among them, the preset percentage can be set based on actual conditions, and the embodiments of the present application do not specifically limit this. For example, the preset percentage is 5% or 10%.

[0070] In some embodiments, when the influence score of the first diagonal element in the diagonal element queue is greater than the influence score of the second diagonal element (the earlier the diagonal element in the diagonal element queue, the higher the influence score), the last target number of diagonal elements in the diagonal element queue are taken as target diagonal elements; when the influence score of the first diagonal element in the diagonal element queue is less than the influence score of the second diagonal element (the earlier the diagonal element in the diagonal element queue, the lower the influence score), the first target number of diagonal elements in the diagonal element queue are taken as target diagonal elements.

[0071] For example, if the target number is 5 and the influence score of the diagonal elements closer to the front in the diagonal element queue is higher, the last 5 diagonal elements in the diagonal element queue can be taken as the target diagonal elements. For another example, if the target number is 5 and the influence score of the diagonal elements closer to the front in the diagonal element queue is lower, the first 5 diagonal elements in the diagonal element queue can be taken as the target diagonal elements.

[0072] Step S105: Update the gating matrix to which each target diagonal element belongs to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model.

[0073] In this embodiment, updating the gating matrix to which each target diagonal element belongs may include: for each target diagonal element, setting the target diagonal element in the gating matrix to which the target diagonal element belongs to 0. For example, assuming that the pre-trained model includes n modules, and each module is configured with a low-rank parameter matrix with a rank number of k and a corresponding gating matrix Λ, then the number of low-rank parameter matrices and gating matrices is also n, and the total number of diagonal elements is n*k, so the diagonal element set can be expressed as [α 11 , α 12 , ..., α 1k , α 21 , α 22 , ..., α 2k , ..., α n1 , α n2 , ..., α nk-1 , α nk ], specifically, let k = 8, then the diagonal element set can be expressed as [α 11 , α 12 , ..., α 18 , α 21 , α 22 , ..., α 28 , ..., α n1 , α n2 , ..., α n7 , α n8 ]. Let the target diagonal element be α 26 , then the target diagonal element α 26 The gating matrix Λ2 is:

[0074]

[0075] Among them, α 21 , α 22 , α 23 , α 24 , α 25 , α 26 , α 27 and α 28The value of is 1, then the target diagonal element α in the gating matrix Λ2 can be 26 Set to 0, thereby adjusting the rank of the low-rank parameter matrix corresponding to the gating matrix Λ2. The updated gating matrix Λ2 is:

[0076]

[0077] Step S106: train the third model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain the fourth model and determine the second loss value of the fourth model according to the verification sample data set.

[0078] In this embodiment, the third model is trained N1 times, and the method used to update the low-rank parameter matrix N1 times is the same as the method used to train the pre-trained model N0 times to update the low-rank parameter matrix N0 times. The specific method for determining the second loss value of the fourth model is the same as the method for determining the first loss value of the second model, which will not be repeated here.

[0079] Step S107: When the second loss value is less than the first loss value, update the first model to the fourth model.

[0080] In this embodiment, when the second loss value is less than the first loss value, the first model is updated to the fourth model, and the process returns to execute step S103, determines the influence score of each diagonal element in each gating matrix on the pre-trained model, and trains the first model N1 times based on the training sample data set to update the low-rank parameter matrix N1 times to obtain the second model, and determines the first loss value of the second model based on the preset verification sample data set.

[0081] Among them, when the second loss value is less than the first loss value, it can be determined that the performance of the model after adjusting the rank is better than the performance of the model before adjusting the rank. Therefore, the rank of the low-rank parameter matrix can be further adjusted, that is, steps S103-S106 are repeated, so as to continuously adjust the rank of the low-rank parameter matrix until the performance of the model after adjusting the rank is worse than the performance of the model before adjusting the rank, thereby preventing the pre-trained model from overfitting and improving the parameter fine-tuning effect of the pre-trained model to ensure the accuracy of the pre-trained model after fine-tuning.

[0082] Step S108: When the second loss value is greater than the first loss value, stop training and revert each current low-rank parameter matrix to each low-rank parameter matrix in the first model.

[0083] In this embodiment, when the second loss value is greater than the first loss value, it can be determined that the performance of the model after rank adjustment is worse than the performance of the model before rank adjustment, and there is no need to further adjust the rank of the low-rank parameter matrix. Therefore, the training is stopped, and each current low-rank parameter matrix is ​​rolled back to the low-rank parameter matrix corresponding to the model with better performance. This can prevent the pre-trained model from overfitting and improve the parameter fine-tuning effect of the pre-trained model to ensure the accuracy of the pre-trained model after fine-tuning.

[0084] See also Figure 3 , Figure 3 It is a schematic block diagram of a model fine-tuning device based on low-rank adaptation provided in an embodiment of the present application.

[0085] like Figure 3 As shown, a model fine-tuning device 100 based on low-rank adaptation includes:

[0086] A configuration module 110, configured to configure a low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix, and the number of diagonal elements in the gating matrix is ​​the same as the rank number;

[0087] A training module 120 is used to train the pre-trained model N0 times according to a preset training sample data set to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2;

[0088] A determination module 130 is used to determine the influence score of each of the diagonal elements in each of the gating matrices on the pre-trained model and train the first model N1 times according to the training sample data set to obtain a second model, and determine a first loss value of the second model according to a preset verification sample data set, where N1 is an integer greater than or equal to 2;

[0089] The determination module 130 is further configured to determine a plurality of target diagonal elements according to the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements;

[0090] An updating module 140, configured to update the gating matrix to which each of the target diagonal elements belongs, so as to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model;

[0091] The training module 120 is further used to train the third model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a fourth model and determine a second loss value of the fourth model according to the verification sample data set;

[0092] A loop module 150 is used to update the first model to the fourth model when the second loss value is less than the first loss value, and return to the determination module to determine the influence score of each diagonal element in each of the gating matrices on the pre-trained model and train the first model N1 times according to the training sample data set to update the low-rank parameter matrix N0 times to obtain a second model, and determine the first loss value of the second model according to a preset verification sample data set;

[0093] The fallback module 160 is used to stop training when the second loss value is greater than the first loss value, and fallback each current low-rank parameter matrix to each low-rank parameter matrix in the first model.

[0094] In some embodiments, Figure 4 As shown, the determination module 130 includes:

[0095] The model determination submodule 131 is used for determining any diagonal element α ij , determine the diagonal element α ij The corresponding first comparison model group and / or second comparison model group, the first comparison model group includes a first reference model and a second reference model, the second comparison model group includes a third reference model and a fourth reference model, the first reference model, the second reference model, the third reference model and the fourth reference model are all related to the first model, i is greater than or equal to 1 and less than or equal to the number of the low-rank parameter matrices, j is greater than or equal to 1 and less than or equal to the rank of the low-rank parameter matrix, and i and j are both positive integers;

[0096] A loss determination submodule 132, configured to determine a loss value of the first reference model and a loss value of the second reference model according to the validation sample data set, and / or determine a loss value of the third reference model and a loss value of the fourth reference model according to the validation sample data set;

[0097] The scoring submodule 133 is used to determine the diagonal element α according to the first loss value group and / or the second loss value group. ij For the influence score of the pre-trained model, the first loss value group includes the loss value of the first reference model and the loss value of the second reference model, and the second loss value group includes the loss value of the third reference model and the loss value of the fourth reference model.

[0098] In some embodiments, the model determination submodule 131 is further used to:

[0099] The diagonal element α in the first model ij Set to 0, except for the diagonal element α in the first model ij All the diagonal elements except for remain unchanged, and the first reference model is obtained;

[0100] The diagonal element α in the first model ij Set to 1, except for the diagonal element α in the first model ij All the diagonal elements except for remain unchanged, and the second reference model is obtained;

[0101] and / or,

[0102] The diagonal element α in the first model ij Set to 0, and remove all elements in the first model except the diagonal element α ij All the diagonal elements except for are set to 0, and a third reference model is obtained;

[0103] The diagonal element α in the first model ij Set to 1, and remove all the diagonal elements α from the first model. ij All the diagonal elements except are set to 0 to obtain the fourth reference model.

[0104] In some embodiments, the scoring submodule 133 is further used to:

[0105] determining a first difference between the loss value of the first reference model and the loss value of the second reference model;

[0106] determining a second difference between the loss value of the third reference model and the loss value of the fourth reference model;

[0107] Determine the diagonal element α according to the first difference and / or the second difference ij The influence score on the pre-trained model.

[0108] In some embodiments, the determining module 130 is further configured to:

[0109] According to the order of the influence scores of each diagonal element in each gating matrix on the pre-trained model, each diagonal element is sorted to obtain a diagonal element queue;

[0110] Determine the total number of the diagonal elements in the first model, and multiply the total number by a preset percentage to obtain a target number;

[0111] The first or last target number of diagonal elements are taken from the diagonal element queue as the target diagonal elements.

[0112] In some embodiments, the verification sample data set includes a plurality of verification sample data, the verification sample data includes verification samples and annotation information, and the determination module 130 is further used to:

[0113] Input each of the verification samples into the second model for processing to obtain a plurality of prediction information, wherein one verification sample corresponds to one piece of prediction information;

[0114] Determine a plurality of loss values ​​of the second model according to each of the prediction information and the corresponding annotation information;

[0115] An average of the plurality of loss values ​​of the second model is calculated, and the average of the plurality of loss values ​​of the second model is determined as a first loss value of the second model.

[0116] In some embodiments, the ranks of the low-rank parameter matrices configured for each module in the pre-trained model to be fine-tuned are the same.

[0117] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the above-described device and each module and unit can refer to the corresponding process in the aforementioned model fine-tuning method embodiment, and will not be repeated here.

[0118] The apparatus provided in the above embodiment may be implemented in the form of a computer program. The computer program may be Figure 5 Run on the server shown.

[0119] See also Figure 5 , Figure 5 It is a schematic block diagram of the structure of a server provided in an embodiment of the present application.

[0120] like Figure 5 As shown, the server includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a storage medium and an internal memory.

[0121] The storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any model fine-tuning method.

[0122] The processor is used to provide computing and control capabilities to support the operation of the entire server.

[0123] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the server to which the solution of the present application is applied. The specific server may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0124] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0125] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0126] Configure a low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix, and the number of diagonal elements in the gating matrix is ​​the same as the rank number;

[0127] According to a preset training sample data set, training the pre-trained model N0 times to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2;

[0128] Determine the influence score of each of the diagonal elements in each of the gating matrices on the pre-trained model and train the first model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a second model, and determine a first loss value of the second model according to a preset validation sample data set, where N1 is an integer greater than or equal to 2;

[0129] Determine a plurality of target diagonal elements according to the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements;

[0130] Updating the gating matrix to which each of the target diagonal elements belongs to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model;

[0131] According to the training sample data set, training the third model N1 times to update the low-rank parameter matrix N1 times to obtain a fourth model and determining a second loss value of the fourth model according to the validation sample data set;

[0132] When the second loss value is less than the first loss value, the first model is updated to the fourth model, and the steps of determining the influence score of each diagonal element in each gating matrix on the pre-trained model and training the first model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a second model, and determining the first loss value of the second model according to a preset verification sample data set are returned to execute;

[0133] When the second loss value is greater than the first loss value, the training is stopped, and each current low-rank parameter matrix is ​​reverted to each low-rank parameter matrix in the first model.

[0134] In some embodiments, when determining the influence score of each of the diagonal elements in each of the gating matrices on the pre-trained model, the processor is used to implement:

[0135] For any of the diagonal elements α ij , determine the diagonal element α ij The corresponding first comparison model group and / or second comparison model group, the first comparison model group includes a first reference model and a second reference model, the second comparison model group includes a third reference model and a fourth reference model, the first reference model, the second reference model, the third reference model and the fourth reference model are all related to the first model, i is greater than or equal to 1 and less than or equal to the number of the low-rank parameter matrices, j is greater than or equal to 1 and less than or equal to the rank number of the low-rank parameter matrix;

[0136] Determine the loss value of the first reference model and the loss value of the second reference model according to the verification sample data set, and / or determine the loss value of the third reference model and the loss value of the fourth reference model according to the verification sample data set;

[0137] Determine the diagonal element α according to the first loss value group and / or the second loss value group ijFor the influence score of the pre-trained model, the first loss value group includes the loss value of the first reference model and the loss value of the second reference model, and the second loss value group includes the loss value of the third reference model and the loss value of the fourth reference model.

[0138] In some embodiments, the processor implements determining the diagonal element α ij The corresponding first comparison model group and / or second comparison model group are used to implement:

[0139] The diagonal element α in the first model ij Set to 0, except for the diagonal element α in the first model ij All the diagonal elements except for remain unchanged, and the first reference model is obtained;

[0140] The diagonal element α in the first model ij Set to 1, except for the diagonal element α in the first model ij All the diagonal elements except for remain unchanged, and the second reference model is obtained;

[0141] and / or,

[0142] The diagonal element α in the first model ij Set to 0, and remove all elements in the first model except the diagonal element α ij All the diagonal elements except for are set to 0, and a third reference model is obtained;

[0143] The diagonal element α in the first model ij Set to 1, and remove all the diagonal elements α from the first model. ij All the diagonal elements except are set to 0 to obtain the fourth reference model.

[0144] In some embodiments, the processor determines the diagonal element α according to the first loss value group and / or the second loss value group. ij The influence score of the pre-trained model is used to achieve:

[0145] determining a first difference between the loss value of the first reference model and the loss value of the second reference model;

[0146] determining a second difference between the loss value of the third reference model and the loss value of the fourth reference model;

[0147] Determine the diagonal element α according to the first difference and / or the second difference ij The influence score on the pre-trained model.

[0148] In some embodiments, when the processor determines the multiple target diagonal elements according to the influence score of each diagonal element in each of the gating matrices on the pre-trained model, it is used to implement:

[0149] According to the order of the influence scores of each diagonal element in each gating matrix on the pre-trained model, each diagonal element is sorted to obtain a diagonal element queue;

[0150] Determine the total number of the diagonal elements in the first model, and multiply the total number by a preset percentage to obtain a target number;

[0151] The first or last target number of diagonal elements are taken from the diagonal element queue as the target diagonal elements.

[0152] In some embodiments, the verification sample data set includes a plurality of verification sample data, the verification sample data includes verification samples and annotation information, and the processor, when determining the first loss value of the second model according to the preset verification sample data set, is used to implement:

[0153] Input each of the verification samples into the second model for processing to obtain a plurality of prediction information, wherein one verification sample corresponds to one piece of prediction information;

[0154] Determine a plurality of loss values ​​of the second model according to each of the prediction information and the corresponding annotation information;

[0155] An average of the plurality of loss values ​​of the second model is calculated, and the average of the plurality of loss values ​​of the second model is determined as a first loss value of the second model.

[0156] In some embodiments, the ranks of the low-rank parameter matrices configured for each module in the pre-trained model to be fine-tuned are the same.

[0157] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and brevity of description, the specific working process of the server described above can refer to the corresponding process in the aforementioned model fine-tuning method embodiment, and will not be repeated here.

[0158] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a server (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0159] An embodiment of the present application also provides a storage medium for computer-readable storage, wherein a computer program is stored on the storage medium, wherein the computer program includes program instructions, and the method implemented when the program instructions are executed can refer to the various embodiments of the model fine-tuning method of the present application.

[0160] The storage medium may be volatile or non-volatile. The storage medium may be an internal storage unit of the server described in the foregoing embodiment, such as a hard disk or memory of the server. The storage medium may also be an external storage device of the server, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the server.

[0161] Furthermore, the storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0162] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0163] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0164] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0165] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A model fine-tuning method based on low-rank adaptation, characterized in that: include: Configure a low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix, and the number of diagonal elements in the gating matrix is ​​the same as the rank number; According to a preset training sample data set, training the pre-trained model N0 times to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2; Determine the influence score of each of the diagonal elements in each of the gating matrices on the pre-trained model and train the first model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a second model, and determine a first loss value of the second model according to a preset validation sample data set, where N1 is an integer greater than or equal to 2; Determine a plurality of target diagonal elements according to the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements; Updating the gating matrix to which each of the target diagonal elements belongs to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model; According to the training sample data set, training the third model N1 times to update the low-rank parameter matrix N1 times to obtain a fourth model and determining a second loss value of the fourth model according to the validation sample data set; When the second loss value is less than the first loss value, the first model is updated to the fourth model, and the steps of determining the influence score of each diagonal element in each gating matrix on the pre-trained model and training the first model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a second model, and determining the first loss value of the second model according to a preset verification sample data set are returned to execute; When the second loss value is greater than the first loss value, training is stopped and each current low-rank parameter matrix is ​​reverted to each low-rank parameter matrix in the first model.

2. The model fine-tuning method according to claim 1, characterized in that: Determining the influence score of each of the diagonal elements in each of the gating matrices on the pre-trained model includes: For any of the diagonal elements α ij , determine the diagonal element α ij The corresponding first comparison model group and / or second comparison model group, the first comparison model group includes a first reference model and a second reference model, the second comparison model group includes a third reference model and a fourth reference model, the first reference model, the second reference model, the third reference model and the fourth reference model are all related to the first model, i is greater than or equal to 1 and less than or equal to the number of the low-rank parameter matrices, j is greater than or equal to 1 and less than or equal to the rank of the low-rank parameter matrix, and i and j are both positive integers; Determine the loss value of the first reference model and the loss value of the second reference model according to the verification sample data set, and / or determine the loss value of the third reference model and the loss value of the fourth reference model according to the verification sample data set; Determine the diagonal element α according to the first loss value group and / or the second loss value group ij For the influence score of the pre-trained model, the first loss value group includes the loss value of the first reference model and the loss value of the second reference model, and the second loss value group includes the loss value of the third reference model and the loss value of the fourth reference model.

3. The model fine-tuning method according to claim 2, characterized in that: The determination of the diagonal element α ij The corresponding first comparison model group and / or second comparison model group include: The diagonal element α in the first model ij Set to 0, except for the diagonal element α in the first model ij All the diagonal elements except for remain unchanged, and the first reference model is obtained; The diagonal element α in the first model ij Set to 1, except for the diagonal element α in the first model ij All the diagonal elements except for remain unchanged, and the second reference model is obtained; and / or, The diagonal element α in the first model ij Set to 0, and remove all elements in the first model except the diagonal element α ij All the diagonal elements except for are set to 0, and a third reference model is obtained; The diagonal element α in the first model ij Set to 1, and remove all the diagonal elements α from the first model. ij All the diagonal elements except are set to 0 to obtain the fourth reference model.

4. The model fine-tuning method according to claim 2, characterized in that: The diagonal element α is determined according to the first loss value group and / or the second loss value group. ij The influence score on the pre-trained model, including: determining a first difference between the loss value of the first reference model and the loss value of the second reference model; determining a second difference between the loss value of the third reference model and the loss value of the fourth reference model; Determine the diagonal element α according to the first difference and / or the second difference ij The influence score on the pre-trained model.

5. The model fine-tuning method according to claim 1, characterized in that: Determining a plurality of target diagonal elements according to the influence score of each diagonal element in each of the gating matrices on the pre-trained model comprises: According to the order of the influence scores of each diagonal element in each gating matrix on the pre-trained model, each diagonal element is sorted to obtain a diagonal element queue; Determine the total number of the diagonal elements in the first model, and multiply the total number by a preset percentage to obtain a target number; The first or last target number of diagonal elements are taken from the diagonal element queue as the target diagonal elements.

6. The model fine-tuning method according to any one of claims 1 to 5, characterized in that: The verification sample data set includes a plurality of verification sample data, the verification sample data includes verification samples and annotation information, and determining the first loss value of the second model according to the preset verification sample data set includes: Input each of the verification samples into the second model for processing to obtain a plurality of prediction information, wherein one verification sample corresponds to one piece of prediction information; Determine a plurality of loss values ​​of the second model according to each of the prediction information and the corresponding annotation information; An average of the plurality of loss values ​​of the second model is calculated, and the average of the plurality of loss values ​​of the second model is determined as a first loss value of the second model.

7. The model fine-tuning method according to any one of claims 1 to 5, characterized in that: The rank of the low-rank parameter matrix configured for each module in the pre-trained model to be fine-tuned is the same.

8. A model fine-tuning device based on low-rank adaptation, characterized in that: The model fine-tuning device comprises: A configuration module, used to configure a low-rank parameter matrix of each module in the pre-trained model to be fine-tuned and a gating matrix of the rank number of the low-rank parameter matrix, wherein the gating matrix is ​​a diagonal matrix, and the number of diagonal elements in the gating matrix is ​​the same as the rank number; A training module, used to train the pre-trained model N0 times according to a preset training sample data set to update the low-rank parameter matrix N0 times to obtain a first model, where N0 is an integer greater than or equal to 2; A determination module, used to determine the influence score of each of the diagonal elements in each of the gating matrices on the pre-trained model and train the first model N1 times according to the training sample data set to obtain a second model, and determine a first loss value of the second model according to a preset verification sample data set, where N1 is an integer greater than or equal to 2; The determination module is further used to determine a plurality of target diagonal elements according to the influence scores of the diagonal elements in each of the gating matrices on the pre-trained model, wherein the influence scores of the plurality of target diagonal elements are all smaller than the influence scores of any of the diagonal elements other than the plurality of target diagonal elements; An updating module, used for updating the gating matrix to which each of the target diagonal elements belongs, so as to adjust the rank of the low-rank parameter matrix corresponding to the updated gating matrix to obtain a third model; The training module is further used to train the third model N1 times according to the training sample data set to update the low-rank parameter matrix N1 times to obtain a fourth model and determine a second loss value of the fourth model according to the verification sample data set; A loop module, used for updating the first model to the fourth model when the second loss value is less than the first loss value, and returning to the determination module, used for determining the influence score of each diagonal element in each of the gating matrices on the pre-trained model and training the first model N1 times according to the training sample data set to update the low-rank parameter matrix N0 times to obtain a second model, and determining the first loss value of the second model according to a preset verification sample data set; A fallback module is used to stop training when the second loss value is greater than the first loss value, and to fallback each current low-rank parameter matrix to each low-rank parameter matrix in the first model.

9. A server, characterized in that: The server includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the model fine-tuning method based on low-rank adaptation as described in any one of claims 1 to 7 are implemented.

10. A storage medium for computer-readable storage, characterized in that: The storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the model fine-tuning method based on low-rank adaptation as described in any one of claims 1 to 7 are implemented.

Images