Large model fine tuning method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention discloses a large model fine tuning method and device, equipment, a storage medium and a program product, and the method comprises the steps: determining a target loss function based on the fine tuning variation of a low-rank adaptation LoRA model and a large model in different rounds; performing fine tuning training on the large model embedded with the LoRA model based on the target loss function and training data; the weight of the large model is adjusted based on the fine-tuning variation of the low-rank matrix of the LoRA model in each fine-tuning training process. Therefore, the gradient of the large model weight does not need to be calculated, video memory resources can be saved, and the fine tuning rate and performance are improved to the maximum extent on the basis of saving the video memory.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to, but is not limited to, the technical field of artificial intelligence, and particularly relates to a large model fine-tuning method, device, equipment, storage medium and program product. BACKGROUND

[0002] Low-Rank Adaptation of Large Language Models (LoRA) is a low-rank adaptation technology for fine-tuning large language models, aiming to efficiently adjust the pre-trained model with a small number of trainable parameters. However, the fine-tuning effect of LoRA is poor. SUMMARY

[0003] Therefore, the embodiments of the present disclosure at least provide a large model fine-tuning method, device, equipment, storage medium and program product.

[0004] The technical solutions of the embodiments of the present disclosure are implemented as follows:

[0005] In one aspect, the present disclosure provides a large model fine-tuning method, which comprises: determining a target loss function based on the low-rank adaptation LoRA model and the fine-tuning variation of the large model at different rounds; fine-tuning training the large model embedded with the LoRA model based on the target loss function and the training data; and adjusting the weight of the large model based on the fine-tuning variation of the low-rank matrix of the LoRA model in each fine-tuning training process.

[0006] In some embodiments, the large model fine-tuning method further comprises: determining the update amount of the low-rank matrix of the LoRA model at the t+1th round and the target parameter; t is a positive integer; the low-rank matrix is adjusted based on the regularization term in the target loss function; and adjusting the weight of the large model at the tth round based on the target parameter and the update amount of the low-rank matrix at the t+1th round to obtain the weight of the large model at the t+1th round.

[0007] In some embodiments, the determination of the update amount of the low-rank matrix of the LoRA model at the t+1th round comprises: determining the first product of the two low-rank matrices of the LoRA model at the tth round; determining the second product of the two low-rank matrices of the LoRA model at the t+1th round; and determining the update amount of the low-rank matrix at the t+1th round based on the first product and the second product.

[0008] In some embodiments, the determination of the update amount of the low-rank matrix of the LoRA model at the t+1th round comprises: determining the product of the gradients of the two low-rank matrices of the LoRA model at the t+1th round; and determining the update amount of the low-rank matrix at the t+1th round based on the product of the gradients of the two low-rank matrices at the t+1th round.

[0009] In some embodiments, the target loss function is determined based on the change in the LoRA model and the large model in different rounds of fine-tuning, including: determining a regularization term based on the change in the LoRA model and the large model in different rounds of fine-tuning; and determining the target loss function based on the regularization term and the loss function of the large model.

[0010] In some embodiments, the regularization term is determined based on the change in the LoRA model and the large model in different rounds of fine-tuning, including: determining a first update expression of the LoRA model in adjacent rounds based on an update expression of a low-rank matrix of the LoRA model; the first update expression representing the change in the LoRA model in adjacent rounds; determining a second update expression of the large model in adjacent rounds based on an update expression of a weight of the large model; the second update expression representing the change in the large model in adjacent rounds; and determining the regularization term based on a difference between the first update expression and the second update expression.

[0011] In some embodiments, the regularization term is determined based on the change in the LoRA model and the large model in different rounds of fine-tuning, including: determining a gradient expression of two low-rank matrices of the LoRA model; the gradient expression of the low-rank matrix representing the change in the LoRA model in adjacent rounds; determining a second update expression of the large model in adjacent rounds based on an update expression of a weight of the large model; the second update expression representing the change in the large model in adjacent rounds; and determining the regularization term based on a difference between a product of the gradient expressions of the two low-rank matrices and the second update expression.

[0012] In another aspect, the embodiments of the present disclosure provide a large model fine-tuning device, which comprises:

[0013] a processing module configured to determine a target loss function based on the change in the LoRA model and the large model in different rounds of fine-tuning;

[0014] a training module configured to fine-tune train the large model embedded with the LoRA model based on the target loss function and training data; and the weight of the large model is adjusted based on the change in the low-rank matrix of the LoRA model in each fine-tuning training process.

[0015] In another aspect, the embodiments of the present disclosure provide a computer device, which comprises a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor implements part or all of the steps of the above method when executing the program.

[0016] In yet another aspect, the embodiments of the present disclosure provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements some or all of the steps of the above method.

[0017] In yet another aspect, the embodiments of the present disclosure provide a computer program comprising computer readable code which, when run in a computer device, causes a processor in the computer device to perform some or all of the steps of the above method.

[0018] In yet another aspect, the embodiments of the present disclosure provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program which, when read and executed by a computer, implements some or all of the steps of the above method.

[0019] In the embodiments of the present disclosure, the fine-tuning change amount of the low-rank matrix based on the LoRA model adjusts the weights of the large model, without the need to calculate the gradient of the large model weights, thereby saving the video memory resources. Meanwhile, the loss function of the large model is updated according to the fine-tuning change amount of the low-rank matrix based on the LoRA model, thereby achieving the same effect as full fine-tuning. In this way, on the basis of saving the video memory, the fine-tuning rate and performance are maximized.

[0020] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the technical solutions of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the technical solutions of the present disclosure.

[0022] Figure 1 An implementation flowchart of a large model fine-tuning method provided by the embodiments of the present disclosure Figure 1

[0023] Figure 2 An implementation flowchart of a large model fine-tuning method provided by the embodiments of the present disclosure Figure 2

[0024] Figure 3 A component structure diagram of a large model fine-tuning device provided by the embodiments of the present disclosure

[0025] Figure 4 A hardware entity diagram of a computer device provided by the embodiments of the present disclosure DETAILED DESCRIPTION

[0026] ​​To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0027] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0028] The terms “first / second / third” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first / second / third” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this disclosure.

[0030] This disclosure provides a method for fine-tuning a large model, which can be executed by a processor of a computer device. The computer device refers to a device with data processing capabilities, such as a server, laptop, tablet, desktop computer, smart TV, set-top box, or mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device).

[0031] like Figure 1 As shown, the method includes the following steps 101 to 102:

[0032] Step 101: Determine the target loss function based on the fine-tuning changes of the low-rank adaptive LoRA model and the large model in different rounds.

[0033] LoRA models are used to fine-tune the weights of large models, reducing the number of parameters during training and improving training speed. Large models refer to models with a massive number of parameters, sometimes reaching billions or even hundreds of billions. Examples of large models include those for image segmentation, speech recognition, or traffic driver detection.

[0034] The large model can include a plurality of network layers. Each network layer can embed the low-rank matrices A and B of the LoRA model. After embedding, the weight of each network layer is W = W0+AB; where W0is the original weight of the network layer. During model training, the original weight of the large model is generally frozen, and only the low-rank matrices A and B in the LoRA model are trained.

[0035] The amount of change in the fine-tuning of the LoRA model and the large model at different rounds can refer to the amount of update of the LoRA model at different rounds and the amount of update of the large model at different rounds. The target loss function refers to the loss function determined according to the amount of change in the fine-tuning of the LoRA model and the large model at different rounds, and is not the loss function of the large model itself.

[0036] In some embodiments, the specific implementation of step 101 can be: adjusting the loss function of the large model based on the amount of change in the fine-tuning of the LoRA model and the large model at different rounds to obtain the target loss function.

[0037] Step 102, based on the target loss function and the training data, fine-tuning training is performed on the large model embedded with the LoRA model; the weight of the large model is adjusted based on the amount of change in the fine-tuning of the low-rank matrix of the LoRA model in each fine-tuning training process.

[0038] In some embodiments, the specific implementation of step 102 can be: inputting the training data into the large model embedded with the LoRA model to obtain the model output; determining the difference (loss value) between the model output and the true value based on the target loss function; updating the low-rank matrices A and B of the LoRA model based on the loss value; adjusting the weight of the large model based on the updated low-rank matrices A and B; updating the adjusted weight to the large model to complete the fine-tuning training of this round; inputting the training data into the updated large model to perform the fine-tuning training of the next round, and so on, until a trained large model is obtained.

[0039] In the prior art, the gradient of the weight of the large model needs to be calculated when adjusting the weight of the large model. Based on the amount of change in the fine-tuning of the low-rank matrix of the LoRA model, the weight of the large model is adjusted, which does not need to calculate the gradient of the weight of the large model, and can maximize the fine-tuning rate and performance on the basis of saving the display memory.

[0040] In some embodiments, the weight of the large model can be adjusted based on the amount of update of the low-rank matrix of the LoRA model. Alternatively, the weight of the large model can also be adjusted based on the gradient of the two low-rank matrices of the LoRA model.

[0041] It should be noted that, since the present disclosure adjusts the weights of the large model based on the fine-tuning change amount of the low-rank matrix of the LoRA model, in order to achieve the same effect as full fine-tuning, the loss function of the large model needs to be updated according to the fine-tuning change amount of the low-rank matrix of the LoRA model. That is, adjusting the weights of the large model based on the fine-tuning change amount of the low-rank matrix of the LoRA model and updating the loss function of the large model according to the fine-tuning change amount of the low-rank matrix of the LoRA model are complementary to each other.

[0042] In the embodiments of the present disclosure, the weights of the large model are adjusted based on the fine-tuning change amount of the low-rank matrix of the LoRA model, without the need to calculate the gradient of the weights of the large model, thereby saving the memory resources. At the same time, the loss function of the large model is updated according to the fine-tuning change amount of the low-rank matrix of the LoRA model, thereby achieving the same effect as full fine-tuning. In this way, the fine-tuning rate and performance are maximized on the basis of saving memory.

[0043] The embodiments of the present disclosure provide a large model fine-tuning method, which can be executed by a processor of a computer device. As shown in Figure 2 The method comprises the following steps 201 to 203:

[0044] Step 201, determining a regularization term based on the fine-tuning change amounts of the LoRA model and the large model in different rounds.

[0045] In some embodiments, the specific implementation of step 201 can be: determining a first update amount expression of the LoRA model in an adjacent round based on an update expression of the low-rank matrix of the LoRA model; the first update amount expression represents the fine-tuning change amount of the LoRA model in the adjacent round; determining a second update amount expression of the large model in the adjacent round based on an update expression of the weights of the large model; the second update amount expression represents the fine-tuning change amount of the large model in the adjacent round; and determining the regularization term based on the difference between the first update amount expression and the second update amount expression.

[0046] The update expression of the low-rank matrix A of the LoRA model can be: The update expression of the low-rank matrix B of the LoRA model can be: wherein, A t is the low-rank matrix A of the LoRA model in the tthround, A t+1 is the low-rank matrix A of the LoRA model in the t+1thround, A t T is the transpose matrix of the low-rank matrix A t , η is the learning rate, is the loss function L to the weight matrix W in the tthroundt the gradient of the low-rank matrix B t Btis the low-rank matrix B of the LoRA model at the t-th round, Bt t+1 Bt+1is the low-rank matrix B of the LoRA model at the t+1-th round, Bt+1 t T is the transpose matrix of the low-rank matrix B t .

[0047] The first update amount expression refers to the update amount expression of the LoRA model at adjacent rounds. The first update amount expression can be:

[0048]

[0049] The second update amount expression refers to the update amount expression of the large model at adjacent rounds. The second update amount expression (full fine-tuning expression) can be: W t＝1 is the weight matrix at the t+1-th round.

[0050] By comparing the first update amount expression with the second update amount expression, it is found that the closer the result is to the unit matrix I, the closer the first update amount expression is to the full fine-tuning. Therefore, according to and the unit matrix I, a regularization term R is determined to make the fine-tuning result close to the full fine-tuning result.

[0051] In some embodiments, if the weight of the large model is adjusted by the update amount of the LoRA model at adjacent rounds, the corresponding regularization term is:

[0052] In some embodiments, the specific implementation of step 201 can also be: determining a gradient expression of two low-rank matrices of the LoRA model; the gradient expression of the low-rank matrix represents the fine-tuning change amount of the LoRA model at adjacent rounds; based on the update expression of the weight of the large model, a second update amount expression of the large model at adjacent rounds is determined; the second update amount expression represents the fine-tuning change amount of the large model at adjacent rounds; based on the difference between the product of the gradient expressions of the two low-rank matrices and the second update amount expression, the regularization term is determined.

[0053] The gradient expression of the low-rank matrix A can be: wherein, is the gradient of the low-rank matrix A. The gradient expression of the low-rank matrix B can be: wherein, is the gradient of the low-rank matrix A. The product of the gradient expressions of the two low-rank matrices can be:

[0054] The product of the gradient expression of the two low-rank matrices is compared with the second update expression, and it is found that The closer the result is to the unit matrix I, the closer the product of the gradient expression of the two low-rank matrices is to the full-amount fine-tuning. Therefore, according to and the unit matrix I, the regularization term R is determined to make the fine-tuning result close to the full-amount fine-tuning result.

[0055] In some embodiments, if the weights of the large model are adjusted by the gradients of the two low-rank matrices of the LoRA model, the corresponding regularization term is:

[0056] Step 202, determining the target loss function based on the regularization term and the loss function of the large model.

[0057] Here, the above steps 201 to 202 correspond to the aforementioned step 101, and in implementation, the specific embodiments of the aforementioned step 101 can be referred to.

[0058] In some embodiments, the specific implementation of step 202 can be: performing addition operation on the regularization term and the loss function of the large model to obtain the target loss function. The expression of the target loss function can be: L = L1 + R; where L1 is the loss function of the large model, and R is the regularization term.

[0059] It should be noted that if the weights of the large model are adjusted by the update of the adjacent round of the LoRA model, the corresponding target loss function is: If the weights of the large model are adjusted by the gradients of the two low-rank matrices of the LoRA model, the corresponding target loss function is:

[0060] Step 203, fine-tuning training the large model embedded with the LoRA model based on the target loss function and the training data; the weights of the large model are adjusted based on the fine-tuning change amount of the low-rank matrix of the LoRA model in each fine-tuning training process.

[0061] Here, the above step 203 corresponds to the aforementioned step 102, and in implementation, the specific embodiments of the aforementioned step 102 can be referred to.

[0062] In some embodiments, the large model fine-tuning method provided by the embodiments of the present disclosure further includes steps 204 to 205:

[0063] Step 204, determining the update of the low-rank matrix of the LoRA model in the t+1 round and the target parameter; t is a positive integer; the low-rank matrix is adjusted based on the regularization term in the target loss function.

[0064] In some embodiments, if the weight of the large model is adjusted by the LoRA model through the update amount of adjacent rounds, the update amount here can refer to the difference between the product of the two low-rank matrices of the adjacent rounds.

[0065] In some embodiments, if the weight of the large model is adjusted by the LoRA model through the product of the gradients of the two low-rank matrices, the update amount here can refer to the product of the gradients of the two low-rank matrices of the current round.

[0066] In some embodiments, the specific implementation of step 204 can be: determining a first product of the two low-rank matrices of the LoRA model in the tth round; determining a second product of the two low-rank matrices of the LoRA model in the t+1th round; and determining an update amount of the low-rank matrix of the t+1th round based on the first product and the second product.

[0067] Here, the update amount of the low-rank matrix of the t+1th round is: △AB=A t+1 B t+1 -A t B t .

[0068] In some embodiments, the specific implementation of step 204 can also be: determining a product of the gradients of the two low-rank matrices of the LoRA model in the t+1th round; and determining an update amount of the low-rank matrix of the t+1th round based on the product of the gradients.

[0069] Here, the update amount of the low-rank matrix of the t+1th round is:

[0070] It should be noted that if the weight of the large model is adjusted by the LoRA model through the update amount of adjacent rounds, in order to achieve the effect of full fine-tuning, the low-rank matrices A and B need to satisfy the regularization term If the weight of the large model is adjusted by the LoRA model through the product of the gradients of the two low-rank matrices, in order to achieve the effect of full fine-tuning, the low-rank matrices A and B need to satisfy the regularization term

[0071] Step 205: adjusting the weight of the large model in the tth round based on the target parameter and the update amount of the low-rank matrix of the t+1th round, to obtain the weight of the large model in the t+1th round.

[0072] Here, the above steps 203 to 205 correspond to the aforementioned step 103, and in implementation, the specific implementation of the aforementioned step 103 can be referred to.

[0073] In some embodiments, the specific implementation of step 205 can be: multiplying the target parameter and the update amount of the low-rank matrix of the t+1th round to obtain an operation result; and performing addition processing on the operation result and the weight of the large model in the tth round to obtain the weight of the large model in the t+1th round. The calculation expression of the weight of the large model in the t+1th round can be: W t+1 t + λ update amount; wherein λ is a hyperparameter.

[0074] In some embodiments, if the weight of the large model is adjusted by the update amount of the LoRA model in adjacent rounds, the calculation expression of the weight of the large model in the t+1th round can be: W t+1 t + λ △ AB.

[0075] In some embodiments, if the weight of the large model is adjusted by the gradient of the two low-rank matrices of the LoRA model, the calculation expression of the weight of the large model in the t+1th round can be:

[0076] In the embodiments of the present disclosure, the weight of the large model is adjusted based on the fine-tuning change amount of the low-rank matrix of the LoRA model, without the need to calculate the gradient of the weight of the large model, thereby saving the video memory resource. At the same time, the loss function of the large model is updated according to the fine-tuning change amount of the low-rank matrix of the LoRA model, thereby achieving the same effect as full fine-tuning. In this way, on the basis of saving the video memory, the fine-tuning rate and performance are maximized.

[0077] The application of the large model fine-tuning method provided in the embodiments of the present disclosure in actual scenarios is described below.

[0078] The LoRA fine-tuning only updates parameters A and B, because the parameter update is relatively small, and the model performance is not up to standard many times. Therefore, the present disclosure proposes an optimization method, which directly updates the weight W of the large model through the update amount of A and B without calculating the gradient of the weight W of the large model, and uses the same video memory overhead of the LoRA fine-tuning to update W in the process of model training, thereby achieving the same effect as full fine-tuning.

[0079] The weight W of the large model has the following two update methods:

[0080] The first update method is: W t+1 = W t + λ △ AB; △ AB = A t+1 B t+1 - A t B t ; wherein λ is a hyperparameter.

[0081] The second update method is:​​

[0082] The derivation principle of the first updating method is as follows:

[0083] The full fine-tuning updates the formula to:

[0084] The updating formula of the low-rank matrix A and B of the LoRA model is: Then,

[0085] Compared with full fine-tuning, the closer the result of the latter part to the unit matrix I, the closer it is to full fine-tuning. Therefore, the present disclosure adds a regularization term and optimizes the loss function, and the optimized loss function is: L = L1 + R, wherein L1 is the original loss function.

[0086] The derivation principle of the second updating method is as follows:

[0087] The full fine-tuning updates the formula to:

[0088] The product of the gradients of the low-rank matrices A and B is:

[0089] Compared with full fine-tuning, the closer to the unit matrix, the closer to full fine-tuning. Therefore, the present disclosure adds a regularization term and optimizes the loss function, and the optimized loss function is: L = L1 + R, wherein L1 is the original loss function.

[0090] The present disclosure tests the above two updating methods on the afqmc dataset using the llama-7B model, and obtains that the accuracy (Accuracy, acc) of the above two updating methods can reach 81.57%, while the accuracy acc of LoRA fine-tuning is 80.69%. Therefore, the updating method provided by the present disclosure is obviously superior to LoRA fine-tuning.

[0091] Based on the foregoing embodiments, the embodiments of the present disclosure provide a large model fine-tuning device, which comprises various units and various modules included in the units, and can be implemented by a processor in a computer device. Of course, it can also be implemented by a specific logic circuit. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA).

[0092] Figure 3 The composition structure diagram of the large model fine-tuning device provided by the embodiments of the present disclosure is shown in FIG. 3, which comprises an obtaining module 310 and a training module 320, wherein: Figure 3

[0093] The processing module 310 is configured to determine a target loss function based on the fine-tuning change amount of the LoRA model and the large model in different rounds.

[0094] The training module 320 is configured to fine-tune train the large model embedded with the LoRA model based on the target loss function and training data. The weight of the large model is adjusted based on the fine-tuning change amount of the low-rank matrix of the LoRA model in each fine-tuning training process.

[0095] In some embodiments, the large model fine-tuning device further comprises an adjusting module 330 configured to determine an update amount of the low-rank matrix of the LoRA model in the t+1th round and a target parameter; t is a positive integer; the low-rank matrix is adjusted based on a regularization term in the target loss function; and the weight of the large model in the tth round is adjusted based on the target parameter and the update amount of the low-rank matrix in the t+1th round to obtain the weight of the large model in the t+1th round.

[0096] In some embodiments, the adjusting module is further configured to determine a first product of two low-rank matrices of the LoRA model in the tth round, determine a second product of two low-rank matrices of the LoRA model in the t+1th round, and determine the update amount of the low-rank matrix in the t+1th round based on the first product and the second product.

[0097] In some embodiments, the adjusting module is further configured to determine a product of gradients of two low-rank matrices of the LoRA model in the t+1th round, and determine the update amount of the low-rank matrix in the t+1th round based on the product of gradients.​

[0098] In some embodiments, the processing module 310 is further configured to determine a regularization term based on the amount of fine-tuning change of the LoRA model and the large model in different rounds, and determine the target loss function based on the regularization term and a loss function of the large model.

[0099] In some embodiments, the processing module 310 is further configured to determine a first update amount expression of the LoRA model in an adjacent round based on an update expression of a low-rank matrix of the LoRA model, the first update amount expression representing the amount of fine-tuning change of the LoRA model in the adjacent round, determine a second update amount expression of the large model in the adjacent round based on an update expression of a weight of the large model, the second update amount expression representing the amount of fine-tuning change of the large model in the adjacent round, and determine the regularization term based on a difference between the first update amount expression and the second update amount expression.

[0100] In some embodiments, the processing module 310 is further configured to determine a gradient expression of two low-rank matrices of the LoRA model, the gradient expression of the low-rank matrix representing the amount of fine-tuning change of the LoRA model in an adjacent round, determine a second update amount expression of the large model in the adjacent round based on an update expression of a weight of the large model, the second update amount expression representing the amount of fine-tuning change of the large model in the adjacent round, and determine the regularization term based on a difference between a product of the gradient expressions of the two low-rank matrices and the second update amount expression.

[0101] The above device embodiments are similar to the description of the above method embodiments, and have similar beneficial effects as the method embodiments. In some embodiments, the device provided by the embodiments of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments. For technical details not disclosed in the device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure.

[0102] It should be noted that, in the embodiments of the present disclosure, if the large model fine-tuning method described above is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product in essence or the part that contributes to the related art, which is stored in a storage medium and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the embodiments of the present disclosure. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read Only Memory, ROM), a magnetic disk or an optical disk, and various media that can store program codes. Thus, the embodiments of the present disclosure are not limited to any specific hardware, software or firmware, or any combination of hardware, software and firmware.

[0103] The embodiments of the present disclosure provide a computer device, including a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor implements part or all of the steps of the above method when executing the program.

[0104] The embodiments of the present disclosure provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement part or all of the steps of the above method. The computer readable storage medium can be transitory or non-transitory.

[0105] The embodiments of the present disclosure provide a computer program, which includes computer readable code, and when the computer readable code runs in a computer device, a processor in the computer device executes part or all of the steps of the above method.

[0106] The embodiments of the present disclosure provide a computer program product, which includes a non-transitory computer readable storage medium storing a computer program, and when the computer program is read and executed by a computer, part or all of the steps of the above method are implemented. The computer program product can be implemented by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

[0107] It should be noted that the above description of the various embodiments tends to emphasize differences between the various embodiments, and the same or similar elements can be mutually referred to. The above description of the device, storage medium, computer program and computer program product embodiments is similar to the description of the method embodiments, and has similar beneficial effects as the method embodiments. For technical details of the device, storage medium, computer program and computer program product embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure.

[0108] It should be noted that, Figure 4 A hardware entity diagram of a computer device in the embodiments of the present disclosure is shown in FIG. 4, which includes a processor 401, a communication interface 402 and a memory 403, wherein: Figure 4 The processor 401 generally controls the overall operation of the computer device 400.

[0109] The processor 401 generally controls the overall operation of the computer device 400.

[0110] The communication interface 402 can enable the computer device to communicate with other terminals or servers through a network.

[0111] The memory 403 is configured to store instructions and applications executable by the processor 401, and can also cache data to be processed by the processor 401 and modules in the computer device 400 (for example, image data, audio data, voice communication data and video communication data) that have been processed or are to be processed, which can be implemented by FLASH or Random Access Memory (RAM). The processor 401, the communication interface 402 and the memory 403 can transmit data through the bus 404.

[0112] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present disclosure. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present disclosure, the size of the serial number of each step / process does not mean the order of execution, and the execution order of each step / process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure. The above serial number of the embodiments of the present disclosure is only for description, not representing the advantages or disadvantages of the embodiments.

[0113] It should be noted that, in the present document, the terms "comprising", "containing" or any other similar term are intended to encompass non-exclusive inclusion, such that processes, methods, articles, or apparatuses that comprise a list of elements are not limited to those elements, but can also include other elements not expressly listed, or also include elements inherent in such processes, methods, articles, or apparatuses. Without further limitation, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0114] In several embodiments provided by the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other manners. The above described device embodiments are merely schematic. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling, or direct coupling or communication connection between the components can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or in other forms.

[0115] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units; they can be located in one place, or distributed on a plurality of network units; and some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0116] In addition, each functional unit in each embodiment of the present disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in the form of hardware, or in the form of hardware plus software function unit.

[0117] Those of ordinary skill in the art can understand that all or part of the above-described method embodiments can be completed by a program instructing related hardware, and the foregoing program can be stored in a computer readable storage medium, and when the program is executed, the steps of the above-described method embodiments are executed; and the foregoing storage medium includes mobile storage devices, read only memory (ROM), magnetic discs or optical discs, and various media that can store program codes.

[0118] Alternatively, the above-mentioned integrated units of the present disclosure, if implemented in the form of software function modules and sold or used as independent products, can also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the various embodiments of the present disclosure. The aforementioned storage medium includes: mobile storage devices, ROM, magnetic disks or optical disks, and various media that can store program codes.

[0119] The above is only an embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present disclosure, which should be covered within the protection scope of the present disclosure.

Claims

1. A method for fine-tuning a large model, characterized in that, The large model fine-tuning method includes: The target loss function is determined based on the fine-tuning changes of the low-rank adaptive LoRA model and the large model in different rounds. The large model embedded with the LoRA model is fine-tuned based on the target loss function and training data; the weights of the large model are adjusted based on the fine-tuning change of the low-rank matrix of the LoRA model during each fine-tuning training process.

2. The large model fine-tuning method according to claim 1, characterized in that, The large model fine-tuning method also includes: Determine the update amount of the low-rank matrix of the LoRA model in round t+1, as well as the target parameters; t is a positive integer; the low-rank matrix is ​​adjusted based on the regularization term in the target loss function; The weights of the large model in round t are adjusted based on the target parameters and the update amount of the low-rank matrix in round t+1 to obtain the weights of the large model in round t+1.

3. The large model fine-tuning method according to claim 2, characterized in that, Determining the update amount of the low-rank matrix of the LoRA model in round t+1 includes: Determine the first product of the two low-rank matrices in the LoRA model at round t; Determine the second product of the two low-rank matrices in the (t+1)th round of the LoRA model; Based on the first product and the second product, the update amount of the low-rank matrix in the (t+1)th round is determined.

4. The large model fine-tuning method according to claim 2, characterized in that, Determining the update amount of the low-rank matrix of the LoRA model in round t+1 includes: Determine the product of the gradients of the two low-rank matrices of the LoRA model in round t+1; Based on the product of the gradients, the update amount of the low-rank matrix in the (t+1)th round is determined.

5. The method for fine-tuning a large model according to any one of claims 1 to 4, characterized in that, The determination of the target loss function based on the fine-tuning changes of the low-rank adaptive LoRA model and the large model in different rounds includes: Based on the fine-tuning changes of the LoRA model and the large model in different rounds, the regularization term is determined; The target loss function is determined based on the regularization term and the loss function of the large model.

6. The large model fine-tuning method according to claim 5, characterized in that, The determination of the regularization term based on the fine-tuning changes of the LoRA model and the large model in different rounds includes: Based on the update expression of the low-rank matrix of the LoRA model, the first update expression of the LoRA model in adjacent rounds is determined; the first update expression represents the amount of fine-tuning change of the LoRA model in adjacent rounds. Based on the update expression of the weights of the large model, a second update expression for the large model in adjacent rounds is determined; the second update expression represents the amount of fine-tuning change of the large model in adjacent rounds. The regular expression is determined based on the difference between the first update expression and the second update expression.

7. The large model fine-tuning method according to claim 5, characterized in that, The determination of the regularization term based on the fine-tuning changes of the LoRA model and the large model in different rounds includes: Determine the gradient expressions of the two low-rank matrices of the LoRA model; the gradient expressions of the low-rank matrices characterize the amount of fine-tuning change of the LoRA model in adjacent rounds; Based on the update expression of the weights of the large model, a second update expression for the large model in adjacent rounds is determined; the second update expression represents the amount of fine-tuning change of the large model in adjacent rounds. The regularization term is determined based on the difference between the product of the gradient expressions of the two low-rank matrices and the expression of the second update quantity.

8. A large model fine-tuning device, characterized in that, The large model fine-tuning device includes: The processing module is configured to determine the target loss function based on the fine-tuning changes of the low-rank adaptive LoRA model and the large model in different rounds. The training module is configured to fine-tune a large model embedded with the LoRA model based on the target loss function and training data; the weights of the large model are adjusted based on the fine-tuning change of the low-rank matrix of the LoRA model during each fine-tuning training process.

9. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes a non-transitory computer-readable storage medium storing a computer program, which, when read and executed by a computer, implements the steps of the method according to any one of claims 1 to 7.

Images