Large model parameter fine tuning method, device, equipment, medium and product

Abstract

The invention provides a large model parameter fine adjustment method and device, equipment, a medium and a product, and belongs to the technical field of artificial intelligence. The method comprises the steps that a data set of a target task is acquired, and a training set is obtained according to the data set; performing parameter fine tuning on the large model according to the training set, a preset fine tuning strategy and an optimization algorithm; wherein the fine tuning strategy comprises performing low-dimensional projection on the gradient of the large model based on a random subspace iteration method. According to the method, low-dimensional projection is carried out on the gradient of the large model through the random subspace iteration method, parameter updating is carried out in the low-dimensional projection space, and compared with a full-parameter fine adjustment method, the needed video memory is reduced, and the fine adjustment time is shortened.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large model parameter fine-tuning method, device, equipment, medium and product. Background Art

[0002] Large models have shown good results in many tasks and have good robustness even for tasks they have not been trained on. In order to make the model work better in a specific task, it is generally fine-tuned using a dataset for that specific task. However, fine-tuning all the parameters of a large model requires a huge amount of video memory and time.

[0003] At present, the efficient fine-tuning method of large model parameters focuses on the amount of video memory occupied during the fine-tuning process, and the emphasis is on how to reduce the amount of video memory occupied. If too many samples need to be trained, the time cost of fine-tuning will be very high. Therefore, it is necessary to provide a technical solution to accelerate the fine-tuning process and reduce the time cost of fine-tuning. Summary of the invention

[0004] The present invention provides a large model parameter fine-tuning method, device, equipment, medium and product, which can accelerate the calculation of the fine-tuning process and reduce the time cost of fine-tuning.

[0005] The present invention provides a large model parameter fine-tuning method, comprising: Obtain a data set for a target task, and obtain a training set based on the data set; Fine-tune the parameters of the large model according to the training set, the preset fine-tuning strategy and the optimization algorithm; The fine-tuning strategy includes performing low-dimensional projection on the gradient of the large model based on a random subspace iteration method.

[0006] As an embodiment, fine-tuning parameters of the large model according to the training set, the preset fine-tuning strategy and the optimization algorithm includes: Determining the gradient of the large model based on the training set; Performing low-dimensional projection on the gradient of the large model based on the random subspace iteration method to obtain a low-dimensional projection gradient; Updating the low-dimensional projected gradient based on the optimization algorithm; Perform high-dimensional projection and gradient update on the updated low-dimensional projected gradient to obtain the updated gradient; Fine-tuning the parameters of the large model according to the updated gradient; Repeat the above steps until the parameters of the large model after fine-tuning meet the preset conditions.

[0007] As an embodiment, the large model includes a large model based on a multi-head attention mechanism, the gradient of the large model includes the gradient of each attention head, the gradient of each attention head includes the gradient of the projected query vector and the gradient of the projection key vector, and the gradient of any attention head is used to determine the low-dimensional projection space shared by the projected query vector and the projection key vector.

[0008] As an embodiment, the step of performing low-dimensional projection on the gradient of the large model based on the random subspace iteration method to obtain the low-dimensional projection gradient includes: If the current gradient step is not the starting point of a period, the orthogonal basis of the previous gradient step is used as the orthogonal basis of the current gradient step, otherwise the next step is executed; If the gradient of the large model to be projected is the gradient of the projected query vector or the gradient of the projected key vector, an orthogonal basis of the current number of gradient steps is obtained based on the random subspace iteration method and the gradient of any attention head; otherwise, an orthogonal basis of the current number of gradient steps is obtained based on the random subspace iteration method and the gradient of the large model; The low-dimensional projected gradient is obtained according to the orthogonal basis of the current gradient step number and the gradient of the large model.

[0009] As an embodiment, after performing low-dimensional projection on the gradient of the large model based on the random subspace iteration method to obtain the low-dimensional projection gradient, the method further includes: Based on the orthogonal basis of the current gradient step number and the gradient of the large model, obtaining a reconstructed gradient projected back to the high-dimensional projection space; Obtaining a gradient residual according to the reconstructed gradient and the gradient of the large model; Correspondingly, fine-tuning the parameters of the large model according to the updated gradient includes: The parameters of the large model are fine-tuned according to the updated gradient and the gradient residual.

[0010] As an embodiment, obtaining the reconstructed gradient projected back to the high-dimensional projection space based on the orthogonal basis of the current gradient step number and the gradient of the large model includes: After a preset number of gradient steps, determining the absolute value of the first-order moment of the gradient of the large model; According to the absolute value of the first-order moment, a certain proportion of the gradient of the large model is selected as the target gradient; A sparse reconstructed gradient is obtained according to an orthogonal basis of the current gradient step number and the target gradient.

[0011] The present invention also provides a large model parameter fine-tuning device, comprising: An acquisition module is used to acquire a data set of a target task and obtain a training set based on the data set; A fine-tuning module, used to fine-tune the parameters of the large model according to the training set, a preset fine-tuning strategy and an optimization algorithm; The fine-tuning strategy includes performing low-dimensional projection on the gradient of the large model based on a random subspace iteration method.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the large model parameter fine-tuning method described above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the large model parameter fine-tuning methods described above.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the large model parameter fine-tuning method as described above is implemented.

[0015] The large model parameter fine-tuning method, device, equipment, medium and product provided by the present invention perform low-dimensional projection on the gradient of the large model through a random subspace iteration method, and update the parameters in the low-dimensional projection space. Compared with the full parameter fine-tuning method, it reduces the required video memory and the fine-tuning time. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is one of the flow charts of the large model parameter fine-tuning method provided by the present invention.

[0018] Figure 2 This is the second flow chart of the large model parameter fine-tuning method provided by the present invention.

[0019] Figure 3 It is a structural schematic diagram of the large model parameter fine-tuning device provided by the present invention.

[0020] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] It should be noted that all actions of acquiring signals, information or data in the present invention are performed in compliance with the corresponding data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0023] The large model parameter fine-tuning method, device, equipment, medium and product provided by the present invention are described in detail below with reference to the accompanying drawings.

[0024] Figure 1 This is one of the flow charts of the large model parameter fine-tuning method provided by the present invention, such as Figure 1 As shown, the present invention provides a large model parameter fine-tuning method, including steps S100 to S200.

[0025] Step S100, obtaining a data set of a target task, and obtaining a training set according to the data set.

[0026] The large model in the present invention refers to a large language model, and the target tasks include but are not limited to text summarization tasks, common sense reasoning tasks, mathematical reasoning tasks and other related scenario tasks of large language model applications, and the data set is the data set required for the above target tasks. The method of obtaining the training set according to the data set is not limited in the present invention and will not be described in detail.

[0027] Step S200, fine-tuning the parameters of the large model according to the training set, the preset fine-tuning strategy and the optimization algorithm. The fine-tuning strategy includes low-dimensional projection of the gradient of the large model based on the random subspace iteration method.

[0028] The gradient of the large model refers to the gradient of the loss function relative to the parameters of the large model. Specifically, the training samples of the training set are input into the large model for forward propagation to generate the predicted output, and then the loss (such as cross entropy loss, mean square error, etc.) is calculated based on the predicted output and the true label. The gradient of the loss function relative to the parameters of the large model can be calculated through the back propagation algorithm.

[0029] The optimization algorithm can be used to maintain a moving average of the first-order moment version (mean) of the gradient and the second-order moment version (variance) of the gradient, and adjust the learning rate of each parameter based on these averages. The gradient is multiplied by the learning rate and subtracted from the current parameter to update the model parameters, finding the direction of minimizing the loss and adjusting the model parameters to reduce the loss.

[0030] The random subspace iteration method uses power iteration and QR decomposition to obtain the matrix The dimension is ( n , q ) , the principle of random subspace iteration method is explained below.

[0031] extract number of rows m and number of columns n , create a shape of ( n , q ) , The elements in follow the standard normal distribution, q is the dimension of the low-dimensional projection space.

[0032] The following steps are iteratively performed until the preset iteration period is reached: First, the orthogonal matrix in the tth iteration is With the matrix and Multiply the transpose of to get the intermediate matrix , as shown in formula (1):

[0033] Then, we can calculate the Perform QR decomposition to obtain an orthogonal matrix and the upper triangular matrix R , orthogonal matrix Can be used as the first t+ Approximate basis for 1 iteration.

[0034] Among them, the implementation steps of QR decomposition are to transform the matrix Convert to upper triangular matrix , and construct an orthogonal matrix First, through the Householder reflection matrix Perform a series of transformations, each of which aims to transform Convert a column of to a column with only non-zero elements at the top, thus gradually building an upper triangular matrix At the same time, each reflection transformation is recorded and accumulated to form the final orthogonal matrix Finally, after a series of transformations, the original matrix Successfully decomposed into and The product of is an orthogonal matrix, is an upper triangular matrix.

[0035] It can be understood that the present invention performs low-dimensional projection on the gradient of the large model through the random subspace iteration method, and updates the parameters in the low-dimensional projection space, thereby reducing the time required for the fine-tuning process while ensuring that the amount of fine-tuning parameters does not increase additionally. Compared with the full parameter fine-tuning method, it not only reduces the required video memory, but also reduces the fine-tuning time.

[0036] Based on the above embodiment, as an optional embodiment, the method of fine-tuning the parameters of the large model according to the training set, the preset fine-tuning strategy and the optimization algorithm includes the following steps.

[0037] Step S210, determining the gradient of the large model according to the training set. The gradient of the large model includes the gradient of each functional module of the large model, and the type of the gradient is determined based on the network structure of the large model, which is not limited in the present invention.

[0038] Optionally, the number of gradient steps for parameter fine-tuning is initialized before step S210.

[0039] Step S220, based on the random subspace iteration method, the gradient of the large model is projected in low dimensions to obtain a low-dimensional projected gradient. The random subspace iteration method is used to project the gradient of the large model in low dimensions to obtain the original gradient in dimension ( n , q ), which is obtained by power iteration and QR decomposition, and its essence is an orthogonal matrix.

[0040] The calculation formula for the low-dimensional projected gradient is as follows:

[0041] in, is the low-dimensional projected gradient, is an approximate basis, is the gradient of the large model, , is the coefficient matrix of the large model, and the elements of the coefficient matrix are the parameters of the large model.

[0042] Step S230, updating the low-dimensional projected gradient based on the optimization algorithm, and updating the low-dimensional projected gradient based on the optimization algorithm can be expressed as .

[0043] Optionally, the optimization algorithm may be an Adam algorithm or an AdamW algorithm, or other optimization algorithms, which is not limited in the present invention.

[0044] Step S240, performing high-dimensional projection and gradient update on the updated low-dimensional projected gradient to obtain an updated gradient; the calculation formulas for the high-dimensional projection and gradient update are shown in formulas (3) and (4), respectively.

[0045]

[0046] in, is the high-dimensional matrix of the updated low-dimensional projected gradient, is the learning rate.

[0047] Step S250, fine-tuning the parameters of the large model according to the update gradient; specifically, substituting the update gradient into the parameter update formula corresponding to the optimization algorithm can achieve fine-tuning of the parameters of the large model.

[0048] Step S260, repeat the above steps until the parameters of the large model after fine-tuning meet the preset conditions.

[0049] It can be understood that the present invention can quickly update the coefficient matrix through the quickly constructed approximate basis and optimization algorithm, thereby achieving fast fine-tuning of large models.

[0050] Based on the above embodiment, as an optional embodiment, the large model includes a large model based on a multi-head attention mechanism, the gradient of the large model includes the gradient of each attention head, the gradient of each attention head includes the gradient of the projected query vector and the gradient of the projection key vector, and the gradient of any attention head is used to determine the low-dimensional projection space shared by the projected query vector and the projection key vector.

[0051] When calculating the projection space, do not select the gradients of all attention head projection query vectors and projection key vectors , but randomly select The corresponding gradient calculation projection space of an attention head in .

[0052] It can be understood that multiple projection query vectors and projection key vectors at the same layer of a large model share the same projection space, which can effectively reduce the amount of calculation.

[0053] Figure 2 This is the second flow chart of the large model parameter fine-tuning method provided by the present invention. Figure 2 As shown, based on the above embodiment, as an optional embodiment, the gradient of the large model is low-dimensionally projected based on the random subspace iteration method to obtain the low-dimensional projection gradient, including the following steps.

[0054] Step S221, if the current gradient step is not the starting point of a period, the orthogonal basis of the previous gradient step is used as the orthogonal basis of the current gradient step, otherwise, the next step is executed.

[0055] Suppose the current gradient step is t and the period is T. The determination of whether the current gradient step is the starting point of the period can be calculated by t%T, where t%T represents the modulus operation (remainder) of the variable t. Suppose the orthogonal basis of the previous gradient step is , the orthogonal basis of the current gradient step can be expressed as , taking the orthogonal basis of the previous gradient step as the orthogonal basis of the current gradient step can be expressed as .

[0056] Step S222: If the gradient of the large model to be projected is the gradient of the projected query vector or the gradient of the projected key vector, an orthogonal basis of the current gradient step is obtained based on the random subspace iteration method and the gradient of any attention head; otherwise, an orthogonal basis of the current gradient step is obtained based on the random subspace iteration method and the gradient of the large model. The gradient of the large model of the current gradient step can be expressed as .

[0057] Step S223, according to the orthogonal basis of the current gradient step number and the gradient of the large model, obtain the low-dimensional projected gradient. Substituting the orthogonal basis of the current gradient step number and the gradient of the large model into formula (2), the low-dimensional projected gradient of the current gradient step number can be obtained. .

[0058] It can be understood that the amount of calculation can be reduced by judging the starting point of the cycle based on the current gradient step number.

[0059] Based on the above embodiment, as an optional embodiment, after performing low-dimensional projection on the gradient of the large model based on the random subspace iteration method to obtain the low-dimensional projection gradient, the following steps are also included.

[0060] Step S224, based on the orthogonal basis of the current gradient step number and the gradient of the large model, obtain the reconstructed gradient projected back to the high-dimensional projection space.

[0061] The original gradient of the current gradient step is , that is, the high-dimensional gradient obtained during the back propagation process; the reconstructed gradient is the low-dimensional gradient by reconstructing the projection matrix The gradient obtained after transformation, the reconstructed gradient can be expressed as .

[0062] Step S225, obtaining a gradient residual according to the reconstructed gradient and the gradient of the large model.

[0063] The residual of the original gradient and the reconstructed gradient is , the calculation formula is as follows:

[0064] in, is the residual of the original gradient and the reconstructed gradient at step t. When t=1, the residual of the first-order moment and the correction amount are shown in formula (6), and when t>1, they are shown in formula (7):

[0065] in, It is The first moment of the step time, It is The residual of the first-order moment at step , It is The correction amount of the first-order moment residual at step time, is the attenuation coefficient of the first-order moment, It is The decay coefficient of the first-order moment at step time.

[0066] When t=1, the residual error and correction value of the second-order moment are shown in formula (8), and when t>1, they are shown in formula (9):

[0067] in, For the The gradient projected back to the high-dimensional space at step : , For the The second moment of the step time, It is The residual of the second-order moment at step time, It is The correction amount of the second-order moment residual at step time, is the attenuation coefficient of the second-order moment, It is The attenuation coefficient of the second-order moment at step time, Represents bitwise multiplication.

[0068] Correspondingly, fine-tuning the parameters of the large model according to the updated gradient includes: Step S250, fine-tuning the parameters of the large model according to the updated gradient and the gradient residual.

[0069] The parameter update formula is as follows:

[0070] in, as well as are the first-order moment and the second-order moment, is the learning rate, is the attenuation coefficient, It is a very small value used to prevent division by 0.

[0071] because The value of is very small, and it can be approximately considered ,therefore It can be approximated as:

[0072] Taking the AdamW optimization algorithm as an example, the parameter update formula of the large model is as follows:

[0073] The present invention is based on the original formula The parameter residuals are added to the basis, and the calculation formula of the parameter residuals is as follows:

[0074] It can be understood that the present invention uses the difference between the original gradient and the gradient projected back to the high-dimensional space to calculate the residual, thereby reducing the error caused by the projection.

[0075] Based on the above embodiment, as an optional embodiment, the orthogonal basis based on the current gradient step number and the gradient of the large model is used to obtain the reconstructed gradient projected back to the high-dimensional projection space, including the following steps.

[0076] Step S2241, after a preset number of gradient steps, determine the absolute value of the first-order moment of the gradient of the large model. After the preset number of gradient steps refers to the end of the warm-up stage, which refers to the early stage of the large model. The learning rate will not reach the set value immediately, but will slowly increase from a very small value to the set value.

[0077] Step S2242, according to the absolute value of the first-order moment, select a certain percentage of the gradient of the large model as the target gradient. Select a certain percentage of the positions with the largest absolute value of the first-order moment as the positions for adding residuals, and do not retain or calculate the data of the other positions.

[0078] Step S2243, obtaining a sparse reconstructed gradient according to the orthogonal basis of the current gradient step number and the target gradient. , as well as They are all sparse matrices, which can reduce the memory usage.

[0079] It can be understood that the present invention selects the gradient of the calculated residual by the absolute value of the first-order moment, constructs a sparse reconstruction gradient, and reduces the occupation of the video memory.

[0080] In experimental evaluation, the present invention can reduce the fine-tuning time of existing large model fine-tuning technology by about 60%, and the performance of the fine-tuned model is comparable to that of the model obtained by the existing technology. While saving training time, there is no additional video memory overhead compared to the existing technology.

[0081] The large model parameter fine-tuning device provided by the present invention is described below. The large model parameter fine-tuning device described below and the large model parameter fine-tuning method described above can refer to each other.

[0082] Figure 3 : is a timing diagram of the large model parameter fine-tuning device provided by the present invention, such as Figure 3 As shown, the present invention also provides a large model parameter fine-tuning device, including the following modules.

[0083] An acquisition module 310 is used to acquire a data set of a target task and obtain a training set according to the data set; A fine-tuning module 320, used to fine-tune the parameters of the large model according to the training set, a preset fine-tuning strategy and an optimization algorithm; The fine-tuning strategy includes performing low-dimensional projection on the gradient of the large model based on a random subspace iteration method.

[0084] As an embodiment, the fine-tuning module 320 is further used for: Determining the gradient of the large model based on the training set; Performing low-dimensional projection on the gradient of the large model based on the random subspace iteration method to obtain a low-dimensional projection gradient; Updating the low-dimensional projected gradient based on the optimization algorithm; Perform high-dimensional projection and gradient update on the updated low-dimensional projected gradient to obtain the updated gradient; Fine-tuning the parameters of the large model according to the updated gradient; Repeat the above steps until the parameters of the large model after fine-tuning meet the preset conditions.

[0085] As an embodiment, the large model includes a large model based on a multi-head attention mechanism, the gradient of the large model includes the gradient of each attention head, the gradient of each attention head includes the gradient of the projected query vector and the gradient of the projection key vector, and the gradient of any attention head is used to determine the low-dimensional projection space shared by the projected query vector and the projection key vector.

[0086] As an embodiment, the fine-tuning module 320 is further used for: If the current gradient step is not the starting point of a period, the orthogonal basis of the previous gradient step is used as the orthogonal basis of the current gradient step, otherwise the next step is executed; If the gradient of the large model to be projected is the gradient of the projected query vector or the gradient of the projected key vector, an orthogonal basis of the current number of gradient steps is obtained based on the random subspace iteration method and the gradient of any attention head; otherwise, an orthogonal basis of the current number of gradient steps is obtained based on the random subspace iteration method and the gradient of the large model; The low-dimensional projected gradient is obtained according to the orthogonal basis of the current gradient step number and the gradient of the large model.

[0087] As an embodiment, the fine-tuning module 320 is further used for: Based on the orthogonal basis of the current gradient step number and the gradient of the large model, obtaining a reconstructed gradient projected back to the high-dimensional projection space; Obtaining a gradient residual according to the reconstructed gradient and the gradient of the large model; Correspondingly, fine-tuning the parameters of the large model according to the updated gradient includes: The parameters of the large model are fine-tuned according to the updated gradient and the gradient residual.

[0088] As an embodiment, the fine-tuning module 320 is further used for: After a preset number of gradient steps, determining the absolute value of the first-order moment of the gradient of the large model; According to the absolute value of the first-order moment, a certain proportion of the gradient of the large model is selected as the target gradient; A sparse reconstructed gradient is obtained according to an orthogonal basis of the current gradient step number and the target gradient.

[0089] It should be noted that the large model parameter fine-tuning device provided by the present invention can execute the large model parameter fine-tuning method described in any of the above embodiments during specific operation, and has the technical effect corresponding to the method, which will not be elaborated in this embodiment.

[0090] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute a large model parameter fine-tuning method, the method comprising: obtaining a data set of a target task, obtaining a training set according to the data set; fine-tuning the parameters of the large model according to the training set, a preset fine-tuning strategy and an optimization algorithm; wherein the fine-tuning strategy includes low-dimensional projection of the gradient of the large model based on a random subspace iteration method.

[0091] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0092] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the large model parameter fine-tuning method provided by the above methods, and the method includes: obtaining a data set of a target task, and obtaining a training set based on the data set; fine-tuning the parameters of the large model according to the training set, a preset fine-tuning strategy and an optimization algorithm; wherein the fine-tuning strategy includes low-dimensional projection of the gradient of the large model based on a random subspace iteration method.

[0093] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the large model parameter fine-tuning method provided by the above-mentioned methods, the method comprising: obtaining a data set of a target task, and obtaining a training set based on the data set; fine-tuning the parameters of the large model according to the training set, a preset fine-tuning strategy and an optimization algorithm; wherein the fine-tuning strategy comprises performing a low-dimensional projection of the gradient of the large model based on a random subspace iteration method.

[0094] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0095] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large model parameter fine-tuning method, characterized in that: include: Obtain a data set for a target task, and obtain a training set based on the data set; Fine-tune the parameters of the large model according to the training set, the preset fine-tuning strategy and the optimization algorithm; The fine-tuning strategy includes performing low-dimensional projection on the gradient of the large model based on a random subspace iteration method.

2. The large model parameter fine-tuning method according to claim 1, characterized in that: The fine-tuning of parameters of the large model according to the training set, the preset fine-tuning strategy and the optimization algorithm includes: Determining the gradient of the large model based on the training set; Performing low-dimensional projection on the gradient of the large model based on the random subspace iteration method to obtain a low-dimensional projection gradient; Updating the low-dimensional projected gradient based on the optimization algorithm; Perform high-dimensional projection and gradient update on the updated low-dimensional projected gradient to obtain the updated gradient; Fine-tuning the parameters of the large model according to the updated gradient; Repeat the above steps until the parameters of the large model after fine-tuning meet the preset conditions.

3. The large model parameter fine-tuning method according to claim 2, characterized in that: The large model includes a large model based on a multi-head attention mechanism, the gradient of the large model includes the gradient of each attention head, the gradient of each attention head includes the gradient of the projected query vector and the gradient of the projection key vector, and the gradient of any attention head is used to determine the low-dimensional projection space shared by the projected query vector and the projection key vector.

4. The large model parameter fine-tuning method according to claim 3, characterized in that: The step of performing low-dimensional projection on the gradient of the large model based on the random subspace iteration method to obtain a low-dimensional projection gradient includes: If the current gradient step is not the starting point of a period, the orthogonal basis of the previous gradient step is used as the orthogonal basis of the current gradient step, otherwise the next step is executed; If the gradient of the large model to be projected is the gradient of the projected query vector or the gradient of the projected key vector, an orthogonal basis of the current number of gradient steps is obtained based on the random subspace iteration method and the gradient of any attention head; otherwise, an orthogonal basis of the current number of gradient steps is obtained based on the random subspace iteration method and the gradient of the large model; The low-dimensional projected gradient is obtained according to the orthogonal basis of the current gradient step number and the gradient of the large model.

5. The large model parameter fine-tuning method according to any one of claims 2 to 4, characterized in that: After performing low-dimensional projection on the gradient of the large model based on the random subspace iteration method to obtain the low-dimensional projection gradient, the method further includes: Based on the orthogonal basis of the current gradient step number and the gradient of the large model, obtaining a reconstructed gradient projected back to the high-dimensional projection space; Obtaining a gradient residual according to the reconstructed gradient and the gradient of the large model; Correspondingly, fine-tuning the parameters of the large model according to the updated gradient includes: The parameters of the large model are fine-tuned according to the updated gradient and the gradient residual.

6. The large model parameter fine-tuning method according to claim 5, characterized in that: The step of obtaining the reconstructed gradient projected back to the high-dimensional projection space based on the orthogonal basis of the current gradient step number and the gradient of the large model comprises: After a preset number of gradient steps, determining the absolute value of the first-order moment of the gradient of the large model; According to the absolute value of the first-order moment, a certain proportion of the gradient of the large model is selected as the target gradient; A sparse reconstructed gradient is obtained according to an orthogonal basis of the current gradient step number and the target gradient.

7. A large model parameter fine-tuning device, characterized in that: include: An acquisition module is used to acquire a data set of a target task and obtain a training set based on the data set; A fine-tuning module, used to fine-tune the parameters of the large model according to the training set, a preset fine-tuning strategy and an optimization algorithm; The fine-tuning strategy includes performing low-dimensional projection on the gradient of the large model based on a random subspace iteration method.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the large model parameter fine-tuning method as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the large model parameter fine-tuning method as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the large model parameter fine-tuning method as claimed in any one of claims 1 to 6 is implemented.

Images