Task adaptation result confirmation method and system based on large language model fine tuning

Abstract

The invention discloses a task adaptation result confirmation method and system based on large language model fine tuning, and the method comprises the steps: obtaining model parameters of a large language model, updating the model parameters through employing an optimization algorithm, and updating the learning rate of the large language model through employing the updated model parameters; and performing iterative training on the large language model by utilizing the updated learning rate and the model parameters until final model parameters are output, updating the large language model according to the final model parameters, executing a plurality of target specific tasks to obtain corresponding target results, and determining a task adaptation result according to all the target results. According to the method, an adaptive learning rate adjustment mechanism is introduced, and the learning rate is dynamically adjusted by using feedback information of the model in a training process, so that the model adaptability is enhanced, the large language model can better adapt to specific tasks, and the performance of the model in practical application is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, system, terminal and computer-readable storage medium for confirming task adaptation results based on fine-tuning of a large language model. Background Art

[0002] With the rapid development of artificial intelligence and natural language processing technologies, large language models (LLMs) have demonstrated powerful capabilities in multiple application fields. These models have acquired rich language understanding and generation capabilities by pre-training on large-scale text data.

[0003] However, since pre-trained models are usually general, their performance on specific tasks may not be satisfactory, such as dependence on labeled data, excessive consumption of computing resources, or complex parameter adjustment. These shortcomings limit the breadth and flexibility of the model in practical applications.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide a task adaptation result confirmation method, system, terminal and computer-readable storage medium based on large language model fine-tuning, aiming to solve the problem in the prior art that the pre-training of large language models has dependence on labeled data, excessive consumption of computing resources or complex parameter adjustment, which leads to low efficiency or low accuracy when the model performs specific tasks.

[0006] To achieve the above object, the present invention provides a method for confirming a task adaptation result based on fine-tuning a large language model, and the method for confirming a task adaptation result based on fine-tuning a large language model comprises the following steps:

[0007] Obtaining initial model parameters, initial distance estimates, and multiple variables of a large language model, and initializing all of the variables to obtain corresponding initial variables;

[0008] Calculating the initial gradients of the initial model parameters, updating a plurality of first initial variables among all the initial variables, and obtaining a plurality of first updated variables;

[0009] Scaling the initial gradient using the plurality of first update variables to obtain an adjustment matrix, and updating the initial model parameters according to the adjustment matrix to obtain model update parameters;

[0010] Updating a plurality of second initial variables to obtain a plurality of second updated variables, updating the initial distance estimate using the plurality of second updated variables to obtain an updated distance estimate, and updating the learning rate of the large language model using the updated distance estimate;

[0011] The large language model is iteratively trained using the updated learning rate until the final model parameters are output, and then the large language model is updated. The updated large language model is used to perform multiple target-specific tasks to obtain corresponding target results, and the task adaptation results are determined based on all the target results.

[0012] Optionally, in the task adaptation result confirmation method based on large language model fine-tuning, the initial variables include initial momentum variables, second-order momentum variables, step accumulation variables and distance estimation variables;

[0013] The obtaining of initial model parameters, initial distance estimation and multiple variables of the large language model, and initializing all the variables to obtain corresponding initial variables specifically includes:

[0014] Acquiring structural information of the large language model, and determining the initial model parameters and the initial distance estimate according to the structural information;

[0015] A plurality of variables of the large language model are obtained, and all the variables are initialized to zero respectively, so as to obtain an initial momentum variable, a second-order momentum variable, a step accumulation variable, and a distance estimation variable.

[0016] Optionally, in the task adaptation result confirmation method based on large language model fine-tuning, the first initial variable includes: the initial momentum variable and the second-order momentum variable; the first update variable includes: the initial momentum update variable and the second-order momentum update variable;

[0017] The calculating the initial gradient of the initial model parameter, updating a plurality of first initial variables among all the initial variables, and obtaining a plurality of first updated variables specifically includes:

[0018] Obtain input parameters input by the user to the large language model, and determine the loss function f(x) of the large language model according to the input parameters k ,ξ k ), where k represents the kth step in each iteration, and x k represents the initial model parameters in the current iteration, ξ k Represents input parameters;

[0019] Calculate the initial gradient of the initial model parameters according to the loss function:

[0020]

[0021] Among them, g k represents the initial gradient in the current iteration, It means to take the derivative of a function;

[0022] A plurality of hyperparameters are obtained, and the initial momentum variable and the second-order momentum variable are updated using the plurality of hyperparameters to obtain an initial momentum update variable and a second-order momentum update variable.

[0023] Optionally, the method for confirming the task adaptation result based on large language model fine-tuning, wherein the obtaining of multiple hyperparameters and using the multiple hyperparameters to update the initial momentum variable and the second-order momentum variable to obtain the initial momentum update variable and the second-order momentum update variable, specifically includes:

[0024] Obtaining multiple hyperparameters of a preset optimization algorithm and obtaining a learning rate scaling factor of the large language model;

[0025] The initial momentum variable is updated using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the initial momentum update variable:

[0026] m k+1 =β1m k +(1-β1)d k γ k g k ;

[0027] Among them, m k+1 represents the initial momentum update variable updated after the k-th iteration, m k represents the initial momentum variable at the kth iteration, β1 represents the first hyperparameter, and d k represents the initial distance estimate at the k-th iteration, γ k represents the learning rate scaling factor;

[0028] The second-order momentum variable is updated using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the second-order momentum update variable:

[0029]

[0030] Among them, v k+1 represents the second-order momentum update variable updated after the k-th iteration, v k represents the second-order momentum variable at the k-th iteration, and β2 represents the second hyperparameter.

[0031] Optionally, the method for confirming the task adaptation result based on fine-tuning of a large language model, wherein the step of scaling the initial gradient using the plurality of first update variables to obtain an adjustment matrix, and updating the initial model parameters according to the adjustment matrix to obtain model update parameters, specifically includes:

[0032] The initial gradient is scaled using the initial momentum update variable and the second-order momentum update variable to obtain an adjustment matrix:

[0033]

[0034] Among them, A k+1 represents the adjustment matrix after the k-th iteration, diag() represents the function of generating a diagonal matrix or extracting the diagonal of a matrix, ∈ represents the third hyperparameter, v k+1 represents the second-order momentum update variable;

[0035] The initial model parameters are updated according to the adjustment matrix to obtain model update parameters:

[0036]

[0037] Among them, x k+1 represents the model update parameters updated after the k-th iteration, represents the inverse matrix of the adjustment matrix, m k+1 Represents the initial momentum update variable updated after the k-th iteration.

[0038] Optionally, in the task adaptation result confirmation method based on large language model fine-tuning, the second initial variable includes: a step accumulation variable and a distance estimation variable; the second update variable includes: a step accumulation update variable and a distance estimation update variable;

[0039] The updating of the plurality of second initial variables to obtain a plurality of second updated variables, using the plurality of second updated variables to update the initial distance estimate to obtain a distance update estimate, and using the distance update estimate to update the learning rate of the large language model specifically includes:

[0040] Update the step size accumulation variable and the distance estimation variable to obtain the step size accumulation update variable and the distance estimation update variable:

[0041]

[0042] Among them, s k+1 represents the cumulative update variable after the k-th iteration, s k represents the step size cumulative variable at the k-th iteration, β2 represents the second hyperparameter, and d k represents the initial distance estimate at the k-th iteration, γ k represents the learning rate scaling factor, g k Represents the initial gradient in the current iteration;

[0043]

[0044] Among them, rk+1 represents the distance estimation update variable updated after the k-th iteration, r k represents the distance estimation variable at the k-th iteration, represents the inverse matrix of the adjustment matrix, T represents the transpose, A represents the diagonal matrix, A -1 represents the inverse matrix of a diagonal matrix;

[0045] The initial distance estimate is updated using the step size cumulative update variable to obtain an updated distance estimate:

[0046]

[0047] in, represents the distance update estimate updated after the k-th iteration, ∥s k+1 ∥1 represents the vector norm of the step-by-step cumulative update variable;

[0048] The learning rate of the large language model is updated using the distance update estimate to obtain a lower bound on the learning rate:

[0049]

[0050] Among them, d k+1 represents the lower bound of the learning rate.

[0051] Optionally, the method for confirming the task adaptation result based on large language model fine-tuning, wherein the large language model is iteratively trained using the updated learning rate until the final model parameters are output and then the large language model is updated, and the updated large language model is used to perform multiple target-specific tasks to obtain corresponding target results, and the task adaptation result is determined according to all the target results, specifically including:

[0052] Acquiring structural information of the large language model, and determining the number of iterative training according to the structural information;

[0053] Respectively using the initial momentum update variable, the second-order momentum update variable, the step accumulation update variable, the distance estimate update variable, the learning rate lower bound, the model update parameters and the initial distance estimate as the initial momentum variable, the second-order momentum variable, the step accumulation variable, the initial learning rate of the distance estimate variable, the initial model parameters and the distance update estimate in the next iterative training process, and iteratively training the large language model until the iteration is completed to obtain the final model parameters;

[0054] Using the final model parameters to update the large language model, obtain a target model, and input the input parameters of multiple target specific results into the target model, and output the corresponding target results;

[0055] If all the target results are qualified, a task adaptation result is obtained, wherein the task adaptation result indicates that the task adaptability of the target model is enhanced.

[0056] In addition, to achieve the above-mentioned purpose, the present invention also provides a task adaptation result confirmation system based on large language model fine-tuning, wherein the task adaptation result confirmation system based on large language model fine-tuning includes:

[0057] A model information acquisition module, used to obtain initial model parameters, initial distance estimation and multiple variables of the large language model, and initialize all the variables to obtain corresponding initial variables;

[0058] A first parameter updating module, used for calculating the initial gradient of the initial model parameter, updating a plurality of first initial variables among all the initial variables, and obtaining a plurality of first updated variables;

[0059] a second parameter updating module, configured to scale the initial gradient using a plurality of the first updating variables to obtain an adjustment matrix, and update the initial model parameters according to the adjustment matrix to obtain model update parameters;

[0060] a third parameter updating module, configured to update the plurality of second initial variables to obtain a plurality of second updated variables, update the initial distance estimate using the plurality of second updated variables to obtain an updated distance estimate, and update the learning rate of the large language model using the updated distance estimate;

[0061] The result testing module is used to iteratively train the large language model using the updated learning rate until the final model parameters are output, and then update the large language model, and use the updated large language model to perform multiple target-specific tasks to obtain corresponding target results, and determine the task adaptation results based on all the target results.

[0062] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a task adaptation result confirmation program based on large language model fine-tuning stored in the memory and executable on the processor, wherein the task adaptation result confirmation program based on large language model fine-tuning implements the steps of the task adaptation result confirmation method based on large language model fine-tuning as described above when executed by the processor.

[0063] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a task adaptation result confirmation program based on large language model fine-tuning, and when the task adaptation result confirmation program based on large language model fine-tuning is executed by a processor, the steps of the task adaptation result confirmation method based on large language model fine-tuning as described above are implemented.

[0064] In the present invention, the initial model parameters, initial distance estimation and multiple variables of the large language model are obtained, and all the variables are initialized to obtain corresponding initial variables; the initial gradient of the initial model parameters is calculated, and multiple first initial variables in all the initial variables are updated to obtain multiple first update variables; the initial gradient is scaled using multiple first update variables to obtain an adjustment matrix, and the initial model parameters are updated according to the adjustment matrix to obtain model update parameters; multiple second initial variables are updated to obtain multiple second update variables, and the initial distance estimation is updated using multiple second update variables to obtain a distance update estimation, and the learning rate of the large language model is updated using the distance update estimation; the large language model is iteratively trained using the updated learning rate until the final model parameters are output and the large language model is updated, and the updated large language model is used to perform multiple target specific tasks to obtain corresponding target results, and the task adaptation results are determined according to all the target results. The present invention introduces an adaptive learning rate adjustment mechanism, uses the feedback information of the model during the training process, dynamically adjusts the learning rate, enhances the model adaptability, enables the large language model to better adapt to specific tasks, and improves the performance of the model in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flow chart of a preferred embodiment of a method for confirming task adaptation results based on fine-tuning of a large language model according to the present invention;

[0066] Figure 2 It is a pseudo code diagram of model training of the task adaptation result confirmation method based on large language model fine-tuning of the present invention;

[0067] Figure 3 It is a structural diagram of a preferred embodiment of the task adaptation result confirmation system based on large language model fine-tuning of the present invention;

[0068] Figure 4 Schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0070] The task adaptation result confirmation method based on large language model fine-tuning described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the task adaptation result confirmation method based on large language model fine-tuning includes the following steps:

[0071] Step S10: obtaining initial model parameters, initial distance estimation and multiple variables of the large language model, and initializing all the variables to obtain corresponding initial variables.

[0072] The initial variables include initial momentum variables, second-order momentum variables, step accumulation variables and distance estimation variables.

[0073] Specifically, the structural information of the large language model is obtained, and the initial model parameters and the initial distance estimation are determined according to the structural information; multiple variables of the large language model are obtained, and all the variables are initialized to zero respectively to obtain initial momentum variables, second-order momentum variables, step accumulation variables and distance estimation variables.

[0074] First, if Figure 2 As shown, multiple model parameters of the large language model that need to be fine-tuned are obtained, wherein the model parameters are generally randomly initialized, or the model parameters are obtained after pre-training the large language model. In addition, it is also necessary to obtain the hyperparameters of the large language model and the learning rate of the model, including the hyperparameters of various optimization algorithms, such as the hyperparameters of the Adam (Adaptive Moment Estimation) optimizer, and then introduce an adaptive learning rate adjustment mechanism to dynamically monitor the performance indicators of the model on the validation set during the training process, so that at the end of each training cycle, the learning rate is automatically adjusted according to the indicators to ensure that the large language model is trained in the optimal state and improve the training efficiency.

[0075] Step S20, calculating the initial gradients of the initial model parameters, updating a plurality of first initial variables among all the initial variables, and obtaining a plurality of first updated variables.

[0076] Among them, the first initial variable includes: the initial momentum variable and the second-order momentum variable; the first updated variable includes: the initial momentum update variable and the second-order momentum update variable.

[0077] Specifically, the input parameters input by the user to the large language model are obtained, and the loss function f(x k ,ξ k ), where k represents the kth step in each iteration, and x k represents the initial model parameters in the current iteration, ξ k represents the input parameters; the initial gradient of the initial model parameters is calculated according to the loss function:

[0078]

[0079] Among them, g k represents the initial gradient in the current iteration, It means to take the derivative of a function;

[0080] A plurality of hyperparameters are obtained, and the initial momentum variable and the second-order momentum variable are updated using the plurality of hyperparameters to obtain an initial momentum update variable and a second-order momentum update variable.

[0081] Among them, the lower bound of the learning rate of the large language model is adaptively adjusted by combining gradient information and momentum information. In this way, the optimization algorithm can fine-tune the learning rate in each iteration according to the change of the current gradient, thereby avoiding the problem of unstable training or slow convergence caused by too large or too small learning rate.

[0082] Further, a plurality of hyperparameters of a preset optimization algorithm are obtained, and a learning rate scaling factor of the large language model is obtained; the initial momentum variable is updated using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the initial momentum update variable:

[0083] m k+1 =β1m k +(1-β1)d k γ k g k ;

[0084] Among them, m k+1 represents the initial momentum update variable updated after the k-th iteration, m k represents the initial momentum variable at the kth iteration, β1 represents the first hyperparameter, and d k represents the initial distance estimate at the k-th iteration, γ k represents the learning rate scaling factor;

[0085] The second-order momentum variable is updated using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the second-order momentum update variable:

[0086]

[0087] Among them, v k+1 represents the second-order momentum update variable updated after the k-th iteration, v k represents the second-order momentum variable at the k-th iteration, and β2 represents the second hyperparameter.

[0088] Among them, the learning rate is automatically adjusted by gradually improving the distance estimation, so there is no need to know the optimal solution distance in advance. Therefore, it has a wider applicability when dealing with different types of optimization problems, and can adapt well to various specific tasks. In practical applications, users do not need to have an in-depth understanding of the objective function to obtain better optimization effects, which reduces the optimization cost of large language models and improves the applicability of optimizing large language models.

[0089] Step S30: Scale the initial gradient using the plurality of first update variables to obtain an adjustment matrix, and update the initial model parameters according to the adjustment matrix to obtain model update parameters.

[0090] Specifically, the initial gradient is scaled using the initial momentum update variable and the second-order momentum update variable to obtain an adjustment matrix:

[0091]

[0092] Among them, A k+1 represents the adjustment matrix after the k-th iteration, diag() represents the function of generating a diagonal matrix or extracting the diagonal of a matrix, ∈ represents the third hyperparameter, v k+1 represents the second-order momentum update variable;

[0093] The initial model parameters are updated according to the adjustment matrix to obtain model update parameters:

[0094]

[0095] Among them, x k+1 represents the model update parameters updated after the k-th iteration, represents the inverse matrix of the adjustment matrix, m k+1 Represents the initial momentum update variable updated after the k-th iteration.

[0096] Among them, multiple hyperparameters used when updating model parameters are all in the D-Adaptation (Dynamic Adaptation) optimization algorithm. By making the optimization algorithm compatible during the training process of the large language model, model training can be integrated as a module in the optimization algorithm without redesigning the entire optimization framework, which greatly reduces the difficulty of application.

[0097] Furthermore, other adaptive algorithms can also be used, including: AdaGrad (Adaptive Gradient Algorithm), which adjusts the learning rate by accumulating the sum of squared gradients and is suitable for processing sparse data; RMSProp (Root Mean Square Propagation), which adjusts the learning rate by the moving average of the squared gradients and is suitable for processing non-stationary targets; Adamax (a variant of Adam), which adjusts the learning rate by using the infinite norm and is more suitable for processing scenarios with large gradient changes.

[0098] Step S40: Update the multiple second initial variables to obtain multiple second updated variables, use the multiple second updated variables to update the initial distance estimate to obtain an updated distance estimate, and use the updated distance estimate to update the learning rate of the large language model.

[0099] The second initial variables include: a step-length accumulation variable and a distance estimation variable; the second updated variables include: a step-length accumulation update variable and a distance estimation update variable.

[0100] Specifically, the step size accumulation variable and the distance estimation variable are updated to obtain the step size accumulation update variable and the distance estimation update variable:

[0101]

[0102] Among them, s k+1 represents the cumulative update variable after the k-th iteration, s k represents the step size cumulative variable at the k-th iteration, β2 represents the second hyperparameter, and d k represents the initial distance estimate at the k-th iteration, γ k represents the learning rate scaling factor, g k Represents the initial gradient in the current iteration;

[0103]

[0104] Among them, r k+1 represents the distance estimation update variable updated after the k-th iteration, r k represents the distance estimation variable at the k-th iteration, represents the inverse matrix of the adjustment matrix, T represents the transpose, A represents the diagonal matrix, A -1 represents the inverse matrix of a diagonal matrix;

[0105] The initial distance estimate is updated using the step size cumulative update variable to obtain an updated distance estimate:

[0106]

[0107] in, represents the distance update estimate updated after the k-th iteration, ∥s k+1 ∥1 represents the vector norm of the step-by-step cumulative update variable;

[0108] The learning rate of the large language model is updated using the distance update estimate to obtain a lower bound on the learning rate:

[0109]

[0110] Among them, d k+1 represents the lower bound of the learning rate.

[0111] Among them, by updating the model variables and the initial distance estimate, the model learning rate is gradually estimated, and the lower bound of the learning rate is dynamically adjusted to obtain the most suitable learning rate.

[0112] Furthermore, an adaptive learning rate method based on Grünwald–Letnikov (G–L) fractional derivatives (a definition of fractional calculus) can be used to adjust the step size in combination with long-term and short-term gradient information to avoid falling into local minima. This method based on fractional derivatives can capture global information and dynamically adjust the step size in combination with short-term gradient changes, making the optimization process more stable and converging quickly.

[0113] Step S50, iteratively train the large language model using the updated learning rate until the final model parameters are output, update the large language model, and use the updated large language model to perform multiple target-specific tasks to obtain corresponding target results, and determine the task adaptation results based on all the target results.

[0114] Specifically, the structural information of the large language model is obtained, and the number of iterative training is determined according to the structural information; the initial momentum update variable, the second-order momentum update variable, the step accumulation update variable, the distance estimation update variable, the learning rate lower bound, the model update parameter and the initial distance estimate are respectively used as the initial momentum variable, the second-order momentum variable, the step accumulation variable, the initial learning rate of the distance estimation variable, the initial model parameters and the distance update estimate in the next iterative training process, and the large language model is iteratively trained until the iteration is completed to obtain the final model parameters; the large language model is updated with the final model parameters to obtain a target model, and the input parameters of multiple target-specific results are input into the target model to output the corresponding target results; if all the target results are qualified, a task adaptation result is obtained, wherein the task adaptation result indicates that the task adaptability of the target model is enhanced.

[0115] Among them, after each training cycle is completed, the model parameters, momentum variables and other information obtained in the previous training cycle are used as the initial model parameters and initial momentum variables in the next training cycle. In each iteration, the learning rate is fine-tuned according to the change of the current gradient, thereby avoiding the problem of unstable training or slow convergence caused by too large or too small learning rate; at the same time, according to the different specific target tasks input by the user, the model learning rate required to be used is also different. According to different specific tasks, the most appropriate learning rate is selected for training, which improves the breadth and applicability of model application.

[0116] The present invention introduces an adaptive learning rate adjustment mechanism and utilizes the feedback information of the model during the training process to dynamically adjust the learning rate, thereby enhancing the adaptability of the model, enabling the large language model to better adapt to specific tasks and improving the performance of the model in practical applications.

[0117] Furthermore, if Figure 3 As shown, based on the above-mentioned task adaptation result confirmation method based on large language model fine-tuning, the present invention also provides a task adaptation result confirmation system based on large language model fine-tuning, wherein the task adaptation result confirmation system based on large language model fine-tuning includes:

[0118] A model information acquisition module 51 is used to obtain initial model parameters, initial distance estimation and multiple variables of the large language model, and initialize all the variables to obtain corresponding initial variables;

[0119] A first parameter updating module 52, used for calculating the initial gradient of the initial model parameter, updating a plurality of first initial variables among all the initial variables, and obtaining a plurality of first updated variables;

[0120] A second parameter updating module 53, configured to scale the initial gradient using a plurality of the first updating variables to obtain an adjustment matrix, and update the initial model parameters according to the adjustment matrix to obtain model update parameters;

[0121] a third parameter updating module 54, configured to update a plurality of second initial variables to obtain a plurality of second updated variables, update the initial distance estimate using the plurality of second updated variables to obtain an updated distance estimate, and update the learning rate of the large language model using the updated distance estimate;

[0122] The result testing module 55 is used to iteratively train the large language model using the updated learning rate until the final model parameters are output, and then update the large language model, and use the updated large language model to perform multiple target-specific tasks to obtain corresponding target results, and determine the task adaptation results based on all the target results.

[0123] Furthermore, if Figure 4 As shown, based on the above-mentioned task adaptation result confirmation method and system based on large language model fine-tuning, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 4 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0124] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a task adaptation result confirmation program 40 based on large language model fine-tuning is stored on the memory 20, and the task adaptation result confirmation program 40 based on large language model fine-tuning can be executed by the processor 10, thereby realizing the task adaptation result confirmation method based on large language model fine-tuning in the present application.

[0125] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the task adaptation result confirmation method based on large language model fine-tuning.

[0126] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0127] In one embodiment, when the processor 10 executes the task adaptation result confirmation program 40 based on large language model fine-tuning in the memory 20, the following steps are implemented:

[0128] Obtaining initial model parameters, initial distance estimates, and multiple variables of a large language model, and initializing all of the variables to obtain corresponding initial variables;

[0129] Calculating the initial gradients of the initial model parameters, updating a plurality of first initial variables among all the initial variables, and obtaining a plurality of first updated variables;

[0130] Scaling the initial gradient using the plurality of first update variables to obtain an adjustment matrix, and updating the initial model parameters according to the adjustment matrix to obtain model update parameters;

[0131] Updating a plurality of second initial variables to obtain a plurality of second updated variables, updating the initial distance estimate using the plurality of second updated variables to obtain an updated distance estimate, and updating the learning rate of the large language model using the updated distance estimate;

[0132] The large language model is iteratively trained using the updated learning rate until the final model parameters are output, and then the large language model is updated. The updated large language model is used to perform multiple target-specific tasks to obtain corresponding target results, and the task adaptation results are determined based on all the target results.

[0133] Wherein, the initial variables include initial momentum variables, second-order momentum variables, step accumulation variables and distance estimation variables;

[0134] The obtaining of initial model parameters, initial distance estimation and multiple variables of the large language model, and initializing all the variables to obtain corresponding initial variables specifically includes:

[0135] Acquiring structural information of the large language model, and determining the initial model parameters and the initial distance estimate according to the structural information;

[0136] A plurality of variables of the large language model are obtained, and all the variables are initialized to zero respectively, so as to obtain an initial momentum variable, a second-order momentum variable, a step accumulation variable, and a distance estimation variable.

[0137] Wherein, the first initial variable includes: the initial momentum variable and the second-order momentum variable; the first update variable includes: the initial momentum update variable and the second-order momentum update variable;

[0138] The calculating the initial gradient of the initial model parameter, updating a plurality of first initial variables among all the initial variables, and obtaining a plurality of first updated variables specifically includes:

[0139] Obtain input parameters input by the user to the large language model, and determine the loss function f(x) of the large language model according to the input parameters k,ξ k ), where k represents the kth step in each iteration, and x k represents the initial model parameters in the current iteration, ξ k Represents input parameters;

[0140] Calculate the initial gradient of the initial model parameters according to the loss function:

[0141]

[0142] Among them, g k represents the initial gradient in the current iteration, It means to take the derivative of a function;

[0143] A plurality of hyperparameters are obtained, and the initial momentum variable and the second-order momentum variable are updated using the plurality of hyperparameters to obtain an initial momentum update variable and a second-order momentum update variable.

[0144] The step of obtaining a plurality of hyperparameters and updating the initial momentum variable and the second-order momentum variable using the plurality of hyperparameters to obtain the initial momentum update variable and the second-order momentum update variable specifically includes:

[0145] Obtaining multiple hyperparameters of a preset optimization algorithm and obtaining a learning rate scaling factor of the large language model;

[0146] The initial momentum variable is updated using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the initial momentum update variable:

[0147] m k+1 =β1m k +(1-β1)d k γ k g k ;

[0148] Among them, m k+1 represents the initial momentum update variable updated after the k-th iteration, m k represents the initial momentum variable at the kth iteration, β1 represents the first hyperparameter, and d k represents the initial distance estimate at the k-th iteration, γ k represents the learning rate scaling factor;

[0149] The second-order momentum variable is updated using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the second-order momentum update variable:

[0150]

[0151] Among them, vk+1 represents the second-order momentum update variable updated after the k-th iteration, v k represents the second-order momentum variable at the k-th iteration, and β2 represents the second hyperparameter.

[0152] The step of scaling the initial gradient using the plurality of first update variables to obtain an adjustment matrix, and updating the initial model parameters according to the adjustment matrix to obtain model update parameters specifically includes:

[0153] The initial gradient is scaled using the initial momentum update variable and the second-order momentum update variable to obtain an adjustment matrix:

[0154]

[0155] Among them, A k+1 represents the adjustment matrix after the k-th iteration, diag() represents the function of generating a diagonal matrix or extracting the diagonal of a matrix, ∈ represents the third hyperparameter, v k+1 represents the second-order momentum update variable;

[0156] The initial model parameters are updated according to the adjustment matrix to obtain model update parameters:

[0157]

[0158] Among them, x k+1 represents the model update parameters updated after the k-th iteration, represents the inverse matrix of the adjustment matrix, m k+1 Represents the initial momentum update variable updated after the k-th iteration.

[0159] The second initial variable includes: a step length accumulation variable and a distance estimation variable; the second update variable includes: a step length accumulation update variable and a distance estimation update variable;

[0160] The updating of the plurality of second initial variables to obtain a plurality of second updated variables, using the plurality of second updated variables to update the initial distance estimate to obtain a distance update estimate, and using the distance update estimate to update the learning rate of the large language model specifically includes:

[0161] Update the step size accumulation variable and the distance estimation variable to obtain the step size accumulation update variable and the distance estimation update variable:

[0162]

[0163] Among them, s k+1 represents the cumulative update variable after the k-th iteration, s krepresents the step size cumulative variable at the k-th iteration, β2 represents the second hyperparameter, and d k represents the initial distance estimate at the k-th iteration, γ k represents the learning rate scaling factor, g k Represents the initial gradient in the current iteration;

[0164]

[0165] Among them, r k+1 represents the distance estimation update variable updated after the k-th iteration, r k represents the distance estimation variable at the k-th iteration, represents the inverse matrix of the adjustment matrix, T represents the transpose, A represents the diagonal matrix, A -1 represents the inverse matrix of a diagonal matrix;

[0166] The initial distance estimate is updated using the step size cumulative update variable to obtain an updated distance estimate:

[0167]

[0168] in, represents the distance update estimate updated after the k-th iteration, ∥s k+1 ∥1 represents the vector norm of the step-by-step cumulative update variable;

[0169] The learning rate of the large language model is updated using the distance update estimate to obtain a lower bound on the learning rate:

[0170]

[0171] Among them, d k+1 represents the lower bound of the learning rate.

[0172] The method of iteratively training the large language model using the updated learning rate until the final model parameters are outputted and then updating the large language model, and performing a plurality of target-specific tasks using the updated large language model to obtain corresponding target results, and determining the task adaptation results according to all the target results, specifically includes:

[0173] Acquiring structural information of the large language model, and determining the number of iterative training according to the structural information;

[0174] Respectively using the initial momentum update variable, the second-order momentum update variable, the step accumulation update variable, the distance estimate update variable, the learning rate lower bound, the model update parameters and the initial distance estimate as the initial momentum variable, the second-order momentum variable, the step accumulation variable, the initial learning rate of the distance estimate variable, the initial model parameters and the distance update estimate in the next iterative training process, and iteratively training the large language model until the iteration is completed to obtain the final model parameters;

[0175] Using the final model parameters to update the large language model, obtain a target model, and input the input parameters of multiple target specific results into the target model, and output the corresponding target results;

[0176] If all the target results are qualified, a task adaptation result is obtained, wherein the task adaptation result indicates that the task adaptability of the target model is enhanced.

[0177] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a task adaptation result confirmation program based on large language model fine-tuning, and when the task adaptation result confirmation program based on large language model fine-tuning is executed by a processor, the steps of the task adaptation result confirmation method based on large language model fine-tuning as described above are implemented.

[0178] In summary, the present invention provides a method for confirming task adaptation results based on large language model fine-tuning and related equipment, the method comprising: obtaining initial model parameters, initial distance estimates and multiple variables of a large language model, and initializing all the variables to obtain corresponding initial variables; calculating the initial gradients of the initial model parameters, updating multiple first initial variables among all the initial variables, and obtaining multiple first updated variables; scaling the initial gradients using the multiple first updated variables to obtain an adjustment matrix, and updating the initial model parameters according to the adjustment matrix to obtain model update parameters; updating multiple second initial variables to obtain multiple second updated variables, using the multiple second updated variables to update the initial distance estimate to obtain a distance update estimate, and using the distance update estimate to update the learning rate of the large language model; iteratively training the large language model using the updated learning rate until the final model parameters are output, then updating the large language model, and using the updated large language model to perform multiple target-specific tasks to obtain corresponding target results, and determining the task adaptation results according to all the target results. The present invention introduces an adaptive learning rate adjustment mechanism and utilizes the feedback information of the model during the training process to dynamically adjust the learning rate, thereby enhancing the adaptability of the model, enabling the large language model to better adapt to specific tasks, improving the performance of the model in practical applications, and improving the breadth and applicability of the model application.

[0179] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0180] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0181] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A task adaptation result confirmation method based on large language model fine-tuning, characterized in that: The task adaptation result confirmation method based on large language model fine-tuning includes: Obtaining initial model parameters, initial distance estimates, and multiple variables of a large language model, and initializing all of the variables to obtain corresponding initial variables; Calculating the initial gradients of the initial model parameters, updating a plurality of first initial variables among all the initial variables, and obtaining a plurality of first updated variables; Scaling the initial gradient using the plurality of first update variables to obtain an adjustment matrix, and updating the initial model parameters according to the adjustment matrix to obtain model update parameters; Updating a plurality of second initial variables to obtain a plurality of second updated variables, updating the initial distance estimate using the plurality of second updated variables to obtain an updated distance estimate, and updating the learning rate of the large language model using the updated distance estimate; The large language model is iteratively trained using the updated learning rate until the final model parameters are output, and then the large language model is updated. The updated large language model is used to perform multiple target-specific tasks to obtain corresponding target results, and the task adaptation results are determined based on all the target results.

2. The task adaptation result confirmation method based on large language model fine-tuning according to claim 1 is characterized in that: The initial variables include an initial momentum variable, a second-order momentum variable, a step accumulation variable, and a distance estimation variable; The obtaining of initial model parameters, initial distance estimation and multiple variables of the large language model, and initializing all the variables to obtain corresponding initial variables specifically includes: Acquiring structural information of the large language model, and determining the initial model parameters and the initial distance estimate according to the structural information; A plurality of variables of the large language model are obtained, and all the variables are initialized to zero respectively, so as to obtain an initial momentum variable, a second-order momentum variable, a step accumulation variable, and a distance estimation variable.

3. The task adaptation result confirmation method based on large language model fine-tuning according to claim 2 is characterized in that: The first initial variable includes: the initial momentum variable and the second-order momentum variable; the first update variable includes: the initial momentum update variable and the second-order momentum update variable; The calculating the initial gradient of the initial model parameter, updating a plurality of first initial variables among all the initial variables, and obtaining a plurality of first updated variables specifically includes: Obtain input parameters input by the user to the large language model, and determine the loss function f(x) of the large language model according to the input parameters k ,ξ k ), where k represents the kth step in each iteration, and x k represents the initial model parameters in the current iteration, ξ k Represents input parameters; Calculate the initial gradient of the initial model parameters according to the loss function: Among them, g k represents the initial gradient in the current iteration, It means to take the derivative of a function; A plurality of hyperparameters are obtained, and the initial momentum variable and the second-order momentum variable are updated using the plurality of hyperparameters to obtain an initial momentum update variable and a second-order momentum update variable.

4. The method for confirming task adaptation results based on large language model fine-tuning according to claim 3, characterized in that: The acquiring of multiple hyperparameters and updating the initial momentum variable and the second-order momentum variable using the multiple hyperparameters to obtain the initial momentum update variable and the second-order momentum update variable specifically includes: Obtaining multiple hyperparameters of a preset optimization algorithm and obtaining a learning rate scaling factor of the large language model; The initial momentum variable is updated using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the initial momentum update variable: m k+1 =β1m k +(1-β1)d k c k g k ; Among them, m k+1 represents the initial momentum update variable updated after the k-th iteration, m k represents the initial momentum variable at the kth iteration, β1 represents the first hyperparameter, and d k represents the initial distance estimate at the k-th iteration, γ k represents the learning rate scaling factor; The second-order momentum variable is updated using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the second-order momentum update variable: Among them, v k+1 represents the second-order momentum update variable updated after the k-th iteration, v k represents the second-order momentum variable at the k-th iteration, and β2 represents the second hyperparameter.

5. The task adaptation result confirmation method based on large language model fine-tuning according to claim 3 is characterized in that: The step of scaling the initial gradient using the plurality of first update variables to obtain an adjustment matrix, and updating the initial model parameters according to the adjustment matrix to obtain model update parameters specifically includes: The initial gradient is scaled using the initial momentum update variable and the second-order momentum update variable to obtain an adjustment matrix: Among them, A k+1 represents the adjustment matrix after the k-th iteration, diag() represents the function of generating a diagonal matrix or extracting the diagonal of a matrix, ∈ represents the third hyperparameter, v k+1 represents the second-order momentum update variable; The initial model parameters are updated according to the adjustment matrix to obtain model update parameters: Among them, x k+1 represents the model update parameters updated after the k-th iteration, represents the inverse matrix of the adjustment matrix, m k+1 Represents the initial momentum update variable updated after the k-th iteration.

6. The method for confirming task adaptation results based on large language model fine-tuning according to claim 1, characterized in that: The second initial variables include: step length accumulation variables and distance estimation variables; the second update variables include: step length accumulation update variables and distance estimation update variables; The updating of the plurality of second initial variables to obtain a plurality of second updated variables, using the plurality of second updated variables to update the initial distance estimate to obtain a distance update estimate, and using the distance update estimate to update the learning rate of the large language model specifically includes: Update the step size accumulation variable and the distance estimation variable to obtain the step size accumulation update variable and the distance estimation update variable: Among them, s k+1 represents the cumulative update variable after the k-th iteration, s k represents the step size cumulative variable at the k-th iteration, β2 represents the second hyperparameter, and d k represents the initial distance estimate at the k-th iteration, γ k represents the learning rate scaling factor, g k Represents the initial gradient in the current iteration; Among them, r k+1 represents the distance estimation update variable updated after the k-th iteration, r k represents the distance estimation variable at the k-th iteration, represents the inverse matrix of the adjustment matrix, T represents the transpose, A represents the diagonal matrix, A -1 represents the inverse matrix of a diagonal matrix; The initial distance estimate is updated using the step size cumulative update variable to obtain an updated distance estimate: in, represents the distance update estimate updated after the k-th iteration, ∥s k+1 ∥1 represents the vector norm of the step-by-step cumulative update variable; The learning rate of the large language model is updated using the distance update estimate to obtain a lower bound on the learning rate: Among them, d k+1 represents the lower bound of the learning rate.

7. The method for confirming task adaptation results based on large language model fine-tuning according to claim 6, characterized in that: The large language model is iteratively trained using the updated learning rate until the final model parameters are output and then the large language model is updated, and the updated large language model is used to perform multiple target-specific tasks to obtain corresponding target results, and the task adaptation results are determined according to all the target results, specifically including: Acquiring structural information of the large language model, and determining the number of iterative training according to the structural information; Respectively using the initial momentum update variable, the second-order momentum update variable, the step accumulation update variable, the distance estimate update variable, the learning rate lower bound, the model update parameters and the initial distance estimate as the initial momentum variable, the second-order momentum variable, the step accumulation variable, the initial learning rate of the distance estimate variable, the initial model parameters and the distance update estimate in the next iterative training process, and iteratively training the large language model until the iteration is completed to obtain the final model parameters; Using the final model parameters to update the large language model to obtain a target model, and inputting input parameters of multiple target specific results into the target model to output corresponding target results; If all the target results are qualified, a task adaptation result is obtained, wherein the task adaptation result indicates that the task adaptability of the target model is enhanced.

8. A task adaptation result confirmation system based on large language model fine-tuning, characterized in that: The task adaptation result confirmation system based on large language model fine-tuning includes: A model information acquisition module, used to obtain initial model parameters, initial distance estimation and multiple variables of the large language model, and initialize all the variables to obtain corresponding initial variables; A first parameter updating module, used for calculating the initial gradient of the initial model parameter, updating a plurality of first initial variables among all the initial variables, and obtaining a plurality of first updated variables; a second parameter updating module, configured to scale the initial gradient using a plurality of the first updating variables to obtain an adjustment matrix, and update the initial model parameters according to the adjustment matrix to obtain model update parameters; a third parameter updating module, configured to update the plurality of second initial variables to obtain a plurality of second updated variables, update the initial distance estimate using the plurality of second updated variables to obtain an updated distance estimate, and update the learning rate of the large language model using the updated distance estimate; The result testing module is used to iteratively train the large language model using the updated learning rate until the final model parameters are output, and then update the large language model, and use the updated large language model to perform multiple target-specific tasks to obtain corresponding target results, and determine the task adaptation results based on all the target results.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a task adaptation result confirmation program based on large language model fine-tuning stored in the memory and executable on the processor. When the task adaptation result confirmation program based on large language model fine-tuning is executed by the processor, the steps of the task adaptation result confirmation method based on large language model fine-tuning according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a task adaptation result confirmation program based on large language model fine-tuning, and when the task adaptation result confirmation program based on large language model fine-tuning is executed by a processor, the steps of the task adaptation result confirmation method based on large language model fine-tuning according to any one of claims 1 to 7 are implemented.

Images