A task adaptation result confirmation method and system based on large language model fine tuning

Abstract

The application discloses a task adaptation result confirmation method and system based on large language model fine tuning, the method comprises the following steps: obtaining model parameters of a large language model, updating the model parameters by using an optimization algorithm, and then updating the learning rate of the large language model by using the updated model parameters; iteratively training the large language model by using the updated learning rate and model parameters until the final model parameters are output, updating the large language model according to the final model parameters, executing a plurality of target specific tasks to obtain corresponding target results, and determining the task adaptation result according to all target results. By introducing an adaptive learning rate adjustment mechanism, the application uses feedback information of the model in the training process to dynamically adjust the learning rate, enhances the model adaptability, enables the large language model to better adapt to specific tasks, and improves the performance of the model in actual application.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a task adaptation result confirmation method and system based on large language model fine-tuning, a terminal and a computer readable storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence and natural language processing technology, large language models (LLM) have shown great capabilities in various application fields. These models have obtained rich language understanding and generation capabilities through pre-training on large-scale text data.

[0003] However, since the pre-trained model is generally universal, its performance on specific tasks may not be satisfactory, such as dependence on labeled data, excessive consumption of computing resources, or complex parameter adjustment, which limits the universality and flexibility of the model in practical applications.

[0004] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0005] The main purpose of the present application is to provide a task adaptation result confirmation method and system based on large language model fine-tuning, a terminal and a computer readable storage medium, which aims to solve the problem of dependence on labeled data, excessive consumption of computing resources or complex parameter adjustment in the pre-training of large language models in the prior art, thereby causing low efficiency or low accuracy when the model performs specific tasks.

[0006] To achieve the above purpose, the present application provides a task adaptation result confirmation method based on large language model fine-tuning, which comprises the following steps:

[0007] Obtain the initial model parameters, initial distance estimates and multiple variables of the large language model, and initialize all the variables to obtain the corresponding initial variables;

[0008] Calculate the initial gradient of the initial model parameters, update multiple first initial variables in all the initial variables to obtain multiple first updated variables;

[0009] Scale the initial gradient using multiple first updated variables to obtain an adjustment matrix, and update the initial model parameters according to the adjustment matrix to obtain model updated parameters;

[0010] Update multiple second initial variables to obtain multiple second updated variables, update the initial distance estimates using multiple second updated variables to obtain distance updated estimates, and update the learning rate of the large language model using the distance updated estimates;

[0011] iteratively training the large language model with the updated learning rate until outputting final model parameters, and updating the large language model, and performing a plurality of target-specific tasks using the updated large language model to obtain corresponding target results, and determining a task adaptation result according to all the target results.

[0012] Optionally, the task adaptation result confirmation method based on large language model fine-tuning, wherein the initial variables include an initial momentum variable, a second-order momentum variable, a step length accumulation variable, and a distance estimation variable.

[0013] The initial model parameters, the initial distance estimation, and a plurality of variables of the large language model are obtained, and all the variables are initialized to obtain corresponding initial variables, specifically including:

[0014] The structure information of the large language model is obtained, and the initial model parameters and the initial distance estimation are determined according to the structure information.

[0015] A plurality of variables of the large language model are obtained, and all the variables are initialized to zero to obtain an initial momentum variable, a second-order momentum variable, a step length accumulation variable, and a distance estimation variable.

[0016] Optionally, the task adaptation result confirmation method based on large language model fine-tuning, wherein the first initial variables include the initial momentum variable and the second-order momentum variable; and the first updated variables include an initial momentum updated variable and a second-order momentum updated variable.

[0017] The initial gradient of the initial model parameters is calculated, and a plurality of first initial variables in all the initial variables are updated to obtain a plurality of first updated variables, specifically including:

[0018] An input parameter input by a user to the large language model is obtained, and a loss function f(x k ,ξ k ) of the large language model is determined according to the input parameter, wherein k represents the kth step in each iteration, x k represents the initial model parameters in the current iteration, and ξ k represents the input parameter.

[0019] The initial gradient of the initial model parameters is calculated according to the loss function:

[0020]

[0021] wherein g k represents the initial gradient in the current iteration, represents derivation of a function.

[0022] obtaining a plurality of hyperparameters, and updating the initial momentum variable and the second-order momentum variable by using the plurality of hyperparameters to obtain an initial momentum update variable and a second-order momentum update variable.

[0023] Optionally, the task adaptation result confirmation method based on large language model fine-tuning, wherein the obtaining a plurality of hyperparameters, and updating the initial momentum variable and the second-order momentum variable by using the plurality of hyperparameters to obtain an initial momentum update variable and a second-order momentum update variable specifically comprises:

[0024] obtaining a plurality of hyperparameters of a preset optimization algorithm, and obtaining a learning rate scaling factor of the large language model;

[0025] updating the initial momentum variable by using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the initial momentum update variable:

[0026] m k+1 = β1m k + (1-β1)d k γ k g k ;

[0027] wherein m k+1 represents the updated initial momentum update variable after the kth iteration, m k represents the initial momentum variable at the kth iteration, β1 represents the first hyperparameter, d k represents the initial distance estimate at the kth iteration, γ k represents the learning rate scaling factor;

[0028] updating the second-order momentum variable by using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the second-order momentum update variable:

[0029]

[0030] wherein v k+1 represents the updated second-order momentum update variable after the kth iteration, v k represents the second-order momentum variable at the kth iteration, and β2 represents the second hyperparameter.

[0031] Optionally, the task adaptation result confirmation method based on large language model fine-tuning, wherein the scaling the initial gradient by using the plurality of first update variables to obtain an adjustment matrix, and updating the initial model parameter according to the adjustment matrix to obtain a model update parameter specifically comprises:

[0032] scaling the initial gradient by the initial momentum update variable and the second order momentum update variable to obtain an adjustment matrix:

[0033]

[0034] wherein A k+1 denotes the adjustment matrix after the k-th iteration, diag() denotes a function of generating a diagonal matrix or extracting a diagonal line of a matrix, ∈ denotes a third hyperparameter, v k+1 denotes the second order momentum update variable;

[0035] updating the initial model parameter according to the adjustment matrix to obtain a model update parameter:

[0036]

[0037] wherein x k+1 denotes the updated model update parameter after the k-th iteration, denotes an inverse matrix of the adjustment matrix, m k+1 denotes the updated initial momentum update variable after the k-th iteration.

[0038] Optionally, the task adaptation result confirmation method based on large language model fine-tuning, wherein the second initial variable comprises: a step length accumulation variable and a distance estimation variable; and the second update variable comprises: a step length accumulation update variable and a distance estimation update variable.

[0039] The method comprises the following steps:

[0040] updating the step length accumulation variable and the distance estimation variable to obtain the step length accumulation update variable and the distance estimation update variable:

[0041]

[0042] wherein s k+1 denotes the updated step length accumulation update variable after the k-th iteration, s k denotes the step length accumulation variable at the k-th iteration, β2 denotes a second hyperparameter, d k denotes the initial distance estimation at the k-th iteration, γ k denotes a learning rate scaling factor, g k denotes the initial gradient in the current iteration;

[0043]

[0044] wherein rk+1 denotes an updated distance estimation update variable after the kth iteration, k denotes a distance estimation variable at the kth iteration, denotes an inverse matrix of an adjustment matrix, T denotes a transpose, A denotes a diagonal matrix, A -1 denotes an inverse matrix of a diagonal matrix;

[0045] an initial distance estimation is updated using the step size accumulated update variable to obtain a distance update estimation:

[0046]

[0047] wherein, denotes an updated distance update estimation after the kth iteration, and k+1 denotes a vector norm of the step size accumulated update variable;

[0048] a learning rate lower bound is obtained by updating the learning rate of the large language model using the distance update estimation:

[0049]

[0050] wherein, d k+1 denotes a learning rate lower bound.

[0051] Optionally, the task adaptation result confirmation method based on large language model fine-tuning, wherein the large language model is iteratively trained using the updated learning rate until the final model parameters are output, the large language model is updated, and a plurality of target-specific tasks are performed using the updated large language model to obtain corresponding target results, and the task adaptation result is determined according to all the target results, specifically comprising:

[0052] obtaining structural information of the large language model, and determining the number of iterations of iterative training according to the structural information;

[0053] respectively, the initial momentum update variable, the second-order momentum update variable, the step size accumulated update variable, the distance estimation update variable, the learning rate lower bound, the model update parameter, and the initial distance estimation are used as the initial momentum variable, the second-order momentum variable, the step size accumulated variable, the distance estimation variable, the initial learning rate, the initial model parameter, and the distance update estimation in the next iteration training process, and the large language model is iteratively trained until the iteration is completed to obtain the final model parameters;

[0054] the large language model is updated using the final model parameters to obtain a target model, and input parameters of a plurality of target-specific results are input into the target model to output corresponding target results;

[0055] If all the target results are qualified, a task adaptation result is obtained, wherein the task adaptation result represents an enhanced task adaptation capability of the target model.

[0056] In addition, to achieve the above object, the present application also provides a task adaptation result confirmation system based on fine-tuning of a large language model, wherein the task adaptation result confirmation system based on fine-tuning of the large language model comprises:

[0057] A model information acquisition module is configured to acquire initial model parameters, initial distance estimation and a plurality of variables of the large language model, and perform initialization processing on all the variables to obtain corresponding initial variables.

[0058] A first parameter updating module is configured to calculate an initial gradient of the initial model parameters, update a plurality of first initial variables among all the initial variables to obtain a plurality of first updated variables.

[0059] A second parameter updating module is configured to scale the initial gradient by using the plurality of first updated variables to obtain an adjustment matrix, and update the initial model parameters according to the adjustment matrix to obtain model updated parameters.

[0060] A third parameter updating module is configured to update a plurality of second initial variables to obtain a plurality of second updated variables, update the initial distance estimation by using the plurality of second updated variables to obtain distance updated estimation, and update a learning rate of the large language model by using the distance updated estimation.

[0061] A result testing module is configured to perform iterative training on the large language model by using the updated learning rate until the large language model is updated after outputting final model parameters, perform a plurality of target specific tasks by using the updated large language model to obtain corresponding target results, and determine a task adaptation result according to all the target results.

[0062] In addition, to achieve the above object, the present application also provides a terminal, wherein the terminal comprises a memory, a processor, and a task adaptation result confirmation program based on fine-tuning of a large language model stored on the memory and executable on the processor, and the task adaptation result confirmation program based on fine-tuning of the large language model implements the steps of the task adaptation result confirmation method based on fine-tuning of the large language model when executed by the processor.

[0063] In addition, to achieve the above object, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a task adaptation result confirmation program based on fine-tuning of a large language model, and the task adaptation result confirmation program based on fine-tuning of the large language model implements the steps of the task adaptation result confirmation method based on fine-tuning of the large language model when executed by a processor.

[0064] In the present application, the initial model parameters, the initial distance estimate and the plurality of variables of the large language model are obtained, and all the variables are initialized to obtain corresponding initial variables; the initial gradient of the initial model parameters is calculated, a plurality of first initial variables in all the initial variables are updated to obtain a plurality of first updated variables; the initial gradient is scaled using a plurality of the first updated variables to obtain an adjustment matrix, and the initial model parameters are updated according to the adjustment matrix to obtain model updated parameters; a plurality of second initial variables are updated to obtain a plurality of second updated variables, the initial distance estimate is updated using a plurality of the second updated variables to obtain a distance updated estimate, and the learning rate of the large language model is updated using the distance updated estimate; the large language model is iteratively trained using the updated learning rate until the final model parameters are output, the large language model is updated, and a plurality of target specific tasks are performed using the updated large language model to obtain corresponding target results, and the task adaptation results are determined according to all the target results. The present application introduces an adaptive learning rate adjustment mechanism, dynamically adjusts the learning rate using the feedback information of the model in the training process, enhances the model adaptability, so that the large language model can better adapt to specific tasks, and improves the performance of the model in practical applications. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 is a flowchart of a preferred embodiment of the task adaptation result confirmation method based on large language model fine-tuning of the present application;

[0066] Figure 2 is a pseudocode diagram of model training of the task adaptation result confirmation method based on large language model fine-tuning of the present application;

[0067] Figure 3 is a structure diagram of a preferred embodiment of the task adaptation result confirmation system based on large language model fine-tuning of the present application;

[0068] Figure 4 is a running environment schematic diagram of a preferred embodiment of the terminal of the present application. DETAILED DESCRIPTION

[0069] To make the purpose, technical scheme and advantages of the present application clearer and more explicit, the present application is further described in detail below with reference to the drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0070] The task adaptation result confirmation method based on large language model fine-tuning according to the preferred embodiment of the present application, as shown in Figure 1 The task adaptation result confirmation method based on large language model fine-tuning includes the following steps:

[0071] Step S10, obtain the initial model parameters, initial distance estimation and multiple variables of the large language model, and initialize all the variables to obtain the corresponding initial variables.

[0072] The initial variables include an initial momentum variable, a second-order momentum variable, a step length accumulation variable and a distance estimation variable.

[0073] Specifically, the structure information of the large language model is obtained, the initial model parameters and the initial distance estimation are determined according to the structure information, and multiple variables of the large language model are obtained. All the variables are initialized to zero to obtain an initial momentum variable, a second-order momentum variable, a step length accumulation variable and a distance estimation variable.

[0074] First, as shown in Figure 2 , multiple model parameters of the large language model that need to be fine-tuned are obtained, wherein the model parameters are generally randomly initialized, or the model parameters obtained after pre-training of the large language model. In addition, the hyperparameters and learning rate of the model of the large language model need to be obtained, including the hyperparameters of various optimization algorithms, such as the hyperparameters of the Adam (Adaptive Moment Estimation) optimizer, and then the adaptive learning rate adjustment mechanism is introduced. The performance indicators of the model on the validation set are dynamically monitored during the training process, so that at the end of each training period, the learning rate is automatically adjusted according to the indicators to ensure that the large language model is trained in the optimal state and improve the training efficiency.

[0075] Step S20, calculate the initial gradient of the initial model parameters, update multiple first initial variables in all the initial variables to obtain multiple first updated variables.

[0076] The first initial variables include the initial momentum variable and the second-order momentum variable, and the first updated variables include an initial momentum updated variable and a second-order momentum updated variable.

[0077] Specifically, the input parameters input by the user into the large language model are obtained, and the loss function f(x k ,ξ k ) of the large language model is determined according to the input parameters, wherein k represents the k-th step in each iteration, x k represents the initial model parameters in the current iteration, and ξ k represents the input parameters; the initial gradient of the initial model parameters is calculated according to the loss function:

[0078]

[0079] wherein g k represents the initial gradient in the current iteration, This indicates taking the derivative of the function;

[0080] Multiple hyperparameters are obtained, and the initial momentum variable and the second-order momentum variable are updated using the multiple hyperparameters to obtain the initial momentum update variable and the second-order momentum update variable.

[0081] In this way, by combining gradient and momentum information, the lower bound of the learning rate of the large language model is adapted and adjusted. In this way, the optimization algorithm can finely adjust the learning rate in each iteration according to the current gradient change, thereby avoiding the problems of unstable training or slow convergence caused by an excessively large or small learning rate.

[0082] Further, multiple hyperparameters of the preset optimization algorithm are obtained, and the learning rate scaling factor of the large language model is obtained; using the multiple hyperparameters, the initial distance estimation, and the learning rate scaling factor, the initial momentum variable is updated to obtain the initial momentum update variable:

[0083] m k+1 =β1m k +(1-β1)d k γ k g k ;

[0084] Where, m k+1 Let m represent the initial momentum update variable updated after the k-th iteration. k Let β1 represent the initial momentum variable at the k-th iteration, and d represent the first hyperparameter. k γ represents the initial distance estimate at the k-th iteration. k This represents the learning rate scaling factor;

[0085] The second-order momentum variable is updated using the aforementioned hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the second-order momentum update variable:

[0086]

[0087] Among them, v k+1 Let v represent the second-order momentum update variable updated after the k-th iteration. k Let β2 represent the second-order momentum variable at the k-th iteration, and let β2 represent the second hyperparameter.

[0088] The learning rate is automatically adjusted by gradually improving the distance estimation, so that the optimal solution distance does not need to be known in advance, and the method has wider applicability when dealing with different types of optimization problems, can well adapt to various specific tasks, and users do not need to have a deep understanding of the target function to obtain good optimization results in practical applications, thereby reducing the optimization cost of the large language model and improving the applicability of optimizing the large language model.

[0089] In step S30, the initial gradient is scaled by using a plurality of first update variables to obtain an adjustment matrix, and the initial model parameters are updated according to the adjustment matrix to obtain model update parameters.

[0090] Specifically, the initial gradient is scaled by using the initial momentum update variable and the second-order momentum update variable to obtain an adjustment matrix:

[0091]

[0092] wherein A k+1 represents the adjustment matrix after the kth iteration, diag() represents a function of generating a diagonal matrix or extracting the diagonal line of the matrix, represents a third hyperparameter, and v k+1 represents the second-order momentum update variable.

[0093] The initial model parameters are updated according to the adjustment matrix to obtain model update parameters:

[0094]

[0095] wherein x k+1 represents the updated model update parameter after the kth iteration, represents the inverse matrix of the adjustment matrix, and m k+1 represents the updated initial momentum update variable after the kth iteration.

[0096] The plurality of hyperparameters used when updating the model parameters are all in the D-Adaptation (Dynamic Adaptation) optimization algorithm, and by making the model training compatible with the optimization algorithm during the training process of the large language model, the model training is integrated as a module in the optimization algorithm, without the need to redesign the entire optimization framework, which greatly reduces the application difficulty.

[0097] Further, other adaptive algorithms can also be used, including: AdaGrad (Adaptive Gradient Algorithm), which adjusts the learning rate by accumulating the square sum of the gradient, suitable for processing sparse data; RMSProp (Root Mean Square Propagation), which adjusts the learning rate by taking the moving average of the square of the gradient, suitable for processing non-stationary targets; Adamax (a variant of Adam), which adjusts the learning rate by using the infinite norm, more suitable for processing scenarios with large gradient changes.

[0098] Step S40, updating a plurality of second initial variables to obtain a plurality of second updated variables, using the plurality of second updated variables, updating the initial distance estimate to obtain a distance updated estimate, and using the distance updated estimate, updating the learning rate of the large language model.

[0099] Among them, the second initial variable includes: a step accumulation variable and a distance estimation variable; the second updated variable includes: a step accumulation updated variable and a distance estimation updated variable.

[0100] Specifically, updating the step accumulation variable and the distance estimation variable to obtain the step accumulation updated variable and the distance estimation updated variable:

[0101]

[0102] Among them, s k+1 represents the updated step accumulation updated variable after the kth iteration, s k represents the step accumulation variable at the kth iteration, β2 represents the second hyperparameter, d k represents the initial distance estimate at the kth iteration, γ k represents the learning rate scaling factor, g k represents the initial gradient in the current iteration;

[0103]

[0104] Among them, r k+1 represents the updated distance estimation updated variable after the kth iteration, r k represents the distance estimation variable at the kth iteration, represents the inverse matrix of the adjustment matrix, T represents the transpose, A represents the diagonal matrix, A -1 represents the inverse matrix of the diagonal matrix;

[0105] Using the step accumulation updated variable, updating the initial distance estimate to obtain a distance updated estimate:

[0106]

[0107] wherein, denotes the distance update estimation after the k-th step iteration, and ||s k+1 denotes the vector norm of the step length cumulative update variable;

[0108] using the distance update estimation, updating the learning rate of the large language model to obtain a learning rate lower bound:

[0109]

[0110] wherein, d k+1 denotes the learning rate lower bound.

[0111] wherein, by updating the model variable and the initial distance estimation, step-by-step estimation of the model learning rate is realized, and the lower bound of the learning rate is dynamically adjusted to obtain the most suitable learning rate.

[0112] Further, an adaptive learning rate method based on Grünwald-Letnikov (G-L) fractional derivative (a definition method of fractional calculus) can also be used to adjust the step size in combination with long-term gradient information to avoid falling into local minimum. This method based on fractional derivative can capture global information and dynamically adjust the step size in combination with short-term gradient changes, making the optimization process more stable and fast converging.

[0113] Step S50, using the updated learning rate to iteratively train the large language model until the final model parameters are output, and updating the large language model, and using the updated large language model to perform a plurality of target-specific tasks to obtain corresponding target results, and determining the task adaptation result according to all the target results.

[0114] Specifically, the structure information of the large language model is obtained, and the number of iteration training is determined according to the structure information; the initial momentum update variable, the second-order momentum update variable, the step length cumulative update variable, the distance estimation update variable, the learning rate lower bound, the model update parameter and the initial distance estimation are respectively used as the initial momentum variable, the second-order momentum variable, the step length cumulative variable, the distance estimation variable initial learning rate, the initial model parameter and the distance update estimation in the next iteration training process, and the large language model is iteratively trained until the iteration is completed to obtain the final model parameters; the final model parameters are used to update the large language model to obtain a target model, and input parameters of a plurality of target-specific results are input into the target model to output corresponding target results; if all the target results are qualified, a task adaptation result is obtained, wherein the task adaptation result represents that the task adaptation ability of the target model is enhanced.

[0115] Wherein, after each training cycle is completed, the model parameters, momentum variables and other information obtained in the last training cycle are used as the initial model parameters and initial momentum variables in the next training cycle, and the learning rate is finely adjusted according to the change of the current gradient in each iteration, thereby avoiding the problems of unstable training or slow convergence caused by excessively large or small learning rate; at the same time, according to different specific target tasks input by the user, the model learning rate required to be used is also different, and according to different specific tasks, the most suitable learning rate is selected for training, thereby improving the universality and applicability of the model application.

[0116] The adaptive learning rate adjustment mechanism is introduced, the learning rate is dynamically adjusted by using the feedback information of the model in the training process, and the adaptability of the model is enhanced, so that the large language model can better adapt to specific tasks and improve the performance of the model in practical application.

[0117] Further, as shown in Figure 3 Based on the above task adaptation result confirmation method based on large language model fine tuning, the present application also correspondingly provides a task adaptation result confirmation system based on large language model fine tuning, wherein the task adaptation result confirmation system based on large language model fine tuning comprises:

[0118] A model information acquisition module 51 is configured to acquire initial model parameters, initial distance estimates and a plurality of variables of a large language model, and initialize all the variables to obtain corresponding initial variables.

[0119] A first parameter updating module 52 is configured to calculate the initial gradient of the initial model parameters, update a plurality of first initial variables in all the initial variables to obtain a plurality of first updated variables.

[0120] A second parameter updating module 53 is configured to scale the initial gradient using a plurality of the first updated variables to obtain an adjustment matrix, and update the initial model parameters according to the adjustment matrix to obtain model updated parameters.

[0121] A third parameter updating module 54 is configured to update a plurality of second initial variables to obtain a plurality of second updated variables, update the initial distance estimates using a plurality of the second updated variables to obtain distance updated estimates, and update the learning rate of the large language model using the distance updated estimates.

[0122] A result testing module 55 is configured to iteratively train the large language model using the updated learning rate until the large language model is updated after outputting the final model parameters, execute a plurality of target specific tasks using the updated large language model to obtain corresponding target results, and determine the task adaptation result according to all the target results.

[0123] Further, as shown in Figure 4 Based on the above task adaptation result confirmation method and system based on large language model fine-tuning, the application further provides a terminal, which comprises a processor 10, a memory 20 and a display 30. Figure 4 Only part of the components of the terminal are shown, but it should be understood that all the shown components are not required to be implemented, and more or less components can be alternatively implemented.

[0124] The memory 20 can be an internal storage unit of the terminal in some embodiments, such as a hard disk or a memory of the terminal. The memory 20 can also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal. Further, the memory 20 can include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software and various data installed on the terminal, such as program codes of the terminal, etc. The memory 20 can also be used to temporarily store data that has been output or will be output. In an embodiment, the memory 20 stores a task adaptation result confirmation program 40 based on large language model fine-tuning, which can be executed by the processor 10, so as to implement the task adaptation result confirmation method based on large language model fine-tuning in the application.

[0125] The processor 10 can be a central processing unit (CPU), a microprocessor or other data processing chip in some embodiments, which is used to run program codes or process data stored in the memory 20, such as to execute the task adaptation result confirmation method based on large language model fine-tuning, etc.

[0126] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 30 is used to display information of the terminal and to display visualized user interfaces. The components of the terminal communicate with each other through a system bus.

[0127] In an embodiment, the following steps are implemented when the processor 10 executes the task adaptation result confirmation program 40 based on large language model fine-tuning in the memory 20:

[0128] Obtaining initial model parameters, initial distance estimation and multiple variables of a large language model, and initializing all the variables to obtain corresponding initial variables;

[0129] Calculating an initial gradient of the initial model parameters, updating multiple first initial variables among all the initial variables to obtain multiple first updated variables;

[0130] Scaling the initial gradient using the multiple first updated variables to obtain an adjustment matrix, and updating the initial model parameters according to the adjustment matrix to obtain model updated parameters;

[0131] Updating multiple second initial variables to obtain multiple second updated variables, updating the initial distance estimation using the multiple second updated variables to obtain distance updated estimation, and updating a learning rate of the large language model using the distance updated estimation;

[0132] Iteratively training the large language model using the updated learning rate until outputting final model parameters, updating the large language model, and performing multiple target specific tasks using the updated large language model to obtain corresponding target results, and determining task adaptation results according to all the target results.

[0133] The initial variables include initial momentum variables, second order momentum variables, step length accumulation variables and distance estimation variables.

[0134] The obtaining initial model parameters, initial distance estimation and multiple variables of a large language model, and initializing all the variables to obtain corresponding initial variables, specifically includes:

[0135] Obtaining structure information of the large language model, and determining the initial model parameters and the initial distance estimation according to the structure information;

[0136] Obtaining multiple variables of the large language model, and initializing all the variables to zero to obtain initial momentum variables, second order momentum variables, step length accumulation variables and distance estimation variables.

[0137] The first initial variables include the initial momentum variables and the second order momentum variables, and the first updated variables include initial momentum updated variables and second order momentum updated variables.

[0138] The calculating an initial gradient of the initial model parameters, updating multiple first initial variables among all the initial variables to obtain multiple first updated variables, specifically includes:

[0139] Obtaining input parameters input by a user to the large language model, and determining a loss function f(x k,ξ k ), where k represents the k-th step in each iteration, x k represents the initial model parameters in the current iteration, ξ k represents the input parameters;

[0140] An initial gradient of the initial model parameters is calculated according to the loss function:

[0141]

[0142] where g k represents the initial gradient in the current iteration, represents the derivative of the function;

[0143] A plurality of hyperparameters are obtained, and the initial momentum variable and the second-order momentum variable are updated using the plurality of hyperparameters to obtain an initial momentum update variable and a second-order momentum update variable.

[0144] The plurality of hyperparameters are obtained, and the initial momentum variable and the second-order momentum variable are updated using the plurality of hyperparameters to obtain an initial momentum update variable and a second-order momentum update variable, and specifically include:

[0145] A plurality of hyperparameters of a preset optimization algorithm are obtained, and a learning rate scaling factor of the large language model is obtained;

[0146] The initial momentum variable is updated using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the initial momentum update variable:

[0147] m k+1 = β1m k +(1-β1)d k γ k g k ;

[0148] where m k+1 represents the updated initial momentum update variable after the k-th step iteration, m k represents the initial momentum variable at the k-th step iteration, β1 represents a first hyperparameter, d k represents the initial distance estimate at the k-th step iteration, γ k represents a learning rate scaling factor;

[0149] The second-order momentum variable is updated using the plurality of hyperparameters, the initial distance estimate, and the learning rate scaling factor to obtain the second-order momentum update variable:

[0150]

[0151] where vk+1 denotes the updated second-order momentum update variable after the kth iteration, v k denotes the second-order momentum variable at the kth iteration, and β2 denotes a second hyperparameter.

[0152] The scaling of the initial gradient by the plurality of first update variables obtains an adjustment matrix, and the updating of the initial model parameter according to the adjustment matrix obtains a model update parameter, and specifically includes:

[0153] The scaling of the initial gradient by the initial momentum update variable and the second-order momentum update variable obtains an adjustment matrix:

[0154]

[0155] wherein A k+1 denotes the adjustment matrix after the kth iteration, diag() denotes a function of generating a diagonal matrix or extracting a diagonal line of a matrix, ∈ denotes a third hyperparameter, and v k+1 denotes the updated second-order momentum update variable;

[0156] The updating of the initial model parameter according to the adjustment matrix obtains a model update parameter:

[0157]

[0158] wherein x k+1 denotes the updated model update parameter after the kth iteration, denotes the inverse matrix of the adjustment matrix, m k+1 denotes the updated initial momentum update variable after the kth iteration.

[0159] The second initial variable includes a step length accumulation variable and a distance estimation variable, and the second update variable includes a step length accumulation update variable and a distance estimation update variable.

[0160] The updating of a plurality of second initial variables obtains a plurality of second update variables, the updating of the initial distance estimation by the plurality of second update variables obtains a distance update estimation, and the updating of the learning rate of the large language model by the distance update estimation, and specifically includes:

[0161] The updating of the step length accumulation variable and the distance estimation variable obtains the step length accumulation update variable and the distance estimation update variable:

[0162]

[0163] wherein s k+1 denotes the updated step length accumulation update variable after the kth iteration, s kdenotes a step size accumulation variable at the kth iteration, and β2denotes a second hyperparameter, d k denotes an initial distance estimate at the kth iteration, and γ k denotes a learning rate scaling factor, and g k denotes an initial gradient in the current iteration;

[0164]

[0165] wherein r k+1 denotes an updated distance estimate after the kth iteration, and r k denotes a distance estimate variable at the kth iteration, denotes an inverse matrix of the adjustment matrix, T denotes a transpose, A denotes a diagonal matrix, and A -1 denotes an inverse matrix of the diagonal matrix;

[0166] using the step size accumulation update variable, the initial distance estimate is updated to obtain a distance update estimate:

[0167]

[0168] wherein, denotes an updated distance update estimate after the kth iteration, and ∥s k+1 denotes a vector norm of the step size accumulation update variable;

[0169] using the distance update estimate, the learning rate of the large language model is updated to obtain a learning rate lower bound:

[0170]

[0171] wherein d k+1 denotes the learning rate lower bound.

[0172] wherein the large language model is iteratively trained using the updated learning rate until the final model parameters are output, and the large language model is updated, and the large language model after the update is used to perform a plurality of target-specific tasks to obtain corresponding target results, and a task adaptation result is determined according to all the target results, specifically comprising:

[0173] obtain the structure information of the large language model, and determine the number of iterations according to the structure information;

[0174] respectively, as initial momentum variables, second-order momentum variables, step size cumulative update variables, distance estimation update variables, lower bounds of learning rates, model update parameters and initial distance estimations in a next iteration training process, and iteratively train the large language model until iteration is completed to obtain final model parameters;

[0175] update the large language model using the final model parameters to obtain a target model, and input input parameters of multiple target-specific results to the target model to output corresponding target results;

[0176] If all the target results are qualified, a task adaptation result is obtained, wherein the task adaptation result represents that the task adaptation capability of the target model is enhanced.

[0177] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a task adaptation result confirmation program based on large language model fine-tuning, and the task adaptation result confirmation program based on large language model fine-tuning, when executed by a processor, implements the steps of the task adaptation result confirmation method based on large language model fine-tuning.

[0178] In summary, the application provides a task adaptation result confirmation method based on large language model fine-tuning and related equipment, which comprises the following steps: obtaining initial model parameters, initial distance estimations and multiple variables of a large language model, and initializing all the variables to obtain corresponding initial variables; calculating initial gradients of the initial model parameters, updating multiple first initial variables among all the initial variables to obtain multiple first update variables; scaling the initial gradients using the multiple first update variables to obtain an adjustment matrix, and updating the initial model parameters according to the adjustment matrix to obtain model update parameters; updating multiple second initial variables to obtain multiple second update variables, updating the initial distance estimations using the multiple second update variables to obtain distance update estimations, and updating learning rates of the large language model using the distance update estimations; iteratively training the large language model using the updated learning rates until the large language model is updated after outputting final model parameters, and performing multiple target-specific tasks using the updated large language model to obtain corresponding target results, and determining a task adaptation result according to all the target results. The application introduces an adaptive learning rate adjustment mechanism, dynamically adjusts the learning rate using feedback information of the model in the training process, enhances the model adaptability, makes the large language model better adapt to specific tasks, improves the performance of the model in actual application, and improves the universality and applicability of the model application.

[0179] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0180] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer readable computer readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The computer readable storage medium can be a memory, a magnetic disc, an optical disc, etc.

[0181] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes shall fall within the protection scope of the appended claims of the present application.

Claims

1. A task adaptation result confirmation method based on large language model fine-tuning, characterized in that, The task adaptation result confirmation method based on large language model fine tuning comprises: Obtain initial model parameters, initial distance estimation and multiple variables of a large language model, and initialize all the variables to obtain corresponding initial variables; Calculate the initial gradient of the initial model parameters, update multiple first initial variables among all the initial variables to obtain multiple first updated variables; Scale the initial gradient using multiple first updated variables to obtain an adjustment matrix, and update the initial model parameters according to the adjustment matrix to obtain model updated parameters; Update multiple second initial variables to obtain multiple second updated variables, update the initial distance estimation using multiple second updated variables to obtain distance updated estimation, and update the learning rate of the large language model using the distance updated estimation; Iteratively train the large language model using the updated learning rate until the final model parameters are output, update the large language model, and execute multiple target specific tasks using the updated large language model to obtain corresponding target results, and determine the task adaptation result according to all the target results.

2. The task adaptation result confirmation method based on large language model fine tuning according to claim 1, characterized in that, The initial variables include initial momentum variables, second order momentum variables, step length cumulative variables and distance estimation variables; The method of obtaining initial model parameters, initial distance estimation and multiple variables of a large language model, and initializing all the variables to obtain corresponding initial variables comprises: Obtain the structure information of the large language model, and determine the initial model parameters and the initial distance estimation according to the structure information; Obtain multiple variables of the large language model, and initialize all the variables to zero to obtain initial momentum variables, second order momentum variables, step length cumulative variables and distance estimation variables. 3.The task adaptation result confirmation method based on large language model fine tuning according to claim 2, characterized in that, The first initial variables include the initial momentum variables and the second order momentum variables, and the first updated variables include initial momentum updated variables and second order momentum updated variables. The method of calculating the initial gradient of the initial model parameters, updating multiple first initial variables among all the initial variables to obtain multiple first updated variables comprises: obtaining an input parameter of user input to the large language model, and determining a loss function f(x k ,ξ k ) of the large language model according to the input parameter, wherein k represents the kth step in each iteration, x k represents the initial model parameter in the current iteration, and ξ k represents the input parameter; Calculate the initial gradient of the initial model parameters according to the loss function: where g k denotes the initial gradient in the current iteration, denotes the derivative of the function; Obtain multiple hyperparameters, and update the initial momentum variables and the second order momentum variables using multiple hyperparameters to obtain initial momentum updated variables and second order momentum updated variables.

4. The task adaptation result confirmation method based on large language model fine tuning according to claim 3, characterized in that, The method of obtaining multiple hyperparameters, and updating the initial momentum variables and the second order momentum variables using multiple hyperparameters to obtain initial momentum updated variables and second order momentum updated variables comprises: Obtain multiple hyperparameters of a preset optimization algorithm, and obtain a learning rate scaling factor of the large language model; Update the initial momentum variables using multiple hyperparameters, the initial distance estimation and the learning rate scaling factor to obtain the initial momentum updated variables: m k+1 = β1m k + (1 - β1)d k γ k g k ; where m k+1 denotes the updated initial momentum update variable after the kth iteration, m k denotes the initial momentum variable at the kth iteration, β1denotes the first hyperparameter, d k denotes the initial distance estimate at the kth iteration, γ k denotes the learning rate scaling factor; Update the second order momentum variables using multiple hyperparameters, the initial distance estimation and the learning rate scaling factor to obtain the second order momentum updated variables: where v k+1 represents the updated second-order momentum update variable after the kth iteration, v k represents the second-order momentum variable at the kth iteration, and β2represents a second hyperparameter. 5.The task adaptation result confirmation method based on large language model fine tuning according to claim 3, characterized in that, The scaling of the initial gradient by the plurality of first update variables obtains an adjustment matrix, and the initial model parameters are updated according to the adjustment matrix to obtain model update parameters, and the specific method comprises: The initial gradient is scaled by the initial momentum update variable and the second-order momentum update variable to obtain an adjustment matrix: where A k+1 denotes the adjustment matrix after the kth iteration, diag() denotes a function that generates a diagonal matrix or extracts the diagonal line of the matrix, ∈ denotes a third hyperparameter, v k+1 denotes a second-order momentum update variable; The initial model parameters are updated according to the adjustment matrix to obtain model update parameters: where x k+1 denotes the updated model update parameter after the kth iteration, denotes the inverse of the adjustment matrix, m k+1 denotes the updated initial momentum update variable after the kth iteration. 6.The task adaptation result confirmation method based on large language model fine tuning according to claim 1, characterized in that, The second initial variable includes a step accumulation variable and a distance estimation variable; the second update variable includes a step accumulation update variable and a distance estimation update variable; The plurality of second update variables are obtained by updating the plurality of second initial variables, the initial distance estimation is updated by using the plurality of second update variables to obtain a distance update estimation, and the learning rate of the large language model is updated by using the distance update estimation, and the specific method comprises: The step accumulation variable and the distance estimation variable are updated to obtain the step accumulation update variable and the distance estimation update variable: where s k+1 denotes the step size accumulated update variable after the kth iteration of the update, s k denotes the step size accumulated variable at the kth iteration, β2denotes the second hyperparameter, d k denotes the initial distance estimate at the kth iteration, γ k denotes the learning rate scaling factor, g k denotes the initial gradient in the current iteration; where r k+1 denotes the updated distance estimate update variable after the kth iteration, r k denotes the distance estimate variable at the kth iteration, denotes the inverse of the adjustment matrix, T denotes the transpose, A denotes a diagonal matrix, A -1 denotes the inverse of the diagonal matrix; The initial distance estimation is updated by using the step accumulation update variable to obtain a distance update estimation: wherein, denotes the updated distance update estimate after the kth step iteration, ∥s k+1 denotes the vector norm of the step size accumulated update variable; The learning rate of the large language model is updated by using the distance update estimation to obtain a lower bound of the learning rate: where d k+1 denotes the lower bound of the learning rate.

7. The task adaptation result confirmation method based on large language model fine tuning according to claim 6, characterized in that, The large language model is iteratively trained by using the updated learning rate until the final model parameters are output, the large language model is updated, and the plurality of target specific tasks are performed by using the updated large language model to obtain corresponding target results, and the task adaptation result is determined according to all the target results, and the specific method comprises: Obtain the structure information of the large language model, and determine the number of iterations according to the structure information; The initial momentum update variable, the second-order momentum update variable, the step accumulation update variable, the distance estimation update variable, the lower bound of the learning rate, the model update parameter and the initial distance estimation are used as the initial momentum variable, the second-order momentum variable, the step accumulation variable, the distance estimation variable, the initial learning rate, the initial model parameter and the distance update estimation in the next iteration training process, and the large language model is iteratively trained until the iteration is completed to obtain the final model parameter; The final model parameter is used to update the large language model to obtain a target model, and the input parameters of the plurality of target specific results are input into the target model to output corresponding target results; If all the target results are qualified, the task adaptation result is obtained, wherein the task adaptation result represents that the task adaptation ability of the target model is enhanced.

8. A task adaptation result confirmation system based on large language model fine-tuning, characterized in that, The task adaptation result confirmation system based on the large language model fine-tuning comprises: A model information acquisition module is configured to acquire initial model parameters, initial distance estimations and a plurality of variables of a large language model, and initialize all the variables to obtain corresponding initial variables; A first parameter update module is configured to calculate an initial gradient of the initial model parameters, update a plurality of first initial variables in all the initial variables to obtain a plurality of first update variables; The second parameter updating module is configured to scale the initial gradient by using the plurality of first updating variables to obtain an adjustment matrix, and update the initial model parameters according to the adjustment matrix to obtain model updating parameters. The third parameter updating module is configured to update a plurality of second initial variables to obtain a plurality of second updating variables, update the initial distance estimation by using the plurality of second updating variables to obtain distance updating estimation, and update the learning rate of the large language model by using the distance updating estimation. The result testing module is configured to iteratively train the large language model by using the updated learning rate until a final model parameter is output, update the large language model, and execute a plurality of target specific tasks by using the updated large language model to obtain corresponding target results, and determine a task adaptation result according to all the target results.

9. A terminal, characterized by The terminal comprises a memory, a processor, and a task adaptation result confirmation program based on fine-tuning of a large language model stored on the memory and executable on the processor, and the task adaptation result confirmation program based on fine-tuning of the large language model, when executed by the processor, implements the steps of the task adaptation result confirmation method based on fine-tuning of the large language model according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a task adaptation result confirmation program based on fine-tuning of a large language model, and the task adaptation result confirmation program based on fine-tuning of the large language model, when executed by the processor, implements the steps of the task adaptation result confirmation method based on fine-tuning of the large language model according to any one of claims 1-7.

Images