Fine tuning training method and system for large language model parameters and medium

By setting adjustment data in the parameter fine-tuning of large language model, dividing training and verification data, calculating initial accuracy, freezing of some parameters and thawing layer by layer, the problem of low fine-tuning efficiency and applicability in the existing technology is solved, and the model is improved accuracy and intelligence of fine-tuning strategies.

CN119067171AInactive Publication Date: 2024-12-03BRAKE AGRI BIG DATA TECH GRP CO LTD

Patent Information

Application Number
CN202410905584.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing large language model parameter fine-tuning method lacks model architecture analysis, which makes it impossible to accurately fine-tune some parameters, resulting in a large amount of memory and calculation budget required for full fine-tuning, and is less efficient and applicable.

Method used

By setting the adjustment data, dividing the training data and verifying data, using the initial language model to calculate the initial accuracy, some parameters are frozen, fine-tuning is used for training data, thawing and fine-tuning layer by layer, retraining is carried out based on the accuracy analysis, and improving the intelligence of parameter fine-tuning is achieved. Sex and accuracy.

Benefits of technology

By calculating and analyzing the accuracy before and after fine-tuning, we can judge the efficiency of layer-by-layer thawing, improve the intelligence of parameter fine-tuning and model accuracy, determine the best freezing level and fine-tuning strategy, and improve fine-tuning efficiency and model performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067171A_ABST
    Figure CN119067171A_ABST
Patent Text Reader

Abstract

The invention discloses a fine-tuning training method and system for large language model parameters and a medium, and relates to the technical field of natural language processing, and the method comprises the steps: setting adjustment data, and dividing the adjustment data; calculating the accuracy rate of the model, and outputting the initial accuracy rate; performing partial parameter freezing on the initial language model, performing model training, and outputting a fine tuning language model; calculating the accuracy rate of the model, and outputting the adjustment accuracy rate; analyzing the adjustment accuracy and the initial accuracy, and performing retraining processing or outputting adjustment completion information; the method is used for solving the problems that in the prior art, due to the fact that model architecture analysis is insufficient, fine adjustment of part of parameters cannot be accurately conducted, enough internal storage and calculation budget need to be provided in the full fine adjustment process, and the fine adjustment efficiency and applicability are low.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a fine-tuning training method, system and medium for large language model parameters. Background Art

[0002] Fine-tuning of large language model parameters refers to the process of adjusting the model parameters of a large language model (LLM) to adapt the large language model to a specific task. Unlike the pre-training phase that uses a large amount of unstructured text data, fine-tuning is a supervised learning process. This means using labeled example datasets to update the weights of the LLM so that the model can better complete specific tasks.

[0003] Existing improvements for fine-tuning of large language model parameters usually focus on the selection of training set data, so that the fine-tuned model outputs more accurate results for specific tasks. For example, in a Chinese patent with invention publication number CN117494786A, a large language model hot search generation method and device based on fine-tuning is disclosed. The scheme is to collect hot search data, extract hot events, construct training sets and verification sets, and then train to obtain a fine-tuned large language model. Existing improvement methods are usually full fine-tuning. Compared with partial fine-tuning, full fine-tuning requires sufficient memory and computing budget to store and process all gradients, optimizers and other update components in the training process, which will result in the full fine-tuning method. The applicability is low. In view of this, it is necessary to improve the existing fine-tuning method of large language model parameters. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems in the prior art to a certain extent. By improving the fine-tuning method of large language model parameters, the present invention is used to solve the problem that due to the lack of model architecture analysis in the prior art, some parameters cannot be accurately fine-tuned, and sufficient memory and computing budget need to be provided during full fine-tuning, resulting in low efficiency and applicability of fine-tuning.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for fine-tuning parameters of a large language model, comprising: Setting a first amount of adjustment data, dividing the adjustment data, and outputting training data and verification data; the verification data includes a data set and prediction output information associated with the data set; Use the initial language model to process the verification data and output the actual output information; calculate the accuracy of the model based on the predicted output information and the actual output information, and output the initial accuracy; Freeze some parameters of the initial language model, use the training data to train the initial language model, and output a fine-tuned language model; Use the fine-tuned language model to process the verification data and output the adjusted output information; calculate the accuracy of the model based on the predicted output information and the adjusted output information, and output the adjusted accuracy; The adjustment accuracy and the initial accuracy are analyzed, and retraining is performed or adjustment completion information is output based on the analysis results.

[0006] Further, setting a first amount of adjustment data, dividing the adjustment data, and outputting training data and verification data includes: Setting a first amount of adjustment data related to a domain based on a specific domain to which the large language model is to be applied; The first amount of adjusted data is equally divided, and the two groups of equally divided data are respectively marked as training data and verification data.

[0007] Furthermore, the verification data is processed using the large language model to output actual output information; the accuracy of the model is calculated based on the predicted output information and the actual output information, and the output initial accuracy includes: Loading pre-trained parameters to obtain an initial language model; the initial language model includes an embedding layer, a hidden layer, and an output layer; the hidden layer includes multiple Transformer layers, and the initial language model can receive training data and perform fine-tuning, as well as receive verification data and output actual output information; Set the first calculated value to an initial value of 0; Executing a first accurate calculation cycle, the first accurate calculation cycle comprising: Input a single verification data set into the initial language model, obtain the actual output information of the initial language model, and mark the input verification data as verified data; The actual output information and the predicted output information are analyzed by using natural language processing technology, and when the actual output information is similar to the predicted output information, the first calculated value is increased by one; when the actual output information is not similar to the predicted output information, no processing is performed; Repeat the first accurate calculation cycle, and when all the verification data are marked as verified data, end the first accurate calculation cycle; mark all the verified data back to verification data; Calculating the first calculated value using the first calculation formula and outputting an initial accuracy rate; The first calculation formula is configured as: ; Wherein ZQ1 is the initial accuracy, NQ is the first calculated value, and DY is the first quantity.

[0008] Furthermore, some parameters of the initial language model are frozen, and the initial language model is trained using the training data. The output fine-tuned language model includes: Get the model architecture of the initial language model, number the Transformer layer closest to the output layer in the model architecture as 1; number the Transformer layer second closest to the output layer in the model architecture as 2, and so on, until the Transformer layer farthest from the output layer is numbered as n, where n is the number of Transformer layers; Define the number of cycles i with an initial value of 0 and the number of adjustments m; Define the number of update layers d with an initial value of 1; Perform a fine-tuning step, the fine-tuning step comprising: Update d to d+i+m; Freeze the model parameters of the Transformer layers numbered 1 to nd and the embedding layer; Define a loss function based on the adjusted data; Input the training data into the initial language model for training, use the back propagation algorithm to update the model parameters, and mark the initial language model after the model parameters are updated as the fine-tuning language model; Update i to i+1.

[0009] Further, the verification data is processed using the fine-tuned language model to output adjusted output information; the accuracy of the model is calculated based on the predicted output information and the adjusted output information, and the output adjusted accuracy includes: Execute an adjustment accuracy calculation step, the adjustment accuracy calculation step comprising: Set the second calculated value to 0 as the initial value; Executing a second accurate calculation cycle, the second accurate calculation cycle comprising: Input a data set of a single verification data into the fine-tuned language model, obtain the adjusted output information output by the fine-tuned language model, and mark the input verification data as verified data; The adjusted output information and the predicted output information are analyzed by using natural language processing technology, and when the adjusted output information is similar to the predicted output information, the second calculated value is increased by one; when the actual output information is not similar to the predicted output information, no processing is performed; Repeat the second accurate calculation cycle, and when all the verification data are marked as verified data, end the second accurate calculation cycle; mark all the verified data back to verification data; Calculating the second calculated value using the second calculation formula, and outputting an adjustment accuracy rate; The second calculation formula is configured as: ; Where ZQ2 is the adjustment accuracy and MQ is the second calculated value.

[0010] Further, analyzing the adjustment accuracy and the initial accuracy, and performing retraining processing or outputting adjustment completion information based on the analysis results include: Execute an accuracy determination step, the accuracy determination step comprising: Calculate the difference between ZQ2 and ZQ1, and mark it as the rising accuracy; Compare the rising accuracy rate with the first rising threshold and the second rising threshold, and when the rising accuracy rate is greater than or equal to the second rising threshold, output adjustment completion information; When the rising accuracy is less than the second rising threshold and greater than or equal to the first rising threshold, outputting first retraining information; When the rising accuracy is less than the first rising threshold, the second retraining information is output.

[0011] Further, analyzing the adjusted accuracy and the initial accuracy, and performing retraining based on the analysis results also includes: When the first retraining information is received, the parameters of all layers are unfrozen, the adjusted language model is marked back to the initial language model, and the fine-tuning step, the accuracy calculation step, and the accuracy judgment step are performed again.

[0012] Further, analyzing the adjusted accuracy and the initial accuracy, and performing retraining based on the analysis results also includes: When the second retraining information is received, m is updated to m+k, where k is a constant; the parameters of all layers are unfrozen, the adjusted language model is marked back to the initial language model, and the fine-tuning step, the accuracy calculation step, and the accuracy judgment step are performed again.

[0013] In a second aspect, the present invention also provides a fine-tuning training system for large language model parameters, including a performance calculation module, a model training module, and a performance judgment module; The performance calculation module is used to set a first amount of adjustment data, divide the adjustment data, and output training data and verification data; the verification data includes a data set and prediction output information associated with the data set; The performance calculation module is also used to process the verification data using the initial language model and output the actual output information; calculate the accuracy of the model based on the predicted output information and the actual output information, and output the initial accuracy; The performance calculation module is also used to process the verification data using the fine-tuned language model and output the adjusted output information; calculate the accuracy of the model based on the predicted output information and the adjusted output information, and output the adjusted accuracy; The model training module is used to freeze some parameters of the initial language model, train the initial language model using the training data, and output a fine-tuned language model; The performance judgment module is used to analyze the adjustment accuracy and the initial accuracy, and perform retraining processing or output adjustment completion information based on the analysis results.

[0014] In a third aspect, the present application provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the above method are executed.

[0015] Beneficial effects of the present invention: The present invention first processes the verification data using the initial language model to calculate the initial accuracy; then calculates the adjusted accuracy of the adjusted language model, and by calculating and analyzing the initial accuracy and the adjusted accuracy, it is possible to determine whether the fine-tuning is completed; the advantage of this is that by calculating and analyzing the accuracy before and after fine-tuning, the efficiency of layer-by-layer unfreezing can be determined, and different subsequent steps can be executed according to different calculated rising accuracy rates, which can improve the intelligence of parameter fine-tuning and the accuracy of the model obtained by fine-tuning; The present invention also freezes parameters based on the model architecture, and unfreezes them layer by layer or multiple layers according to the calculated rising accuracy, and after unfreezing, retraining can continue on the original model; the advantage of this is that, starting from the initial frozen level, more layers can be gradually unfrozen and fine-tuned; through gradual unfreezing and fine-tuning, the performance changes of the model on the validation set can be observed to determine the optimal freezing level and fine-tuning strategy.

[0016] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a functional block diagram of the system of the present invention; Figure 2 is a flow chart of the steps of the method of the present invention; Figure 3 It is a partial data flow diagram of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Example 1, first aspect, please refer to Figure 1As shown, the present invention provides a fine-tuning training system for large language model parameters, including a performance calculation module, a model training module and a performance judgment module; The performance calculation module is used to set a first amount of adjustment data, divide the adjustment data, and output training data and verification data; the training data and verification data respectively include a data set and predicted output information associated with the data set; the performance calculation module is also used to process the verification data using the initial language model and output actual output information; calculate the accuracy of the model based on the predicted output information and the actual output information, and output the initial accuracy; the performance calculation module is also used to process the verification data using the fine-tuning language model and output the adjustment output information; calculate the accuracy of the model based on the predicted output information and the adjustment output information, and output the adjustment accuracy; The performance calculation module is configured with a fine-tuning preparation strategy, and the fine-tuning preparation strategy is configured to: set a first amount of adjustment data related to the domain based on a special domain to which the large language model needs to be applied; Dividing the first amount of adjusted data equally, and marking the two groups of data after the equal division as training data and verification data, respectively, where the verification data includes a data set and prediction output information associated with the data set; It should be noted that training data also includes data sets and prediction output information associated with the data sets, but is only used to train large language models in the model training strategy and does not participate in calculating the initial accuracy or adjusting the accuracy. It should also be noted that the pre-training data of large language models is usually obtained from large-scale general texts such as the Internet; this means that the model may not have enough training data for specialized terms in special fields, semantic relationships and contexts in specific fields; by fine-tuning the model, data from specific fields can be used to correct data bias and improve the performance of the model in that field; Special fields include but are not limited to the legal field, medical field, and educational field. Taking the legal field as an example, this field involves legal terminology, regulations, case law, and specific legal language. By fine-tuning the large language model to the legal field, it can be applied to tasks such as abstract extraction of legal documents, legal consultation, and contract analysis. The first amount of adjustment data related to the field is set, and the first amount can be set to 200. Since the subsequent step is to adjust some parameters of the large language model, unlike full fine-tuning, the first amount does not need to be set too large; the setting of the lower limit of the first amount needs to be determined in combination with the specific pre-trained large language model loaded; the step of setting the adjustment data can use network communication to obtain the latest or most reference cases.

[0020] The performance calculation module is also configured with a first calculation strategy, and the first calculation strategy is configured as follows: Load the pre-trained parameters to obtain an initial language model; the initial language model includes an embedding layer, a hidden layer, and an output layer; the hidden layer includes multiple Transformer layers, and the initial language model can receive training data and perform fine-tuning, as well as receive verification data and output actual output information; It should be noted that the loaded parameters can be parameters pre-trained on a large-scale corpus, such as using the GPT-3.5 model; Set the first calculated value to an initial value of 0; It should be noted that the large language model processes the data set of the validation data and outputs the actual output information. When the actual output information is similar to the predicted output information, it is necessary to count, so the first calculated value with an initial value of 0 is set; the second calculated value in the subsequent processing is similar, and is used to count the fine-tuned language model; Execute a first accurate calculation cycle, the first accurate calculation cycle includes: Input a single verification data set into the initial language model, obtain the actual output information of the initial language model, and mark the input verification data as verified data; The actual output information and the predicted output information are analyzed by using natural language processing technology, and when the actual output information is similar to the predicted output information, the first calculated value is increased by one; when the actual output information is not similar to the predicted output information, no processing is performed; In the specific implementation, taking the field of education as an example, for example, the input data set is: Why do apples on a fruit tree fall vertically downward after they mature instead of falling in other directions? The predicted output information is: The gravity of the earth causes objects to move toward the center of the earth. The apple is affected by the gravity of the earth and moves vertically downward. One possible actual output information is: The apple is affected by gravity. Natural language processing technology can be used to identify that the actual output information and the predicted output information have the same final meaning, but the predicted output information includes an explanation of the final meaning, so the actual output information is not similar to the predicted output information; Repeat the first accurate calculation cycle, and when all the verification data are marked as verified data, end the first accurate calculation cycle; mark all the verified data back to verification data; Calculating the first calculated value using the first calculation formula and outputting an initial accuracy rate; The first calculation formula is configured as: ; Wherein ZQ1 is the initial accuracy, NQ is the first calculated value, and DY is the first quantity.

[0021] The model training module is used to freeze some parameters of the initial language model, use the training data to train the initial language model, and output a fine-tuned language model; See also Figure 3 As shown, Figure 3The left side of Figure 3 is a flowchart of the model training strategy, the dashed box is a flowchart of the fine-tuning step, and the right side of Figure 4 is a flowchart of the retraining processing strategy and the improvement analysis strategy; The model training module is configured with a model training strategy, which includes: Get the model architecture of the initial language model, number the Transformer layer closest to the output layer in the model architecture as 1; number the Transformer layer second closest to the output layer in the model architecture as 2, and so on, until the Transformer layer farthest from the output layer is numbered as n, where n is the number of Transformer layers; In specific implementation, the model architecture of the initial language model can be visualized by viewing the model code or documentation or using visualization tools (such as TensorBoard) to finally determine the stacking order of the Transformer layer and determine the distance relationship between the Transformer layer and the output layer; It should be noted that the parameters of the Transformer layer in the hidden layer that is closer to the output layer carry most of the context understanding capabilities and have a greater impact on the performance and expression capabilities of the model. Therefore, the parameters of the Transformer layer closest to the output layer are preferentially selected for fine-tuning, and the parameters are unfrozen layer by layer or multiple layers based on the accuracy after fine-tuning. Define the number of cycles i with an initial value of 0 and the number of adjustments m; Define the number of update layers d with an initial value of 1; Perform fine-tuning steps, which include: Update d to d+i+m; Freeze the model parameters of the Transformer layers numbered 1 to nd and the embedding layer; In specific implementation, you can use the parameter freezing mechanism provided by the deep learning framework, such as PyTorch or TensorFlow, to freeze the parameters. For example, if the initial language model includes 10 Transformer layers, the Python code for freezing the parameters of the Transformer layers numbered 1 to 9 is as follows: for param in model.encoder.layer[:9].parameters(): param.requires_grad = False; In the above code, model.encoder.layer[:9].parameters() selects the parameters of the first 9 Transformer layers and sets them to be non-trainable by requiring_grad = False; It should be noted that the specific freezing code needs to be set according to the specific initial language model; Define a loss function based on the adjusted data; It should be noted that the specific definition steps of the loss function need to be based on the specific application scenario. Common loss functions include cross entropy loss and mean square error. For special requirements of some tasks, other evaluation indicators can be defined as loss functions. Input the training data into the initial language model for training, use the back propagation algorithm to update the model parameters, and mark the initial language model after the model parameters are updated as the fine-tuning language model; Update i to i+1.

[0022] The performance calculation module is also configured with a second calculation strategy, and the second calculation strategy is configured as follows: The adjustment accuracy calculation step is performed, and the adjustment accuracy calculation step includes: Set the second calculated value to 0 as the initial value; Execute a second accurate calculation cycle, the second accurate calculation cycle includes: Input a data set of a single verification data into the fine-tuned language model, obtain the adjusted output information output by the fine-tuned language model, and mark the input verification data as verified data; The adjusted output information and the predicted output information are analyzed by using natural language processing technology, and when the adjusted output information is similar to the predicted output information, the second calculated value is increased by one; when the actual output information is not similar to the predicted output information, no processing is performed; Repeat the second accurate calculation cycle, and when all the verification data are marked as verified data, end the second accurate calculation cycle; mark all the verified data back to verification data; Calculating the second calculated value using the second calculation formula, and outputting an adjustment accuracy rate; The second calculation formula is configured as: ; Where ZQ2 is the adjustment accuracy and MQ is the second calculated value.

[0023] The performance judgment module is used to analyze the adjustment accuracy and the initial accuracy, and perform retraining processing or output adjustment completion information based on the analysis results; The performance judgment module is configured with an improvement analysis strategy, which includes: Execute the accuracy judgment step, which includes: Calculate the difference between ZQ2 and ZQ1, and mark it as the rising accuracy; Compare the rising accuracy rate with the first rising threshold and the second rising threshold, and when the rising accuracy rate is greater than or equal to the second rising threshold, output adjustment completion information; In specific implementation, the first rising threshold is set to 10%, and the second rising threshold is set to 30%; the second rising threshold is specifically determined by the difference between the target accuracy and ZQ1. For example, if the final expected model accuracy reaches 95% and ZQ1 is 70%, the second rising threshold is 25%; the first rising threshold is a fixed value, which is used to predict the fine-tuning efficiency of the model when unfreezing layer by layer. For example, when the rising accuracy is 12%, only the parameters of the Transformer layer numbered 10 are adjusted. When the adjustment of numbers 10 and 9 is predicted, the rising accuracy is 24%, and so on. It is expected to output the adjustment completion information after the second retraining is completed; When the rising accuracy is 8%, if layer-by-layer unfreezing is used, it may take until the fourth retraining to output the adjustment completion information. At this time, the fine-tuning efficiency is low. Therefore, the initial value is defined as 0 adjustment times m. When the rising accuracy is less than 10%, k is set to 2 in the subsequent steps, and the original layer-by-layer unfreezing is adjusted to unfreezing two layers each time. At this time, it is expected that the adjustment completion information will be output after the second retraining. Set the value of k according to the specific situation to improve the efficiency of fine-tuning; When the rising accuracy is less than the second rising threshold and greater than or equal to the first rising threshold, outputting first retraining information; When the rising accuracy is less than the first rising threshold, outputting second retraining information; The performance judgment module is also configured with a retraining processing strategy, which includes: When the first retraining information is received, the parameters of all layers are unfrozen, the adjusted language model is marked back to the initial language model, and the fine-tuning step, the accuracy calculation step, and the accuracy judgment step are performed again.

[0024] When the second retraining information is received, m is updated to m+k, where k is a constant; the parameters of all layers are unfrozen, the adjusted language model is marked back to the initial language model, and the fine-tuning step, the accuracy calculation step, and the accuracy judgment step are performed again.

[0025] Example 2, second aspect, please refer to Figure 2 As shown, the present invention also provides a fine-tuning training method for large language model parameters, including: step S1, setting a first amount of adjustment data, dividing the adjustment data, and outputting training data and verification data; the training data and the verification data respectively include a data set and predicted output information associated with the data set; step S1 includes the following sub-steps: Step S101, setting a first amount of adjustment data related to a field based on a special field to which the large language model needs to be applied; Step S102: equally divide the first amount of adjusted data, and mark the two equally divided groups of data as training data and verification data, respectively. The training data and the verification data respectively include a data set and prediction output information associated with the data set.

[0026] Step S2, using the initial language model to process the verification data and output the actual output information; based on the predicted output information and the actual output information, the accuracy of the model is calculated and the initial accuracy is output; Step S2 includes the following sub-steps: Step S201, loading pre-training parameters to obtain an initial language model; the initial language model includes an embedding layer, a hidden layer, and an output layer; the hidden layer includes multiple Transformer layers, and the initial language model can receive training data and perform fine-tuning, as well as receive verification data and output actual output information; Step S202, setting the first calculated value with an initial value of 0; Step S203, executing a first accurate calculation cycle, the first accurate calculation cycle includes: Input a single verification data set into the initial language model, obtain the actual output information of the initial language model, and mark the input verification data as verified data; The actual output information and the predicted output information are analyzed by using natural language processing technology, and when the actual output information is similar to the predicted output information, the first calculated value is increased by one; when the actual output information is not similar to the predicted output information, no processing is performed; Repeat the first accurate calculation cycle, and when all the verification data are marked as verified data, end the first accurate calculation cycle; mark all the verified data back to verification data; Calculating the first calculated value using the first calculation formula and outputting an initial accuracy rate; The first calculation formula is configured as: ; Wherein ZQ1 is the initial accuracy, NQ is the first calculated value, and DY is the first quantity.

[0027] Step S3, partially freeze the parameters of the initial language model, use the training data to train the initial language model, and output a fine-tuned language model; Step S3 includes the following sub-steps: Step S301, obtaining the model architecture of the initial language model, numbering the Transformer layer closest to the output layer in the model architecture as 1; numbering the Transformer layer second closest to the output layer in the model architecture as 2, and so on, until the Transformer layer farthest from the output layer is numbered as n, where n is the number of Transformer layers; Step S302, defining the number of cycles i with an initial value of 0 and the number of adjustments m; Step S303, defining the number of update layers d with an initial value of 1; Step S304, performing a fine-tuning step, the fine-tuning step includes: Update d to d+i+m; Freeze the model parameters of the Transformer layers numbered 1 to nd and the embedding layer; Define a loss function based on the adjusted data; Input the training data into the initial language model for training, use the back propagation algorithm to update the model parameters, and mark the initial language model after the model parameters are updated as the fine-tuning language model; Update i to i+1.

[0028] Step S4, using the fine-tuned language model to process the verification data and output the adjusted output information; based on the predicted output information and the adjusted output information, the accuracy of the model is calculated and the adjusted accuracy is output; Step S4 includes the following sub-steps: Step S401, executing an adjustment accuracy calculation step, the adjustment accuracy calculation step includes: Set the second calculated value to 0 as the initial value; Execute a second accurate calculation cycle, the second accurate calculation cycle includes: Input a data set of a single verification data into the fine-tuned language model, obtain the adjusted output information output by the fine-tuned language model, and mark the input verification data as verified data; The adjusted output information and the predicted output information are analyzed by using natural language processing technology, and when the adjusted output information is similar to the predicted output information, the second calculated value is increased by one; when the actual output information is not similar to the predicted output information, no processing is performed; Repeat the second accurate calculation cycle, and when all the verification data are marked as verified data, end the second accurate calculation cycle; mark all the verified data back to verification data; Calculating the second calculated value using the second calculation formula, and outputting an adjustment accuracy rate; The second calculation formula is configured as: ; Where ZQ2 is the adjustment accuracy and MQ is the second calculated value.

[0029] Step S5, analyzing the adjustment accuracy and the initial accuracy, and performing retraining processing or outputting adjustment completion information based on the analysis results; Step S5 includes the following sub-steps: Step S501, executing the accuracy determination step, the accuracy determination step includes: Calculate the difference between ZQ2 and ZQ1, and mark it as the rising accuracy; Compare the rising accuracy rate with the first rising threshold and the second rising threshold, and when the rising accuracy rate is greater than or equal to the second rising threshold, output adjustment completion information; When the rising accuracy is less than the second rising threshold and greater than or equal to the first rising threshold, outputting first retraining information; When the rising accuracy is less than the first rising threshold, the second retraining information is output.

[0030] Step S502, when the first retraining information is received, the parameters of all layers are unfrozen, the adjusted language model is marked back to the initial language model, and the fine-tuning step, the accuracy calculation step and the accuracy judgment step are performed again.

[0031] Step S503, when the second retraining information is received, m is updated to m+k, where k is a constant; the parameters of all layers are unfrozen, the adjusted language model is marked back to the initial language model, and the fine-tuning step, the accuracy calculation step, and the accuracy judgment step are performed again.

[0032] Embodiment 3, in the third aspect, the present application provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method are executed. Through the above technical solution, when the computer program is executed by the processor, the method in any optional implementation of the above embodiment is executed to achieve the following functions: set a first amount of adjustment data, divide the adjustment data, output training data and verification data; process the verification data using the initial language model, and output actual output information; calculate the accuracy of the model based on the predicted output information and the actual output information, and output the initial accuracy; freeze some parameters of the initial language model, use the training data to train the initial language model, and output the fine-tuning language model; process the verification data using the fine-tuning language model, and output the adjustment output information; calculate the accuracy of the model based on the predicted output information and the adjustment output information, and output the adjustment accuracy; analyze the adjustment accuracy and the initial accuracy, and perform retraining processing or output adjustment completion information based on the analysis results.

[0033] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0034] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. Among them, the storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0035] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

Claims

1. A fine-tuning training method for large language model parameters, characterized in that: include: Setting a first amount of adjustment data, dividing the adjustment data, and outputting training data and verification data; The verification data includes a data set and prediction output information associated with the data set; Use the initial language model to process the verification data and output the actual output information; Calculate the accuracy of the model based on the predicted output information and the actual output information, and output the initial accuracy; Freeze some parameters of the initial language model, use the training data to train the initial language model, and output a fine-tuned language model; Use the fine-tuned language model to process the verification data and output the adjusted output information; The accuracy of the model is calculated based on the predicted output information and the adjusted output information, and the output adjustment accuracy is output; The adjustment accuracy and the initial accuracy are analyzed, and retraining is performed or adjustment completion information is output based on the analysis results.

2. The fine-tuning training method for large language model parameters according to claim 1, characterized in that: Setting a first amount of adjustment data, dividing the adjustment data, and outputting training data and verification data include: Setting a first amount of adjustment data related to a domain based on a specific domain to which the large language model is to be applied; The first amount of adjusted data is equally divided, and the two groups of equally divided data are respectively marked as training data and verification data.

3. The fine-tuning training method for large language model parameters according to claim 2, characterized in that: Use the initial language model to process the verification data and output the actual output information; The accuracy of the model is calculated based on the predicted output information and the actual output information. The output initial accuracy includes: Loading pre-trained parameters to obtain an initial language model; the initial language model includes an embedding layer, a hidden layer, and an output layer; the hidden layer includes multiple Transformer layers, and the initial language model can receive training data and perform fine-tuning, as well as receive verification data and output actual output information; Set the first calculated value to an initial value of 0; Executing a first accurate calculation cycle, the first accurate calculation cycle comprising: Input a single verification data set into the initial language model, obtain the actual output information of the initial language model, and mark the input verification data as verified data; The actual output information and the predicted output information are analyzed by using natural language processing technology, and when the actual output information is similar to the predicted output information, the first calculated value is increased by one; when the actual output information is not similar to the predicted output information, no processing is performed; Repeat the first accurate calculation cycle, and when all the verification data are marked as verified data, end the first accurate calculation cycle; mark all the verified data back to verification data; Calculating the first calculated value using the first calculation formula and outputting an initial accuracy rate; The first calculation formula is configured as: ; Wherein ZQ1 is the initial accuracy, NQ is the first calculated value, and DY is the first quantity.

4. The fine-tuning training method for large language model parameters according to claim 3, characterized in that: Freeze some parameters of the initial language model, use the training data to train the initial language model, and output the fine-tuned language model including: Get the model architecture of the initial language model, number the Transformer layer closest to the output layer in the model architecture as 1; number the Transformer layer second closest to the output layer in the model architecture as 2, and so on, until the Transformer layer farthest from the output layer is numbered as n, where n is the number of Transformer layers; Define the number of cycles i with an initial value of 0 and the number of adjustments m; Define the number of update layers d with an initial value of 1; Perform a fine-tuning step, the fine-tuning step comprising: Update d to d+i+m; Freeze the model parameters of the Transformer layers numbered 1 to nd and the embedding layer; Define a loss function based on the adjusted data; Input the training data into the initial language model for training, use the back propagation algorithm to update the model parameters, and mark the initial language model after the model parameters are updated as the fine-tuning language model; Update i to i+1.

5. The fine-tuning training method for large language model parameters according to claim 4, characterized in that: Use the fine-tuned language model to process the verification data and output the adjusted output information; The accuracy of the model is calculated based on the predicted output information and the adjusted output information. The output adjustment accuracy includes: Execute an adjustment accuracy calculation step, the adjustment accuracy calculation step comprising: Set the second calculated value to 0 as the initial value; Executing a second accurate calculation cycle, the second accurate calculation cycle comprising: Input a data set of a single verification data into the fine-tuned language model, obtain the adjusted output information output by the fine-tuned language model, and mark the input verification data as verified data; The adjusted output information and the predicted output information are analyzed by using natural language processing technology, and when the adjusted output information is similar to the predicted output information, the second calculated value is increased by one; when the actual output information is not similar to the predicted output information, no processing is performed; Repeat the second accurate calculation cycle, and when all the verification data are marked as verified data, end the second accurate calculation cycle; mark all the verified data back to verification data; Calculating the second calculated value using the second calculation formula, and outputting an adjustment accuracy rate; The second calculation formula is configured as: ; Where ZQ2 is the adjustment accuracy and MQ is the second calculated value.

6. The fine-tuning training method for large language model parameters according to claim 5, characterized in that: Analyze the adjustment accuracy and initial accuracy, and perform retraining or output adjustment completion information based on the analysis results, including: Execute an accuracy determination step, the accuracy determination step comprising: Calculate the difference between ZQ2 and ZQ1, and mark it as the rising accuracy; Compare the rising accuracy rate with the first rising threshold and the second rising threshold, and when the rising accuracy rate is greater than or equal to the second rising threshold, output adjustment completion information; When the rising accuracy is less than the second rising threshold and greater than or equal to the first rising threshold, outputting first retraining information; When the rising accuracy is less than the first rising threshold, the second retraining information is output.

7. The fine-tuning training method for large language model parameters according to claim 6, characterized in that: Analyzing the adjusted accuracy and the initial accuracy, and performing retraining based on the analysis results also includes: When the first retraining information is received, the parameters of all layers are unfrozen, the adjusted language model is marked back to the initial language model, and the fine-tuning step, the accuracy calculation step, and the accuracy judgment step are performed again.

8. The method for fine-tuning parameters of a large language model according to claim 7, characterized in that: Analyzing the adjusted accuracy and the initial accuracy, and performing retraining based on the analysis results also includes: When the second retraining information is received, m is updated to m+k, where k is a constant; the parameters of all layers are unfrozen, the adjusted language model is marked back to the initial language model, and the fine-tuning step, the accuracy calculation step, and the accuracy judgment step are performed again.

9. A fine-tuning training system for large language model parameters, implemented based on a fine-tuning training method for large language model parameters according to any one of claims 1 to 8, characterized in that: Includes performance calculation module, model training module and performance judgment module; The performance calculation module is used to set a first amount of adjustment data, divide the adjustment data, and output training data and verification data; the verification data includes a data set and prediction output information associated with the data set; The performance calculation module is also used to process the verification data using the initial language model and output the actual output information; calculate the accuracy of the model based on the predicted output information and the actual output information, and output the initial accuracy; The performance calculation module is also used to process the verification data using the fine-tuned language model and output adjusted output information; The accuracy of the model is calculated based on the predicted output information and the adjusted output information, and the output adjustment accuracy is output; The model training module is used to freeze some parameters of the initial language model, train the initial language model using the training data, and output a fine-tuned language model; The performance judgment module is used to analyze the adjustment accuracy and the initial accuracy, and perform retraining processing or output adjustment completion information based on the analysis results.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are executed.

Citation Information

Patent Citations

  • Language task model training method and device, electronic equipment and storage medium

    CN111159416A

  • Foundation cloud picture recognition model training method and foundation cloud picture recognition method

    CN111695640A

  • System and method of training a neural network

    US20210216872A1

Cited By

  • Construction method and device of coal mine vertical field large model

    CN119848553A

  • Visual language model fine tuning method and device based on adaptive parameter selection

    CN120562479A

  • Vision language model fine-tuning method and device based on adaptive parameter selection

    CN120562479B

  • Method for evaluating fine tuning effect of large language model based on multimedia learning cognitive theory and application thereof

    CN120562524A

  • Evaluation method of large language model fine-tuning effect based on multimedia learning cognitive theory and application thereof

    CN120562524B