A large language model training method and device

Abstract

The embodiment of the present application discloses a large language model training method and device. The method includes: inputting all training data into the large model for training; determining the large model performance data of the large model; selecting at least part of the training data from the full training data as the target training data according to the large model performance data, and optimizing and adjusting the parameters based on the large model performance data corresponding to the target training data; if the preset conditions are not met, returning to continue the execution from inputting all training data into the large model for training. The above scheme selects relatively high-quality training data according to the large model performance data, and first optimizes the parameters based on the large model performance data corresponding to the part of the training data, so that the model can be optimized quickly. When the model capability is improved, the learning ability of other training data is further improved, and the full training data is continuously input to participate in parameter optimization and adjustment, which improves the model training efficiency and generalization ability.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a large language model training method and device. Background Art

[0002] Large Language Model (LLM) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, and dialogue, and are an important path to artificial intelligence.

[0003] Large language models require a large amount of training data for training. Although the training data has been cleaned, the data quality is still uneven, and the model training efficiency is low. Multiple rounds of model optimization are required to learn the training data of uneven quality, and training failure is easy to occur. However, if the data quality is strictly unified for model training, the model's generalization ability may be insufficient, and it may not be able to correctly identify a variety of model requests, resulting in poor training results. At the same time, when training complex models, because complex training data is required and the model parameters are large, model training is very difficult and more likely to fail, resulting in a large waste of manpower, material resources and computing resources, inefficient model training, and poor results. Summary of the invention

[0004] The embodiments of the present application provide a large language model training method, device, electronic device and medium to improve the training efficiency and training success rate of the large language model while ensuring the generalization ability of the large language model.

[0005] According to one aspect of the present application, a large language model training method is provided, the method comprising:

[0006] Input the full amount of training data into the large model for training, and obtain the output data corresponding to the training round;

[0007] Determining large model performance data of the large model according to the training data and the output data;

[0008] Selecting at least part of the training data from the full amount of training data as target training data according to the large model performance data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data;

[0009] Determine whether the large model meets the preset conditions. If not, return to continue execution from inputting the full amount of training data into the large model for training; if so, obtain the final large model.

[0010] According to one aspect of the present application, a large language model training device is provided, the device comprising:

[0011] The output data determination module is used to input the full amount of training data into the large model for training and obtain the output data corresponding to the training round;

[0012] A large model performance data determination module, used to determine the large model performance data of the large model according to the training data and the output data;

[0013] a parameter optimization and adjustment module, configured to select at least part of the training data from the full amount of training data as target training data according to the large model performance data, and perform parameter optimization and adjustment on the large model based on the large model performance data corresponding to the target training data;

[0014] The judgment module is used to determine whether the large model meets the preset conditions. If not, it returns to continue the execution from inputting the full amount of training data into the large model for training; if so, the final large model is obtained.

[0015] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] A memory connected to at least one processor for large language model training; wherein,

[0018] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the large language model training method of any embodiment of the present application.

[0019] According to another aspect of the present application, a computer-readable storage medium is provided, which stores computer instructions, and the computer instructions are used to enable a processor to implement the large language model training method of any embodiment of the present application when executed.

[0020] The technical solution of the embodiment of the present application inputs the full amount of training data into the large model for training, and obtains the output data corresponding to the model training round; determines the large model performance data of the large model based on the training data and the output data; selects at least part of the training data from the full amount of training data as the target training data based on the large model performance data, and optimizes and adjusts the parameters of the large model based on the large model performance data corresponding to the target training data; determines whether the large model meets the preset conditions, and if not, returns to continue the execution from inputting the full amount of training data into the large model for training; if so, obtains the final large model. After the above scheme performs model training based on the full amount of data, relatively high-quality training data is selected according to the large model performance data, and parameters are first tuned based on the large model performance data corresponding to the part of the training data, so that the model can be quickly optimized. When the model capability is improved, the learning ability of other training data is further improved, and the model training efficiency is improved. Continue to input the full amount of training data to participate in parameter optimization and adjustment, so as to improve the performance and generalization ability of the model.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flowchart of a large language model training method provided in an embodiment of the present application;

[0024] Figure 2 A flowchart of a large language model training method provided by another embodiment of the present application;

[0025] Figure 3 A schematic diagram of the structure of a large language model training device provided in an embodiment of the present application;

[0026] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", "third", "fourth", "actual", "preset", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Figure 1 This is a flowchart of a large language model training method provided in an embodiment of the present application. The embodiment of the present application can be applied to the case of training a large language model. The method can be executed by a large language model training device, which can be implemented in the form of hardware and / or software, and the large language model training device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0030] S110: Input the full amount of training data into the large model for training, and obtain output data corresponding to the training round.

[0031] The full amount of training data may be all the training data that has been acquired, or may be all the training data in a training data set. The large model may be a large language model, that is, a large model that can accurately learn and deeply understand human language. A training round may be a process of training the large model once and optimizing its parameters.

[0032] The large language model needs to handle a variety of natural language tasks during application, and the natural languages ​​that need to be recognized are also diverse. Therefore, a variety of training samples are needed to train the large model to improve the generalization ability of the large model, so that it can recognize and understand a variety of input languages ​​during application. In this case, in the embodiment of the present application, the full amount of training data is input into the large model for training, so as to train the large model with rich and comprehensive training samples and improve the generalization ability of the large model.

[0033] In the embodiment of the present application, the full amount of training data is input into the large model. For each training data, the training data can be used as an input, and the large model outputs one output data accordingly. For each training data, the training data can also be split into multiple data unit tokens, and multiple tokens are input into the large model, and each token corresponds to one output data.

[0034] The network structure used by the large model can be selected according to the actual situation. For example, for a large language model that needs to recognize and understand human language, the Transformer architecture can be used as the main architecture of the large model. You can also choose the Mixture of Experts (MoE) architecture, prompt-based learning, and other architectures, which can be selected adaptively according to the actual application scenario.

[0035] S120. Determine large model performance data of the large model according to the training data and the output data.

[0036] Exemplarily, for each training data, the large model performance data of the large model can be determined based on the training data and the output data to reflect the performance of the large model. The expression of the large model performance data is not limited. Generally, the performance of the large model can be reflected from aspects such as accuracy, precision, F1 score, mixed matrix, ROC curve and AUC true value, loss function, Perplexity value, BLEU score, etc. The value of at least one of the above indicators can be calculated, and the large model performance data can be determined based on the value of at least one indicator. If a training data corresponds to one output data, then one training data corresponds to a large model performance data. If a training data corresponds to multiple tokens, multiple output data are obtained when input into the large model. Multiple large model performance data can be obtained based on multiple output data and training data, or the descriptive statistics of multiple large model performance data can be used as the large model performance data corresponding to the training data. Descriptive statistics can be mean, median, mode, etc.

[0037] Specifically, each indicator is deformed so that the changing trend of each indicator is consistent with the performance trend of the large model reflected. For example, the accuracy is positively correlated with the performance of the large model, and the loss function value is negatively correlated with the performance of the large model. The reciprocal of the loss function value can be taken, and the reciprocal of the loss function value is positively correlated with the performance of the large model. Then the reciprocal of the accuracy and / or the loss function value is calculated in a preset manner to obtain the performance data of the large model. In this case, the performance data of the large model is positively correlated with the performance of the large model. The reciprocal of the accuracy can also be taken, so that the reciprocal of the accuracy and / or the loss function value are both negatively correlated with the performance of the large model. The reciprocal of the accuracy and / or the loss function value are calculated in a preset manner to obtain the performance data of the large model. In this case, the performance data of the large model is negatively correlated with the performance of the large model. The calculation of the preset method can be a method in which individual indicators such as addition, weighted summation, multiplication, power, index, and pairing work together to produce a greater effect.

[0038] S130. Select at least part of the training data from the full amount of training data as target training data according to the large model performance data, and optimize and adjust parameters of the large model based on the large model performance data corresponding to the target training data.

[0039] Exemplarily, the amount of full training data used by users to train a large model is huge, generally reaching billions to hundreds of billions of tokens. If the full amount of training data is always used for parameter adjustment and optimization, the amount of data required to be learned is large, the learning speed is slow, and the uneven quality of the training data may lead to training failure. In an embodiment of the present application, at least part of the training data is selected from the full amount of training data as target training data based on the performance data of the large model, and the parameters of the large model are optimized and adjusted only based on the performance data of the large model corresponding to the target training data. This can reduce the adaptability of the data volume of the large model learning, improve the efficiency of the large model parameter optimization adjustment, and quickly achieve the improvement of model performance.

[0040] In an embodiment of the present application, training data corresponding to the large model performance data that reflects the better performance of the large model can be preferentially selected as the target training data, that is, training data that makes the large model easier to learn and has fewer or smaller model parameter adjustments as the target training data, so as to optimize the parameters of the large model based on the large model performance data corresponding to the target training data, so that the large model can be optimized more quickly and the performance of the model can be improved. In general, the quality of the target training data acquired in the early stage is relatively high. When the performance of the large model has been improved after optimization, it may have better learning ability for other uneven training data, thereby improving the efficiency of large model training optimization and avoiding the problem of directly learning the full amount of training data of uneven quality with a large model with lower performance, resulting in low learning efficiency and even leading to the failure of large model training.

[0041] S140, determining whether the large model satisfies a preset condition; if not, returning to continue the execution from inputting the full amount of training data into the large model for training; if so, obtaining the final large model.

[0042] Among them, the preset conditions can be the conditions for terminating the iterative optimization of the large model, such as the number of iterative optimizations reaching the maximum number of iterations, the performance of the large model on the validation set reaching a certain threshold, the performance of the large model on the test set starting to decline, a certain performance indicator of the large model reaching a stable state, etc.

[0043] Exemplarily, if the large model meets the preset conditions, it reflects that the performance of the large model has reached the preset performance requirements, and the iterative optimization can be stopped, and the large model at this time is used as the final large model trained by the training data set. If the large model does not meet the preset conditions, it reflects that the performance of the large model does not meet the preset performance requirements, and it is necessary to continue iterative optimization. Specifically, since the parameters of the large model have been optimized and adjusted based on the target training data in the above embodiment, the performance of the large model has been improved, and the learning ability of the training data with uneven quality is stronger, so it can return to continue execution from S110, so that the large model continues to learn the full amount of training data and improve the generalization ability of the large large model.

[0044] The technical solution of the embodiment of the present application inputs the full amount of training data into the large model for training, and obtains the output data corresponding to the model training round; determines the large model performance data of the large model based on the training data and the output data; selects at least part of the training data from the full amount of training data as the target training data based on the large model performance data, and optimizes and adjusts the parameters of the large model based on the large model performance data corresponding to the target training data; determines whether the large model meets the preset conditions, and if not, returns to continue the execution from inputting the full amount of training data into the large model for training; if so, obtains the final large model. After the above scheme performs model training based on the full amount of data, relatively high-quality training data is selected according to the large model performance data, and parameters are first tuned based on the large model performance data corresponding to the part of the training data, so that the model can be quickly optimized. When the model capability is improved, the learning ability of other training data is further improved, and the model training efficiency is improved. Continue to input the full amount of training data to participate in parameter optimization and adjustment, so as to improve the performance and generalization ability of the model.

[0045] Figure 2 This is a flowchart of a large language model training method provided in another embodiment of the present application. The present embodiment is optimized based on the above embodiment. For solutions not described in detail in the present embodiment, please refer to the above embodiment. Figure 2 As shown, the method of the embodiment of the present application specifically includes the following steps:

[0046] S210: Split the entire training data into at least one training batch including at least one piece of training data.

[0047] For example, when training a large model based on the full amount of training data, inputting the full amount of training data into the large model at the same time will consume a lot of memory and computing resources. In order to improve the training efficiency of the large model and reduce the computing pressure, the full amount of training data can be split to obtain at least one training batch, each training batch contains at least one training data, and the training data is input into the large model in batches for training.

[0048] Specifically, assuming that the full training data contains 50,000 training data, it can be divided into 5,000 training batches, each of which contains 10 training data. Assuming that each training data is encoded as 250 tokens, 5,000 batches of [10*250] training data can be obtained.

[0049] S220, input the training data of each training batch into the large model to obtain the output data corresponding to each training batch.

[0050] Exemplarily, the training data of each training batch can be input into the large model in sequence to obtain the output data corresponding to each training batch.

[0051] For example, if 10 training data are input into the large model at a time, and 5000 training batches of training data are input into the large model in sequence, 5000 output data can be obtained, and the dimension of each output data is [10*1]. Alternatively, if the training data is encoded into 250 tokens, [10*250] training data are input into the large model at a time, and 5000 training batches are input into the large model in sequence, 5000 output data are obtained, and the dimension corresponding to each output data is [10*250].

[0052] In an embodiment of the present application, determining the large model performance data of the large model according to the training data and the output data includes:

[0053] For each training batch, at least one large model performance data corresponding to the training batch is determined based on the output data corresponding to the training batch and the fitting data in the training data.

[0054] In general, the training data includes fitting data. The fitting data is the true value in the training data. Exemplarily, for each training batch, at least one large model performance data corresponding to the training batch can be determined based on the output data corresponding to the training batch and the fitting data in the training data to represent the performance of the large model. Exemplarily, the fitting data and the output data can be brought into a predetermined loss function, and the loss function value is calculated as the large model performance data. Each training batch contains at least one training data, and each training data corresponds to a large model performance data, so each training batch corresponds to at least one large model performance data.

[0055] In an embodiment of the present application, selecting at least part of the training data from the full amount of training data as target training data according to the large model performance data includes:

[0056] For at least one large model performance data corresponding to each of the training batches, the training data corresponding to the large model performance data that is higher than a preset performance data threshold is retained; or, the training data with the highest large model performance data of a preset number or a preset number ratio in the training data is retained; wherein the large model performance data is positively correlated with the quality of the large model performance;

[0057] The training data retained in each training batch is used as the target training data.

[0058] In an achievable solution, in the process of selecting target training data, a preset performance data threshold can be set in advance, and for at least one large model performance data corresponding to each training batch, the training data corresponding to the large model performance data that is higher than the preset performance data threshold is retained, and the retained training data is used as the target training data. For a training batch, the descriptive statistics of the large model performance data of each training data can be used as the large model performance data of the training batch, and the training batch can be selected according to the large model performance data of each training batch. It is also possible to determine the large model performance data of each training data in each training batch, and select the training data according to the large model performance data. It is also possible to determine the large model performance data corresponding to each token for each training data encoded in each training batch, and select the token according to the large model performance data. For example, case 1: for the large model performance data of the [10*250] dimension corresponding to the training data of the [10*250] dimension of the first training batch, the average value of each large model performance data is calculated as the large model performance data corresponding to the training batch. If the large model performance data is higher than the preset performance data threshold, the training batch corresponding to the large model performance data is retained. If the large model performance data is not higher than the preset performance data threshold, the training batch corresponding to the large model performance data will not be retained. For example, case 2: for the large model performance data of [10*250] dimension corresponding to the training data of [10*250] dimension of the first training batch, compare the large model performance data with the preset performance data threshold. If the large model performance data is higher than the preset performance data threshold, the training data corresponding to the large model performance data will be retained. If the large model performance data is not higher than the preset performance data threshold, the training data corresponding to the large model performance data will not be retained. Assuming that 8 large model performance data are higher than the preset performance data threshold, the training data of [8*250] dimension corresponding to the large model performance data of [8*250] dimension will be retained at last, and the training data of [8*250] dimension will be used as the target training data. For the large model performance data of [10*250] dimension corresponding to the training data of [10*250] dimension of the second training batch, the large model performance data that may be higher than the preset performance data threshold is 5, then the training data of [5*250] dimension corresponding to the large model performance data of [5*250] dimension is finally retained, and the training data of [5*250] dimension is used as the target training data. The dimensions of the large model performance data finally retained in each training batch may be different, and the dimensions of the corresponding target training data may also be different. Case three: When the training data is encoded as tokens, the training data of each training batch is [10*250] tokens. For the training data of the first training batch, training data can be selected for each token to be retained.For example, each token is input into the big model to obtain an output data, and the big model performance data can be determined based on the output data and fitting data of each token. One token corresponds to one big model performance data. For each token, determine whether the big model performance data is higher than the preset performance data threshold. If it is higher, it is retained, otherwise it is not retained. Assuming that in the first training batch, [10*200] tokens correspond to the big model performance data higher than the preset performance data threshold, the training data corresponding to [10*200] big model performance data are retained as the target training data. The same is true for other training data. The tokens retained in the next training batch may be [10*220] dimensions. The dimensions of the tokens retained in different training batches and / or different training data may be different. In the subsequent application process, the unaligned elements of the retained different training data can be filled with preset values.

[0059] In another feasible solution, in the process of selecting the target training data, a preset number or a preset number ratio can be predetermined, and the highest large model performance data can be selected according to the preset number and the preset number ratio, and the corresponding training data can be retained as the target training data. For example, case 1: Assuming that the preset number is 60 and the preset number ratio is 60%, for the 100 [10*500] dimensional large model performance data corresponding to the 100 [10*500] dimensional training data of all training batches, the average value of the large model performance data of a training batch is calculated as the large model performance data corresponding to the training batch. Sort the training batches according to the high or low large model performance data, retain the 60 training batches with the highest ranking, and retain the [10*500] dimensional training data of 60 training batches as the target training data. Case 2: Assuming that the preset number is 6 and the preset number ratio is 60%. For the large model performance data of [10*500] dimension corresponding to the training data of [10*500] dimension of the first training batch, the large model performance data are sorted in order, and 6 or 60% (10*60%=6) large model performance data corresponding to the training data are selected, and finally the training data of [6*500] dimension corresponding to the large model performance data of [6*500] dimension are retained, and the training data of [6*500] dimension are used as the target training data. For the large model performance data of [10*500] dimension corresponding to the training data of [10*500] dimension of the second training batch, the large model performance data are also sorted in order, and 6 or 60% (10*60%=6) large model performance data corresponding to the training data of [6*500] dimension are selected and retained, and the training data of [6*500] dimension are used as the target training data. The number of target training data in each batch determined according to this scheme is consistent. Case 3: When the training data is encoded as tokens, the training data of each training batch is [10*500] tokens. For the training data of the first training batch, the training data can be selected for each token to be retained. For example, each token is input into the large model to obtain an output data, and the large model performance data can be determined based on the output data and fitting data of each token. One token corresponds to one large model performance data. Assuming that the preset number is 400 or the preset number ratio is 80%, the training data corresponding to the highest large model performance data of [10*400] is retained as the target training data.

[0060] It should be noted that the basis for the implementation of the above embodiment is that the large model performance data is positively correlated with the performance quality of the large model. If the indicator used to represent the large model performance data is negatively correlated with the performance quality of the large model, the indicator can be taken as the inverse. Alternatively, if the large model performance data is negatively correlated with the performance quality of the large model, the condition of "higher than the preset performance data threshold" in the above embodiment needs to be adjusted to "lower than the preset performance data threshold", and "the training data with the highest large model performance data of a preset number or a preset number ratio is retained" is adjusted to "the training data with the lowest large model performance data of a preset number or a preset number ratio is retained".

[0061] S230. Determine large model performance data of the large model according to the training data and the output data.

[0062] S240. Select at least part of the training data from the full amount of training data as target training data according to the large model performance data, and optimize and adjust parameters of the large model based on the large model performance data corresponding to the target training data.

[0063] S250, determining whether the large model satisfies a preset condition; if not, returning to continue the execution from inputting the full amount of training data into the large model for training; if so, obtaining the final large model.

[0064] The embodiment of the present application provides a large language model training method, which splits the full amount of training data to obtain at least one training batch containing at least one training data; inputs the training data of each training batch into the large model to obtain the output data corresponding to each training batch. The above scheme can effectively reduce the processing pressure and improve the training efficiency of the large model by splitting the full amount of training data to obtain individual training batches and inputting them into the large model in sequence. The full amount of training data is input into the large model for training to obtain the output data corresponding to the training round; the large model performance data of the large model is determined according to the training data and the output data; at least part of the training data is selected from the full amount of training data as the target training data according to the large model performance data, and the parameters of the large model are optimized and adjusted based on the large model performance data corresponding to the target training data. The large model performance data corresponding to the target training data can be adaptively selected to optimize the parameters of the large model, thereby improving the efficiency of the large model optimization and achieving rapid performance improvement, and then the full amount of training data is input for training, thereby improving the generalization ability of the large model.

[0065] As a non-limiting implementation method, selecting at least part of the training data from the full amount of training data as target training data according to the large model performance data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data, includes:

[0066] For the big model performance data corresponding to the full amount of training data, mark the training data corresponding to the big model performance data that is higher than the preset performance data threshold, or mark the training data corresponding to the largest big model performance data of a preset number or a preset ratio;

[0067] Using all labeled training data as target training data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data; or,

[0068] For the big model performance data corresponding to the full amount of training data, mark the training data corresponding to the big model performance data below the preset performance data threshold, or mark the training data corresponding to the big model performance data with the lowest preset number or preset number ratio;

[0069] Using the unlabeled training data as target training data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data;

[0070] Among them, the large model performance data is positively correlated with the quality of the large model performance.

[0071] In an embodiment of the present application, for each training data in the full amount of training data, the corresponding output data is input into the large model, and the large model performance data is determined according to the training data and the output data. The training data can be selected according to the large model performance data, and the selected training data is marked, and then all the marked training data is used as the target training data. The form of marking is not limited. For example, one-dimensional data is added to the training data matrix to mark the training data. If the training data is selected, the corresponding data is set to 1, and if the training data is not selected, the corresponding data is set to 0. Specifically, the process of selecting the training data to be marked can be to mark the training data corresponding to the large model performance data that is higher than the preset performance data threshold, or to mark the training data corresponding to the largest large model performance data with a preset number or a preset number ratio. The preset performance data threshold, the preset number, and the preset number ratio are determined as shown in the above embodiment. All marked training data are used as target training data. The process of selecting the training data to be marked can also be to mark the training data corresponding to the large model performance data below the preset performance data threshold, or to mark the training data corresponding to the lowest large model performance data of a preset number or a preset number ratio. The method for determining the preset performance data threshold, the preset number, and the preset number ratio is shown in the above embodiment. All unlabeled training data are used as target training data. The premise of the above scheme is that the large model performance data is positively correlated with the quality of the large model performance.

[0072] As a non-limiting implementation, the method further includes:

[0073] During the large model training process with multiple training rounds, for the same training data, count the number of first training rounds that were not selected as target training data;

[0074] If the number of the first training rounds reaches the first round number threshold, the training data is removed from the total training data, or the training data is marked so as not to participate in parameter optimization adjustment any more.

[0075] In an embodiment of the present application, if there is training data that is not selected as target training data during the training process of the large model with multiple training rounds, the number of the first training rounds that are not selected as target training data is counted. If the number of the first training rounds reaches the first round number threshold, it reflects that the quality of the training data is poor, and the large model cannot learn the characteristics of the training data after multiple training rounds of training optimization. Therefore, the training data can be removed from the total amount of training data, or the training data can be marked so as not to participate in parameter optimization and adjustment, so as to avoid the large model in multiple training rounds being unable to learn the characteristics of the training data, affecting the progress of the large model training optimization. It should be noted that the number of training data that is removed or marked as no longer participating in the optimization adjustment needs to be limited to a certain extent to avoid removing too much training data and failing to improve the generalization ability of the large model. For example, assuming that the first round number threshold is 3, the training data numbered 1-900 is not selected as the target training data in the first training round, the second training round, and the third training round, and is deleted from the full training data. In the fourth training round, even if the full training data is input into the large model for training, the training data numbered 1-900 will not be input. The first training round number obtained by statistics can be continuous or discontinuous.

[0076] As a non-limiting implementation method, selecting at least part of the training data from the full amount of training data as target training data according to the large model performance data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data, includes:

[0077] Selecting a first portion of training data from the full amount of training data as first target training data according to the large model performance data, wherein the first portion of training data is training data that has been used as the target training data at least once in a previous training round;

[0078] A second part of the training data is selected from the training data that is not selected from the full amount of training data as the second target training data, and the parameters of the large model after parameter optimization and adjustment are further optimized based on the large model performance data corresponding to the first target training data and the second target training data.

[0079] In the embodiment of the present application, in the process of selecting target training data to optimize the parameters of the large model, the first part of the training data can also be selected from the full amount of training data as the first target training data, that is, the training data that has been selected as the target training data in the previous training round is selected, and the part of the training data is assumed to be relatively high-quality training data, and the part of the training data is still retained. From the training data other than the first target training data of the full amount of training data, the second target training data is selected, and the first target training data and the second target training data are used together as training data. The parameters of the large model after the parameter optimization adjustment are continued to be optimized and adjusted based on the large model performance data corresponding to the target training data, so as to retain the previously selected high-quality training data, and continue to select high-quality training data from the unselected training data, increase the number of high-quality training data, and jointly constitute the target training data in this training round, and optimize the parameters of the large model, so as to gradually improve the generalization ability of the large model, and give priority to the large model to learn high-quality training data, thereby improving training efficiency.

[0080] As a non-limiting implementation, the method further includes:

[0081] In the large model training process of multiple training rounds, for the same training data, counting the number of second training rounds selected as target training data, or counting the number of second training rounds in which the large model performance data corresponding to the training data is continuously higher than a preset performance data threshold;

[0082] If the number of the second training rounds reaches the second round number threshold, the training data is removed from the total training data, or the training data is marked so as not to participate in parameter optimization adjustment any more;

[0083] Among them, the large model performance data is positively correlated with the quality of the large model performance.

[0084] Exemplarily, in order to prevent overfitting, the number of second training rounds selected as target training data can be counted for the training data selected as target training data, or the number of second training rounds in which the performance data of the large model corresponding to the training data is higher than the preset performance data threshold. If the number of second training rounds reaches the second round number threshold, it reflects that the training data has been used for the target training data to participate in the parameter optimization adjustment of the large model many times. If too many rounds use the same large model performance data corresponding to the target training data for parameter optimization adjustment, it may cause overfitting of the large model and reduce the generalization ability of the large model. Therefore, this part of the training data is removed from the full amount of data, or marked to no longer participate in parameter optimization adjustment. The number of second training rounds can be continuous or discontinuous.

[0085] As a non-limiting implementation, the method further includes:

[0086] In the case where training data is excluded or marked as not selected for model training, the number of third training rounds that have been trained on the large model is counted;

[0087] If the number of the third training rounds reaches the third round number threshold, the eliminated training data is added to the full training data or unmarked, and continues to be input into the large model for training or parameter optimization adjustment.

[0088] Exemplarily, in the above parameter optimization and adjustment process, a part of the training data is excluded or marked, and none of them participate in the input training work or parameter optimization and adjustment of the large model in the third training round. However, with the performance improvement and optimization of the large model, the learning ability of the training data that did not participate in the input training work or parameter optimization and adjustment of the large model may change. Therefore, after the preset rounds of training, it is necessary to continue to add the previously unselected training data to the full amount of data to participate in the input training work or parameter optimization and adjustment of the large model to improve the generalization ability of the large model, and the processing of the full amount of training data reflects a relatively ideal performance. Specifically, assuming that the third round quantity threshold is 2, in the first training round, 9000 training data are excluded from the full amount of training data or marked to no longer participate in the parameter optimization and adjustment of the large model. After the second training round and the third training round, two training rounds have been passed without the part of the training data participating in the input training or parameter optimization and adjustment of the large model. In the fourth training round, continue to add the part of the training data to the full amount of training data to participate in the model training process or parameter optimization and adjustment process of the subsequent training rounds. The number of the third training rounds can be continuous or discontinuous. The above solution can continue to add the eliminated data to the full data for training optimization when the performance of the large model has been improved to a certain extent, thereby improving the generalization ability of the large model without affecting the training efficiency.

[0089] Figure 3 This is a schematic diagram of the structure of a large language model training device provided in an embodiment of the present application. The device can execute the large language model training method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method. Figure 3 As shown, the device comprises:

[0090] The output data determination module 310 is used to input the full amount of training data into the large model for training and obtain the output data corresponding to the training round;

[0091] A large model performance data determination module 320, configured to determine large model performance data of the large model based on the training data and the output data;

[0092] A parameter optimization and adjustment module 330 is used to select at least part of the training data from the full amount of training data as target training data according to the large model performance data, and optimize and adjust the parameters of the large model based on the large model performance data corresponding to the target training data;

[0093] The judgment module 340 is used to determine whether the large model meets the preset conditions. If not, it returns to continue the execution from inputting the full amount of training data into the large model for training; if so, the final large model is obtained.

[0094] In the embodiment of the present application, the output data determination module 310 inputs the full amount of training data into the large model for training, and obtains the output data corresponding to the model training round, including:

[0095] Splitting the full amount of training data to obtain at least one training batch including at least one piece of training data;

[0096] The training data of each training batch is input into the large model to obtain the output data corresponding to each training batch.

[0097] In the embodiment of the present application, the output data determination module 310 determines the large model performance data of the large model according to the training data and the output data, including:

[0098] For each training batch, at least one large model performance data corresponding to the training batch is determined based on the output data corresponding to the training batch and the fitting data in the training data.

[0099] In the embodiment of the present application, the output data determination module 310 selects at least part of the training data from the full amount of training data as target training data according to the large model performance data, including:

[0100] For at least one large model performance data corresponding to each of the training batches, the training data corresponding to the large model performance data that is higher than a preset performance data threshold is retained; or, the training data with the highest large model performance data of a preset number or a preset number ratio in the training data is retained; wherein the large model performance data is positively correlated with the quality of the large model performance;

[0101] The training data retained in each training batch is used as the target training data.

[0102] In the embodiment of the present application, the parameter optimization and adjustment module 330 selects at least part of the training data from the full amount of training data as target training data according to the large model performance data, and optimizes and adjusts the parameters of the large model based on the large model performance data corresponding to the target training data, including:

[0103] For the big model performance data corresponding to the full amount of training data, mark the training data corresponding to the big model performance data that is higher than the preset performance data threshold, or mark the training data corresponding to the largest big model performance data of a preset number or a preset ratio;

[0104] Using all labeled training data as target training data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data; or,

[0105] For the big model performance data corresponding to the full amount of training data, mark the training data corresponding to the big model performance data below the preset performance data threshold, or mark the training data corresponding to the big model performance data with the lowest preset number or preset number ratio;

[0106] Using the unlabeled training data as target training data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data;

[0107] Among them, the large model performance data is positively correlated with the quality of the large model performance.

[0108] In the embodiment of the present application, the device further includes:

[0109] A first statistical module is used to count the number of first training rounds that are not selected as target training data for the same training data during the large model training process with multiple training rounds;

[0110] The first elimination or marking module is used to eliminate the training data from the total training data if the number of the first training rounds reaches a first round number threshold, or to mark the training data so as not to participate in parameter optimization adjustment any more.

[0111] In the embodiment of the present application, the parameter optimization and adjustment module 330 selects at least part of the training data from the full amount of training data as target training data according to the large model performance data, and optimizes and adjusts the parameters of the large model based on the large model performance data corresponding to the target training data, including:

[0112] Selecting a first portion of training data from the full amount of training data as first target training data according to the large model performance data, wherein the first portion of training data is training data that has been used as the target training data at least in a previous training round;

[0113] A second part of the training data is selected from the training data that is not selected from the full amount of training data as the second target training data, and the parameters of the large model after parameter optimization and adjustment are further optimized based on the large model performance data corresponding to the first target training data and the second target training data.

[0114] In the embodiment of the present application, the device further includes:

[0115] A second statistical module is used to count the number of second training rounds selected as target training data for the same training data during the large model training process of multiple training rounds, or to count the number of second training rounds in which the large model performance data corresponding to the training data is continuously higher than a preset performance data threshold;

[0116] A second elimination or marking module, configured to eliminate the training data from the total amount of training data, or mark the training data so as not to participate in parameter optimization adjustment if the number of the second training rounds reaches a second round number threshold;

[0117] Among them, the large model performance data is positively correlated with the quality of the large model performance.

[0118] In the embodiment of the present application, the device further includes:

[0119] A third statistical module is used to count the number of third training rounds that have been trained on the large model in the case where training data is excluded or marked as not selected for model training;

[0120] The adding module is used to add the training data to the full amount of training data or cancel the mark if the number of the third training rounds reaches the third round number threshold, and continue to participate in parameter optimization adjustment.

[0121] A large language model training device provided in an embodiment of the present application can execute a large language model training method provided in any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.

[0122] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0123] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0124] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a large language model training unit 19, such as a network card, a modem, a wireless large language model training transceiver, etc. The large language model training unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0125] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 performs the various methods and processes described above, such as a large language model training method.

[0126] In some embodiments, the large language model training method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the large language model training unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the large language model training method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the large language model training method in any other appropriate manner (e.g., by means of firmware).

[0127] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0128] The computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable large language model training device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0129] In the context of the present application, a computer readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device or equipment. A computer readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium may be a machine readable signal medium. A more specific example of a machine readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0130] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0131] The systems and techniques described herein can be implemented in a computing system that includes a backend component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a frontend component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data large language model training (e.g., a large language model training network) in any form or medium. Examples of large language model training networks include: local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0132] The computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a large language model training network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0133] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the information expected by the technical solution of this application can be achieved, and this document is not limited here.

[0134] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A large language model training method, characterized in that: The method comprises: Input the full amount of training data into the large model for training, and obtain the output data corresponding to the training round; Determining large model performance data of the large model according to the training data and the output data; Selecting at least part of the training data from the full amount of training data as target training data according to the large model performance data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data; Determine whether the large model meets the preset conditions. If not, return to continue execution from inputting the full amount of training data into the large model for training; if so, obtain the final large model.

2. The method according to claim 1, characterized in that Input the full amount of training data into the large model for training, and obtain the output data corresponding to the model training round, including: Splitting the full amount of training data to obtain at least one training batch including at least one piece of training data; The training data of each training batch is input into the large model to obtain the output data corresponding to each training batch.

3. The method according to claim 2, characterized in that Determining large model performance data of the large model according to the training data and the output data includes: For each training batch, at least one large model performance data corresponding to the training batch is determined based on the output data corresponding to the training batch and the fitting data in the training data.

4. The method according to claim 3, characterized in that Selecting at least part of the training data from the entire training data as target training data according to the large model performance data includes: For at least one large model performance data corresponding to each of the training batches, the training data corresponding to the large model performance data that is higher than a preset performance data threshold is retained; or, the training data with the highest large model performance data of a preset number or a preset number ratio in the training data is retained; wherein the large model performance data is positively correlated with the quality of the large model performance; The training data retained in each training batch is used as the target training data.

5. The method according to any one of claims 1 to 4, characterized in that: Selecting at least part of the training data from the full amount of training data as target training data according to the large model performance data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data, including: For the big model performance data corresponding to the full amount of training data, mark the training data corresponding to the big model performance data that is higher than the preset performance data threshold, or mark the training data corresponding to the largest big model performance data of a preset number or a preset ratio; Using all labeled training data as target training data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data; or, For the big model performance data corresponding to the full amount of training data, mark the training data corresponding to the big model performance data below the preset performance data threshold, or mark the training data corresponding to the big model performance data with the lowest preset number or preset number ratio; Using the unlabeled training data as target training data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data; Among them, the large model performance data is positively correlated with the quality of the large model performance.

6. The method according to claim 1, characterized in that The method further comprises: During the large model training process with multiple training rounds, for the same training data, count the number of first training rounds that were not selected as target training data; If the number of the first training rounds reaches the first round number threshold, the training data is removed from the total training data, or the training data is marked so as not to participate in parameter optimization adjustment any more.

7. The method according to claim 1, characterized in that Selecting at least part of the training data from the full amount of training data as target training data according to the large model performance data, and optimizing and adjusting parameters of the large model based on the large model performance data corresponding to the target training data, including: Selecting a first portion of training data from the full amount of training data as first target training data according to the large model performance data, wherein the first portion of training data is training data that has been used as the target training data at least once in a previous training round; A second part of the training data is selected from the training data that is not selected from the full amount of training data as the second target training data, and the parameters of the large model after parameter optimization and adjustment are further optimized based on the large model performance data corresponding to the first target training data and the second target training data.

8. The method according to claim 1, characterized in that The method further comprises: In the large model training process of multiple training rounds, for the same training data, counting the number of second training rounds selected as target training data, or counting the number of second training rounds in which the large model performance data corresponding to the training data is continuously higher than a preset performance data threshold; If the number of the second training rounds reaches the second round number threshold, the training data is removed from the total training data, or the training data is marked so as not to participate in parameter optimization adjustment any more; Among them, the large model performance data is positively correlated with the quality of the large model performance.

9. The method according to any one of claims 6 to 8, characterized in that: The method further comprises: In the case where training data is excluded or marked as not selected for model training, the number of third training rounds that have been trained on the large model is counted; If the number of the third training rounds reaches the third round number threshold, the eliminated training data is added to the full training data or unmarked, and continues to be input into the large model for training or parameter optimization adjustment.

10. A large language model training device, characterized in that: The device comprises: The output data determination module is used to input the full amount of training data into the large model for training and obtain the output data corresponding to the training round; A large model performance data determination module, used to determine the large model performance data of the large model according to the training data and the output data; a parameter optimization and adjustment module, configured to select at least part of the training data from the full amount of training data as target training data according to the large model performance data, and perform parameter optimization and adjustment on the large model based on the large model performance data corresponding to the target training data; The judgment module is used to determine whether the large model meets the preset conditions. If not, it returns to continue the execution from inputting the full amount of training data into the large model for training; if so, the final large model is obtained.

Images