Model training method and device, task processing method and device, program product and equipment
By collecting difficult samples from the initial training samples centrally and dynamically adjusting the category labels of the sample evaluation model, and combining large language models for batch training, the problems of long training time and high cost of large language models are solved, and efficient and accurate models are achieved train.
Patent Information
- Application Number
- CN202411322812.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-09-20
AI Technical Summary
The existing large language model training methods cannot meet the needs of high precision, efficiency and low cost at the same time. The traditional methods take months or even six months, and the computing resources and storage space are consumed hugely.
By collecting difficult samples from the initial training samples for training, dynamically adjusting the category label of the sample evaluation model, combining large language models for training, and optimizing the training process batch by batch, realizing the integration of model training and sample screening, and improving training efficiency and accuracy.
While maintaining the model training accuracy, accelerate model training, reduce costs, improve the quality of training samples in subsequent training batches, and achieve efficient and accurate training of large language models.
Smart Images

Figure CN118917440A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a model training method, a task processing method, a model training device, a task processing device, a computer program product, and an electronic device. Background Art
[0002] In recent years, large language models have emerged, bringing revolutionary changes to the fields of natural language processing and artificial intelligence. Large language models can be applied to many application scenarios, such as text generation, intelligent question answering, etc. Large language models require a lot of computing resources, storage space and time to train. Traditional training methods may take months or even half a year. In order to solve this problem, distributed training, computing optimization and hardware acceleration are currently used to improve the convergence speed and computing efficiency of the training process. However, the current methods for training large language models cannot simultaneously meet the high-precision, high-efficiency and low-cost training requirements for training large language models.
[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0004] The purpose of the present disclosure is to provide a model training method, a task processing method, a model training device, a task processing device, a computer program product and an electronic device, thereby improving the training efficiency and training accuracy of a large language model and reducing the training cost at least to a certain extent.
[0005] According to a first aspect of the present disclosure, a model training method is provided, comprising: collecting training samples of a current training batch from an initial training sample set, the training samples comprising difficult samples obtained from the initial training sample set based on a sample evaluation model; training a large language model based on the training samples, and labeling the training samples with category labels according to training loss values of the training samples, the category labels being used to indicate whether the training samples are difficult samples; training the sample evaluation model according to the training samples and the corresponding category labels to complete the current training batch; entering the next training batch until the large language model completes training and stops, thereby obtaining a trained large language model.
[0006] In an exemplary embodiment of the present disclosure, the sample evaluation model shares the target network parameters in the large language model; before training the large language model, the method further includes: determining an initial network based on the target network parameters of the large language model to be trained; and adding a sample classification network on the output side of the initial network to obtain a sample evaluation model.
[0007] In an exemplary embodiment of the present disclosure, training samples for a current training batch are collected from an initial training sample set, including: collecting a first training sample from the initial training sample set according to an initial sample retention rate; for uncollected samples in the initial training sample set, using a sample evaluation model to perform difficulty evaluation on the uncollected samples, and determining the uncollected samples whose evaluation results are difficult samples as second training samples; and determining training samples for the current training batch based on the first training samples and the second training samples.
[0008] In an exemplary embodiment of the present disclosure, a large language model is trained based on training samples, and category labels are annotated on the training samples according to the training loss values of the training samples, including: obtaining a current difficulty threshold; if the training loss value of the training sample is greater than the current difficulty threshold, determining the category label of the training sample as a difficult sample; otherwise, determining the category label of the training sample as a non-difficult sample.
[0009] In an exemplary embodiment of the present disclosure, obtaining the current difficulty threshold includes: obtaining a first training loss of a current training batch and a second training loss of a training batch before the current training batch; and determining the current difficulty threshold according to the first training loss and the second training loss.
[0010] In an exemplary embodiment of the present disclosure, a large language model is trained based on the training samples, including: inputting the training samples into the large language model, determining a prediction loss value based on a prediction result; adjusting parameters of the large language model based on the prediction loss value; wherein the parameters of the initial network in the sample evaluation model are adjusted along with the adjustment of the parameters of the large language model.
[0011] In an exemplary embodiment of the present disclosure, a sample evaluation model is trained based on training samples and corresponding category labels, including: inputting the training samples into the sample evaluation model to obtain sample prediction results; constructing a loss function based on the sample prediction results and category labels, and adjusting the parameters of the sample classification network of the sample evaluation model based on the loss function.
[0012] According to a second aspect of the present disclosure, a task processing method is provided, comprising: obtaining task information to be processed; inputting the task information to be processed into a target large language model to obtain a processing result of the task information to be processed; wherein the target large language model is obtained by training a large language model based on any of the above methods.
[0013] According to a third aspect of the present disclosure, a model training device is provided, including: a sample processing module, used to collect training samples of a current training batch from an initial training sample set, the training samples including difficult samples obtained from the initial training sample set based on a sample evaluation model; a first training module, used to train a large language model based on the training samples, and annotate category labels for the training samples according to the training loss values of the training samples, the category labels being used to indicate whether the training samples are difficult samples; a second training module, used to train the sample evaluation model according to the training samples and the corresponding category labels to complete the current training batch; and a third training module, used to control the model training process to enter the next training batch until the large language model completes the training and stops, thereby obtaining a trained large language model.
[0014] According to a fourth aspect of the present disclosure, a task processing device is provided, comprising: an information acquisition module, used to acquire task information to be processed; a task processing module, used to input the task information to be processed into a target large language model to obtain a processing result of the task information to be processed; wherein the target large language model is obtained by training a large language model based on any of the above methods.
[0015] According to one aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, any one of the above methods is implemented.
[0016] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any one of the above methods by executing the executable instructions.
[0017] In the technical solution provided by the embodiment of the present disclosure, on the one hand, during the training of each training batch, the training samples of the current training batch are collected from the initial training sample set, so that the training samples include the difficult samples obtained from the initial training sample set based on the sample evaluation model, and the training samples used for model training are difficult samples containing rich information, so as to properly filter redundant data in model training, accelerate model training while maintaining the accuracy of model training, and reduce costs. On the other hand, during the training process of the current training batch, after the large language model is trained based on the training samples, the training samples are labeled with category labels according to the training loss values of the training samples, and the category labels used for training the sample evaluation model are dynamically adjusted, so as to facilitate the accurate removal of low-value training samples for training the large language model in subsequent training batches through the sample evaluation model, thereby improving the quality of the training samples in the subsequent training batches, and by using the same training samples to complete the training of the large language model and the sample evaluation model during the training process, the model training and sample screening processes are integrated, so that the selected samples are more in line with the training requirements of the large language model, and the model training accuracy is improved.
[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0020] Figure 1 A schematic diagram of the stages involved in the large language model training solution of an embodiment of the present disclosure is shown;
[0021] Figure 2 A flowchart schematically illustrates a model training method of an exemplary embodiment of the present disclosure;
[0022] Figure 3 A flowchart schematically illustrates an implementation method of building a sample evaluation model according to an exemplary embodiment of the present disclosure;
[0023] Figure 4 A schematic diagram schematically illustrates a method of constructing a sample evaluation model in an exemplary embodiment of the present disclosure;
[0024] Figure 5 A flowchart schematically illustrates an implementation method of determining training samples of a current training batch in an exemplary embodiment of the present disclosure;
[0025] Figure 6 A flowchart schematically illustrates a method for implementing dynamic labeling of training samples in an exemplary embodiment of the present disclosure;
[0026] Figure 7 A flowchart schematically illustrates an implementation method of determining a current difficulty threshold in an exemplary embodiment of the present disclosure;
[0027] Figure 8 A complete flow chart of training a large language model in an exemplary embodiment of the present disclosure is schematically shown;
[0028] Fig. 9 A flowchart schematically illustrates a task processing method in an exemplary embodiment of the present disclosure;
[0029] Fig.10 A schematic diagram schematically shows the composition of a model training device in an exemplary embodiment of the present disclosure;
[0030] Fig.11 A schematic diagram schematically shows the composition of a task processing device in an exemplary embodiment of the present disclosure;
[0031] Fig.12 A schematic diagram of an electronic device to which the embodiments of the present disclosure can be applied is shown. DETAILED DESCRIPTION
[0032] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0033] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0034] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.
[0035] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0036] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.
[0037] The technical solution provided by the exemplary embodiment of the present disclosure involves the machine learning technology of artificial intelligence, which uses the machine learning technology to collect training samples including difficult samples of the current training batch from the initial training sample set, and then trains the large language model based on the training samples, and annotates the training samples with category labels according to the training loss values of the training samples, and trains the sample evaluation model according to the training samples and the corresponding category labels to complete the current training batch. Before the large language model is trained, the above process is executed repeatedly to finally obtain the trained large language model. The model training method of the exemplary embodiment of the present disclosure and the large language model obtained by training can be applied to various natural language processing or artificial intelligence scenarios, such as text translation, text generation, question answering and other tasks, which can improve the training accuracy and efficiency of the large language model, reduce training costs, and improve the quality of task processing.
[0038] Figure 1A schematic diagram of the stages involved in the technical solution of the exemplary embodiment of the present disclosure is shown, such as Figure 1 As shown, the technical solution of the exemplary embodiment of the present disclosure includes a model training stage and a model application stage.
[0039] In an exemplary embodiment of the present disclosure, the provided model training method can be executed by a terminal device. During the training stage of the large language model, the terminal device can collect training samples including difficult samples of the current training batch from the initial training sample set, and then train the large language model based on the training samples, and annotate the training samples with category labels according to the training loss values of the training samples, and train the sample evaluation model according to the training samples and the corresponding category labels to complete the current training batch. Before the large language model is trained, the above process is executed in a loop to finally obtain a trained large language model, so as to use the trained large language model to process the task information to be processed.
[0040] In this way, when all steps in the model training method and task processing method provided by the exemplary embodiment of the present disclosure can be executed by the terminal device, all steps can be executed by the processor of the terminal device.
[0041] The terminal device can be an intelligent device with data processing capabilities, such as monitoring equipment, smart phones, computers, tablet computers, vehicle-mounted devices, wearable devices and other intelligent devices. The terminal device can also be called a mobile terminal, terminal, mobile device, etc. The present disclosure does not limit the type of terminal device.
[0042] Furthermore, the technical solution provided by the exemplary embodiment of the present disclosure may also be executed by a server. Correspondingly, in this mode of execution by the server, the server may start executing the steps in the technical solution of the exemplary embodiment of the present disclosure in response to a trigger instruction, wherein the trigger instruction may be sent by a terminal device used by a user, or may be triggered locally by the server in response to some automation events. The server may be a background system that provides related services in the exemplary embodiment of the present disclosure, and may include one electronic device with computing functions such as a portable computer, a desktop computer, a smart phone, or a cluster formed by multiple electronic devices.
[0043] In addition, the technical solution of the exemplary embodiment of the present disclosure can also be executed collaboratively by the terminal device and the server. In this way of collaborative execution by the electronic device and the server, some steps in the technical solution provided by the exemplary embodiment of the present disclosure are executed by the terminal device, while other steps are executed by the server. For example, the technical solution provided by the exemplary embodiment of the present disclosure can be executed by the server in the model training phase, and the trained large language model is sent to the terminal device, and the terminal device executes the model application phase, that is, the trained large language model is used to perform task processing.
[0044] It should be noted that in this mode of collaborative execution by the terminal device and the server, the steps respectively executed by the terminal device and the server can be dynamically adjusted according to actual conditions, and the exemplary embodiments of the present disclosure do not impose special restrictions on this.
[0045] Training a large language model requires a lot of computing resources, storage space and time. Traditional training methods may take months or even half a year. In order to solve this problem, there are currently methods for developing and improving training acceleration. For example, distributed training, computing optimization and hardware acceleration. Among them, distributed training distributes the training of the model to multiple computers for parallel computing. Each computer is responsible for processing a part of the data or different parts of the model, and then updates the parameters and fuses the model through communication and synchronization mechanisms. However, although this method greatly accelerates the process of model training, the total amount of computing required for training has not changed, and the training cost is still very high. Computational optimization improves the convergence speed and computing efficiency of the training process through technologies such as optimizer improvement and operator optimization fusion. However, the adjustment of the hyperparameters of the optimization algorithm is a difficult and time-consuming process, and the optimization of the computing layout depends on the specific structural design of the model, which is highly customized and difficult to meet the training needs of various models. Hardware acceleration methods, such as using higher-performance GPUs (graphics processing units, graphics processors) and other dedicated chips, but high-performance computing chips are usually expensive, and the adaptation and application of new chips may require additional work and resources.
[0046] Therefore, current methods for training large language models cannot meet the requirements of high efficiency, high precision and low cost.
[0047] Exemplarily, the large language model of the exemplary embodiment of the present disclosure can be a large model of the billion level, which refers to a deep learning model trained with a large amount of sample data, which can generate natural language text or understand the meaning of language text. It can also be constructed using lightweight neural networks, such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Networks), DNN (Deep Neural Networks), etc. Relative to lightweight neural networks, language models with more complex network structures can also be used, for example, heavyweight neural networks can be constructed, such as BERT (Bidirectional Encoder Representation from Transformers, feature encoder of bidirectional Transformers), Transformer (a language processing model), and any heavyweight neural network that can be used for task processing. The exemplary embodiment of the present disclosure takes the training of a large language model with a large model of the billion level as an example.
[0048] The exemplary embodiment of the present disclosure first provides a model training method. Figure 2 As shown, the model training method of the embodiment of the present disclosure may include the following steps S210 and S240:
[0049] In step S210, training samples of the current training batch are collected from the initial training sample set, and the training samples include difficult samples obtained from the initial training sample set based on the sample evaluation model.
[0050] In an exemplary embodiment of the present disclosure, the initial training samples are all samples used to train the model, and the training samples may be obtained from various channels, without limitation.
[0051] In most cases, training samples need to be labeled. In the process of training the iterative model, a large part of the training samples are easy for the model to judge, and are useless or of little use to the model training. At the same time, the labeling of these sample data takes up a high cost. Compared with such "easy samples", training samples that can provide useful information for model training and ensure the accuracy of model training are called difficult samples. Difficult samples can be used to guide model learning judgment, and continuously improve the ability to handle difficult samples in the learning judgment process, thereby improving the accuracy of model training.
[0052] It should be understood that the difficulty of a sample refers to the difficulty of learning the sample data. For example, the difficulty of learning a text / image sample. The difficulty is for a specific model, and different models may have different learning difficulties for the same text / image.
[0053] The sample evaluation model is a model used to evaluate the difficulty of training samples, and can be a classification model, such as a binary classification model, which can evaluate whether a training sample is a difficult sample or a non-difficult sample (also referred to as an easy sample). In each training batch, the exemplary embodiment of the present disclosure obtains a difficult sample from the initial training sample set based on the sample evaluation model as a training sample of the current training batch.
[0054] In some optional embodiments, when collecting training samples for the current training batch, a sample evaluation model can be used to perform a difficulty evaluation on all initial samples in the initial training sample set, and difficult samples can be obtained from the initial training sample set based on the evaluation results as training samples for the current training batch. This method can filter out redundant samples (such as non-difficult samples), thereby reducing costs and improving efficiency while ensuring model training accuracy.
[0055] In some optional embodiments, some initial samples may be retained in the training samples of the current training batch, and some difficult samples may be collected to jointly construct the training samples of the current training batch, so that the model can ensure the stability of model training under the joint training of non-difficult samples and difficult samples.
[0056] Batch: refers to the combination of multiple training samples used in a training process. A batch can include training samples corresponding to different categories, and a batch includes at least two training samples. In some cases, the ratio of each type of sample in a batch is fixed. For example, there are 4 types of categories included in the use scenario of the model, namely category 1, category 2, category 3 and category 4, and the set sample ratio corresponding to each category is 1:2:1:1. Then, when constructing a batch, the ratio of the training samples corresponding to category 1, the training samples corresponding to category 2, the training samples corresponding to category 3 and the training samples corresponding to category 4 included in the batch can still be maintained at 1:2:1:1. The training samples corresponding to each training batch in the exemplary embodiment of the present disclosure can be understood as a batch. Batches of different training batches can be completely identical batches, completely different batches, or batches with some identical training samples. Completely identical here means that the training samples of different training batches are the same.
[0057] The model selects batches of samples for processing each time during the training process, and then in each training batch, the training samples of the training batch are collected from the initial training sample set. The size of the batch training (batch) affects the selection of the learning rate. The exemplary embodiment of the present disclosure determines the size of the batch training according to actual needs and is not limited to this.
[0058] In step S220, the large language model is trained based on the training samples, and the training samples are labeled with category labels according to the training loss values of the training samples, where the category labels are used to indicate whether the training samples are difficult samples.
[0059] In an exemplary embodiment of the present disclosure, the large language model is trained using training samples, and the parameters in the large language model are adjusted. In order to ensure that the difficult samples identified by the sample evaluation model during the entire training process of the model are all difficult for the large language model, after the large language model is trained and updated, the sample evaluation model is further trained using the training samples of the current training batch to adjust the parameters in the sample evaluation model.
[0060] Among them, the category labels of whether the training samples are difficult samples can be marked according to the training loss values of the training samples for use in the training process of the sample evaluation model. By dynamically adjusting the category labels used for training the sample evaluation model, it is beneficial to accurately remove training samples with low value for training the large language model through the sample evaluation model in subsequent training batches, thereby improving the quality of training samples in subsequent training batches.
[0061] In step S230, the sample evaluation model is trained according to the training samples and the corresponding category labels to complete the current training batch.
[0062] In an exemplary embodiment of the present disclosure, after the current training batch completes the update of the parameters of the large language model, the sample evaluation model is trained according to the training samples and the corresponding category labels. When the sample evaluation model completes this round of training, the current training batch is completed, that is, the exemplary embodiment of the present disclosure includes the training of the large language model and the training of the sample evaluation model in one training batch.
[0063] In step S240, the next training batch is entered until the training of the large language model is completed, thereby obtaining a trained large language model.
[0064] In an exemplary embodiment of the present disclosure, after completing the current training batch, if the training of the large language model has not yet ended, the next training batch can be entered, and the next training batch can continue to be the current training batch, that is, returning to the step of collecting training samples for the current training batch from the initial training sample set.
[0065] The above steps are executed repeatedly until the large language model completes training. The large language model completes training when the large language model reaches convergence, or the training batch reaches a set number of times, or the sample evaluation model completes processing of the initial samples in all initial training sample sets. The exemplary embodiments of the present disclosure can flexibly select the conditions for stopping model training according to actual needs without limiting this.
[0066] In the technical solution provided by the embodiment of the present disclosure, on the one hand, during the training of each training batch, the training samples of the current training batch are collected from the initial training sample set, so that the training samples include the difficult samples obtained from the initial training sample set based on the sample evaluation model, and the training samples used for model training are difficult samples containing rich information, so as to properly filter redundant data in model training, accelerate model training while maintaining the accuracy of model training, and reduce costs. On the other hand, during the training process of the current training batch, after the large language model is trained based on the training samples, the training samples are labeled with category labels according to the training loss values of the training samples, and the category labels used for training the sample evaluation model are dynamically adjusted, so as to facilitate the accurate removal of low-value training samples for training the large language model in subsequent training batches through the sample evaluation model, thereby improving the quality of the training samples in the subsequent training batches, and by using the same training samples to complete the training of the large language model and the sample evaluation model during the training process, the model training and sample screening processes are integrated, so that the selected samples are more in line with the training requirements of the large language model, and the model training accuracy is improved.
[0067] In an exemplary embodiment, a method for constructing a sample evaluation model is also provided. The sample evaluation model shares the target network parameters in the large language model, such as Figure 3 As shown, before training the large language model, it also includes:
[0068] Step S310: Determine an initial network based on target network parameters of the large language model to be trained.
[0069] Step S320: Add a sample classification network to the output side of the initial network to obtain a sample evaluation model.
[0070] Among them, the sample evaluation model shares the target network parameters in the large language model as the initial network, so that the base of the sample evaluation model is the large language model, and the target network parameters can be selected according to actual needs (such as the amount of computation). The sample classification network is a classifier used to score the difficulty of sample data. The specific network structure is not restricted, and it can output the probability that the training sample is a "difficult sample". In this case, when training the large language model, as the parameters of the large language model are adjusted, the parameters of the initial network in the sample evaluation model can be dynamically adjusted, and the parameters of the sample classification network can be frozen. Correspondingly, when training the sample evaluation model, the parameters of the initial network are frozen, and the parameters of the sample classification network are adjusted.
[0071] As an example, Figure 4 A schematic diagram of constructing a sample evaluation model is shown. Figure 4 For a large language model with L layers and each layer width W, its first L / 2 layers (LX / 2) and the first 1 / 4 width (WX / 4) are used as the initial network of the sample evaluation model, and a sample classification network is added on the output side of the initial network. If the sample classification network is a binary classification network, the probability of the training sample being a difficult sample is determined, thereby determining whether the training sample is a difficult sample or a non-difficult sample.
[0072] At the same time, it should be understood that in the current training batch, the large language model is first trained based on the training samples, and the parameters in the large language model are updated. After the large language model is trained, the parameters of the initial network of the sample evaluation model are updated accordingly, because the initial network of the sample evaluation model shares the target network parameters of the large language model. Therefore, in the current training batch, when the sample evaluation model is trained according to the training samples and the corresponding category labels, the parameters of the initial network in the sample evaluation model have been updated based on the training of the large language model.
[0073] That is to say, since the sample evaluation model shares the target network parameters in the large language model, it can dynamically adjust the parameters of the initial network in the sample evaluation model according to the state of the large language model, so that the difficult samples subsequently predicted by the sample evaluation model are always "difficult" for the training of the large language model, that is, they are training samples that contain rich information and are difficult to distinguish for the large language model, thereby improving the sample quality of subsequent training of the large language model and reducing the cost and workload by repeatedly building new network models.
[0074] In an exemplary embodiment, a method for determining the training samples of the current training batch is provided. Figure 5 As shown, collecting training samples of the current training batch from the initial training sample set may include:
[0075] Step S510: collecting a first training sample from an initial training sample set according to the initial sample retention rate.
[0076] The initial sample retention rate refers to the proportion of initial samples that have not been evaluated and selected by the sample evaluation model in the training samples of the current training batch to the total number of samples in the initial training sample set. Samples can be randomly selected from the initial training samples as the first training samples based on the initial sample retention rate, or other selection methods can be used, and there is no restriction on this.
[0077] Step S520: For the uncollected samples in the initial training sample set, use the sample evaluation model to perform difficulty evaluation on the uncollected samples, and determine the uncollected samples whose evaluation results are difficult samples as the second training samples.
[0078] For uncollected samples, difficult samples can be obtained from the uncollected samples to construct the second training samples. The uncollected samples can be input into the sample evaluation model, and whether the uncollected samples are difficult samples can be determined according to the output results, so as to construct the second training samples according to the difficult samples.
[0079] As an example, random sampling can be used to randomly sample from uncollected samples, and the sampling results can be input into the sample evaluation model to determine whether the sampling result is a difficult sample based on the evaluation results. For example, random sampling c~U(0,1), that is, random sampling from 0 to 1, can be used to satisfy The sampling result is used as the second training sample, otherwise it is discarded. Indicates the sampling result x n When x is the input, the output value of the sample evaluation model is used to represent the sampling result x n is the probability of difficult samples. Among them, The larger the value, the greater the n The greater the probability of being a difficult sample, on the contrary, The smaller the value, the more x n The less likely it is to be a difficult sample.
[0080] Among them, for all unsampled samples, after determining whether they are difficult samples through the sample evaluation model, the second training sample can be determined from the difficult samples according to the sample data required for the actual batch processing. Optionally, all difficult samples can be sorted according to the size of the probability value, and difficult samples with high probability values can be obtained as the second training sample. Optionally, random selection can also be performed from all difficult samples to improve the randomness of difficult samples in the training samples of the current training batch. Exemplary embodiments of the present disclosure include but are not limited to the above-mentioned method of determining the second training sample.
[0081] Step S530: Determine the training samples of the current training batch according to the first training samples and the second training samples.
[0082] The first training samples and the second training samples are used to construct the training samples of the current training batch. By retaining a certain degree of the original initial samples (first training samples) in the training samples of the current training batch, the training samples include both difficult samples and guide the model to learn the ability to discriminate difficult samples. At the same time, the model can smoothly transition between learning easy samples and difficult samples, ensuring the stability of model training.
[0083] In an exemplary embodiment, a method for implementing training a large language model is provided. Training the large language model based on training samples includes:
[0084] First, the training sample is input into the large language model, and the prediction loss value is determined according to the prediction result; then the parameters of the large language model are adjusted according to the prediction loss value, wherein the parameters of the initial network in the sample evaluation model are adjusted along with the adjustment of the parameters of the large language model.
[0085] Specifically, the process of training the large language model is described by training the large language model in a training batch, wherein the parameters of the large language model are adjusted according to the prediction loss value, and the first training loss of the large language model can be constructed according to the prediction loss. The first training loss can be a standard large language model training loss, and there is no specific limitation on this.
[0086] As an example, the first training loss can be determined by the following formula (1):
[0087]
[0088] Among them, L O is the first training loss, E is the expectation, through which the average value of the predicted loss value of all training samples can be obtained, B is the number of training samples in the current training batch, and p o (x i |x <i ; θ) means that the large language model with θ as parameter predicts the current word as x when the previous i words are known i probability.
[0089] It should also be understood that inputting a training sample into formula (1) can obtain the predicted loss value of the training sample. Of course, formula (1) is only an example of the first training loss, and the method of constructing the first training loss can be adjusted according to actual needs.
[0090] An exemplary embodiment of the present disclosure, when training and updating parameters of a large language model, simultaneously completes the adjustment of parameters of an initial network in a sample evaluation model, thereby achieving dynamic adjustment of the sample evaluation model parameters, which is beneficial for the sample evaluation model to always predict the most "difficult" training samples for the current training of the large language model, thereby improving the accuracy and efficiency of sample selection.
[0091] In an exemplary embodiment, a method for dynamically labeling training samples is provided. Figure 6 As shown, the large language model is trained based on the training samples, and the category labels of the training samples are annotated according to the training loss values of the training samples, including:
[0092] Step S610: Obtain the current difficulty threshold.
[0093] Step S620: Compare the training loss value of the training sample with the current difficulty threshold, and label the training sample according to the comparison result.
[0094] The current difficulty threshold is a loss threshold used to distinguish the categories of training samples. It is not fixed during the entire training process, but is determined based on the loss of the large language model during the training process. If the training loss value of the training sample is greater than the current difficulty threshold, the category label of the training sample is determined to be a difficult sample; otherwise, the category label of the training sample is determined to be a non-difficult sample.
[0095] As an example, the category label of the training sample can be determined by the following formula (2):
[0096]
[0097] Among them, L O (fo(xb;)θ) is the training loss value of the training sample, which can be obtained by substituting the training sample into formula (1), T is the current difficulty threshold, z b is the category label of the training sample. Based on this, when the training sample x b When the training loss value caused to the large language model is greater than T, the training sample is determined to be a difficult sample, otherwise it is a non-difficult sample.
[0098] In order to enable the sample evaluation model to accurately predict the difficulty type of training samples, the exemplary embodiment of the present disclosure uses the loss caused by each training sample to the large language model when training the large language model to determine the category label of the training sample, thereby realizing dynamic labeling of the training samples used to train the sample evaluation model and improving the flexibility and accuracy of sample labeling.
[0099] In an exemplary embodiment, an implementation method for determining a current difficulty threshold is provided.
[0100] like Figure 7 As shown, obtaining the current difficulty threshold may include:
[0101] Step S710: Obtain the first training loss of the current training batch and the second training loss of the training batch before the current training batch.
[0102] At least one second training loss of a training batch before the current training batch may be obtained. The method for obtaining the second training loss may refer to the method for obtaining the first training loss, which will not be described in detail.
[0103] As an example, if the current training batch is the 4th training batch, in addition to obtaining the first training loss of the 4th training batch, the second training losses of the 1st to 3rd training batches are also obtained.
[0104] Step S720: Determine a current difficulty threshold according to the first training loss and the second training loss.
[0105] The current difficulty threshold can be determined based on the smooth sliding of the loss during the training process of the large language training model.
[0106] Specifically, still taking the current training batch as the fourth training batch as an example, the second training losses corresponding to the first training batch to the third training batch are L1, L2 and L3 respectively, and the first training loss of the fourth training batch is L4. The current difficulty threshold can be determined according to the following formula (3):
[0107] αL4+(1-α)[αL3+(1-α)(αL2+(1-α)L1)](3)
[0108] Among them, α is a training hyperparameter.
[0109] In an optional embodiment, since the current difficulty threshold needs to be re-determined in each training batch, when obtaining the current difficulty threshold in the fourth training batch, it can be directly determined by the following formula (4):
[0110] T4=αL4+(1-α)T3 (4)
[0111] T3 is the current difficulty threshold corresponding to the third training batch, and T4 is the current difficulty threshold corresponding to the fourth training batch. The current difficulty threshold corresponding to the third training batch can also be determined based on the current difficulty threshold corresponding to the second training batch and the first training loss of the third training batch, and so on. The current difficulty thresholds of other training batches will not be described in detail.
[0112] Of course, the above is only an example of obtaining the sliding average of the loss during the training process of the large language model. The current difficulty threshold can also be determined according to the loss during the training process by other calculation methods.
[0113] During the model training process, the loss value of training the large language model gradually decreases. By redetermining the current difficulty threshold in each training batch, the category label of the training sample based on the dynamically updated current difficulty threshold can be accurately determined in each training batch, thereby avoiding the situation where the training sample cannot be labeled as a difficult sample.
[0114] In an exemplary embodiment, a method for implementing a training sample evaluation model is provided. Training the sample evaluation model according to the training samples and corresponding category labels includes:
[0115] First, the training samples are input into the sample evaluation model to obtain the sample prediction results. Then, a loss function is constructed based on the sample prediction results and category labels. The parameters of the sample classification network of the sample evaluation model are adjusted according to the loss function.
[0116] Specifically, the parameters of the initial network in the sample evaluation model can be frozen, and only the parameters of the sample classification network are adjusted during the training process, that is, the trainable parameters of the sample evaluation model are the parameters of the sample classification network.
[0117] As an example, the sample evaluation model can be trained using the following formula (5):
[0118]
[0119] Among them, L c is the cross entropy loss, z b is a binary label (i.e., a dynamically determined category label), and the training sample x b Whether it is a difficult sample is determined by formula (2), where min is the minimum.
[0120] It should be noted that formula (5) is only an example of constructing a loss function based on sample prediction results and category labels, and the exemplary embodiments of the present disclosure do not limit the specific method of constructing a loss function based on sample prediction results and category labels.
[0121] By using training samples in the same training batch to train the sample evaluation model and the large language model, it can be ensured that the sample evaluation model can timely and accurately reflect the large language model's demand for training samples, so that the sample evaluation model can subsequently correctly predict the difficulty category of the training samples, thereby improving the accuracy of the training samples in each training batch.
[0122] Figure 8A complete flow chart for training a large language model is shown below. Figure 8 The model training method of the exemplary embodiment of the present disclosure is described. In which, a sample evaluation model is pre-constructed, the sample evaluation model shares the target network parameters in the large language model, and a sample classification network is added to the output side of the initial network determined by the target network parameters to serve as the sample evaluation model. The following is a detailed description.
[0123] Execute the first training batch.
[0124] First, training samples of the first training batch are collected from the initial training sample set, so that the training samples include difficult samples obtained from the initial training sample set based on the sample evaluation model.
[0125] Secondly, the large language model is trained based on the training samples, and the parameters in the large language model are updated. At the same time, the parameters of the initial network in the sample evaluation model are updated, and the category labels of the training samples are annotated according to the training loss values of the training samples.
[0126] Then, the sample evaluation model is trained according to the training samples and the corresponding category labels to adjust the parameters of the sample classification network in the sample evaluation model. It should be understood that the parameters in the initial network in the sample evaluation model here have been updated and frozen.
[0127] The above completes the first training batch and enters the second training batch.
[0128] First, training samples of the second training batch are collected from the initial training sample set, so that the training samples include difficult samples obtained from the initial training sample set based on the sample evaluation model.
[0129] Secondly, the large language model trained in the first training batch is trained based on the training samples, and the parameters in the large language model are updated. At the same time, the parameters of the initial network in the sample evaluation model trained in the first training batch are updated, and the category labels of the training samples are marked according to the training loss values of the training samples.
[0130] Then, the sample evaluation model obtained in the previous step is further trained according to the training samples and the corresponding category labels to adjust the parameters of the sample classification network in the sample evaluation model. It should be understood that the parameters in the initial network in the sample evaluation model here have been updated based on the previous step and frozen.
[0131] The above completes the second training batch and enters the third training batch.
[0132] The above process is executed repeatedly until the large language model is trained and stopped, and the trained large language model is obtained. The training process of the subsequent training batches can refer to the training process of the second training batch, which will not be described in detail. At the same time, in each training batch, the method of obtaining training samples, the method of classifying the training samples, and the method of determining the current difficulty threshold can all refer to the specific content of obtaining the current training batch, which will not be described in detail here.
[0133] In the technical solution provided by the embodiment of the present disclosure, on the one hand, during the training of each training batch, the training samples of the current training batch are collected from the initial training sample set, so that the training samples include the difficult samples obtained from the initial training sample set based on the sample evaluation model, and the training samples used for model training are difficult samples containing rich information, so as to realize the proper filtering of redundant data in model training, maintain the accuracy of model training, reduce the computing consumption, accelerate model training, and reduce the cost. On the other hand, during the training process of the current training batch, after the large language model is trained based on the training samples, the training samples are labeled with category labels according to the training loss value of the training samples, and the category labels used for training the sample evaluation model are dynamically adjusted, so as to facilitate the accurate removal of low-value training samples for training the large language model in subsequent training batches through the sample evaluation model, thereby improving the quality of the training samples in the subsequent training batches, and by using the same training samples to complete the training of the large language model and the sample evaluation model during the training process, the model training and the sample screening process are integrated, so that the selected samples are more in line with the training requirements of the large language model, and the model training accuracy is improved. In addition, the model acceleration training method based on sample selection proposed in the exemplary embodiment of the present disclosure is compatible with the current acceleration training method and has universal applicability.
[0134] For further reference, Fig. 9 As shown, in an exemplary embodiment of the present disclosure, a task processing method is provided, and the task processing method may include:
[0135] Step S910: Obtaining task information to be processed;
[0136] Step S920: The task information to be processed is input into the target large language model to obtain a processing result of the task information to be processed; wherein the target large language model is obtained by training a large language model based on the model training method of any of the above exemplary embodiments.
[0137] Among them, the task information to be processed can be determined according to the specific task scenario. The task scenario includes but is not limited to text generation, intelligent question answering, etc. In the specific task scenario, the large language model is trained using the initial training sample set corresponding to the task scenario to obtain the target large language model that can execute the corresponding task scenario.
[0138] As an example, the descriptive text content used to indicate text generation can be input into the target large language model as the task information to be processed, and the large language model can generate text that conforms to the descriptive text content. Of course, similar to other scenarios, the model training method of the exemplary embodiment of the present disclosure can be adapted to any scenario that requires training of a large language model to improve the accuracy and efficiency of model training.
[0139] It is worth noting that the process of training a large language model can refer to the exemplary embodiment of the above-mentioned training method, which will not be repeated here.
[0140] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.
[0141] For further reference, Fig.10 As shown, in an exemplary embodiment of the present disclosure, a model training device 1000 is provided, comprising a sample processing module 1010, a first training module 1020, a second training module 1030 and a third training module 1040. Among them:
[0142] The sample processing module 1010 is used to collect training samples of the current training batch from the initial training sample set, and the training samples include difficult samples obtained from the initial training sample set based on the sample evaluation model; the first training module 1020 is used to train the large language model based on the training samples, and mark the training samples with category labels according to the training loss values of the training samples, and the category labels are used to indicate whether the training samples are difficult samples; the second training module 1030 is used to train the sample evaluation model according to the training samples and the corresponding category labels to complete the current training batch; the third training module 1040 is used to control the model training process to enter the next training batch until the large language model completes the training and stops, thereby obtaining a trained large language model.
[0143] In an exemplary embodiment, the sample evaluation model shares the target network parameters in the large language model; the exemplary embodiment of the present disclosure also includes;
[0144] The model building module is configured to perform: determining an initial network based on target network parameters of a large language model to be trained; and adding a sample classification network to an output side of the initial network to obtain a sample evaluation model.
[0145] In an exemplary embodiment, the sample processing module 1010 is configured to perform: collecting a first training sample from an initial training sample set according to an initial sample retention rate; for uncollected samples in the initial training sample set, performing a difficulty evaluation on the uncollected samples using a sample evaluation model, and determining the uncollected samples with an evaluation result of a difficult sample as a second training sample; and determining the training samples of the current training batch according to the first training sample and the second training sample.
[0146] In an exemplary embodiment, the first training module 1020 is configured to perform: obtaining a current difficulty threshold; if the training loss value of the training sample is greater than the current difficulty threshold, determining the category label of the training sample as a difficult sample; otherwise, determining the category label of the training sample as a non-difficult sample.
[0147] In an exemplary embodiment, the first training module 1020 is configured to perform: obtaining a first training loss of a current training batch and a second training loss of a training batch before the current training batch; and determining a current difficulty threshold according to the first training loss and the second training loss.
[0148] In an exemplary embodiment, the first training module 1020 is configured to perform: inputting the training sample into the large language model, determining the prediction loss value based on the prediction result; adjusting the parameters of the large language model based on the prediction loss value; wherein the parameters of the initial network in the sample evaluation model are adjusted along with the adjustment of the parameters of the large language model.
[0149] In an exemplary embodiment, the second training module 1030 is configured to perform: inputting the training sample into the sample evaluation model to obtain a sample prediction result;
[0150] A loss function is constructed based on the sample prediction results and category labels, and the parameters of the sample classification network of the sample evaluation model are adjusted according to the loss function.
[0151] Since the details of each functional module of the model training device of the exemplary embodiment of the present disclosure have been recorded in the exemplary embodiment of the above-mentioned model training method, they will not be repeated here.
[0152] For further reference, Fig.11 As shown, in an exemplary embodiment of the present disclosure, a task processing device 1100 is provided, comprising an information acquisition module 1110 and a task processing module 1120. Among them:
[0153] The information acquisition module 1110 is used to acquire the task information to be processed; the task processing module 1120 is used to input the task information to be processed into the target large language model to obtain the processing result of the task information to be processed; wherein the target large language model is obtained by training the large language model based on the model training method of any of the above exemplary embodiments.
[0154] Since the details of the functional modules of the task processing device of the exemplary embodiment of the present disclosure have been described in the exemplary embodiment of the task processing method described above, they will not be repeated here.
[0155] It should be noted that although several modules or units of the model training device and task processing are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.
[0156] The exemplary embodiments of the present disclosure also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the above method is implemented.
[0157] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.
[0158] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.
[0159] The code of the computer program can be written in one or more programming languages. Programming languages such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).
[0160] The computer program may be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. The electronic device may convert the signal carrying the computer program into a digital signal, thereby running the computer program. When the computer program is run on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device may execute) the method steps of various exemplary embodiments of the present disclosure, such as the above-mentioned method.
[0161] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module", or "system".
[0162] Refer to the following Fig.12 1200 according to this embodiment of the present disclosure is described. Fig.12 The electronic device 1200 shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure. Fig.12 As shown, the electronic device 1200 is in the form of a general computing device. The components of the electronic device 1200 may include, but are not limited to: the at least one processing unit 1210, the at least one storage unit 1220, a bus 1230 connecting different system components (including the storage unit 1220 and the processing unit 1210), and a display unit 1240.
[0163] The storage unit stores program codes, which can be executed by the processing unit 1210, so that the processing unit 1210 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.
[0164] The storage unit 1220 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 1221 and / or a cache storage unit 1222, and may further include a read-only storage unit (ROM) 1223. The storage unit 1220 may also include a program / utility 1224 having a set (at least one) of program modules 1225, such program modules 1225 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include the implementation of a network environment.
[0165] Bus 1230 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0166] The electronic device 1200 may also communicate with one or more external devices 1300 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1200, and / or communicate with any device that enables the electronic device 1200 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1250. In addition, the electronic device 1200 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1260. As shown, the network adapter 1260 communicates with other modules of the electronic device 1200 via a bus 1230. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0167] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.
[0168] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.
[0169] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.
Claims
1. A model training method, characterized in that: include: Collecting training samples of a current training batch from an initial training sample set, wherein the training samples include difficult samples obtained from the initial training sample set based on a sample evaluation model; Training a large language model based on the training samples, and labeling the training samples with category labels according to the training loss values of the training samples, wherein the category labels are used to indicate whether the training samples are difficult samples; Training the sample evaluation model according to the training samples and corresponding category labels to complete the current training batch; Enter the next training batch until the large language model completes training and stops, thereby obtaining a trained large language model.
2. The method according to claim 1, characterized in that The sample evaluation model shares the target network parameters in the large language model; before training the large language model, the method further includes: Determining an initial network based on target network parameters of the large language model to be trained; A sample classification network is added to the output side of the initial network to obtain the sample evaluation model.
3. The method according to claim 1, characterized in that The step of collecting training samples of the current training batch from the initial training sample set includes: Collecting a first training sample from the initial training sample set according to the initial sample retention rate; For the uncollected samples in the initial training sample set, the sample evaluation model is used to perform difficulty evaluation on the uncollected samples, and the uncollected samples whose evaluation results are difficult samples are determined as second training samples; The training samples of the current training batch are determined according to the first training samples and the second training samples.
4. The method according to claim 1, characterized in that: The step of training the large language model based on the training samples and labeling the training samples with category labels according to the training loss values of the training samples includes: Get the current difficulty threshold; If the training loss value of the training sample is greater than the current difficulty threshold, determining that the category label of the training sample is a difficult sample; Otherwise, the category label of the training sample is determined as a non-difficult sample.
5. The method according to claim 4, characterized in that The obtaining of the current difficulty threshold comprises: Get the first training loss of the current training batch and the second training loss of the training batch before the current training batch; The current difficulty threshold is determined according to the first training loss and the second training loss.
6. The method according to claim 2, characterized in that The training of the large language model based on the training sample includes: Inputting the training sample into the large language model, and determining a prediction loss value according to the obtained prediction result; Adjusting parameters of the large language model according to the prediction loss value; The parameters of the initial network in the sample evaluation model are adjusted along with the adjustment of the parameters of the large language model.
7. The method according to claim 6, characterized in that The step of training the sample evaluation model according to the training samples and the corresponding category labels includes: Inputting the training sample into the sample evaluation model to obtain a sample prediction result; A loss function is constructed according to the sample prediction result and the category label, and the parameters of the sample classification network of the sample evaluation model are adjusted according to the loss function.
8. A task processing method, characterized in that: include: Get the task information to be processed; Inputting the task information to be processed into the target large language model to obtain a processing result of the task information to be processed; The target large language model is obtained by training a large language model based on the method according to any one of claims 1 to 7.
9. A model training device, characterized in that: include: A sample processing module, used for collecting training samples of a current training batch from an initial training sample set, wherein the training samples include difficult samples obtained from the initial training sample set based on a sample evaluation model; A first training module is used to train a large language model based on the training samples, and to label the training samples with category labels according to the training loss values of the training samples, wherein the category labels are used to indicate whether the training samples are difficult samples; A second training module, used for training the sample evaluation model according to the training samples and corresponding category labels to complete the current training batch; The third training module is used to control the model training process to enter the next training batch until the large language model is trained and stops, thereby obtaining a trained large language model.
10. A task processing device, characterized in that: include: An information acquisition module is used to obtain task information to be processed; A task processing module, used for inputting the task information to be processed into a target large language model to obtain a processing result of the task information to be processed; The target large language model is obtained by training a large language model based on the method according to any one of claims 1 to 7.
11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
12. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 8 by executing the executable instructions.
Citation Information
Patent Citations
Speech recognition method and system, electronic equipment and storage medium
CN111613212A
Construction method of multi-label hybrid classification model and news classification method and system
CN112070138A
Difficult sample mining method and system, terminal and storage medium
CN113920079A
Model training method and device, equipment and storage medium
CN116756577A
Hyperspectral image classification method based on active deep learning
CN116797810A
