Model training method, task processing method, device, program product and equipment

Abstract

The present disclosure provides a model training method and device, a task processing method and device, a computer program product, and an electronic device, and relates to the field of computer technology. The model training method includes: collecting training samples of the current training batch from the initial training sample set, the training samples including difficult samples obtained from the initial training sample set based on a sample evaluation model; training a large language model based on the training samples, and labeling the training samples with category labels based on the training loss values ​​of the training samples, the category labels being used to indicate whether the training samples are difficult samples; training the sample evaluation model based on the training samples and the corresponding category labels to complete the current training batch; entering the next training batch until the large language model completes training and stops, thereby obtaining a trained large language model. The present disclosure can improve the training accuracy and training efficiency of the large language model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a model training method, a task processing method, a model training device, a task processing device, a computer program product, and an electronic device. Background Art

[0002] In recent years, the emergence of large language models has revolutionized the fields of natural language processing and artificial intelligence. These models are applicable to a wide range of scenarios, such as text generation and intelligent question-answering. Training large language models requires extensive computing resources, storage space, and time. Traditional training methods can take months or even half a year. To address this, distributed training, computational optimization, and hardware acceleration are currently being used to improve the convergence speed and computational efficiency of the training process. However, current methods for training large language models cannot simultaneously meet the requirements for high accuracy, high efficiency, and low cost.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0004] The purpose of the present disclosure is to provide a model training method, a task processing method, a model training device, a task processing device, a computer program product, and an electronic device, thereby improving the training efficiency and training accuracy of large language models, at least to a certain extent, and reducing training costs.

[0005] According to a first aspect of the present disclosure, a model training method is provided, comprising: collecting training samples for a current training batch from an initial training sample set, the training samples including difficult samples obtained from the initial training sample set based on a sample evaluation model; training a large language model based on the training samples, and labeling the training samples with category labels based on training loss values ​​of the training samples, the category labels being used to indicate whether the training samples are difficult samples; training the sample evaluation model based on the training samples and corresponding category labels to complete the current training batch; and entering the next training batch until the large language model completes training and stops, thereby obtaining a trained large language model.

[0006] In an exemplary embodiment of the present disclosure, the sample evaluation model shares the target network parameters in the large language model; before training the large language model, the method further includes: determining an initial network based on the target network parameters of the large language model to be trained; and adding a sample classification network to the output side of the initial network to obtain a sample evaluation model.

[0007] In an exemplary embodiment of the present disclosure, training samples for a current training batch are collected from an initial training sample set, including: collecting a first training sample from the initial training sample set according to an initial sample retention rate; for uncollected samples in the initial training sample set, performing a difficulty evaluation on the uncollected samples using a sample evaluation model, and determining the uncollected samples whose evaluation results are difficult samples as second training samples; and determining the training samples for the current training batch based on the first training samples and the second training samples.

[0008] In an exemplary embodiment of the present disclosure, a large language model is trained based on training samples, and category labels are labeled on the training samples according to the training loss values ​​of the training samples, including: obtaining a current difficulty threshold; if the training loss value of the training sample is greater than the current difficulty threshold, determining the category label of the training sample as a difficult sample; otherwise, determining the category label of the training sample as a non-difficult sample.

[0009] In an exemplary embodiment of the present disclosure, obtaining the current difficulty threshold includes: obtaining a first training loss of a current training batch and a second training loss of a training batch before the current training batch; and determining the current difficulty threshold based on the first training loss and the second training loss.

[0010] In an exemplary embodiment of the present disclosure, a large language model is trained based on the training samples, including: inputting the training samples into the large language model, determining a prediction loss value based on the obtained prediction results; adjusting the parameters of the large language model based on the prediction loss value; wherein the parameters of the initial network in the sample evaluation model are adjusted as the parameters of the large language model are adjusted.

[0011] In an exemplary embodiment of the present disclosure, a sample evaluation model is trained based on training samples and corresponding category labels, including: inputting the training samples into the sample evaluation model to obtain sample prediction results; constructing a loss function based on the sample prediction results and category labels, and adjusting the parameters of the sample classification network of the sample evaluation model based on the loss function.

[0012] According to a second aspect of the present disclosure, a task processing method is provided, comprising: obtaining task information to be processed; inputting the task information to be processed into a target large language model to obtain a processing result of the task information to be processed; wherein the target large language model is obtained by training a large language model based on any of the above methods.

[0013] According to a third aspect of the present disclosure, a model training device is provided, comprising: a sample processing module for collecting training samples of a current training batch from an initial training sample set, the training samples including difficult samples obtained from the initial training sample set based on a sample evaluation model; a first training module for training a large language model based on the training samples, and labeling the training samples with category labels based on the training loss values ​​of the training samples, the category labels being used to indicate whether the training samples are difficult samples; a second training module for training the sample evaluation model based on the training samples and the corresponding category labels to complete the current training batch; and a third training module for controlling the model training process to enter the next training batch until the large language model completes training and stops, thereby obtaining a trained large language model.

[0014] According to a fourth aspect of the present disclosure, a task processing device is provided, comprising: an information acquisition module for acquiring task information to be processed; a task processing module for inputting the task information to be processed into a target large language model to obtain a processing result of the task information to be processed; wherein the target large language model is obtained by training a large language model based on any of the above methods.

[0015] According to one aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, any one of the above methods is implemented.

[0016] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any one of the above methods by executing the executable instructions.

[0017] In the technical solutions provided by the embodiments of the present disclosure, on the one hand, during the training of each training batch, training samples for the current training batch are collected from the initial training sample set, so that the training samples include difficult samples obtained from the initial training sample set based on the sample evaluation model. Therefore, the training samples used for model training are difficult samples containing rich information, thereby properly filtering redundant data during model training, maintaining model training accuracy while accelerating model training and reducing costs. On the other hand, during the training process of the current training batch, after the large language model is trained based on the training samples, the training samples are labeled with category labels based on the training loss values ​​of the training samples, and the category labels used for training the sample evaluation model are dynamically adjusted. This facilitates the accurate removal of low-value training samples for training the large language model in subsequent training batches through the sample evaluation model, thereby improving the quality of training samples in subsequent training batches. Furthermore, by using the same training samples to complete the training of the large language model and the sample evaluation model during the training process, the model training and sample screening processes are integrated, so that the selected samples are more in line with the training requirements of the large language model, thereby improving the model training accuracy.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0020] Figure 1 A schematic diagram showing the stages involved in the large language model training solution of an embodiment of the present disclosure is shown;

[0021] Figure 2 A flowchart schematically illustrates a model training method according to an exemplary embodiment of the present disclosure;

[0022] Figure 3 A flowchart schematically illustrates an implementation method of building a sample evaluation model according to an exemplary embodiment of the present disclosure;

[0023] Figure 4 A schematic diagram schematically illustrates a method for constructing a sample evaluation model in an exemplary embodiment of the present disclosure;

[0024] Figure 5 A flowchart schematically illustrates an implementation method for determining training samples of a current training batch in an exemplary embodiment of the present disclosure;

[0025] Figure 6 A flowchart schematically illustrates a method for dynamically labeling training samples in an exemplary embodiment of the present disclosure;

[0026] Figure 7 A flowchart schematically illustrates an implementation method for determining a current difficulty threshold according to an exemplary embodiment of the present disclosure;

[0027] Figure 8 Schematically illustrates a complete flow chart of training a large language model in an exemplary embodiment of the present disclosure;

[0028] Figure 9 A flowchart schematically illustrates a task processing method in an exemplary embodiment of the present disclosure;

[0029] Figure 10 A schematic diagram schematically illustrates the composition of a model training device in an exemplary embodiment of the present disclosure;

[0030] Figure 11 A schematic diagram schematically illustrates the composition of a task processing device in an exemplary embodiment of the present disclosure;

[0031] Figure 12 A schematic diagram of an electronic device to which the embodiments of the present disclosure can be applied is shown. DETAILED DESCRIPTION

[0032] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0033] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0034] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0035] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0036] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning through demonstration.

[0037] The technical solution provided by the exemplary embodiment of the present disclosure relates to machine learning technology for artificial intelligence, which utilizes machine learning technology to collect training samples of the current training batch, including difficult samples, from the initial training sample set, and then trains a large language model based on the training samples. The training samples are labeled with category labels according to the training loss values ​​of the training samples, and the sample evaluation model is trained based on the training samples and the corresponding category labels to complete the current training batch. Before the large language model is trained, the above process is executed in a loop to finally obtain a trained large language model. The model training method of the exemplary embodiment of the present disclosure and the large language model obtained by training can be applied to various natural language processing or artificial intelligence scenarios, such as text translation, text generation, question answering and other tasks, and can improve the training accuracy and efficiency of the large language model, reduce training costs, and improve the quality of task processing.

[0038] Figure 1A schematic diagram of the stages involved in the technical solution of the exemplary embodiment of the present disclosure is shown, as Figure 1 As shown, the technical solution of the exemplary embodiment of the present disclosure includes a model training stage and a model application stage.

[0039] In an exemplary embodiment of the present disclosure, the provided model training method can be executed by a terminal device. During the training phase of the large language model, the terminal device can collect training samples including difficult samples of the current training batch from the initial training sample set, and then train the large language model based on the training samples, label the training samples with category labels according to the training loss values ​​of the training samples, and train the sample evaluation model based on the training samples and the corresponding category labels to complete the current training batch. Before the large language model is trained, the above process is executed in a loop to finally obtain a trained large language model, so as to use the trained large language model to process the task information to be processed.

[0040] In this way, when all steps in the model training method and task processing method provided by the exemplary embodiment of the present disclosure can be executed by the terminal device, all steps can be executed by the processor of the terminal device.

[0041] The terminal device can be an intelligent device with data processing capabilities, such as a monitoring device, a smart phone, a computer, a tablet computer, a vehicle-mounted device, a wearable device, etc. The terminal device can also be called a mobile terminal, a terminal, a mobile device, etc. The present disclosure does not limit the type of terminal device.

[0042] Furthermore, the technical solutions provided by the exemplary embodiments of the present disclosure may also be executed by a server. Accordingly, in this server-side execution mode, the server may begin executing the steps of the technical solutions of the exemplary embodiments of the present disclosure in response to a trigger instruction, wherein the trigger instruction may be sent by a terminal device used by a user, or may be triggered locally by the server in response to some automated events. The server may be a backend system that provides the relevant services in the exemplary embodiments of the present disclosure, and may include a single electronic device with computing capabilities, such as a portable computer, a desktop computer, or a smartphone, or a cluster of multiple electronic devices.

[0043] In addition, the technical solutions of the exemplary embodiments of the present disclosure can also be executed collaboratively by the terminal device and the server. In this manner of collaborative execution by the electronic device and the server, some steps in the technical solutions provided by the exemplary embodiments of the present disclosure are executed by the terminal device, while other steps are executed by the server. For example, the technical solutions provided by the exemplary embodiments of the present disclosure can be executed by the server in the model training phase, and the trained large language model is sent to the terminal device, and the terminal device executes the model application phase, that is, the trained large language model is used to perform task processing.

[0044] It should be noted that in this method of collaborative execution by the terminal device and the server, the steps respectively executed by the terminal device and the server can be dynamically adjusted according to actual conditions, and the exemplary embodiments of the present disclosure do not impose special restrictions on this.

[0045] Training large language models requires extensive computing resources, storage space, and time. Traditional training methods can take months or even half a year. To address this, methods are being developed and improved to accelerate training. These include distributed training, computational optimization, and hardware acceleration. Distributed training distributes model training across multiple computers for parallel computation, with each computer responsible for processing a portion of the data or a different part of the model. Parameter updates and model fusion are then performed through communication and synchronization mechanisms. While this approach significantly accelerates model training, the total computational effort required remains unchanged, and training costs remain very high. Computational optimization improves the convergence speed and computational efficiency of the training process through techniques such as optimizer improvements and operator optimization fusion. However, adjusting the hyperparameters of the optimization algorithm is difficult and time-consuming, and optimizing the computational layout depends on the specific structural design of the model, requiring high customization and difficulty meeting the training requirements of various models. Hardware acceleration methods, such as the use of specialized chips such as higher-performance GPUs (graphics processing units), are also available. However, high-performance computing chips are typically expensive, and adapting and implementing new chips can require additional effort and resources.

[0046] Therefore, current methods for training large language models cannot meet the requirements of high efficiency, high precision and low cost.

[0047] Exemplarily, the large language model of the exemplary embodiment of the present disclosure can be a large model of the billion level, which refers to a deep learning model trained with a large amount of sample data, which can generate natural language text or understand the meaning of language text. It can also be constructed using lightweight neural networks, such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Networks), DNN (Deep Neural Networks), etc. Compared with lightweight neural networks, a language model with a more complex network structure can also be used, for example, it can be constructed using a heavyweight neural network, such as BERT (Bidirectional Encoder Representation from Transformers, feature encoder of bidirectional Transformers), Transformer (a language processing model), and any heavyweight neural network that can be used for task processing. The exemplary embodiment of the present disclosure takes the training of a large language model with a large model of the billion level as an example.

[0048] The exemplary embodiment of the present disclosure first provides a model training method. Figure 2 As shown, the model training method of the embodiment of the present disclosure may include the following steps S210 and S240:

[0049] In step S210 , training samples of a current training batch are collected from the initial training sample set, where the training samples include difficult samples obtained from the initial training sample set based on a sample evaluation model.

[0050] In an exemplary embodiment of the present disclosure, the initial training samples are all samples used to train the model, and the training samples can be obtained from various channels, without limitation.

[0051] In most cases, training samples require labeling. During the iterative model training process, a significant portion of these samples are easily misidentified by the model, rendering them useless or of minimal use. Labeling these samples is also very costly. In contrast to these "easy samples," training samples that provide useful information for model training and ensure accuracy are called difficult samples. These difficult samples can guide model learning and judgment, continuously improving the model's ability to handle difficult samples and ultimately improving model training accuracy.

[0052] It should be understood that the difficulty of a sample refers to the difficulty of learning the sample data. For example, the difficulty of learning a text or image sample. The difficulty is specific to a particular model; different models may have different learning difficulties for the same text or image.

[0053] The sample evaluation model is a model used to evaluate the difficulty of training samples. It can be a classification model, such as a binary classification model, that can evaluate whether a training sample is a difficult sample or a non-difficult sample (also referred to as an easy sample). In each training batch, the exemplary embodiments of the present disclosure obtain difficult samples from the initial training sample set based on the sample evaluation model as training samples for the current training batch.

[0054] In some optional embodiments, when collecting training samples for the current training batch, a sample evaluation model can be used to perform a difficulty evaluation on all initial samples in the initial training sample set, and difficult samples can be obtained from the initial training sample set based on the evaluation results as training samples for the current training batch. This method can filter out redundant samples (such as non-difficult samples), thereby reducing costs and improving efficiency while ensuring model training accuracy.

[0055] In some optional embodiments, some initial samples may be retained in the training samples of the current training batch, and some difficult samples may be collected to jointly construct the training samples of the current training batch, so that the model can ensure the stability of model training under the joint training of non-difficult samples and difficult samples.

[0056] Batch: refers to the combination of multiple training samples used in a training process. A batch can include training samples corresponding to different categories, and a batch includes at least two training samples. In some cases, the ratio of each type of sample in a batch is fixed. For example, there are 4 categories included in the model usage scenario, namely category 1, category 2, category 3 and category 4, and the set sample ratio corresponding to each category is 1:2:1:1. Then, when constructing a batch, the ratio of the training samples corresponding to category 1, the training samples corresponding to category 2, the training samples corresponding to category 3 and the training samples corresponding to category 4 included in the batch can still be maintained at 1:2:1:1. The training samples corresponding to each training batch in the exemplary embodiment of the present disclosure can be understood as a batch. Batches of different training batches can be exactly the same batches, completely different batches, or batches with some of the same training samples. Completely the same here means that the training samples of different training batches are the same.

[0057] During training, the model selects batches of samples for processing, and in each training batch, training samples for that batch are collected from the initial training sample set. The size of the training batch (batch) affects the choice of learning rate. The exemplary embodiments of the present disclosure determine the size of the training batch based on actual needs and are not limited to this.

[0058] In step S220, the large language model is trained based on the training samples, and the training samples are labeled with category labels according to the training loss values ​​of the training samples. The category labels are used to indicate whether the training samples are difficult samples.

[0059] In an exemplary embodiment of the present disclosure, a large language model is trained using training samples, and parameters within the large language model are adjusted. To ensure that all difficult samples identified by the sample evaluation model throughout the training process are difficult for the large language model, after the large language model is trained and updated, the sample evaluation model is further trained using training samples from the current training batch to adjust parameters within the sample evaluation model.

[0060] Among them, the training samples can be marked with category labels to indicate whether they are difficult samples based on their training loss values, so as to be used in the training process of the sample evaluation model. By dynamically adjusting the category labels used for training the sample evaluation model, it is beneficial to accurately remove training samples with low value for training the large language model in subsequent training batches through the sample evaluation model, thereby improving the quality of training samples in subsequent training batches.

[0061] In step S230 , the sample evaluation model is trained according to the training samples and the corresponding category labels to complete the current training batch.

[0062] In an exemplary embodiment of the present disclosure, after the parameters of the large language model are updated in the current training batch, the sample evaluation model is trained based on the training samples and their corresponding category labels. When the sample evaluation model completes this round of training, the current training batch is complete. In other words, an exemplary embodiment of the present disclosure includes training of both the large language model and the sample evaluation model in a single training batch.

[0063] In step S240 , the next training batch is entered until the large language model is trained and stops, thereby obtaining a trained large language model.

[0064] In an exemplary embodiment of the present disclosure, after completing the current training batch, if the training of the large language model has not yet ended, the next training batch can be entered, and the next training batch can continue to be used as the current training batch, that is, returning to the step of collecting training samples for the current training batch from the initial training sample set.

[0065] The above steps are repeated until the large language model completes training. Completion of large language model training may mean that the large language model reaches convergence, or the number of training batches reaches a set number, or the sample evaluation model completes processing of all initial samples in the initial training sample set. The exemplary embodiments of the present disclosure can flexibly select the conditions for stopping model training based on actual needs and are not limited to this.

[0066] In the technical solutions provided by the embodiments of the present disclosure, on the one hand, during the training of each training batch, training samples for the current training batch are collected from the initial training sample set, so that the training samples include difficult samples obtained from the initial training sample set based on the sample evaluation model. Therefore, the training samples used for model training are difficult samples containing rich information, thereby properly filtering redundant data during model training, maintaining model training accuracy while accelerating model training and reducing costs. On the other hand, during the training process of the current training batch, after the large language model is trained based on the training samples, the training samples are labeled with category labels based on the training loss values ​​of the training samples, and the category labels used for training the sample evaluation model are dynamically adjusted. This facilitates the accurate removal of low-value training samples for training the large language model in subsequent training batches through the sample evaluation model, thereby improving the quality of training samples in subsequent training batches. Furthermore, by using the same training samples to complete the training of the large language model and the sample evaluation model during the training process, the model training and sample screening processes are integrated, so that the selected samples are more in line with the training requirements of the large language model, thereby improving the model training accuracy.

[0067] In an exemplary embodiment, a method for constructing a sample evaluation model is also provided. The sample evaluation model shares the target network parameters in the large language model, such as Figure 3 As shown, before training the large language model, it also includes:

[0068] Step S310: Determine an initial network based on target network parameters of the large language model to be trained.

[0069] Step S320: Add a sample classification network to the output side of the initial network to obtain a sample evaluation model.

[0070] Among them, the sample evaluation model shares the target network parameters in the large language model as the initial network, making the base of the sample evaluation model the large language model. The target network parameters can be selected according to actual needs (such as the amount of computation). The sample classification network is a classifier used to score the difficulty of sample data. The specific network structure is not restricted, and it can output the probability that the training sample is a "difficult sample". In this case, when training the large language model, as the parameters of the large language model are adjusted, the parameters of the initial network in the sample evaluation model can be dynamically adjusted, and the parameters of the sample classification network can be frozen. Accordingly, when training the sample evaluation model, the parameters of the initial network are frozen, and the parameters of the sample classification network are adjusted.

[0071] As an example, Figure 4 A schematic diagram of constructing a sample evaluation model is shown. Figure 4 For a large language model with L layers and each layer width W, its first L / 2 layers (LX / 2) and the first 1 / 4 width (WX / 4) are used as the initial network of the sample evaluation model, and a sample classification network is added on the output side of the initial network. If the sample classification network is a binary classification network, the probability of the training sample being a difficult sample is determined, thereby determining whether the training sample is a difficult sample or a non-difficult sample.

[0072] At the same time, it should be understood that in the current training batch, the large language model is first trained based on the training samples, and the parameters in the large language model are updated. After training the large language model, the parameters of the initial network of the sample evaluation model are updated accordingly, because the initial network of the sample evaluation model shares the target network parameters of the large language model. Therefore, in the current training batch, when the sample evaluation model is trained according to the training samples and the corresponding category labels, the parameters of the initial network in the sample evaluation model have been updated based on the training of the large language model.

[0073] That is to say, since the sample evaluation model shares the target network parameters in the large language model, it can dynamically adjust the parameters of the initial network in the sample evaluation model according to the state of the large language model, so that the difficult samples subsequently predicted by the sample evaluation model are always "difficult" for the training of the large language model, that is, they are training samples that contain rich information and are difficult to distinguish for the large language model, thereby improving the sample quality of subsequent training of the large language model, and at the same time reducing the repeated construction of new network models, reducing costs and workload.

[0074] In an exemplary embodiment, a method for determining the training samples of the current training batch is provided. Figure 5 As shown, collecting training samples of the current training batch from the initial training sample set may include:

[0075] Step S510: collecting a first training sample from the initial training sample set according to the initial sample retention rate.

[0076] The initial sample retention rate refers to the proportion of initial samples in the training samples of the current training batch that have not been evaluated and selected by the sample evaluation model to the total number of samples in the initial training sample set. A sample can be randomly selected from the initial training samples as the first training sample based on the initial sample retention rate, or other selection methods can be used, and there is no restriction on this.

[0077] Step S520: For the uncollected samples in the initial training sample set, the sample evaluation model is used to perform difficulty evaluation on the uncollected samples, and the uncollected samples evaluated as difficult samples are determined as second training samples.

[0078] For uncollected samples, difficult samples can be obtained from the uncollected samples to construct a second training sample. The uncollected samples can be input into a sample evaluation model, and the output results can be used to determine whether the uncollected samples are difficult samples, thereby constructing the second training sample based on the difficult samples.

[0079] As an example, random sampling can be used to randomly sample from uncollected samples, and the sampling results can be input into the sample evaluation model to determine whether the sampling result is a difficult sample based on the evaluation results. For example, random sampling c~U(0,1), that is, random sampling from 0 to 1, can be used to satisfy The sampling result is used as the second training sample, otherwise it is discarded. Indicates the sampling result x n When it is input, the output value of the sample evaluation model is used to represent the sampling result x n is the probability of difficult samples. The larger the value, the greater the x n The greater the probability of being a difficult sample, on the contrary, The smaller the value, the more x n The less likely it is to be a difficult sample.

[0080] For all unsampled samples, after determining whether they are difficult samples using the sample evaluation model, a second training sample can be determined from the difficult samples based on the sample data required for the actual batch processing. Optionally, all difficult samples can be sorted by probability value, and difficult samples with high probability values ​​can be obtained as the second training sample. Optionally, a random selection can be made from all difficult samples to improve the randomness of difficult samples in the training samples of the current training batch. Exemplary embodiments of the present disclosure include but are not limited to the above-mentioned method for determining the second training sample.

[0081] Step S530: Determine the training samples of the current training batch according to the first training samples and the second training samples.

[0082] The obtained first training samples and second training samples are used to construct the training samples of the current training batch. By retaining a certain degree of original initial samples (first training samples) in the training samples of the current training batch, the training samples include both difficult samples and guide the model to learn the ability to discriminate difficult samples. At the same time, the model can smoothly transition between learning easy samples and difficult samples, ensuring the stability of model training.

[0083] In an exemplary embodiment, a method for training a large language model is provided. Training the large language model based on training samples includes:

[0084] First, the training sample is input into the large language model, and the prediction loss value is determined based on the prediction result; then, the parameters of the large language model are adjusted according to the prediction loss value, wherein the parameters of the initial network in the sample evaluation model are adjusted along with the adjustment of the parameters of the large language model.

[0085] Specifically, the above-described process of training a large language model is described by training the large language model in a single training batch. The parameters of the large language model are adjusted based on the prediction loss value. A first training loss for the large language model can be constructed based on the prediction loss. The first training loss can be a standard large language model training loss, and this is not specifically limited.

[0086] As an example, the first training loss can be determined by the following formula (1):

[0087]

[0088] Among them, L O is the first training loss, E is the expectation, through which the average value of the predicted loss value of all training samples can be obtained, B is the number of training samples in the current training batch, and p o (x i |x ＜i ; θ) means that the large language model with θ as the parameter predicts the current word as x when the previous i words are known i probability.

[0089] It should also be understood that inputting a training sample into formula (1) can obtain the predicted loss value of the training sample. Of course, formula (1) is only an example of the first training loss, and the method of constructing the first training loss can be adjusted according to actual needs.

[0090] In an exemplary embodiment of the present disclosure, when training and updating parameters of a large language model, the parameters of the initial network in the sample evaluation model are adjusted simultaneously, thereby achieving dynamic adjustment of the sample evaluation model parameters. This is beneficial for the sample evaluation model to always predict the training samples that are most "difficult" for the current training of the large language model, thereby improving the accuracy and efficiency of sample selection.

[0091] In an exemplary embodiment, a method for dynamically labeling training samples is provided. Figure 6 As shown, the large language model is trained based on the training samples, and the category labels of the training samples are annotated according to the training loss values ​​of the training samples, including:

[0092] Step S610: Obtain the current difficulty threshold.

[0093] Step S620: Compare the training loss value of the training sample with the current difficulty threshold, and label the training sample according to the comparison result.

[0094] The current difficulty threshold is the loss threshold used to distinguish the categories of training samples. It is not fixed throughout the training process, but is determined based on the loss of the large language model during training. If the training loss value of a training sample is greater than the current difficulty threshold, the training sample is classified as a difficult sample; otherwise, the training sample is classified as a non-difficult sample.

[0095] As an example, the category label of the training sample can be determined by the following formula (2):

[0096]

[0097] Among them, L O (fo(xb;)θ) is the training loss value of the training sample, which can be obtained by substituting the training sample into formula (1), T is the current difficulty threshold, z b is the category label of the training sample. Based on this, when the training sample x b When the training loss value caused to the large language model is greater than T, the training sample is determined to be a difficult sample, otherwise it is a non-difficult sample.

[0098] In order to enable the sample evaluation model to accurately predict the difficulty type of training samples, the exemplary embodiment of the present disclosure uses the loss caused by each training sample to the large language model when training the large language model to determine the category label of the training sample, thereby realizing dynamic labeling of the training samples used to train the sample evaluation model and improving the flexibility and accuracy of sample labeling.

[0099] In an exemplary embodiment, an implementation for determining a current difficulty threshold is provided.

[0100] like Figure 7 As shown, obtaining the current difficulty threshold may include:

[0101] Step S710: Obtain the first training loss of the current training batch and the second training loss of the training batch before the current training batch.

[0102] At least one second training loss of a training batch before the current training batch can be obtained. The method for obtaining the second training loss can refer to the method for obtaining the first training loss above, and will not be described in detail here.

[0103] As an example, if the current training batch is the 4th training batch, in addition to obtaining the first training loss of the 4th training batch, the second training losses of the 1st to 3rd training batches are also obtained.

[0104] Step S720: Determine a current difficulty threshold according to the first training loss and the second training loss.

[0105] The current difficulty threshold can be determined based on the smooth sliding of the loss during the training process of the large language training model.

[0106] Specifically, still taking the current training batch as the fourth training batch as an example, the second training losses corresponding to the first to third training batches are L1, L2 and L3 respectively, and the first training loss of the fourth training batch is L4. The current difficulty threshold can be determined according to the following formula (3):

[0107] αL4+(1-α)[αL3+(1-α)(αL2+(1-α)L1)](3)

[0108] Among them, α is a training hyperparameter.

[0109] In an optional embodiment, since the current difficulty threshold needs to be re-determined in each training batch, when obtaining the current difficulty threshold in the fourth training batch, it can be directly determined by the following formula (4):

[0110] Τ4=αL4+(1-α)Τ3 (4)

[0111] T3 is the current difficulty threshold corresponding to the third training batch, and T4 is the current difficulty threshold corresponding to the fourth training batch. The current difficulty threshold corresponding to the third training batch can also be determined based on the current difficulty threshold corresponding to the second training batch and the first training loss of the third training batch, and so on. The current difficulty thresholds for other training batches are not described in detail here.

[0112] Of course, the above is only an example of obtaining a sliding average of the loss during the training process of a large language model. Other calculation methods can also be used to determine the current difficulty threshold based on the loss during the training process.

[0113] During the model training process, the loss value of training the large language model gradually decreases. By re-determining the current difficulty threshold in each training batch, the current difficulty threshold based on dynamic updates can accurately determine the category label of the training sample in each training batch, thereby avoiding the situation where the training sample cannot be labeled as a difficult sample.

[0114] In an exemplary embodiment, a method for implementing a training sample evaluation model is provided. Training the sample evaluation model based on training samples and corresponding category labels includes:

[0115] First, the training samples are input into the sample evaluation model to obtain the sample prediction results. Then, a loss function is constructed based on the sample prediction results and category labels, and the parameters of the sample classification network of the sample evaluation model are adjusted according to the loss function.

[0116] Specifically, the parameters of the initial network in the sample evaluation model can be frozen, and only the parameters of the sample classification network are adjusted during the training process, that is, the trainable parameters of the sample evaluation model are the parameters of the sample classification network.

[0117] As an example, the sample evaluation model can be trained using the following formula (5):

[0118]

[0119] Among them, L c is the cross entropy loss, z b is a binary label (i.e., a dynamically determined category label), and the training sample x b Whether it is a difficult sample is determined by formula (2), and min is the minimum.

[0120] It should be noted that formula (5) is only an example of constructing a loss function based on sample prediction results and category labels. The exemplary embodiments of the present disclosure do not limit the specific method of constructing a loss function based on sample prediction results and category labels.

[0121] By using training samples in the same training batch to train the sample evaluation model and the large language model, it can be ensured that the sample evaluation model reflects the large language model's demand for training samples in a timely and accurate manner, so that the sample evaluation model can subsequently correctly predict the difficulty category of the training samples, thereby improving the accuracy of the training samples in each training batch.

[0122] Figure 8A complete flow chart for training a large language model is shown below. Figure 8 The model training method of an exemplary embodiment of the present disclosure is described. A sample evaluation model is pre-built, sharing the target network parameters of the large language model. A sample classification network is added to the output of the initial network, which is determined by the target network parameters, to serve as the sample evaluation model. This is described in detail below.

[0123] Execute the first training batch.

[0124] First, training samples of the first training batch are collected from the initial training sample set, so that the training samples include difficult samples obtained from the initial training sample set based on the sample evaluation model.

[0125] Secondly, the large language model is trained based on the training samples, and the parameters in the large language model are updated. At the same time, the parameters of the initial network in the sample evaluation model are updated, and the category labels of the training samples are marked according to the training loss values ​​of the training samples.

[0126] Then, the sample evaluation model is trained based on the training samples and the corresponding category labels to adjust the parameters of the sample classification network in the sample evaluation model. It should be understood that the parameters in the initial network in the sample evaluation model here have been updated and frozen.

[0127] The above completes the first training batch and enters the second training batch.

[0128] First, training samples of the second training batch are collected from the initial training sample set, so that the training samples include difficult samples obtained from the initial training sample set based on the sample evaluation model.

[0129] Secondly, the large language model trained in the first training batch is trained based on the training samples, and the parameters in the large language model are updated. At the same time, the parameters of the initial network in the sample evaluation model trained in the first training batch are updated, and the training samples are labeled with category labels according to the training loss values ​​of the training samples.

[0130] Then, the sample evaluation model obtained in the previous step is trained based on the training samples and corresponding category labels to adjust the parameters of the sample classification network in the sample evaluation model. It should be understood that the parameters of the initial network in the sample evaluation model here have been updated based on the previous step and frozen.

[0131] The above completes the second training batch and enters the third training batch.

[0132] The above process is repeated until the large language model is trained, resulting in a fully trained large language model. The training process for subsequent training batches can be found in the training process for the second training batch and will not be described in detail here. Furthermore, in each training batch, the methods for obtaining training samples, labeling training samples, and determining the current difficulty threshold can all be found in the details of obtaining the current training batch described above and will not be described in detail here.

[0133] In the technical solutions provided by the embodiments of the present disclosure, on the one hand, during the training of each training batch, training samples for the current training batch are collected from the initial training sample set, so that the training samples include difficult samples obtained from the initial training sample set based on the sample evaluation model. The training samples used for model training are difficult samples containing rich information, thereby properly filtering redundant data during model training, maintaining model training accuracy while reducing computational consumption, accelerating model training, and reducing costs. On the other hand, during the training process of the current training batch, after the large language model is trained based on the training samples, the training samples are labeled with category labels based on the training loss values ​​of the training samples, and the category labels used for training the sample evaluation model are dynamically adjusted. This facilitates the accurate removal of low-value training samples for training the large language model in subsequent training batches through the sample evaluation model, thereby improving the quality of training samples in subsequent training batches. Moreover, by using the same training samples to complete the training of the large language model and the sample evaluation model during the training process, the model training and sample screening processes are integrated, so that the selected samples are more in line with the training requirements of the large language model, thereby improving the model training accuracy. In addition, the model acceleration training method based on sample selection proposed in the exemplary embodiment of the present disclosure is compatible with the current acceleration training method and has universal applicability.

[0134] For further reference, Figure 9 As shown, an exemplary embodiment of the present disclosure provides a task processing method, which may include:

[0135] Step S910: Obtaining task information to be processed;

[0136] Step S920: The task information to be processed is input into the target large language model to obtain a processing result of the task information to be processed; wherein, the target large language model is obtained by training the large language model based on the model training method of any of the above exemplary embodiments.

[0137] Among them, the task information to be processed can be determined according to the specific task scenario. The task scenario includes but is not limited to text generation, intelligent question answering, etc. In the specific task scenario, the large language model is trained using the initial training sample set corresponding to the task scenario to obtain the target large language model that can execute the corresponding task scenario.

[0138] As an example, the descriptive text content used to instruct text generation can be input into the target large language model as the task information to be processed, and the large language model can generate text that conforms to the descriptive text content. Of course, for other similar scenarios, the model training method of the exemplary embodiment of the present disclosure can be adapted to any scenario that requires training a large language model to improve the accuracy and efficiency of model training.

[0139] It is worth noting that the process of training a large language model can be referred to the exemplary embodiment of the training method described above, and will not be described in detail here.

[0140] It should be noted that the above figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present disclosure and are not intended to be limiting. It is readily understood that the processes illustrated in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0141] For further reference, Figure 10 As shown, in an exemplary embodiment of the present disclosure, a model training device 1000 is provided, comprising a sample processing module 1010, a first training module 1020, a second training module 1030, and a third training module 1040. In particular:

[0142] The sample processing module 1010 is used to collect training samples for the current training batch from the initial training sample set, where the training samples include difficult samples obtained from the initial training sample set based on the sample evaluation model; the first training module 1020 is used to train the large language model based on the training samples, and label the training samples with category labels based on the training loss values ​​of the training samples, where the category labels are used to indicate whether the training samples are difficult samples; the second training module 1030 is used to train the sample evaluation model based on the training samples and the corresponding category labels to complete the current training batch; the third training module 1040 is used to control the model training process to enter the next training batch until the large language model completes training and stops, thereby obtaining a trained large language model.

[0143] In an exemplary embodiment, the sample evaluation model shares target network parameters in the large language model; exemplary embodiments of the present disclosure further include;

[0144] The model building module is configured to perform: determining an initial network based on target network parameters of the large language model to be trained; and adding a sample classification network to the output side of the initial network to obtain a sample evaluation model.

[0145] In an exemplary embodiment, the sample processing module 1010 is configured to perform: collecting a first training sample from an initial training sample set according to an initial sample retention rate; for uncollected samples in the initial training sample set, performing a difficulty evaluation on the uncollected samples using a sample evaluation model, and determining the uncollected samples whose evaluation results are difficult samples as second training samples; and determining the training samples of the current training batch based on the first training samples and the second training samples.

[0146] In an exemplary embodiment, the first training module 1020 is configured to perform: obtaining a current difficulty threshold; if the training loss value of the training sample is greater than the current difficulty threshold, determining the category label of the training sample as a difficult sample; otherwise, determining the category label of the training sample as a non-difficult sample.

[0147] In an exemplary embodiment, the first training module 1020 is configured to perform: obtaining a first training loss of a current training batch and a second training loss of a training batch before the current training batch; and determining a current difficulty threshold according to the first training loss and the second training loss.

[0148] In an exemplary embodiment, the first training module 1020 is configured to perform: inputting a training sample into a large language model, determining a prediction loss value based on a prediction result; adjusting parameters of the large language model based on the prediction loss value; wherein the parameters of the initial network in the sample evaluation model are adjusted along with the adjustment of the parameters of the large language model.

[0149] In an exemplary embodiment, the second training module 1030 is configured to perform: inputting the training sample into the sample evaluation model to obtain a sample prediction result;

[0150] A loss function is constructed based on the sample prediction results and category labels, and the parameters of the sample classification network of the sample evaluation model are adjusted according to the loss function.

[0151] Since the details of the various functional modules of the model training device of the exemplary embodiment of the present disclosure have been recorded in the exemplary embodiment of the above-mentioned model training method, they will not be repeated here.

[0152] For further reference, Figure 11 As shown, in an exemplary embodiment of the present disclosure, a task processing device 1100 is provided, comprising an information acquisition module 1110 and a task processing module 1120.

[0153] The information acquisition module 1110 is used to obtain task information to be processed; the task processing module 1120 is used to input the task information to be processed into the target large language model to obtain the processing result of the task information to be processed; wherein, the target large language model is obtained by training a large language model based on the model training method of any of the above exemplary embodiments.

[0154] Since the details of the functional modules of the task processing device of the exemplary embodiment of the present disclosure have been described in the exemplary embodiment of the task processing method described above, they will not be repeated here.

[0155] It should be noted that although several modules or units of the model training device and task processing are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0156] The exemplary embodiments of the present disclosure further provide a computer program product, which includes a computer program, and implements the above method when the computer program is executed by a processor.

[0157] In one embodiment, a computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing the computer program. The computer-readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk drive (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing the computer program, such as a read-only memory, NAND flash memory, and the like.

[0158] In one embodiment, the computer program product may be an intangible product containing a computer program. For example, the computer program product may be implemented as a virtual digital product, such as a digital file such as an executable file or installation package storing the computer program.

[0159] The code of the computer program can be written in one or more programming languages. Programming languages ​​include C, Java, C++, etc. The program code can be executed entirely on the user computing device, partially on the user computing device, or as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device via any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., via an Internet connection provided by a carrier).

[0160] Computer programs can be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. Electronic devices can convert signals carrying computer programs into digital signals to run the computer programs. When the computer program is run on an electronic device, its code is used to cause the electronic device to execute (more specifically, to cause the processor of the electronic device to execute) the method steps of various exemplary embodiments of the present disclosure, such as the method described above.

[0161] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. Those skilled in the art will appreciate that various aspects of the present disclosure can be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to herein as a "circuit," "module," or "system."

[0162] Refer to the following Figure 12 12 is a diagram to describe an electronic device 1200 according to such an embodiment of the present disclosure. Figure 12 The electronic device 1200 shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure. Figure 12 As shown, electronic device 1200 is implemented as a general-purpose computing device. Components of electronic device 1200 may include, but are not limited to, the aforementioned at least one processing unit 1210, the aforementioned at least one storage unit 1220, a bus 1230 connecting various system components (including storage unit 1220 and processing unit 1210), and a display unit 1240.

[0163] The storage unit stores program codes, which can be executed by the processing unit 1210, so that the processing unit 1210 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0164] The storage unit 1220 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 1221 and / or a cache memory unit 1222, and may further include a read-only memory unit (ROM) 1223. The storage unit 1220 may also include a program / utility 1224 having a set (at least one) of program modules 1225. Such program modules 1225 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each of these examples or some combination thereof may include the implementation of a network environment.

[0165] The bus 1230 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0166] The electronic device 1200 can also communicate with one or more external devices 1300 (e.g., a keyboard, a pointing device, a Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 1200, and / or any device that enables the electronic device 1200 to communicate with one or more other computing devices (e.g., a router, a modem, etc.). Such communication can occur via an input / output (I / O) interface 1250. Furthermore, the electronic device 1200 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 1260. As shown, the network adapter 1260 communicates with other modules of the electronic device 1200 via a bus 1230. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 1200, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0167] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0168] Furthermore, the figures above are merely illustrative of the processes included in the methods according to exemplary embodiments of the present disclosure and are not intended to be limiting. It is readily understood that the processes illustrated in the figures above do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0169] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow from the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

Claims

1. A model training method, applied to text generation, characterized in that: include: Collecting training samples for a current training batch from an initial training sample set for text generation, the training samples including difficult samples obtained from the initial training sample set for text generation based on a sample evaluation model; wherein the sample evaluation model is a model for evaluating the difficulty of training samples, where the difficulty of the samples refers to the difficulty of learning the text samples; Training a large language model based on the training samples, and labeling the training samples with category labels according to the training loss values ​​of the training samples, wherein the category labels are used to indicate whether the training samples are difficult samples; Training the sample evaluation model according to the training samples and corresponding category labels to complete the current training batch; Enter the next training batch until the large language model is trained and stops, thereby obtaining a trained large language model; The training samples of the current training batch are collected from the initial training sample set for text generation, including: Collecting a first training sample from the initial training sample set for text generation according to the initial sample retention rate; For the uncollected samples in the initial training sample set for text generation, using the sample evaluation model to perform difficulty evaluation on the uncollected samples, and determining the uncollected samples whose evaluation results are difficult samples as second training samples; Determine the training samples of the current training batch according to the first training samples and the second training samples; The training of the large language model based on the training samples and labeling the training samples with category labels according to the training loss values ​​of the training samples include: Obtaining a current difficulty threshold, which is a loss threshold used to distinguish the categories of training samples, and which is not fixed throughout the training process but is determined based on the loss during the training process of the large language model; If the training loss value of the training sample is greater than the current difficulty threshold, determining that the category label of the training sample is a difficult sample; Otherwise, the category label of the training sample is determined to be a non-difficult sample.

2. The method according to claim 1, characterized in that The sample evaluation model shares the target network parameters in the large language model; before training the large language model, the method further includes: Determine an initial network based on target network parameters of the large language model to be trained; A sample classification network is added to the output side of the initial network to obtain the sample evaluation model.

3. The method according to claim 1, characterized in that The obtaining of the current difficulty threshold includes: Get the first training loss of the current training batch and the second training loss of the training batch before the current training batch; The current difficulty threshold is determined according to the first training loss and the second training loss.

4. The method according to claim 2, characterized in that The training of the large language model based on the training samples includes: Inputting the training sample into the large language model, and determining a prediction loss value according to the obtained prediction result; Adjusting parameters of the large language model according to the prediction loss value; The parameters of the initial network in the sample evaluation model are adjusted along with the adjustment of the parameters of the large language model.

5. The method according to claim 4, characterized in that The training of the sample evaluation model according to the training samples and the corresponding category labels includes: Inputting the training sample into the sample evaluation model to obtain a sample prediction result; A loss function is constructed according to the sample prediction result and the category label, and parameters of the sample classification network of the sample evaluation model are adjusted according to the loss function.

6. A text generation task processing method, characterized in that: include: Get the pending text generation task information; The text generation task information to be processed includes a description text content for instructing text generation; Inputting the to-be-processed text generation task information into a target large language model to obtain a processing result of the to-be-processed text generation task information; The target large language model is obtained by training a large language model based on the method according to any one of claims 1 to 5.

7. A model training device, applied to text generation, characterized in that: include: a sample processing module, configured to collect training samples of a current training batch from an initial training sample set for text generation, wherein the training samples include difficult samples obtained from the initial training sample set for text generation based on a sample evaluation model; A first training module is configured to train a large language model based on the training samples and label the training samples with category labels according to the training loss values ​​of the training samples, wherein the category labels are used to indicate whether the training samples are difficult samples; wherein the sample evaluation model is a model for evaluating the difficulty of the training samples, wherein the difficulty of the samples refers to the difficulty of learning the text samples; A second training module is used to train the sample evaluation model according to the training samples and corresponding category labels to complete the current training batch; The third training module is used to control the model training process to enter the next training batch until the large language model is trained and stops, thereby obtaining a trained large language model; The sample processing module is configured to execute: Collecting a first training sample from the initial training sample set for text generation according to the initial sample retention rate; For the uncollected samples in the initial training sample set for text generation, using the sample evaluation model to perform difficulty evaluation on the uncollected samples, and determining the uncollected samples whose evaluation results are difficult samples as second training samples; Determine the training samples of the current training batch according to the first training samples and the second training samples; The first training module is configured to perform: Obtaining a current difficulty threshold, which is a loss threshold for distinguishing the categories of training samples, which is not fixed throughout the training process and is determined based on the loss during the training process of the large language model; If the training loss value of the training sample is greater than the current difficulty threshold, determining that the category label of the training sample is a difficult sample; Otherwise, the category label of the training sample is determined to be a non-difficult sample.

8. A text generation task processing device, characterized in that: include: An information acquisition module is used to obtain information about the text generation task to be processed; The text generation task information to be processed includes a description text content for instructing text generation; a task processing module, configured to input the to-be-processed text generation task information into a target large language model and obtain a processing result of the to-be-processed text generation task information; The target large language model is obtained by training a large language model based on the method according to any one of claims 1 to 5.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 6 by executing the executable instructions.

Images