Model training method and device

Abstract

A model training method and device relate to the technical field of artificial intelligence. The method comprises the steps that in the process of fine tuning of a language model, training samples are determined from one of a plurality of data subsets in each iteration, each data subset is a set of samples with the same kind of generality, and the samples of different data subsets are different in generality; and inputting the training sample into the language model to train the language model. In this way, the samples used in each iteration are common samples. And through small deviation between common samples, the understanding ability of the model to the samples is enhanced, and the adaptability of the model to various tasks is further improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model training method and device. Background Technology

[0002] Instruction fine-tuning is a training method for large-scale pre-trained language models. Its main purpose is to improve the model's ability to understand and execute specific instructions. It can be applied to multiple fields such as natural language processing, computer vision, and multimodal tasks, including image caption generation, visual question answering, and text generation. Typically, instruction fine-tuning is based on a dataset covering various instructions and is performed using a randomized training strategy. This enables a model to complete multiple downstream tasks, aiming to obtain a general-purpose model.

[0003] However, this instruction fine-tuning strategy is prone to problems such as insufficient training and overfitting, resulting in poor final model performance. Summary of the Invention

[0004] This application provides a model training method, apparatus, computing device, device cluster, computer storage medium, and computer program product that can improve model performance.

[0005] In a first aspect, embodiments of this application provide a model training method, the method comprising: in the process of fine-tuning a language model, determining training samples from one of multiple data subsets in each iteration, each data subset being a set of samples with the same common characteristics, and the common characteristics of samples in different data subsets being different; and inputting the training samples into the language model to train the language model.

[0006] In this embodiment, during the fine-tuning process with the pre-trained language model, the dataset can first be categorized according to commonalities. Samples of the same commonality form a subset; for example, samples used for translation tasks form one subset, samples used for sentiment analysis tasks form another subset, and so on. In each iteration, a batch of commonalities (i.e., training samples) from one subset is selected and input into the language model for fine-tuning. Because of the similarity among these batches of training samples sharing the same commonalities, the bias is very small, helping the language model learn common features and patterns in the samples. This eliminates the need to expend more computational power and time to distinguish subtle differences between different samples. This not only allows the language model to focus more quickly on learning the core features of the samples but also reduces the difficulty of generalization, making it easier for the language model to extend the learned patterns to new similar samples and utilize the learned core features to exhibit stronger robustness when encountering similar situations. Similarly, in the next iteration, a batch of commonalities is randomly selected from all data subsets for iterative training, and so on, until the language model converges, completing the fine-tuning.

[0007] In other words, in this embodiment, samples within the same subset share commonalities, while samples from different subsets have unique characteristics. Therefore, during the fine-tuning process, common samples are used in each iteration to enhance the learning ability of the AI ​​model, but training samples from different batches also have unique characteristics. Thus, macroscopically, it is still fine-tuning a large-scale dataset containing multiple characteristics, ensuring the language model's adaptability to complex and diverse data, improving its generalization ability, avoiding overfitting, and ultimately improving the overall performance of the language model.

[0008] In some examples, fine-tuning involves multiple training rounds, each round including multiple iterations. Specifically, the method includes: during the fine-tuning of the language model, determining training samples from one of multiple data subsets at each iteration of the current training round; inputting the training samples into the language model to train it; obtaining the first loss data of the language model after completing the current training round; and determining the sampling probability of each data subset in the next training round based on the first loss data, such that in each iteration of the next training round, one data subset is selected from the multiple data subsets according to the sampling probability to determine the corresponding training data again.

[0009] In this way, during multiple training rounds, the sampling probabilities of different subsets can be dynamically adjusted to dynamically adjust the training based on the relevance and difficulty of the tasks. For example, if the model demonstrates good processing ability for a task in the current round, the sampling of samples for this task can be reduced in the next round, which will help to adjust the language model more efficiently and ensure the language model's ability to process each single task in a multi-task scenario.

[0010] In some examples, the first loss data includes the loss value obtained from each iteration of the language model during the current training round. During each iteration of the current training round, training samples are determined from one of multiple data subsets, including: selecting a data subset to determine training samples according to a first sampling probability configured for each data subset during each iteration of the current training round; the first sampling probability represents the probability that a given data subset is used to determine training samples during the current training round; and determining the sampling probability of each data subset for the next training round based on the first loss data, including: comparing each loss value in the first loss data with a preset threshold; if any loss value does not reach the threshold, decreasing the first sampling probability corresponding to the data subset used to obtain that loss value; if any loss value reaches the threshold, increasing the first sampling probability corresponding to the data subset used to obtain that loss value, or keeping the first sampling probability unchanged.

[0011] In this way, the learning effect of the language model on a batch of common samples can be determined based on the loss value obtained in each iteration. This allows for dynamic adjustment of the sampling probability of the subset to which the common samples belong during subsequent fine-tuning, which helps to ensure the fine-tuning effect while improving the efficiency of fine-tuning.

[0012] For example, in each iteration, the training data is randomly selected from multiple data subsets. This balances the similarity of the samples with the randomness of the training, accelerating convergence while ensuring generalization ability and avoiding overfitting.

[0013] For example, in the process of fine-tuning a language model, before determining the training samples from one of multiple data subsets in each iteration, the method includes: obtaining a dataset containing several samples; classifying the samples according to commonalities to obtain multiple data subsets.

[0014] In other words, before fine-tuning the language model, this embodiment can first process the samples in the dataset according to commonalities, such as classifying them according to task categories, sample statistics, sources, etc., to obtain subsets that meet the requirements and adapt to different scenarios.

[0015] For example, a sample is instruction data used to guide a language model to perform at least one type of task; several samples are classified according to commonalities to obtain multiple data subsets, including: determining the task type of each sample; and dividing samples with the same task type into the same data subset to obtain multiple data subsets.

[0016] In this way, if the fine-tuned language model is used in a vertical domain scenario, such as deploying the language model as a question-answering system in the education industry or as a machine translation system, then by dividing the large-scale data into common task types, it is possible to sample from relevant data subsets in a targeted manner according to the characteristics of the domain or industry. For example, the sampling probability of these data subsets can be set to be higher to obtain relevant training samples, which will focus on improving the language model's task processing capability in the vertical domain scenario. The model training is more flexible and the training effect is greatly enhanced.

[0017] For example, a sample is instruction data used to guide a language model to perform at least one type of task; classifying several samples according to commonalities to obtain multiple data subsets includes: generating a sample sequence for each sample; determining the length of each sample sequence; and dividing samples whose sample sequence lengths are distributed in the same length range into the same data subset to obtain multiple data subsets.

[0018] In this way, if the fine-tuned language model is used for a general task scenario and the statistical information of the sample data varies greatly, such as the dataset containing several samples of different lengths, all of which are translation tasks, then these samples can be divided into common sequence lengths. In this way, when fine-tuning the language model, the input size of the language model is basically similar in each iteration. This can better utilize the parallel computing capabilities of the hardware that performs the language model training task, reduce the construction of the computation graph, and also help reduce gradient fluctuations, improve stability, and thus improve the efficiency of language model fine-tuning.

[0019] For example, a sample is instruction data used to guide a language model to perform at least one type of task; classifying several samples according to commonalities to obtain multiple data subsets includes: converting the source information of each sample into a feature vector, where the source information is used to describe the task corresponding to the sample; performing clustering calculations on each feature vector to divide samples with similar sources into the same data subset, thus obtaining multiple data subsets.

[0020] In this way, if the task types, sequence lengths, etc. of the samples in the dataset are similar during the fine-tuning of the language model, they can be classified by commonalities of origin, that is, the characteristics of the samples can be obtained from the perspective of language representation. This allows samples with similar characteristics (i.e., commonalities) to be clustered into one class, thereby enhancing the learning ability of the language model.

[0021] Secondly, embodiments of this application provide a model training apparatus, comprising: a determining module, used to determine training samples from one of multiple data subsets in each iteration during the fine-tuning of a language model, wherein each data subset is a set of samples with similar commonalities, and the commonalities of samples in different data subsets are different; and a processing module, used to input the training samples into the language model to train the language model.

[0022] For example, fine-tuning includes multiple training rounds, each round of training including multiple iterations; the determination module is specifically used to determine training samples from one of multiple data subsets during each iteration of the current training round during the fine-tuning of the language model; the processing module is specifically used to input the training samples into the language model to train the language model; the processing module is also used to obtain the first loss data of the language model after the current training round of the language model is completed; the determination module is also used to determine the sampling probability of each data subset in the next training round based on the first loss data, so that in each iteration of the next training round, one data subset is selected from multiple data subsets according to the sampling probability to determine the corresponding training data again.

[0023] For example, the first loss data includes the loss value obtained from each iteration of the language model during the current training round. The determining module is specifically used to select a data subset to determine training samples from each data subset according to the first sampling probability configured for each data subset during each iteration of the current training round. The first sampling probability represents the probability that the data subset is used to determine training samples during the current training round. The determining module is further used to: compare each loss value in the first loss data with a preset threshold; if any loss value does not reach the threshold, reduce the first sampling probability corresponding to the data subset used to obtain the loss value; if any loss value reaches the threshold, increase the first sampling probability corresponding to the data subset used to obtain the loss value, or keep the first sampling probability unchanged.

[0024] For example, in each iteration, the training data is randomly selected from multiple subsets of data.

[0025] For example, the processing module is also used to: obtain a dataset, which includes several samples; classify the samples according to commonalities to obtain multiple data subsets.

[0026] For example, a sample is instruction data used to guide a language model to perform at least one type of task; the processing module is specifically used to: determine the task type of each sample; and divide samples with the same task type into the same data subset to obtain multiple data subsets.

[0027] For example, a sample is instruction data used to guide a language model to perform at least one type of task; the processing module is specifically used to: generate a sample sequence for each sample; determine the length of each sample sequence; and divide samples whose sample sequence lengths are distributed in the same length range into the same data subset to obtain multiple data subsets.

[0028] For example, a sample is instruction data used to guide a language model to perform at least one type of task; the processing module is specifically used to: convert the source information of each sample into a feature vector, the source information being used to describe the task corresponding to the sample; perform clustering calculations on each feature vector, dividing samples with similar sources into the same data subset, resulting in multiple data subsets.

[0029] Thirdly, embodiments of this application provide a computing device, including a processor and a memory; the processor is configured to execute instructions stored in the memory to cause the processor to perform the method described in the first aspect or any possible implementation of the first aspect.

[0030] Fourthly, embodiments of this application provide a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

[0031] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0032] In a sixth aspect, embodiments of this application provide a computer program product, characterized in that, when the computer program product is run on a processor, it causes the processor to execute the method described in the first aspect or any possible implementation of the first aspect.

[0033] In a seventh aspect, embodiments of this application provide a chip, characterized in that it includes at least one processor and an interface; at least one processor obtains program instructions or data through the interface; at least one processor is used to execute program line instructions to implement the method described in the first aspect or any possible implementation of the first aspect.

[0034] It is understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of an artificial intelligence main framework provided in an embodiment of this application;

[0036] Figure 2A This is a schematic diagram of the architecture of a distributed training system provided in an embodiment of this application;

[0037] Figure 2B This is a schematic diagram of the architecture of another distributed training system provided in an embodiment of this application;

[0038] Figure 3 This is a schematic flowchart of a model training method provided in an embodiment of this application;

[0039] Figure 4 This is a schematic flowchart of a model training method provided in an embodiment of this application;

[0040] Figure 5 This is a schematic flowchart of a model training method provided in an embodiment of this application;

[0041] Figure 6This is a schematic flowchart of a model training method provided in an embodiment of this application;

[0042] Figure 7 This is a schematic flowchart of a model training method provided in an embodiment of this application;

[0043] Figure 8 This is a schematic flowchart of a model training method provided in an embodiment of this application;

[0044] Figure 9 This is a schematic diagram illustrating the commonalities of samples in a dataset as provided in an embodiment of this application;

[0045] Figure 10 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application;

[0046] Figure 11 This is a schematic diagram of the structure of a chip provided in an embodiment of this application; Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0048] To facilitate understanding of the technical solutions of this application, the technical terms used in this document are explained below.

[0049] Artificial intelligence (AI) refers to the ability of machine or software systems to learn, reason, solve problems, and perform tasks that typically require human intelligence, such as visual recognition, language understanding, decision-making, and translation.

[0050] Chain-of-Thought (CoT) is a method in artificial intelligence for solving complex problems. It breaks down a problem into a series of subproblems and solves these subproblems step by step to arrive at the answer to the problem.

[0051] Natural Language Processing (NLP) is the process of processing and understanding text data at multiple levels, combined with machine learning techniques, to enable computers to understand, interpret, and generate human language content, so as to perform tasks such as translation, text analysis, sentiment analysis, information extraction, and question answering systems.

[0052] Massive-Multitask-Language-Understanding (MMLU) is a research approach in Natural Language Processing (NLP) that aims to train general-purpose pre-trained models capable of handling multiple language tasks. The core idea behind this approach is to train the model on a large number of different tasks, enabling it to learn broader language knowledge and patterns, thereby improving the model's performance across various tasks.

[0053] Large Language Models (LLMs) are advanced machine learning models. They are typically based on the Transformer architecture, a neural network structure consisting of an encoder and decoder, capable of processing sequential data and discovering long-range dependencies. These models are trained on massive amounts of text data, often possessing hundreds of millions or even trillions of parameters. They are usually pre-trained on large datasets and then fine-tuned for specific tasks to learn to understand and generate natural language, handling various natural language processing tasks such as text generation, text classification, commonsense tasks, rewriting tasks, question answering systems, machine translation, and sentiment analysis.

[0054] Pre-training is a crucial concept in deep learning, particularly in Natural Language Processing (NLP). It refers to training a model with a large amount of unlabeled (or weakly labeled) data before a specific task, enabling the model to learn general properties of language, such as grammar, vocabulary, and contextual relationships. Pre-trained models can then be fine-tuned on subsequent tasks, thereby improving their performance and generalization ability.

[0055] Instruction Turning (IT) is a training method for large-scale pre-trained language models. In this stage, supervised learning is performed using labeled data with correct answers (including specific instructions and expected outputs (i.e., correct answers)) to adjust the model's parameters so that it can better understand and execute instructions, that is, learn how to generate correct outputs based on input instructions.

[0056] Large language models can handle various natural language processing tasks. To improve the adaptability of large language models to different tasks and enhance their overall performance, this application provides a data processing method. This method can be implemented using artificial intelligence (AI) technology. It primarily involves dividing samples in the dataset into multiple subsets based on commonalities. During fine-tuning, the training samples input to the model in each iteration are from the same subset; that is, the samples used in each iteration share common characteristics. This small deviation between these common samples helps strengthen the model's understanding of the samples, thereby improving its adaptability to various tasks.

[0057] The following section first describes the overall workflow of the artificial intelligence system. Please refer to [link / reference]. Figure 1 , Figure 1 A schematic diagram of an artificial intelligence framework is shown, which describes the overall workflow of an artificial intelligence system and is applicable to general artificial intelligence domain needs.

[0058] The above-mentioned artificial intelligence framework will be elaborated from two dimensions: "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0059] The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it could be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom."

[0060] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence, information (provided and processed by technology) to the industrial ecosystem of systems.

[0061] (1) Infrastructure:

[0062] Infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. This communication occurs through sensors; computing power is provided by intelligent chips (hardware acceleration chips such as CPUs, NPUs, GPUs, ASICs, and FPGAs); and the basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0063] (2) Data

[0064] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0065] (3) Data processing

[0066] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0067] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0068] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0069] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0070] (4) General ability

[0071] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0072] (5) Smart Products and Industry Applications

[0073] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They encapsulate overall artificial intelligence solutions, productize intelligent information decision-making, and realize practical applications. Their application areas mainly include: intelligent manufacturing, intelligent transportation, smart home, intelligent healthcare, intelligent security, autonomous driving, safe city, and intelligent terminals.

[0074] The following is a schematic diagram of the architecture of a distributed training system provided in an embodiment of this application.

[0075] Figure 2AThis is a schematic diagram of the architecture of a distributed training system provided in an embodiment of this application. The distributed training system architecture is a model-parallel strategy (MP) architecture, which may include multiple worker servers 100. In the MP architecture, the model to be trained is divided into several parts (or sub-models), and each part (or sub-model) can be processed independently on different worker servers 100. For example, model M is divided into two sub-models, m1 and m2. Worker server 100A processes m1 independently, and worker server 100 processes m2 independently. Each worker server 100 obtains the model parameters (mainly including weights w) of the part of the model it processes and independently calculates the gradient of that part of the model accordingly.

[0076] During model training, each worker server 100 acquires its own training samples shard0, shard1, ..., which are from the same batch of instruction data and come from the same dataset. Next, each worker server 100 uses its own training samples shard0, shard1, ... as input, performs forward propagation on its input based on its own model parameters to obtain a loss value, and then performs backpropagation based on the loss value to obtain the gradient corresponding to its respective weight w. Then, the worker servers 100 communicate with each other over the network to transmit their gradients to other nodes. In this way, each worker server 100 can calculate the gradients obtained from each other, update its assigned part of the model, and update the model parameters of that part of the model. Then, each worker server 100 acquires a new batch of training samples and repeats the above process to recalculate the corresponding gradients until the model converges, resulting in a trained model.

[0077] Figure 2B This is a schematic diagram of another distributed training system architecture provided in an embodiment of this application. The distributed training system architecture is a parameter server (PS) architecture, which includes a parameter server 200 and multiple worker servers 100. Worker servers 100 can communicate with parameter server 200. In the PS architecture, each worker server 100 obtains a complete model with the same model parameters (mainly including weights w). Before model training, the initialized model parameters can be loaded into parameter server 200.

[0078] During model training, parameter server 200 sends the initialized model parameters to each worker server 100. Each worker server 100 then acquires its own training samples (shard0, shard1, ...). Next, each worker server 100, based on its own model parameters, performs forward propagation on its input training samples (shard0, shard1, ...) to obtain a loss value, and then performs backpropagation based on the loss value to obtain the gradient corresponding to its respective weights (w). These gradients are then sent to parameter server 200. Next, parameter server 200 updates the model parameters based on these gradients and redistributes them to each worker server 100. Then, each worker server 100 acquires a new batch of training samples and repeats the above process to recalculate the corresponding gradients, ..., until the model converges, resulting in a trained model.

[0079] The aforementioned working server 100 and parameter server 200 can be a single computing device or a processor in a computing device, such as a graphics processing unit (GPU), neural network processing unit (NPU), tensor processing unit (TPU), data processing unit (DPU), etc. The embodiments of this application do not specifically limit them.

[0080] The above Figure 2A , 2B The process of training a model in the distributed training system shown can be a process of fine-tuning a pre-trained language model with instructions. The training samples obtained by each working server 100 are instruction data. This instruction data is used to guide the language model to perform at least one type of task, such as translation task, common sense task, sentiment analysis task, question answering task, rewriting task, etc., but is not limited to these.

[0081] Unlike small models used for vision tasks, the language models described above, when large, may only be trained in two training epochs. This means that the data order or training scheme can significantly impact the model's final performance. For example, if the language model is a very large model (65B), better performance may be achieved by drawing only a few percent of the samples from these datasets. Conversely, a smaller model (7B) fine-tuned with a small amount of data (less than 100K) can exhibit strong capabilities.

[0082] However, regardless of the size of the language model, the dataset used must cover a variety of data instructions (i.e., samples) to enhance the generalization ability of the language model to follow instructions. Due to the limited number of training rounds, if the inherent relationship and learning order of the samples are not considered, it is easy to cause problems such as insufficient training and overfitting of the language model, which reduces the adaptability of the language model to various tasks.

[0083] To improve the adaptability of large language models to handle different tasks and enhance the overall performance of the models, this application provides a data processing method. This method aims to fine-tune the language model through common samples of different tasks, thereby potentially building a more effective model to adapt to complex instructions, and ultimately achieving better multi-task generalization results for model training with the existing amount of data.

[0084] Next, based on the content described above, a model training method provided by an embodiment of this application will be introduced. It is understood that this method is proposed based on the content described above, and some or all of the content of this method can be found in the description above.

[0085] Please see Figure 3 , Figure 3 This is a schematic flowchart illustrating a model training method provided in an embodiment of this application. It is understood that this method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities. Figure 3 As shown, the method may include:

[0086] S310, in the process of fine-tuning the language model, each iteration determines the training samples from one of multiple data subsets. Each data subset is a set of samples with the same common characteristics, and the common characteristics of samples in different data subsets are different.

[0087] In this embodiment, the language model can first undergo pre-training, for example, training on a large amount of unlabeled text to learn to predict the next character or word. This pre-training stage allows the language model to learn rich linguistic rules and knowledge, but it does not optimize for specific tasks. After pre-training, the language model can be fine-tuned using instruction data (i.e., samples) to guide it in performing specific tasks, thereby improving its processing performance across multiple tasks.

[0088] For example, multiple data subsets can be pre-constructed, and the resulting dataset is the collection of data from these subsets. Each data subset is a collection of samples with similar common characteristics. For instance, the 2000 samples in data subset 00 share common characteristic 1, the 10000 samples in data subset 01 share common characteristic 2, and so on. As specific examples, the common characteristics of the samples could be the same task type, similar statistical information (such as the sequence length of the samples), or the same source of the samples, etc., but are not limited to these.

[0089] In this example, fine-tuning the language model requires training it through multiple epochs. Each epoch involves multiple iterations of the language model. Furthermore, in each iteration, a subset of samples is randomly sampled from one of these subsets as training samples (i.e., a batch of samples). For example, in the first iteration of the first epoch, samples are randomly sampled from subset 00; in the second iteration of the first epoch, samples are randomly sampled from subset 05, and so on.

[0090] In this way, the training samples sampled in each iteration are all samples with a certain commonality.

[0091] For example, during each training round, a subset of data can be randomly selected from multiple subsets for sampling in each iteration, or the subsets can be sorted and selected sequentially for sampling in each iteration.

[0092] S320: Input the training samples into the language model to train the language model.

[0093] In this embodiment, a batch of training samples sampled from a subset of data is input into the language model for training. It should be understood that a single training iteration generally includes the language model performing one forward and one backward propagation on the input batch of training samples. In addition, during this training iteration, the language model calculates the loss based on the current model parameters (mainly including weights) and updates the model parameters based on the loss value.

[0094] This process is repeated multiple times until the entire dataset (i.e., all subsets of data) is trained, which constitutes one training round. Multiple training rounds continue until the language model converges, completing the fine-tuning of the language model.

[0095] In this embodiment, when fine-tuning the language model, the training samples input to the language model in each iteration are instruction data with common characteristics. For example, each batch of training data corresponds to a specific task (such as translation, rewriting, question answering, etc.). Since the common instruction data often have a fixed bias and the bias is small, the language model can obtain a lower loss value based on the small bias instruction data of a single batch. This makes the gradient fluctuation within the language model smaller and more balanced in each iteration, which helps to enhance the model's learning ability on a single batch of training samples. In the multi-task learning process of the language model, it improves its adaptability to a single task, and ultimately improves the overall performance of the language model.

[0096] This embodiment draws inspiration from the human learning process of understanding through analogy, utilizing similarities across different contexts to enhance comprehension. The training method in this embodiment can train the model on various tasks and encourages the development of more similar samples, thereby continuously improving the model's factual accuracy, reasoning ability, multilingualism, and coding capabilities.

[0097] The following is a detailed description of a model training method provided by an embodiment of this application, with reference to the accompanying drawings. For example, Figure 4 The diagram shown is a flowchart of a model training method provided in an embodiment of this application. Figure 4 yes Figure 3 A further detailed description of the process shown. For example... Figure 4 As shown, the method may specifically include:

[0098] S410, in the process of fine-tuning the language model, determines the training samples from one of multiple data subsets during each iteration of the current training round.

[0099] In this step, training samples can be determined from one of multiple data subsets at each iteration during the current round of training of the language model.

[0100] As a concrete example, during each iteration of the current training round of the language model, a subset of data can be selected to determine the training samples according to a first sampling probability configured for each subset of data. This first sampling probability represents the probability that the data subset is used to determine the training samples during the current training round.

[0101] Detailed explanation: In the current training round, there are data subsets 00, 01, 02, and 03, and the sampling probability (i.e., the first sampling probability) of these three data subsets can be set to be equal, i.e., 1 / 4 for each. Therefore, in each iteration, each data subset 00, 01, 02, and 03 has a 1 / 4 probability of being selected, and then a portion of samples is randomly sampled from one of the selected data subsets as training samples.

[0102] Of course, in other examples, the first sampling probabilities of data subsets may not be equal during a single training round.

[0103] S420: Input the training samples into the language model to train the language model.

[0104] This step is similar to the implementation principle of S320 in the above embodiment, and will not be repeated here.

[0105] S430, after completing the current round of training on the language model, obtain the first loss data of the language model;

[0106] In this embodiment, the language model is iterated multiple times according to steps S410 to S420 until the current training round is completed, at which point the first loss data of the language model can be obtained. The first loss data includes the loss value obtained from each iteration of the language model during the current training round.

[0107] S440, based on the first loss data, determine the sampling probability of each data subset in the next round of training, so that in each iteration of the next round of training, select one from multiple data subsets according to the sampling probability to determine the corresponding training data again.

[0108] In this step, the loss data after the current round (i.e. the first loss data) can be used as a reference for adjusting the sampling probability of each data subset in the next round of training, so that in each iteration of the next round of training, one data subset is selected to sample training samples according to the adjusted sampling probability.

[0109] Specifically, step S440 may include:

[0110] S441, compare each loss value in the first loss data with a preset threshold.

[0111] In this example, a threshold can be preset, and each loss value obtained in the current round is compared with the threshold. Since the training samples used to obtain each loss value have commonalities and come from the same data subset, if the loss value reaches the threshold, it indicates that the language model's training effect on the data subset corresponding to that loss value has not met expectations. Conversely, if the loss value does not reach the threshold, it can be considered that the language model's training effect on the data subset corresponding to that loss value has met expectations. Therefore, based on the comparison results, the following steps S442 or S443 can be executed:

[0112] S442, If any loss value does not reach the threshold, reduce the first sampling probability corresponding to the subset of data used to obtain that loss value;

[0113] S443, if any loss value reaches the threshold, then increase the first sampling probability corresponding to the subset of data used to obtain the loss value, or keep the first sampling probability unchanged.

[0114] In this example, if any loss value fails to reach the corresponding threshold, the first sampling probability of the data subset used to obtain that loss value can be reduced. For example, after the first round of training, if the loss value of the training samples from data subset 00 is small and fails to reach the corresponding threshold, the data subset 00 can be adjusted from the first sampling probability to the second sampling probability. The first sampling probability is greater than the second sampling probability. In this way, in the next round of training, data subset 00 will be selected according to the second sampling probability, reducing the possibility of sampling from data subset 00.

[0115] S443, if any loss value does not reach the threshold, then increase the first sampling probability corresponding to the subset of data used to obtain the loss value, or keep the first sampling probability unchanged.

[0116] Similarly, in this example, if any loss value reaches the corresponding threshold, the first sampling probability corresponding to the data subset used to obtain that loss value can be increased. For example, after the first round of training, if the loss value of the training samples from data subset 00 is large and exceeds the corresponding threshold, the data subset 00 can be adjusted from the first sampling probability to the third sampling probability. The first sampling probability is less than the third sampling probability. In this way, in the next round of training, data subset 00 will be selected according to the third sampling probability, increasing the possibility of sampling from data subset 00, strengthening the language model's learning effect on the corresponding common samples in data subset 00, and improving the performance of the language model.

[0117] Thus, in this embodiment, by dynamically adjusting the sampling probability of common samples during multiple training rounds, the language model can better adapt to the constantly changing data flow and learning needs in online learning scenarios. This helps the model update and adapt to new information in real time. Furthermore, in multi-task learning frameworks, such as when the language model needs to handle question answering, machine translation, sentiment analysis, etc., the sampling probability can be dynamically adjusted based on the relevance and difficulty of these tasks (which can be determined by the loss value obtained from the corresponding training samples) to optimize cross-task knowledge sharing and transfer learning effects.

[0118] In this embodiment, since the samples used for different tasks in the dataset affect the final performance of the model, especially in multi-task generalization, this embodiment can improve the model's multi-task processing performance by performing common classification of data subsets on a given dataset and iterative training based on single-batch common samples.

[0119] For example, Figure 5 A schematic flowchart of a model training method provided in an embodiment of this application is shown. Figure 5 As shown, this method is similar to the one described above. Figure 3 The main difference in the method shown is that, before performing S310, i.e., before determining the training samples from one of multiple data subsets in each iteration during the fine-tuning of the language model described above, the method may include S510 to S520:

[0120] S510, Obtain the dataset, which includes several samples.

[0121] In this example, a dataset can be prepared in advance and stored for direct use. Alternatively, a dataset can be constructed by collecting samples from the internet, databases, or other sources. In this example, the dataset includes several samples, each of which can be instruction data used to guide the language model in performing at least one type of task. The types of tasks can include, but are not limited to, question answering, translation, sentiment analysis, commonsense, rewriting, text generation, and so on.

[0122] For example, instruction data a1 used for a translation task may include:

[0123] Instruction: Please translate the Chinese sentence "The weather is very nice today" into English;

[0124] Expected output: The weather is very nice today.

[0125] For example, instruction data a2 used for a question-answering task may include:

[0126] Instruction: When was the Big Bang theory proposed?

[0127] Expected output: The Big Bang theory was proposed in the 20th century.

[0128] Thus, the dataset includes several samples for multiple task types. These samples are used to fine-tune the pre-trained language model, thereby training its performance on these task types. It is understandable that pre-trained language models typically possess general language learning properties, such as the ability to understand grammar, vocabulary, and contextual relationships.

[0129] like Figure 5 As shown, in the dataset obtained in step S510, samples from these different tasks are usually randomly mixed together, such as... Figure 5 Triangles, circles, squares, etc., represent samples with commonalities across different categories. These commonalities could be shared by the tasks corresponding to the samples, shared by the statistical information of the samples, shared by the coding features of the sample sources, and so on.

[0130] S520 classifies several samples according to their commonalities, resulting in multiple data subsets.

[0131] In this example, all samples in the dataset are classified according to their commonalities. Samples with commonalities are grouped into a subset of data, so that the samples in each subset have a commonality. For example, the samples in subset A are all used for translation tasks, the samples in subset B are all used for solving math problems, and so on, but not limited to this.

[0132] You can continue to refer to this. Figure 5 As shown in S520, samples with similar common characteristics are grouped into a subset. For example, a subset is represented by a square, a subset is represented by a triangle, and so on. Optionally, the common samples within the same data subset can be further divided into multiple partitions 1, 2, ..., each of which can include multiple samples. For example, if data subset A contains 10,000 common samples for translation tasks, these 10,000 common samples are further divided into multiple partitions based on the similarity of their respective domains, topics, or formats (or tags). Each partition includes 1,000 to 2,000 common samples with similar attributes. For example, partition 1 includes 1,000 samples for Chinese-English translation, partition 2 includes 1,500 samples for Chinese-Japanese translation, and so on.

[0133] Next, steps S310 to S320 above can be performed. That is, during the fine-tuning of the language model, in each iteration, a subset of these data is randomly selected, and then a portion of samples is extracted from this selected subset as a batch of training samples. For example... Figure 5 As shown in S310, samples from a random subset (represented by circles) are collected as a batch t of training samples. A batch of training samples is input into a language model (LLM) for one iteration of training. Due to the similarity among these batch of training samples sharing common characteristics, the bias is small, helping the language model learn common features and patterns in the samples. This eliminates the need to expend excessive computational power and time to distinguish subtle differences between samples. This not only allows the language model to focus more quickly on learning the core features of the samples but also reduces the difficulty of generalization, making it easier for the language model to extend the learned patterns to new similar samples and utilize the learned core features to exhibit stronger robustness in similar situations. Similarly, in the next iteration, another batch of common samples is randomly selected from all data subsets for iterative training, and so on, until the language model converges, completing the fine-tuning.

[0134] In other words, this embodiment mimics the phenomenon of human learning: when exposed to related topics simultaneously, individuals often find it easier to grasp concepts by leveraging inherent similarities. Conversely, different topics present learning difficulties for the model. Analogously, this embodiment can divide a given large-scale dataset into subsets based on task information, statistical information, and feature encoding. Samples within the same subset share commonalities, while samples in different subsets exhibit unique characteristics. Thus, during fine-tuning, each iteration uses common samples to enhance the AI ​​model's learning ability, while training samples from different batches possess unique characteristics. Therefore, macroscopically, it remains a fine-tuning of a large-scale dataset containing diverse data, ensuring the language model's adaptability to complex and diverse data, improving its generalization ability, avoiding overfitting, and ultimately enhancing the overall performance of the language model.

[0135] In some possible implementations, such as Figure 6 This document illustrates a flowchart of a model training method provided in an embodiment of this application. Figure 6 Yes Figure 5 For a more detailed description of the process shown, please refer to [link / reference]. Figure 6In step S520, samples can be classified based on their corresponding task types. This ensures that each batch of samples input into the language model in each iteration belongs to the same task type. Utilizing these similar task-type samples enhances the language model's understanding of individual tasks during fine-tuning, thereby improving its performance on those tasks. Specifically, for example... Figure 6 As shown, in S520, several samples are classified according to commonalities to obtain multiple data subsets, which may include:

[0136] S521a, determine the task type for each sample.

[0137] In this step, the task information for each sample can be parsed to determine the task type corresponding to each sample. Task information refers to the instructions or descriptions of the specific tasks that the language model needs to identify and perform in a natural language processing task. As a concrete example, task information can be represented by fixed structures or prompts in the instruction data.

[0138] For example, task information can be a hint in the instruction data (it could be the question itself, or keywords, etc.). For instance, the instruction data a2 contains the question: "When was the Big Bang theory proposed?" This question itself is a hint, and based on this hint, we can know that instruction data a2 is a sample used for a question-and-answer task.

[0139] Task information can also have a fixed structure. For example, if the expected output label in instruction data a3 is "positive" or "negative", then instruction data a3 can be determined to be a sample for sentiment analysis based on the label.

[0140] Task information can also be in the form of corresponding instructions. For example, the instruction data may include specific instruction formats such as "Please translate a sentence from Chinese to English" or "Generate a summary for a paragraph". This can help determine whether the task is for translation or rewriting.

[0141] In this way, the task information of the sample can be parsed based on the sample content or labels, and then the task type of each sample can be determined.

[0142] S522a divides samples with the same task type into the same data subset, resulting in multiple data subsets.

[0143] In this step, samples with the same task type can be grouped into the same data subset, thus dividing the dataset into multiple data subsets. Each data subset consists of samples with a common characteristic. For example, data subset 1 contains samples corresponding to question-answering tasks, data subset 2 contains samples corresponding to sentiment analysis tasks, data subset 3 contains samples corresponding to translation tasks, and so on.

[0144] Next, during the fine-tuning of the language model, the obtained data subset can be shuffled, meaning it no longer has a specific sampling order. Then, steps S310 to S320 are executed. In each iteration, a random sample is selected from this subset to obtain training samples, which are then input into the language model for training. This ensures both the randomness of the language model's input to train its generalization ability and avoid overfitting, and also guarantees that the samples input within a batch share commonalities, allowing the language model to better learn the properties of similar data.

[0145] In this implementation, if the fine-tuned language model is used in a vertical domain scenario, such as deploying the language model as a question-answering system in the education industry or as a machine translation system, then by dividing the large-scale data into common task types, it is possible to sample from relevant data subsets in a targeted manner according to the characteristics of the domain or industry. For example, the sampling probability of these data subsets can be set to be higher to obtain relevant training samples, thereby improving the language model's task processing capability in the vertical domain scenario. The model training is more flexible and the training effect is greatly enhanced.

[0146] In some possible implementations, such as Figure 7 This document illustrates a flowchart of a model training method provided in an embodiment of this application. Figure 7 Yes Figure 5 For a more detailed description of the process shown, please refer to [link / reference]. Figure 7 In step S520, common classification can also be performed based on the statistical information corresponding to the samples (such as the length of the sample sequence), so that the batch of samples input to the language model in each iteration belongs to the same statistical interval, thereby strengthening the language model's preference for output length and meeting the user's personalized needs.

[0147] Specifically, such as Figure 7 As shown, in this implementation, S520, several samples are classified according to commonalities to obtain multiple data subsets, which may include:

[0148] S521b, determine the sequence length for each sample.

[0149] In this step, the instruction data serves as samples, possessing a specific word order and length. Therefore, each instruction data item can be considered a sample sequence, and the length of each sample sequence can be calculated. For example, instruction data a4 includes:

[0150] Instruction: Please determine the sentiment of the following comment: This movie is fantastic!

[0151] Expected output: Positive.

[0152] After statistical analysis, the instruction length is 15 (number of characters), the output length is 6 (number of characters), and the total length is 21 (number of characters).

[0153] S522b divides samples whose sequence lengths are distributed in the same length range into the same data subset, resulting in multiple data subsets.

[0154] In this step, some length ranges can be set, such as 1 to 10 (number of characters), 11 to 20 (number of characters), and so on. Then, based on which length range the total sequence length of the samples falls into, the data is divided into subsets. Each subset contains samples with the same length range. For example, data subset 11 contains samples whose total sequence length falls within the range of 1 to 10 (number of characters), data subset 12 contains samples whose total sequence length falls within the range of 11 to 20 (number of characters), and so on.

[0155] Next, during the fine-tuning of the language model, the obtained data subset can be shuffled, i.e. it does not have a specific sampling order, so that the steps S310 to S320 above can be executed. In each iteration, a sample is randomly selected from these data subsets to obtain training samples to be input into the language model for training.

[0156] In this implementation, if the fine-tuned language model is used for a general task scenario and the statistical information of the sample data varies greatly, such as the dataset containing several samples of different lengths, all of which are translation tasks, then these samples can be divided into common sequence lengths. In this way, when fine-tuning the language model, the input size of the language model is basically similar in each iteration, which can better utilize the parallel computing capabilities of the hardware that performs the language model training task, reduce the construction of the computation graph, and also help reduce gradient fluctuations, improve stability, and thus improve the efficiency of language model fine-tuning.

[0157] In some possible implementations, such as Figure 8 This document illustrates a flowchart of a model training method provided in an embodiment of this application. Figure 8 Yes Figure 5 For a more detailed description of the process shown, please refer to [link / reference]. Figure 8 In step S520, common classification can also be performed based on the source information of the samples, so that the batch of samples input into the language model in each iteration are samples from the same or similar sources, thereby enhancing the language model's ability to process specific tasks.

[0158] Specifically, such as Figure 8 As shown, in this implementation, S520, several samples are classified according to commonalities to obtain multiple data subsets, which may include:

[0159] S521c converts the source information of each sample into a feature vector.

[0160] In this step, the source information of the samples can be used to describe the task corresponding to the samples. For example, if each sample in the dataset is a test question and its corresponding answer, then the source of these samples can be information describing them as a test task of the corresponding subject. For example, instruction data a21 includes a math calculation problem and its corresponding answer, so its source can be described as a math subject test task; instruction data a22 includes a physics problem and its corresponding answer, so its source can be described as a physics subject test task.

[0161] For example, after parsing the source information of the sample, the source of the sample can be converted into the corresponding feature vector (embedding) through a sentence encoder or similar means.

[0162] S522c performs clustering calculations for each feature vector, dividing samples from similar sources into the same data subset, resulting in multiple data subsets.

[0163] In this step, clustering can be performed on each feature vector to divide samples from the same or similar sources into the same data subset, thus obtaining multiple data subsets.

[0164] For example, refer to Figure 9 As shown, a dataset used to train an exam-type model contains samples T1, T2, ..., Tn that cover 57 subject exam task categories, including algebra, geometry, calculus, biology, historical events, literature, legal provisions, and microeconomics. Each task category can be represented as Cx, where x = 1, 2, ..., 57. Since different tasks share certain similarities—for example, tasks involving algebra, geometry, and calculus are mathematically-based—clustering can be performed based on this similarity to obtain corresponding subsets. Specifically, in this example, the feature vectors of samples T1, T2, ..., Tn can be clustered using the K-Nearest Neighbors (KNN) clustering algorithm. First, the parameter k is set; k is a positive integer representing the number of neighbors to consider during classification. Then, for each sample, the distance between its source feature vectors and all other samples in the dataset is calculated. For example, and not a limitation, Euclidean distance or cosine similarity can be used for distance calculation. Then, for each sample, based on the distance of its source feature vectors to other samples, the k nearest neighbors s1, s2, ..., sk are retrieved. Next, the category with the most occurrences among the k nearest neighbors for each sample is taken as the sample's category. In this way, samples are grouped into subsets based on their category, resulting in multiple subsets of data with similar sources.

[0165] Next, during the fine-tuning of the language model, the obtained data subset can be shuffled, i.e. it does not have a specific sampling order, so that the steps S310 to S320 above can be executed. In each iteration, a sample is randomly selected from these data subsets to obtain training samples to be input into the language model for training.

[0166] In this implementation, if the task type, sequence length, etc. of the samples in the dataset are similar during the fine-tuning of the language model, they can be classified by common source characteristics, that is, the characteristics of the samples can be obtained from the perspective of language representation. In this way, samples with similar characteristics (i.e., having commonalities) can be clustered into one class to enhance the learning ability of the language model.

[0167] For example, this application embodiment also provides a model training apparatus 1000, which may include: a determining module 1001, which can be used to determine training samples from one of multiple data subsets in each iteration during the fine-tuning of the language model, wherein each data subset is a set of samples with similar commonalities, and the commonalities of samples in different data subsets are different; and a processing module 1002, which can be used to input the training samples into the language model to train the language model.

[0168] For example, fine-tuning includes multiple training rounds, each round of training including multiple iterations; then, the determining module 1001 is specifically used to determine training samples from one of multiple data subsets during each iteration of the current training round during the fine-tuning of the language model; the processing module 1002 is specifically used to input the training samples into the language model to train the language model; the processing module 1002 is also used to obtain the first loss data of the language model after the current training round of the language model is completed; the determining module 100 is also used to determine the sampling probability of each data subset in the next training round based on the first loss data, so that in each iteration of the next training round, one data subset is selected from multiple data subsets according to the sampling probability to determine the corresponding training data again.

[0169] For example, the first loss data includes the loss value obtained from each iteration of the language model during the current training round. The determining module 1001 is specifically used to select a data subset to determine training samples from each data subset according to the first sampling probability configured for each data subset during each iteration of the current training round. The first sampling probability represents the probability that the data subset is used to determine training samples during the current training round. The determining module 1001 is also specifically used to: compare each loss value in the first loss data with a preset threshold; if any loss value does not reach the threshold, reduce the first sampling probability corresponding to the data subset used to obtain the loss value; if any loss value reaches the threshold, increase the first sampling probability corresponding to the data subset used to obtain the loss value, or keep the first sampling probability unchanged.

[0170] For example, in each iteration, the training data is randomly selected from multiple subsets of data.

[0171] For example, the processing module 1002 is also used to obtain a dataset, which includes several samples; and to classify the samples according to their commonalities to obtain multiple data subsets.

[0172] For example, a sample is instruction data used to guide a language model to perform at least one type of task; the processing module 1002 is specifically used to: determine the task type of each sample; divide samples with the same task type into the same data subset to obtain multiple data subsets.

[0173] For example, a sample is instruction data used to guide a language model to perform at least one type of task; the processing module 1002 is specifically used to: generate a sample sequence for each sample; determine the length of each sample sequence; and divide samples whose sample sequence lengths are distributed in the same length range into the same data subset to obtain multiple data subsets.

[0174] For example, a sample is instruction data used to guide a language model to perform at least one type of task; the processing module 1002 is specifically used to: convert the source information of each sample into a feature vector, the source information being used to describe the task corresponding to the sample; perform clustering calculations on each feature vector, dividing samples with similar sources into the same data subset, resulting in multiple data subsets.

[0175] Both the determining module 1001 and the processing module 1002 can be implemented in software or hardware. For example, these modules, as software functional units, can all include code running on a computing instance. The computing instance can include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance can be one or more. For example, the processing module 1002 can include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region can include multiple AZs.

[0176] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0177] These modules, as examples of hardware functional units, can all include at least one computing device, such as a server. Alternatively, these modules can also be devices implemented using application-specific integrated circuits (ASICs) or programmable logic devices (PLDs). The aforementioned PLDs can be implemented using complex programmable logical devices (CPLDs), field-programmable gate arrays (FPGAs), generic array logic (GALs), or any combination thereof.

[0178] These modules include multiple computing devices that can be distributed within the same region or in different regions. They can also be distributed within the same Availability Zone (AZ) or in different AZs. Similarly, they can be distributed within the same Virtual Private Cloud (VPC) or multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0179] Based on the methods in the above embodiments, this application provides a computing device. The computing device may include: at least one memory for storing a program; and at least one processor for executing the program stored in the memory; wherein, when the program stored in the memory is executed, the processor executes the methods in the above embodiments.

[0180] Based on the methods in the above embodiments, this application provides a computing device cluster. The computing device cluster may include at least one computing device. The computing device may include: at least one memory for storing a program; and at least one processor for executing the program stored in the memory; wherein, when the program stored in the memory is executed, the processor is used to execute the methods in the above embodiments.

[0181] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0182] Based on the methods in the above embodiments, this application provides a computer program product, characterized in that, when the computer program product is run on a processor, the processor executes the methods in the above embodiments.

[0183] Based on the methods described in the above embodiments, this application also provides a chip. Please refer to... Figure 11 , Figure 11 This is a schematic diagram of a chip structure provided in an embodiment of this application. Figure 11 As shown, chip 900 includes one or more processors 901 and interface circuitry 902. Optionally, chip 900 may also include a bus 903. Wherein:

[0184] The processor 901 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of the processor 901 or through software instructions. The processor 901 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods and steps disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0185] The interface circuit 902 can be used to send or receive data, instructions or information. The processor 901 can use the data, instructions or other information received by the interface circuit 902 to process the data, instructions or other information, and can send the processed information out through the interface circuit 902.

[0186] Optionally, chip 900 also includes memory, which may include read-only memory and random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).

[0187] Optionally, the memory stores executable software modules or data structures, and the processor can execute corresponding operations by calling the operation instructions stored in the memory (which may be stored in the operating system).

[0188] Optionally, the interface circuit 902 can be used to output the execution results of the processor 901.

[0189] It should be noted that the functions of the processor 901 and the interface circuit 902 can be implemented through hardware design, software design, or a combination of hardware and software; no restrictions are imposed here.

[0190] It should be understood that each step of the above method embodiments can be completed by hardware logic circuits or software instructions in a processor.

[0191] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In addition, in some possible implementations, each step in the above embodiments may be selectively executed according to the actual situation, and may be partially or fully executed, which is not limited here.

[0192] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0193] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0194] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0195] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

Claims

1. A model training method, characterized in that, The method includes: In the process of fine-tuning the language model, each iteration determines the training samples from one of multiple data subsets. Each data subset is a collection of samples with similar commonalities, but the commonalities of samples in different data subsets are different. The training samples are input into the language model to train the language model.

2. The method according to claim 1, characterized in that, The fine-tuning includes multiple rounds of training, and each round of training includes multiple iterations; The method specifically includes: During the fine-tuning of the language model, the training samples are determined from one of the plurality of data subsets during each iteration of the current training round. The training samples are input into the language model to train the language model; After completing the current round of training on the language model, the first loss data of the language model is obtained; Based on the first loss data, the sampling probability of each data subset in the next round of training is determined, so that in each iteration of the next round of training, one data subset is selected from the plurality of data subsets according to the sampling probability, so as to determine the corresponding training data again.

3. The method according to claim 2, characterized in that, The first loss data includes the loss value obtained from each iteration of the language model during the current training round. The step of determining the training samples from one of the plurality of data subsets during each iteration of the current training round includes: In each iteration of the current training round, a data subset is selected to determine the training sample according to a first sampling probability configured for each data subset. The first sampling probability represents the probability that the data subset is used to determine the training sample in the current training round. The step of determining the sampling probability of each data subset in the next training round based on the first loss data includes: Each loss value in the first loss data is compared with a preset threshold; If any of the loss values ​​fails to reach the threshold, the first sampling probability corresponding to the subset of data used to obtain the loss value is reduced. If any of the loss values ​​reaches the threshold, the first sampling probability corresponding to the subset of data used to obtain the loss value is increased, or the first sampling probability is kept unchanged.

4. The method according to any one of claims 1-3, characterized in that, In the process of fine-tuning the language model, before determining training samples from one of multiple data subsets in each iteration, the method includes: Obtain a dataset, which includes several samples; The samples are classified according to their commonalities to obtain the multiple data subsets.

5. The method according to claim 4, characterized in that, The sample is instruction data used to guide the language model to perform at least one type of task; The samples are classified according to their commonalities to obtain the multiple data subsets, including: Determine the task type for each of the samples; Samples with the same task type are divided into the same data subset to obtain the multiple data subsets.

6. The method according to claim 4 or 5, characterized in that, The sample is instruction data used to guide the language model to perform at least one type of task; The samples are classified according to their commonalities to obtain the multiple data subsets, including: Generate a sample sequence for each of the samples; Determine the length of each of the sample sequences; Samples whose sample sequence lengths are distributed within the same length range are divided into the same data subset to obtain the multiple data subsets.

7. The method according to any one of claims 4-6, characterized in that, The sample is instruction data used to guide the language model to perform at least one type of task; The samples are classified according to their commonalities to obtain the multiple data subsets, including: The source information of each sample is converted into a feature vector, and the source information is used to describe the task corresponding to the sample; Clustering calculations are performed on each feature vector to divide samples from similar sources into the same data subset, resulting in multiple data subsets.

8. A model training device, characterized in that, The device includes: The determination module is used to determine training samples from one of multiple data subsets in each iteration during the fine-tuning of the language model. Each data subset is a set of samples with similar commonalities, and the commonalities of samples in different data subsets are different. The processing module is used to input the training samples into the language model to train the language model.

9. The apparatus according to claim 8, characterized in that, The fine-tuning includes multiple rounds of training, and each round of training includes multiple iterations; The determining module is specifically used to determine the training sample from one of the plurality of data subsets during each iteration of the current training round in the process of fine-tuning the language model. The processing module is specifically used to input the training samples into the language model to train the language model; The processing module is also used to obtain the first loss data of the language model when the current round of training of the language model is completed; The determining module is further configured to determine the sampling probability of each data subset in the next training round based on the first loss data, so that in each iteration of the next training round, one data subset is selected from the plurality of data subsets according to the sampling probability to determine the corresponding training data again.

10. The apparatus according to claim 9, characterized in that, The first loss data includes the loss value obtained from each iteration of the language model during the current training round. The determining module is specifically used to select a data subset to determine the training sample in each iteration of the current training round according to a first sampling probability configured for each data subset. The first sampling probability represents the probability that the data subset is used to determine the training sample in the current training round. The determining module is further configured to: Each loss value in the first loss data is compared with a preset threshold; If any of the loss values ​​fails to reach the threshold, the first sampling probability corresponding to the subset of data used to obtain the loss value is reduced. If any of the loss values ​​reaches the threshold, the first sampling probability corresponding to the subset of data used to obtain the loss value is increased, or the first sampling probability is kept unchanged.

11. The apparatus according to any one of claims 8-10, characterized in that, The processing module is also used for: Obtain a dataset, which includes several samples; The samples are classified according to their commonalities to obtain the multiple data subsets.

12. The apparatus according to claim 11, characterized in that, The sample is instruction data used to guide the language model to perform at least one type of task; The processing module is specifically used for: Determine the task type for each of the samples; Samples with the same task type are divided into the same data subset to obtain the multiple data subsets.

13. The apparatus according to claim 11 or 12, characterized in that, The sample is instruction data used to guide the language model to perform at least one type of task; The processing module is specifically used for: Generate a sample sequence for each of the samples; Determine the length of each of the sample sequences; Samples whose sample sequence lengths are distributed within the same length range are divided into the same data subset to obtain the multiple data subsets.

14. The apparatus according to any one of claims 11 to 3, characterized in that, The sample is instruction data used to guide the language model to perform at least one type of task; The processing module is specifically used for: The source information of each sample is converted into a feature vector, and the source information is used to describe the task corresponding to the sample; Clustering calculations are performed on each feature vector to divide samples from similar sources into the same data subset, resulting in multiple data subsets.

15. A computing device, characterized in that, Including processor and memory; The processor is configured to execute instructions stored in the memory such that the processor performs the method as described in any one of claims 1-7.

16. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-7.

17. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1-7.

18. A computer-readable storage medium, characterized in that, Includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1-7.

Images