Model training method, device, program product, and storage medium

Abstract

Embodiments of the present specification provide a model training method, device, program product and storage medium, the method comprising: obtaining, according to at least one preset category, an initial data set corresponding to the at least one preset category respectively; if it is determined that the initial data set does not satisfy a preset balance condition, adjusting the initial data set so that the adjusted data set satisfies the balance condition; wherein the balance condition comprises one or more of the following: a condition that the data amount of the data set is balanced in different languages, a condition that the data amount of the data set is balanced in different lengths, or a condition that the data amount of the data set is balanced in different data complexities; determining at least one training task of a preset model, and training the preset model based on the adjusted data set of the preset category corresponding to the training task.

Description

Technical Field

[0001] This manual relates to the field of large model technology, and in particular to model training methods, equipment, program products and storage media. Background Technology

[0002] Fine-tuning of large language models refers to the process of further training a pre-trained large language model using datasets specific to a particular domain or task to optimize and adapt it for that specific task. Supervised fine-tuning determines the breadth and depth of skills and knowledge the model ultimately acquires. However, current training methods still have shortcomings, resulting in poor out-of-distribution generalization ability of the model. Summary of the Invention

[0003] To overcome the problems existing in related technologies, embodiments of this specification provide model training methods, devices, program products, and storage media.

[0004] According to a first aspect of the embodiments of this specification, a model training method is provided, the method comprising:

[0005] Based on at least one preset category, obtain an initial dataset corresponding to each of the at least one preset category;

[0006] If it is determined that the initial dataset does not meet the preset balance condition, the initial dataset is adjusted so that the adjusted dataset meets the balance condition; wherein, the balance condition includes one or more of the following: the condition of balancing the amount of data in different languages, the condition of balancing the amount of data in different lengths, or the condition of balancing the amount of data in different data complexities.

[0007] Determine at least one training task for the preset model, and train the preset model based on an adjusted dataset of preset categories corresponding to the training task.

[0008] According to a second aspect of the embodiments of this specification, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method embodiments described in the first aspect above.

[0009] According to a third aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method embodiments described in the first aspect above.

[0010] According to a fourth aspect of the embodiments of this specification, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method embodiments described in the first aspect above.

[0011] The technical solutions provided in the embodiments of this specification may include the following beneficial effects:

[0012] In this embodiment, multiple preset datasets of various categories can be prepared in advance, each containing multiple data entries within that category. This embodiment designs balancing conditions, including one or more of the following: balancing the amount of data in different languages, balancing the amount of data in different lengths, or balancing the amount of data in different complexities. Thus, the adjusted dataset satisfies the balancing conditions, thereby achieving data diversity and balance in any of the three aspects: language, length, or complexity. Therefore, by utilizing a widely distributed dataset, when training the model, the preset model can be trained based on the adjusted dataset corresponding to the preset category of the training task, and the model can possess good out-of-distribution generalization ability.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a model training method according to an exemplary embodiment of this specification.

[0015] Figure 2 This is a schematic diagram illustrating an exemplary embodiment of constructing training data, as shown in this specification.

[0016] Figure 3 This is a hardware structure diagram of a computer device containing a model training apparatus, as illustrated in this specification according to an exemplary embodiment.

[0017] Figure 4 This is a block diagram illustrating a model training apparatus according to an exemplary embodiment of this specification. Detailed Implementation

[0018] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0019] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0020] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0021] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points shall be provided for users to choose to authorize or refuse.

[0022] Large language models are artificial intelligence models that utilize deep learning techniques and are trained on massive amounts of text data to understand and generate natural language. They have a wide range of applications, including language translation, dialogue systems, and text generation.

[0023] Supervised fine-tuning is a crucial process in the knowledge distillation of large language models. Fine-tuning of large language models refers to the process of further training a pre-trained large language model using datasets specific to a particular domain or task to optimize and adapt it for that specific task. Supervised fine-tuning determines the breadth and depth of skills and knowledge the model ultimately acquires. At this stage, introducing diverse training data can comprehensively develop and enhance the model's various capabilities, including mathematical reasoning, translation, programming, commonsense understanding, and domain knowledge.

[0024] However, the selection and use of training data requires extreme caution. Because data from different domains and sources vary significantly in distribution, size, and quality, careful consideration must be given to avoid any data source exerting an excessive influence on the model, leading to overfitting on certain tasks and underfitting on others.

[0025] Furthermore, an imbalance in the scale of different types of data can also lead to underdevelopment of certain capabilities in the model. For example, if the number of programming-related data samples far exceeds that of other types of data, the model may ultimately perform well on programming tasks but lag behind in other capabilities such as mathematical reasoning. Therefore, balancing and allocating multivariate data is crucial during the supervised fine-tuning phase.

[0026] While knowledge distillation and supervised fine-tuning of small language models for single tasks have been explored to improve specific capabilities, enhancing the versatility of models across multiple tasks through fine-tuning remains a challenge. The relationships between the performance of fine-tuned models and the amount of data, the proportion of data combinations, model size, and fine-tuning strategies are not yet fully understood.

[0027] Research has found that fine-tuning a model across multiple tasks can lead to conflicting capabilities or catastrophic forgetting, meaning it's difficult to simultaneously improve the capabilities of multiple models. Furthermore, the distribution of the original data is also closely related to the effectiveness of fine-tuning. An insufficiently broad distribution can result in poor out-of-distribution generalization ability in the fine-tuned model.

[0028] Currently, massive amounts of raw data can be obtained from the internet or business data sources. However, filtering and combining this raw, massive data to obtain the required fine-tuning dataset is a significant challenge. Research has found that balancing the difficulty, diversity, and long-tail distribution of data in terms of length and language is crucial. Based on this, this specification provides a model training method. Using this method, high-quality data can be obtained, and the model can be trained based on this data, ensuring the breadth and diversity of the fine-tuning data and improving the model's robustness. The embodiments of this specification will now be described in detail.

[0029] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a model training method according to an exemplary embodiment, comprising the following steps:

[0030] In step 102, an initial dataset corresponding to each of the at least one preset category is obtained.

[0031] In step 104, if it is determined that the initial dataset does not meet the preset balance condition, the initial dataset is adjusted so that the adjusted dataset meets the balance condition.

[0032] The balance conditions include one or more of the following: the data volume balance of the dataset in different languages, the data volume balance of the dataset in different lengths, or the data volume balance of the dataset in different data complexities.

[0033] In step 106, at least one training task of the preset model is determined, and the preset model is trained based on the adjusted dataset of the preset category corresponding to the training task.

[0034] As can be seen from the above embodiments, this embodiment can prepare multiple preset initial datasets, each containing multiple data points within that category. This embodiment designs balancing conditions, including one or more of the following: balancing the amount of data in different languages, balancing the amount of data in different lengths, or balancing the amount of data in different complexities. Thus, the adjusted dataset satisfies the balancing conditions, thereby achieving data diversity and balance in any of the three aspects: language, length, or complexity. Therefore, by utilizing a widely distributed dataset, when training the model, the preset model can be trained based on the adjusted dataset corresponding to the preset category of the training task, and the model can possess good out-of-distribution generalization ability.

[0035] As an example, the model training method of this embodiment can run on a computer device, which includes, but is not limited to, a server, a server cluster consisting of multiple servers, or a cloud server based on cloud computing consisting of a large number of hosts or servers, a tablet computer, a personal digital assistant, a laptop computer, a desktop computer, or a mobile device, etc.

[0036] As an example, the preset model used in this embodiment can be a Large Language Model (LLM). This language model can be self-built or an open-source language model. This embodiment does not limit this.

[0037] As an example, the language model can be a pre-trained language model, which can be supervisedly fine-tuned using this embodiment.

[0038] As another example, this embodiment can be applied to knowledge distillation scenarios. For instance, the preset model in this embodiment can be the student model in a teacher-student model. This embodiment is used to perform supervised training on the student model, which is already trained on the teacher model. The teacher model in the teacher-student model can be trained according to actual needs; for example, it can be obtained by training the language model using other methods, or it can be obtained by performing supervised fine-tuning on a pre-trained language model using this embodiment.

[0039] In some cases, data can be collected in advance to obtain large-scale and diverse raw data, which is text. For example, massive amounts of source data can be obtained from the internet or business data sources. For instance, the OpenHermes 2.5 dataset (a dataset name) contains over one million high-quality prompts, responses, and multi-turn dialogues from multiple different sources, covering various domains. In some scenarios, short, non-instructional dialogues may be required, which can be achieved using datasets such as the Large-scale Cleaned Chinese Conversation Dataset (LCCD). Alternatively, for specialized tasks such as mathematical reasoning, more relevant datasets can be obtained from the internet as alternatives. This embodiment does not limit the specific source of the dataset.

[0040] Given that LLMs are models pre-trained on massive amounts of text data, capable of understanding and generating natural language, and widely used in language translation, dialogue systems, text generation, content summarization, and programming assistance, etc., supervised fine-tuning, based on pre-trained large language models, is the process of additionally training the model using datasets specific to a particular domain or task to improve its performance on certain tasks (such as legal document drafting, medical diagnosis, etc.). Therefore, to build high-quality training for the model on various specific tasks, the collected massive amounts of data can be categorized. For example, each piece of data can be categorized according to different training task categories, such as mathematics, reasoning, code generation, etc. As an example, this embodiment can have multiple preset categories, for example, it can include the following 33 categories:

[0041] Mathematics, reasoning, code generation, code debugging, writing, computer science, complex formats, common sense, hypothetical problems, multilingual translation, role-playing, biology, engineering, ethics, sports, law, medicine, literature, entertainment, art, music, malicious attacks, economics, physics, history, academic writing, chemistry, philosophy, health, ecology, grammar, rewriting, and others.

[0042] These categories cover a wide range of application areas and, when applied to training specific tasks, can meet the training needs of different types of models. When applied to training comprehensive models, they can ensure the comprehensive capabilities of large language models across diverse tasks. The collected source data can be classified according to the above categories. An initial dataset can be constructed for each category, and each data point can be assigned to the corresponding initial dataset based on the classification results.

[0043] In practical applications, the collected source data can be classified in several ways. For example, it can be classified manually. Alternatively, existing language models or pre-trained classification models can be used for classification.

[0044] As an example, obtaining the initial dataset corresponding to each of the at least one preset category may include:

[0045] Acquire multiple source data, and select a portion of target source data from the multiple source data;

[0046] Using the target source data and preset prompt information, prompt content is constructed and input into a preset language model. Based on the response of the preset language model, the preset category to which the target source data belongs is determined. The preset prompt information is used to instruct the preset language model to identify the preset category to which the target source data belongs from at least one preset category.

[0047] Based on the target source data and its preset category, a preset data classification model is trained;

[0048] The trained data classification model is used to identify the preset category to which the source data belongs, thereby obtaining an initial dataset corresponding to each of the at least one preset category.

[0049] In this embodiment, in order to accurately classify the massive source data, a portion of the target source data can be selected from the massive source data, and these target source data can be classified using a preset language model.

[0050] As an example, the preset prompt information could be used to prompt the preset language model to determine the preset category to which the data belongs among multiple preset categories (e.g., determining which of the 33 categories the data belongs to).

[0051] As an example, taking Chinese as the case, a prompt template can be prepared in advance. The prompt template can be:

[0052] Please classify the following given data;

[0053] Categories include: Mathematics, Reasoning, Code Generation, Code Debugging, Writing, Computer Science, Complex Formatting, Common Sense, Hypothesis Problems, Multilingual Translation, Role-Playing, Biology, Engineering, Ethics, Sports, Law, Medicine, Literature, Entertainment, Art, Music, Harmful Attacks, Economics, Physics, History, Academic Writing, Chemistry, Philosophy, Health, Ecology, Grammar, Rewriting, and Others;

[0054] The given data is: ***.

[0055] The content within the double quotes represents the prompt information; "***" represents the target source data. Based on this, prompt content can be generated for each piece of target source data and the preset prompt information. The prompt content is then input into a preset language model, and the response content from the language model can be obtained. Based on the response content, the preset category to which the target source data belongs can be determined.

[0056] Therefore, the target source data has corresponding category labels, and a data classification model can be trained using this labeled data. This data classification model can then be used to identify the category of the source data. The classification model can be selected or constructed as needed; this embodiment does not impose any limitations on this.

[0057] Through the above embodiments, multiple initial datasets can be obtained, and the initial datasets store data for the corresponding categories.

[0058] In practical applications, the datasets obtained through the above embodiments may contain data that is imbalanced in dimensions such as language, length, or complexity. To ensure the diversity and balance of the fine-tuned data in terms of language, length, and complexity, and to improve the generalization ability and robustness of the model, this embodiment designs a balance condition to determine whether the dataset meets the balance condition. If the condition is not met, the dataset can be adjusted. For ease of distinction, this embodiment refers to the dataset before adjustment as the initial dataset and the dataset after adjustment as the adjusted dataset. The initial dataset can be adjusted based on the preset balance condition so that the adjusted dataset meets the balance condition; wherein, the balance condition includes one or more of the following: a balance of data volume in different languages, a balance of data volume in different lengths, or a balance of data volume in different complexities.

[0059] Considering that the wide distribution of fine-tuning data plays a crucial role in the performance robustness of the fine-tuned model, this embodiment designs three diversity balance conditions: 1. Balance of language diversity 2. Balance of length diversity 3. Balance of complexity diversity.

[0060] Based on this, the training process in this embodiment can be: collecting a large-scale dataset → constructing datasets of multiple categories → balancing length diversity / language diversity / data complexity → fusing according to a set method → ​​supervised fine-tuning.

[0061] As an example, in many scenarios, it is necessary to train a model to be capable of multiple languages, such as Chinese and English, but it may not be possible to collect multilingual data with a sufficiently wide distribution.

[0062] Based on this, in this embodiment, the condition for balancing the amount of data in different languages ​​in the dataset may include: the amount of data in each language in the dataset is basically the same.

[0063] If it is determined that the initial dataset does not meet the preset balance condition, adjusting the initial dataset so that the adjusted dataset meets the balance condition may include:

[0064] Obtain the language of the data in the initial dataset and the amount of data for each language;

[0065] If there is a target language with a lower data volume than other languages, use the data from other languages ​​to generate extended data corresponding to the target language, so that the data volume of each language in the adjusted dataset is basically the same.

[0066] As an example, the distribution of language counts in the dataset for each task can be determined. If the amount of data for each language in a task's dataset is roughly the same, no adjustment is needed. If the amount of data for different languages ​​in a task's dataset varies significantly—for example, if the difference in the amount of data for one language is large compared to other languages—expanded data can be generated for the languages ​​with smaller datasets.

[0067] Existing language models can be used to expand the language range of the data. For example, if there is less data in language A and more data in language B, the language model can use the data in language B to generate data in language A. For instance, prompts can be generated containing data in language B and information indicating that the model has generated language A data based on the language B data. This method can be used to generate data with different content and languages ​​until the amount of data in both languages ​​is approximately equal.

[0068] In practical applications, more languages ​​are also optional, ultimately resulting in a dataset where the amount of data for each language is roughly the same. Here, "roughly the same" means that the difference in data volume between different languages ​​is less than a set threshold, not that they must be completely identical. Exposing the model to text of varying lengths during training can improve its performance in different application scenarios. For example, the model requires different processing strategies when handling short texts (such as SMS messages or short instructions) and long texts (such as articles or reports). If the training data is mostly short text, the model may perform poorly when handling long text, and vice versa.

[0069] In practical applications, users may input text of varying lengths. Models need to be able to adapt flexibly to provide accurate and useful responses. Some tasks (such as long-form summarization and complex question answering) require handling longer inputs, while others (such as simple question answering and instruction rewriting) rely more on shorter texts. If a model is exposed to data of a certain length too frequently during training, it may overfit to that length and perform poorly on other lengths. Conversely, if a model lacks training on certain lengths of data, it may fail to effectively understand and process that data, leading to performance degradation.

[0070] Research has revealed that datasets may have a smaller long-tail portion. Ensuring training data diversity in length (e.g., text length, instruction length) is crucial. For example, the conditions for balancing the amount of data in different length ranges include: the amount of data in the dataset within several preset length intervals conforms to a normal distribution.

[0071] If it is determined that the initial dataset does not meet the preset balance condition, adjusting the initial dataset to make the adjusted dataset meet the balance condition includes:

[0072] The amount of data in the initial dataset in the multiple preset length intervals is determined based on the text length of the data in the initial dataset.

[0073] If it is determined that the amount of data in the initial dataset does not conform to a normal distribution in the multiple length intervals, one or more target length intervals are determined, and similar extended data is generated based on the data in the target length intervals so that the amount of data in the adjusted dataset conforms to a normal distribution in the multiple length intervals.

[0074] As an example, the text length of data can be measured using various metrics; for example, these could be:

[0075] Character count; This can be used to count the total number of all characters in the text as the text length. Characters can include letters, numbers, punctuation marks, etc.

[0076] Word count; can be used to count the number of words or vocabulary in a text as the text length.

[0077] Tokens: In Natural Language Processing (NLP), a token is the basic unit of text after it has been segmented; it can be a word or even a character. For example, a tokenizer in a language model can segment text into multiple tokens, each representing a portion of the text. For instance, the sentence "Hello,world!" might be segmented into `["Hello",",","world","!"]`, a total of 4 tokens, by some tokenizers. You can input each data point into the tokenizer in a language model, obtain the number of tokens in the output data, and thus determine the text length of each data point.

[0078] Multiple length ranges can be set, and the size and number of length ranges can be set as needed. This embodiment does not limit this.

[0079] For each dataset, we can first obtain the text length of each data point and determine the length range to which each data point belongs. This allows us to determine the amount of data in each length range. Furthermore, we can determine whether the data distribution across the various length ranges is balanced.

[0080] As an example, this embodiment uses a normal distribution to measure whether the data volume distribution of the dataset is balanced across different length intervals. For instance, if the data volume of the dataset conforms to a normal distribution across multiple length intervals, then no adjustment to the dataset is necessary. If it does not conform to a normal distribution, it can be determined which length intervals have insufficient data volume; in this embodiment, these are referred to as target length intervals. Based on the data in the target length intervals, similar extended data is generated so that the data volume of the extended dataset conforms to a normal distribution across the multiple length intervals.

[0081] As an example, the plurality of length intervals include: a very short interval and a very long interval; wherein, the very short interval refers to a length interval in which the data length is greater than the mean text length of the initial dataset minus twice the standard deviation; and the very long interval refers to a length interval in which the data length is greater than the mean text length of the initial dataset plus twice the standard deviation.

[0082] If it is determined that the amount of data in the initial dataset does not conform to a normal distribution in the multiple length intervals, one or more target length intervals are determined, and similar extended data is generated based on the data in the target length intervals, including:

[0083] Based on the mean and standard deviation of the text length of the initial dataset, determine the expected data volume of the extremely short interval and the expected data volume of the extremely long interval under the normal distribution;

[0084] If the actual data volume of the extremely short interval is less than the expected data volume of the extremely short interval, similar extended data is generated based on the data of the extremely short interval so that the actual data volume of the extended extremely short interval is basically the same as the expected data volume of the extremely short interval.

[0085] If the actual data volume of the extremely long interval is less than the expected data volume of the extremely long interval, similar extended data is generated based on the data of the extremely long interval so that the actual data volume of the extended extremely long interval is basically the same as the expected data volume of the extremely long interval.

[0086] For example, the normal distribution of a dataset can be estimated in the following way: or

[0087] Calculate the mean text length of all data in the dataset.

[0088] Where M is the total amount of data in the dataset, i is the i-th data item in the dataset, and l i Let be the text length of the i-th data item.

[0089] Calculate the text length variance of all data in the dataset.

[0090] The standard deviation σ is the square root of the variance.

[0091] The detection requires expanding the data; statistically, data satisfying L>μ+2σ or L<μ-2σ are counted, where L is the text length. That is, data with L>μ+2σ represents the upper tail of the distribution, which is the extremely long interval in this embodiment; typically, in a normal distribution, approximately 95% of the data lies within ±2σ of the mean. Data with L<μ-2σ represents the lower tail of the distribution, which is the extreme interval in this embodiment. The upper and lower tails may have relatively small data volumes; therefore, the portions of text length located at the upper and lower tails can be examined. The normal interval, located between the extremely short and extremely long intervals, usually has a normal data volume; of course, the actual data volume of the normal interval can also be checked according to the method described in this embodiment to see if it matches the expected data volume of a normal interval under a normal distribution.

[0092] If the amount of data P(L>μ+2σ) in the dataset is less than or equal to the expected value of the normal distribution estimate... Small, or, the amount of data P (L<μ-2σ) in the dataset is less than the expected value of the normal distribution estimate. If the data is too small, then this portion of the data needs to be expanded. For example, the actual data volume of the expanded extremely short interval should be roughly the same as the expected data volume of the extremely short interval; the actual data volume of the expanded extremely long interval should be roughly the same as the expected data volume of the extremely long interval.

[0093] For example, more data can be generated in various ways, thus supplementing the upper and lower tails of the dataset so that the amount of data in multiple length intervals of the dataset conforms to a normal distribution.

[0094] As an example, suppose that in a certain dataset, the length distribution of the data is as follows:

[0095] Mean: 100

[0096] Variance: 25

[0097] Standard deviation: 5

[0098] Check the upper and lower tail sections:

[0099] L>100+2*5=110; L<100-2*5=90

[0100] L>100+2×5=110 or L<100-2×5=90

[0101] Data composition in the dataset:

[0102] The number of data points satisfying L>110 is 1500.

[0103] The number of data points satisfying L<90 is 1800.

[0104] Based on the normal distribution, theoretically, the amount of time-limited data with L>110 or L<90 is about 5%.

[0105] Assuming the total data volume is N = 100,000, the expected data volume for the upper tail and the lower tail are 100,000 * 5% = 2,500 records respectively.

[0106] The dataset currently contains 1500 data points (L>110) but less than 2500 (insufficient).

[0107] The current data volume for L<90 is 1800<2500 (insufficient);

[0108] Therefore, for extremely long intervals where L>110, 2500-1500=1000 additional data entries are needed;

[0109] For extremely short intervals where L < 90, 2500 - 1800 = 700 additional data points are needed.

[0110] By supplementing this data, the dataset can be evenly distributed in length, improving the robustness and generalization ability of the model when processing extremely long or short texts.

[0111] As an example, when supplementary data is needed, existing data can be used to instruct the existing language model to generate supplementary data of the required length.

[0112] In other examples, this embodiment also ensures, from the perspective of data complexity, that the dataset meets the condition of data volume balance at different data complexities. As an example, adjusting the initial dataset to ensure that the adjusted dataset meets the balance condition includes:

[0113] Obtain the complexity of the data in the initial dataset and sort them according to complexity;

[0114] Initialize an empty target set, and extract data from the initial dataset in descending order of complexity; wherein, the first extracted data is stored in the target set, and for each data extracted after the first data, if the similarity between the extracted data and the adjacent data in the target set is less than or equal to a preset similarity threshold, the extracted data is placed in the target set; otherwise, it is not placed.

[0115] Once the amount of data stored in the target set reaches a first threshold, the target set is used as an adjusted dataset so that the adjusted dataset satisfies the condition of balancing the amount of data in datasets with different data complexities.

[0116] In this embodiment, considering that each piece of data in the dataset may have different complexities, if there is a large amount of data with some complexities and a small amount of data with other complexities, it will lead to an uneven distribution of complexity in the dataset.

[0117] As an example, the complexity of data in a dataset can be assessed using various metrics. In the natural language processing domain, for instance, the data in a dataset typically consists of text-based question-and-answer pairs, containing a question Q and its corresponding response A. Therefore, the complexity of the data can be assessed using the Instruction-Following Difficulty (IFD), where...

[0118]

[0119] in:

[0120]

[0121] Where Q represents the question, A represents the answer, and θ represents the model; ω i A' ω represents the probability of each word generated by model θ in response to the input A; i A Let θ be the probability of each word generated by model θ for the input question and response A.

[0122] Perplexity θ(A|Q) represents the average of the logarithmic sum of the probabilities of each word in generating the answer A given question Q. Perplexity measures the difficulty of the model predicting the answer; a lower value indicates a more accurate prediction.

[0123] Perplexity θ (A) indicates that, without a question Q, the average of the logarithmic sums of the probabilities of each word is calculated based solely on the answer A itself, which can be used to measure the generation complexity of the answer itself.

[0124] In other words, when calculating Perplexityθ(A|Q), the question Q and the answer A can be input into the model θ together to calculate the perplexity of the model generating A given Q.

[0125] When calculating Perplexityθ(A), you can simply input the answer A into the model θ and calculate the perplexity of A generated by the model without considering Q.

[0126] Based on this, the complexity of each data point in the dataset can be obtained, and then the data in the dataset can be sorted according to the complexity, for example, from high to low. In order to obtain a training dataset with uniform complexity, data can be selected from complex to simple in sequence, and similar or duplicate data need to be reduced.

[0127] This embodiment designs a method of first initializing an empty target set, and then retrieving data from the dataset corresponding to the training task in descending order of complexity. For example, the data with the highest complexity is retrieved first from the dataset and then stored in the target set. Next, the second data is retrieved from the dataset. Starting from the second data, each retrieved data can be used to determine whether it can be stored in the target set through a similarity judgment operation.

[0128] Similarity calculation algorithms can be used to determine the similarity between each retrieved data point and its neighboring data in the target set. For example, for the second retrieved data point, the target set currently only stores the first data point, which is a neighbor of the second data point. Therefore, the similarity between the second and first data points can be calculated. Metrics for measuring the similarity between data points can be pre-set, such as a preset similarity threshold to measure whether the data points are relatively close. If the similarity is very high, the second data point does not need to be stored in the target set; if the similarity is only moderate, the second data point can be stored in the target set.

[0129] Assuming the second data can be stored in the target set, then the third data is retrieved. The neighbor data of the third data in the target set is the second data. Similarly, the similarity between the second and third data is calculated. If the similarity between the two is very high, greater than the preset similarity threshold, then the third data does not need to be stored in the target set.

[0130] Similarly, the fourth data point is retrieved from the dataset. The neighbor of the fourth data point in the target set is the second data point. Similarly, the similarity between the second and fourth data points is calculated. If the similarity between the two is less than the preset similarity threshold, the fourth data point can be stored in the target set.

[0131] The above process can continue until the amount of data stored in the target set reaches the first quantity threshold.

[0132] In practical applications, algorithms for determining the similarity between two data points can include:

[0133] A sentence embedding model can be used to obtain the embedding representation of the data; then the cosine similarity between the embeddings of two data points can be calculated, and the cosine similarity represents the similarity between the two data points.

[0134] Alternatively, the similarity between two datasets can be characterized using the F1 (F-measure, the harmonic mean of recall and precision) score, which is based on Rouge-L (Recall-Oriented Understudy for Gisting Evaluation - Longest Common Subsequence, an evaluation metric for natural language processing tasks such as automatic text summarization or machine translation).

[0135] In practical applications, the adjusted dataset can satisfy one or more of the three balance conditions mentioned above. In cases with multiple conditions, the adjustment order can be set as needed. First, the initial dataset is adjusted according to one balance condition so that the adjusted dataset satisfies that balance condition. Then, the adjusted dataset is used as the initial dataset again and adjusted according to another balance condition, and so on. The adjustment order can be set as needed, and this embodiment does not limit it.

[0136] Based on this, the adjusted datasets of the aforementioned multiple predefined categories can be applied to model training. In the supervised fine-tuning or knowledge distillation process of LLM, different application scenarios and task requirements may be involved. Based on these different application scenarios and task requirements, training data can also be rationally designed and allocated using the adjusted datasets of multiple predefined categories.

[0137] As an example, in a practical application scenario designed to improve the overall capabilities of a model, the step of determining at least one training task for a preset model, and training the preset model based on an adjusted dataset of preset categories corresponding to the training task, includes:

[0138] Obtain the current training objective of the preset model. If the current training objective is to train a model with general input and general output capabilities, determine that the training tasks of the preset model include inference tasks and non-inference tasks.

[0139] Based on the adjusted datasets of each preset category corresponding to the reasoning task, the training dataset of the reasoning task is obtained, and the preset model is trained on the reasoning task; and based on the adjusted datasets of each preset category corresponding to the non-reasoning task, the training dataset of the non-reasoning task is obtained, and the preset model is trained on the non-reasoning task, so that the preset model is trained to be a model with general input and general output capabilities.

[0140] The total amount of training data for reasoning tasks is basically the same as the total amount of training data for non-reasoning tasks; the reasoning tasks contain training data for multiple preset categories, and the amount of training data for each preset category is basically the same; the non-reasoning tasks contain training data for multiple preset categories, and the amount of training data for each preset category is basically the same.

[0141] For example, in scenarios where it is necessary to improve the overall capabilities of the model, this embodiment divides the model training tasks into two categories: one is reasoning tasks that require analysis and the other is simple non-reasoning tasks. The corresponding categories can be determined for these two types of tasks based on the adjusted dataset of the aforementioned multiple preset categories.

[0142] For example, reasoning tasks, which require the model to perform logical reasoning, analysis, and solve complex problems, can be categorized into the aforementioned 33 categories: mathematics, reasoning, code generation, and code debugging. Non-reasoning tasks, such as information retrieval, content generation, translation, and summarization, can be categorized into the 29 categories excluding mathematics, reasoning, code generation, and code debugging.

[0143] For example, such as Figure 2 The diagram shown is a schematic of constructing training data according to an embodiment of this specification. Taking 33 categories as an example, the reasoning task corresponds to categories 1 to 4, a total of 4 categories, while the non-reasoning task corresponds to categories 5 to 33, a total of 29 categories; the amount of data in the adjusted dataset under each category may be different.

[0144] Because the model has different data requirements for reasoning tasks and non-reasoning tasks, this embodiment can adopt the following weighted task allocation method, for example, it can be designed as follows:

[0145] In the overall probability distribution, training data for reasoning tasks accounts for 50% of the total, and training data for non-reasoning tasks accounts for 50%. That is, when constructing training data based on adjusted data from multiple predefined categories, data for reasoning tasks accounts for half, and data for non-reasoning tasks also accounts for half. This ensures that the model has sufficient training for complex reasoning while also performing well in routine tasks, thus achieving a balance in multi-tasking capabilities.

[0146] Within the same type of task, an equal distribution design is adopted, meaning that the amount of training data from each preset category is basically the same within reasoning tasks; similarly, the amount of training data from each preset category is basically the same within non-reasoning tasks. For example, in reasoning tasks, if there are datasets corresponding to 4 different categories, the amount of training data from each of the 4 categories is basically the same. In non-reasoning tasks, if there are datasets corresponding to 29 different categories, the amount of training data from each of the 29 categories is basically the same.

[0147] Assuming 10,000 training data points are needed, the reasoning task requires 5,000 data points, which corresponds to 4 categories, with each category having 5,000 / 4 data points. The non-reasoning task requires 5,000 data points, which corresponds to 29 categories, with each category having 5,000 / 29 data points.

[0148] The above design can be expressed by the following formula:

[0149] P(T i |T i ∈Reasoning Tasks)=α i

[0150] P(T i |T i ∈Non-Reasoning Tasks)=β i

[0151] ∑α i +∑β i =1

[0152]

[0153]

[0154] P refers to the probability function, T i Represents the i-th category; αi Representing reasoning tasks T i The sampling probability; β i Represents a non-reasoning task T i The sampling probability;

[0155] For each category corresponding to a reasoning task, any category T i The probability is α i .

[0156] For each category corresponding to a non-reasoning task, any category T i The probability is β i .

[0157] The sum of probabilities for all reasoning tasks and the sum of probabilities for non-reasoning tasks each account for 50% of the total probability. That is, when extracting training data, the amount of data for reasoning tasks accounts for half, and the amount of data for non-reasoning tasks accounts for half. Within the same type of task, the probability of each preset category is equal.

[0158] Based on this, the preset model can be trained on the training dataset for reasoning tasks. Similarly, the preset model can be trained on the training dataset for non-reasoning tasks. Therefore, the model trained in this embodiment possesses comprehensive capabilities, meaning it exhibits superior performance across multiple tasks.

[0159] As an example, in practical applications, when a model faces training on multiple training tasks, the training data from two types of tasks can be mixed together for fine-tuning as needed. Of course, other methods are also optional in practical applications, and this embodiment does not limit this approach.

[0160] In other examples, the user's model training objective may be to achieve general input and specific output capabilities; that is, the model can accept input from multiple categories of data, but the output requirement is singular. For example, the model's training task may be a text summarization task, where the model can perform text summarization on various categories of data, such as summarizing mathematical data, fiction data, and medical data.

[0161] Based on this, in this embodiment, determining at least one training task of the preset model and training the preset model based on an adjusted dataset of preset categories corresponding to the training task includes:

[0162] Obtain the current training objective of the preset model. If the current training objective is to train a model with general input and specific output capabilities, determine that the training task of the preset model includes specific training tasks.

[0163] Based on the adjusted dataset of the at least one preset category, after obtaining the training dataset corresponding to the specific training task, the preset model is trained for the specific training task so that the preset model is trained to be a model with general input and specific output capabilities.

[0164] The training dataset for the specific training task contains training data for multiple preset categories, and the amount of training data for each preset category is basically the same.

[0165] In this embodiment, the goal is not to enhance the model's multi-task capabilities, but rather to ensure that a single, specific training task can be applied to general scenarios. In this case, the actual data distribution needs to be considered, and data from multiple categories should be sampled evenly. Although the output task is singular, to ensure the model can efficiently perform this task under various input scenarios, the training data needs to include diverse input types. This can be achieved using uniform multi-task sampling, i.e., sampling different types of data evenly within a single task.

[0166] As an example, in the application of this embodiment, a single task that needs to train the preset model can be determined, which is referred to as the target training task in this embodiment. The dataset of the target training task can contain adjusted datasets from different preset categories, and data can be sampled evenly from the adjusted datasets of each preset category to make the training data in each domain basically the same.

[0167] The above design can be expressed by the following formula:

[0168]

[0169] Here, N represents the N preset categories corresponding to a specific training task, such as the 33 preset categories mentioned above. An equal number of data points can be sampled from each preset category to construct a training dataset with roughly the same amount of training data for each category, and then training for the specific training task can be performed.

[0170] Optionally, since the adjusted datasets for each category are constructed based on massive amounts of data, the label of each data point may not be applicable to the specific training task. Therefore, this embodiment can also label each training data point in the training dataset with a corresponding label as needed, and the specific label can be configured according to the specific training task.

[0171] For example, for the adjusted dataset of the novel category, if the specific training task is text summarization, then a corresponding summary text label needs to be generated for each data point; if the specific training task is rewriting, then a corresponding rewriting text label needs to be generated for each data point.

[0172] In this way, the model can perform a single task under different types of inputs, improving its applicability in diverse application scenarios.

[0173] In some examples, determining at least one training task for the preset model, and training the preset model based on an adjusted dataset of preset categories corresponding to the training task, may include:

[0174] Obtain the current training objective of the preset model. If the current training objective is to train a model with specific input and specific output capabilities, determine that the training task of the preset model includes the specific training task.

[0175] From the adjusted datasets of the at least one preset category, determine the adjusted dataset of the specific category corresponding to the specific category input;

[0176] If the amount of data in the adjusted dataset of the specific category is less than the second quantity threshold, training data with a data amount that is basically the same as that in the adjusted dataset of the specific category shall be obtained from the adjusted datasets of other preset categories.

[0177] Using the adjusted dataset of the specific category and the acquired training data, the preset model is trained for the specific training task, so that the preset model is trained to be a model with specific category input and specific output capabilities.

[0178] As an example, the second quantity threshold can be set as needed, such as a value of 2000.

[0179] In this embodiment, a model capable of specific input and output categories can be trained using an adjusted dataset of any of the aforementioned preset categories. For example, a model with historical knowledge-based question-answering capabilities can be trained using an adjusted dataset of the historical category; a model with legal knowledge-based question-answering capabilities can be trained using an adjusted dataset of the legal category. In other words, the model is trained for a single task using a single type of data.

[0180] Based on this, in this embodiment, when the current training objective is to train data with specific category input and specific output capabilities, an adjusted dataset for the specific category corresponding to the specific category input can be determined. Considering that in practical applications, the amount of data in the adjusted dataset for a specific category may be relatively small, this embodiment proposes using general data augmentation for such low-data-volume scenarios in single-task low-resource scenarios. When the amount of data in a low-resource scenario is below a threshold, this embodiment can mix in an equal amount of other data before training, which is also helpful for the effect of a single task. For example, training data with a data volume that is basically the same as the adjusted dataset for the specific category can be obtained from other preset category adjusted datasets. For example, it can be expressed by the following formula:

[0181]

[0182] In this implementation, TargetTasks refers to the training dataset of a specific category corresponding to a specific training task, while GeneralTasks refers to datasets from other categories. The training data corresponding to TargetTasks accounts for half of the total training data; the training datasets from other categories account for the other half of the total training data. The training data from other categories can be of one category or multiple categories; this embodiment does not limit this. Thus, this embodiment designs a mixture of data from other categories in scenarios with low training data volume, ensuring the model has sufficient data for training and improving the model's training performance.

[0183] Corresponding to the embodiments of the aforementioned model training methods, this specification also provides embodiments of the model training apparatus and the computer equipment used thereon.

[0184] The embodiments of the model training device described in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by its processor reading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 3 The diagram shown is a hardware structure diagram of the computer device on which the model training device of this manual is located. (Except for...) Figure 3 In addition to the processor 310, network interface 320, memory 330, and non-volatile memory 340 shown, the computer device where the model training device is located in the embodiment may also include other hardware depending on the actual function of the computer device, which will not be described in detail here.

[0185] like Figure 4 As shown, Figure 4This is a block diagram illustrating a model training apparatus according to an exemplary embodiment of this specification, the apparatus comprising:

[0186] The acquisition module 41 is used to: acquire an initial dataset corresponding to each of the at least one preset category.

[0187] The adjustment module 42 is configured to: if it is determined that the initial dataset does not meet the preset balance condition, adjust the initial dataset so that the adjusted dataset meets the balance condition. The balance condition includes one or more of the following: a balance of data volume in different languages, a balance of data volume in different lengths, or a balance of data volume in different data complexities.

[0188] Training module 43 is used to: determine at least one training task of the preset model, and train the preset model based on an adjusted dataset of preset categories corresponding to the training task.

[0189] In some examples, the acquisition module 41 is also used for:

[0190] Acquire multiple source data, and select a portion of target source data from the multiple source data;

[0191] Using the target source data and preset prompt information, prompt content is constructed and input into a preset language model. Based on the response of the preset language model, the preset category to which the target source data belongs is determined. The preset prompt information is used to instruct the preset language model to identify the preset category to which the target source data belongs from at least one preset category.

[0192] Based on the target source data and its preset category, a preset data classification model is trained;

[0193] The trained data classification model is used to identify the preset category to which the source data belongs, thereby obtaining an initial dataset corresponding to each of the at least one preset category.

[0194] In some examples, the conditions for balancing the amount of data in different languages ​​in the dataset include: the amount of data in each language in the dataset is basically the same;

[0195] The adjustment module 42 is also used for:

[0196] Obtain the language of the data in the initial dataset and the amount of data for each language;

[0197] If there is a target language with a lower data volume than other languages, use the data from other languages ​​to generate extended data corresponding to the target language, so that the data volume of each language in the adjusted dataset is basically the same.

[0198] In some examples, the conditions for balancing the amount of data in the dataset at different lengths include: the amount of data in the dataset across multiple preset length intervals conforms to a normal distribution;

[0199] The adjustment module 42 is also used for:

[0200] The amount of data in the initial dataset in the multiple preset length intervals is determined based on the text length of the data in the initial dataset.

[0201] If it is determined that the amount of data in the initial dataset does not conform to a normal distribution in the multiple length intervals, one or more target length intervals are determined, and similar extended data is generated based on the data in the target length intervals so that the amount of data in the adjusted dataset conforms to a normal distribution in the multiple length intervals.

[0202] In some examples, the plurality of length intervals includes: a very short interval and a very long interval; wherein, the very short interval refers to a length interval in which the data length is greater than the mean text length of the initial dataset minus twice the standard deviation; and the very long interval refers to a length interval in which the data length is greater than the mean text length of the initial dataset plus twice the standard deviation.

[0203] The adjustment module 42 is also used for:

[0204] Based on the mean and standard deviation of the text length of the initial dataset, determine the expected data volume of the extremely short interval and the expected data volume of the extremely long interval under the normal distribution;

[0205] If the actual data volume of the extremely short interval is less than the expected data volume of the extremely short interval, similar extended data is generated based on the data of the extremely short interval so that the actual data volume of the extended extremely short interval is basically the same as the expected data volume of the extremely short interval.

[0206] If the actual data volume of the extremely long interval is less than the expected data volume of the extremely long interval, similar extended data is generated based on the data of the extremely long interval so that the actual data volume of the extended extremely long interval is basically the same as the expected data volume of the extremely long interval.

[0207] In some examples, the adjustment module 42 is also used for:

[0208] Obtain the complexity of the data in the initial dataset and sort them according to complexity;

[0209] An empty target set is initialized, and data is extracted from the initial dataset in descending order of complexity. The first extracted data is stored in the target set. For each data extracted after the first data, if the similarity between the extracted data and its adjacent data in the target set is less than or equal to a preset similarity threshold, the extracted data is placed in the target set. The adjacent data is the data in the target set whose complexity is closest to that of the extracted data.

[0210] Once the amount of data stored in the target set reaches a first threshold, the target set is used as an adjusted dataset so that the adjusted dataset satisfies the condition of balancing the amount of data in datasets with different data complexities.

[0211] In some examples, the training module 43 is used for:

[0212] Obtain the current training objective of the preset model. If the current training objective is to train a model with general input and general output capabilities, determine that the training tasks of the preset model include inference tasks and non-inference tasks.

[0213] Based on the adjusted datasets of each preset category corresponding to the reasoning task, the training dataset of the reasoning task is obtained, and the preset model is trained on the reasoning task; and based on the adjusted datasets of each preset category corresponding to the non-reasoning task, the training dataset of the non-reasoning task is obtained, and the preset model is trained on the non-reasoning task, so that the preset model is trained to be a model with general input and general output capabilities.

[0214] The total amount of training data for reasoning tasks is basically the same as the total amount of training data for non-reasoning tasks; the reasoning tasks contain training data for multiple preset categories, and the amount of training data for each preset category is basically the same; the non-reasoning tasks contain training data for multiple preset categories, and the amount of training data for each preset category is basically the same.

[0215] In some examples, the training module 43 is used for:

[0216] Obtain the current training objective of the preset model. If the current training objective is to train a model with general input and specific output capabilities, determine that the training task of the preset model includes specific training tasks.

[0217] Based on the adjusted dataset of the at least one preset category, after obtaining the training dataset corresponding to the specific training task, the preset model is trained for the specific training task so that the preset model is trained to be a model with general input and specific output capabilities.

[0218] The training dataset for the specific training task contains training data for multiple preset categories, and the amount of training data for each preset category is basically the same.

[0219] In some examples, the training module 43 is used for:

[0220] Obtain the current training objective of the preset model. If the current training objective is to train a model with specific input and specific output capabilities, determine that the training task of the preset model includes the specific training task.

[0221] From the adjusted datasets of the at least one preset category, determine the adjusted dataset of the specific category corresponding to the specific category input;

[0222] If the amount of data in the adjusted dataset of the specific category is less than the second quantity threshold, training data with a data amount that is basically the same as that in the adjusted dataset of the specific category shall be obtained from the adjusted datasets of other preset categories.

[0223] Using the adjusted dataset of the specific category and the acquired training data, the preset model is trained for the specific training task, so that the preset model is trained to be a model with specific category input and specific output capabilities.

[0224] The specific implementation process of the functions and roles of each module in the above-mentioned model training device can be found in the implementation process of the corresponding steps in the above-mentioned model training method, and will not be repeated here.

[0225] Accordingly, this specification also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned model training method embodiments.

[0226] Accordingly, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the model training method embodiment.

[0227] Accordingly, embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the model training method embodiments.

[0228] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0229] The above embodiments can be applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the computer device includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0230] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0231] The computer equipment may also include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0232] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0233] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0234] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0235] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0236] The terms "specific example" or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with the embodiments or examples, which are included in at least one embodiment or example of this specification. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0237] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.

[0238] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

[0239] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A model training method, the method comprising: Based on at least one preset category, obtain an initial dataset corresponding to each of the at least one preset category; If it is determined that the initial dataset does not meet the preset balance condition, the initial dataset is adjusted so that the adjusted dataset meets the balance condition; wherein, the balance condition includes one or more of the following: the condition of balancing the amount of data in different languages, the condition of balancing the amount of data in different lengths, or the condition of balancing the amount of data in different data complexities. Determine at least one training task for the preset model, and train the preset model based on an adjusted dataset of preset categories corresponding to the training task.

2. The method according to claim 1, wherein obtaining an initial dataset corresponding to each of the at least one preset category comprises: Acquire multiple source data, and select a portion of target source data from the multiple source data; Using the target source data and preset prompt information, prompt content is constructed and input into a preset language model. Based on the response of the preset language model, the preset category to which the target source data belongs is determined. The preset prompt information is used to instruct the preset language model to identify the preset category to which the target source data belongs from at least one preset category. Based on the target source data and its preset category, a preset data classification model is trained; The trained data classification model is used to identify the preset category to which the source data belongs, thereby obtaining an initial dataset corresponding to each of the at least one preset category.

3. The method according to claim 1, wherein the conditions for balancing the data volume of the dataset in different languages ​​include: The amount of data for each language in the dataset is roughly the same. If it is determined that the initial dataset does not meet the preset balance condition, adjusting the initial dataset to make the adjusted dataset meet the balance condition includes: Obtain the language of the data in the initial dataset and the amount of data for each language; If there is a target language with a lower data volume than other languages, use the data from other languages ​​to generate extended data corresponding to the target language, so that the data volume of each language in the adjusted dataset is basically the same.

4. The method according to claim 1, wherein the conditions for balancing the data volume of the dataset at different lengths include: The dataset contains data in multiple preset length intervals that conform to a normal distribution. If it is determined that the initial dataset does not meet the preset balance condition, adjusting the initial dataset to make the adjusted dataset meet the balance condition includes: Based on multiple preset length intervals and the text length of the data in the initial dataset, the actual data volume of the initial dataset in the multiple length intervals is determined; If it is determined that the actual amount of data in the initial dataset does not conform to a normal distribution in the multiple length intervals, one or more target length intervals are determined, and similar extended data is generated based on the data in the target length intervals so that the amount of data in the adjusted dataset in the multiple length intervals conforms to a normal distribution.

5. The method according to claim 4, wherein the plurality of length intervals comprises: Extremely short intervals and extremely long intervals; wherein, the extremely short interval refers to a data length greater than the mean text length of the initial dataset minus twice the standard deviation; the extremely long interval refers to a data length greater than the mean text length of the initial dataset plus twice the standard deviation; If it is determined that the amount of data in the initial dataset does not conform to a normal distribution in the multiple length intervals, one or more target length intervals are determined, and similar extended data is generated based on the data in the target length intervals, including: Based on the mean and standard deviation of the text length of the initial dataset, determine the expected data volume of the extremely short interval and the expected data volume of the extremely long interval under the normal distribution; If the actual data volume of the extremely short interval is less than the expected data volume of the extremely short interval, similar extended data is generated based on the data of the extremely short interval so that the actual data volume of the extended extremely short interval is basically the same as the expected data volume of the extremely short interval. If the actual data volume of the extremely long interval is less than the expected data volume of the extremely long interval, similar extended data is generated based on the data of the extremely long interval so that the actual data volume of the extended extremely long interval is basically the same as the expected data volume of the extremely long interval.

6. The method according to claim 1, wherein adjusting the initial dataset to make the adjusted dataset satisfy the balance condition comprises: Obtain the complexity of the data in the initial dataset and sort them according to complexity; An empty target set is initialized, and data is extracted from the initial dataset in descending order of complexity. The first extracted data is stored in the target set. For each data extracted after the first data, if the similarity between the extracted data and its adjacent data in the target set is less than or equal to a preset similarity threshold, the extracted data is placed in the target set; otherwise, it is not placed. The adjacent data is the data in the target set whose complexity is closest to that of the extracted data. Once the amount of data stored in the target set reaches a first threshold, the target set is used as an adjusted dataset so that the adjusted dataset satisfies the condition of balancing the amount of data in datasets with different data complexities.

7. The method according to claim 1, wherein determining at least one training task of the preset model and training the preset model based on an adjusted dataset of preset categories corresponding to the training task comprises: Obtain the current training objective of the preset model. If the current training objective is to train a model with general input and general output capabilities, determine that the training tasks of the preset model include inference tasks and non-inference tasks. Based on the adjusted datasets of each preset category corresponding to the reasoning task, the training dataset of the reasoning task is obtained, and the preset model is trained on the reasoning task; and based on the adjusted datasets of each preset category corresponding to the non-reasoning task, the training dataset of the non-reasoning task is obtained, and the preset model is trained on the non-reasoning task, so that the preset model is trained to be a model with general input and general output capabilities. The total amount of training data for reasoning tasks is basically the same as the total amount of training data for non-reasoning tasks; the reasoning tasks contain training data for multiple preset categories, and the amount of training data for each preset category is basically the same; the non-reasoning tasks contain training data for multiple preset categories, and the amount of training data for each preset category is basically the same.

8. The method according to claim 1, wherein determining at least one training task of the preset model and training the preset model based on an adjusted dataset of preset categories corresponding to the training task comprises: Obtain the current training objective of the preset model. If the current training objective is to train a model with general input and specific output capabilities, determine that the training task of the preset model includes specific training tasks. Based on the adjusted dataset of the at least one preset category, after obtaining the training dataset corresponding to the specific training task, the preset model is trained for the specific training task so that the preset model is trained to be a model with general input and specific output capabilities. The training dataset for the specific training task contains training data for multiple preset categories, and the amount of training data for each preset category is basically the same.

9. The method according to claim 1, wherein determining at least one training task of the preset model and training the preset model based on an adjusted dataset of preset categories corresponding to the training task comprises: Obtain the current training objective of the preset model. If the current training objective is to train a model with specific input and specific output capabilities, determine that the training task of the preset model includes the specific training task. From the adjusted datasets of the at least one preset category, determine the adjusted dataset of the specific category corresponding to the specific category input; If the amount of data in the adjusted dataset of the specific category is less than the second quantity threshold, training data with a data amount that is basically the same as that in the adjusted dataset of the specific category shall be obtained from the adjusted datasets of other preset categories. Using the adjusted dataset of the specific category and the acquired training data, the preset model is trained for the specific training task, so that the preset model is trained to be a model with specific category input and specific output capabilities.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

11. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

Images