Model training method and apparatus, and computing device cluster

Abstract

A model training method, comprising: constructing a training data set required for the current iteration of training, and training a first model on the basis of the training data set to obtain a second model; constructing a regression test set required for the current iteration of evaluation, and evaluating the second model on the basis of the regression test set to obtain a first evaluation result; and on the basis of the first evaluation result and a second evaluation result, assessing the correctness of the second model in an optimization direction of the current iteration, wherein the second evaluation result is obtained by evaluating the second model on the basis of a general test set. In this way, during model training, on the basis of the regression test set constructed using sampled live network data in each iteration and a general evaluation set containing assessments of various basic capabilities, two levels of tests are performed to fully assess the correctness in the optimization direction and guide follow-up planning, thereby preventing bias in the optimization direction and degradation of model performance.

Description

Model training method and device and computing device cluster

[0001] The present application claims priority to the Chinese patent application No. 2024112575465, filed on September 6, 2024, entitled "A model training method, device and computing device cluster", the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a model training method, device and computing device cluster. BACKGROUND

[0003] At present, generative large language models (LLMs) represented by chat generative pre-trained transformers (ChatGPT) have made significant research progress in natural language understanding and generation. Such large language models have excellent language understanding and generation capabilities, rich knowledge reserves, and high instruction compliance, enabling them to effectively perform complex analysis and reasoning tasks. The training of large language models usually includes two steps of pre-training and fine-tuning. In some practical application scenarios, due to the need for knowledge updating, error correction, etc., continuous training iteration of existing large language models is required, which usually includes two key steps of optimization direction exploration (evaluation set formulation and updating, evaluation execution and problem analysis) and solution formulation (training set construction and model training). However, in the traditional solution, the cost of manually customizing the evaluation set is high and the efficiency is low, resulting in a long time to fix an evaluation set or slow updating, which is prone to overfitting the evaluation set, causing the large language model training optimization direction to be biased. SUMMARY

[0004] The present application provides a model training method, device, computing device cluster, computer storage medium and computer product, which can ensure the correctness of the optimization direction of each iteration in the model training process.

[0005] In a first aspect, the present application provides a model training method, comprising: constructing a training data set required for the current iteration training, and training a first model based on the training data set to obtain a second model; constructing a regression test set required for the current iteration evaluation, and evaluating the second model based on the regression test set to obtain a first evaluation result; and evaluating the correctness of the second model in the optimization direction of the current iteration based on the first evaluation result and a second evaluation result, wherein the second evaluation result is obtained by evaluating the second model based on a general test set.

[0006] In this way, in the model training process, the regression test set constructed based on the sampled live network data in each iteration and the general evaluation set containing various basic capability evaluations are used to perform two-level testing to fully evaluate the correctness of the optimization direction and guide the next step of planning, so that the optimization direction can be prevented from being biased and the model effect can be prevented from being degraded.

[0007] In a possible implementation, based on the first evaluation result and the second evaluation result, the correctness of the second model in the optimization direction of the current iteration is evaluated, including: in the case that the first evaluation result is better than the third evaluation result and the second evaluation result is better than the fourth evaluation result, it is determined that the second model is correct in the optimization direction of the current iteration, and the current training and evaluation is ended, and the next round of model training and evaluation is entered, wherein the third evaluation result is obtained by evaluating the first model based on the regression test set, and the fourth evaluation result is obtained by evaluating the first model based on the general test set; in the case that the first evaluation result is better than the third evaluation result and the second evaluation result is worse than the fourth evaluation result, it is determined that the second model is incorrect in the optimization direction of the current iteration, and the training data set is reconstructed, and the first model is trained; in the case that the first evaluation result is worse than the third evaluation result, it is determined that the second model is incorrect in the optimization direction of the current iteration, and the regression test set is reconstructed, and the second model is evaluated. In this way, by comparing the two types of evaluation results, the correctness of the model in the optimization direction of the current iteration can be determined, and the next step of planning can be automatically made based on the evaluation results, so that the optimization direction can be prevented from being biased and the model effect can be prevented from being degraded.

[0008] In a possible implementation, the regression test set required for the current iteration evaluation is constructed, including: constructing a first test set based on live network data; evaluating the data in the first test set to obtain a wrong example evaluation report, the wrong example evaluation report including the scores of the data in the first test set in at least one dimension and the error description of the data in the first test set; based on the wrong example evaluation report, the regression test set and the first wrong example data set are selected from the error data in the first test set, wherein the first wrong example data set is used to construct the training data set required for the next iteration training. In this way, different regression test sets can be constructed in different iterations, so that the problems existing in the model can be better found, and the optimization direction can be prevented from being biased and the model effect can be prevented from being degraded.

[0009] In a possible implementation, based on the live data, the first test set is constructed, including: based on a first data set in a regression test set constructed in a last iteration, sampling from the live data to obtain a second data set, wherein the first data set is a failed example in the regression test set constructed in the last iteration indicated by a third evaluation result, the third evaluation result is obtained by evaluating the model trained in the last iteration based on the regression test set constructed in the last iteration; and taking the first data set and the second data set as the first test set. In this way, the regression test set can be constructed in a targeted manner, and the correctness of the test can be improved.

[0010] In a possible implementation, the training data set required for the current iteration training is constructed, including: performing error root cause analysis on the data in the second error example data set obtained by evaluating the last iteration to obtain an error root cause analysis result; based on the error root cause analysis result, correcting the data in the second error example data set to obtain a third data set; based on the error root cause analysis result and the second error example data set, generalizing to obtain an error example generalization data set; sampling from the general test set to obtain a fourth data set; and taking the third data set, the fourth data set, and the error example generalization data set as the training data set. In this way, the error example found by the evaluation based on the large language model is automatically corrected, and the result of the error root cause analysis is used to automatically generalize new data with similar error root causes, thereby constructing the training data set required for the current iteration, and realizing efficient and targeted training data construction and large model iteration.

[0011] In a possible implementation, before performing error root cause analysis on the data in the second error example data set obtained by evaluating the last iteration, the method further includes: clustering the data in the second error example data set to obtain error example clustering clusters and / or discrete error examples, wherein the error root cause analysis result is a summary of the error root causes of the error example clustering clusters and the discrete error examples. In this way, similar error examples can be uniformly analyzed by clustering, and the error root cause analysis efficiency is improved.

[0012] In a second aspect, the present application provides a model training device, including: a training module and an evaluation module. The training module is configured to construct a training data set required for the current iteration training, and train a first model based on the training data set to obtain a second model. The evaluation module is configured to construct a regression test set required for the current iteration evaluation, and evaluate the second model based on the regression test set to obtain a first evaluation result; and based on the first evaluation result and a second evaluation result, evaluate the correctness of the second model in the optimization direction of the current iteration, wherein the second evaluation result is obtained by evaluating the second model based on a general test set.

[0013] In a possible implementation, when evaluating the correctness of the second model in the optimization direction of the current iteration based on the first evaluation result and the second evaluation result, the evaluation module is specifically configured to: in the case that the first evaluation result is better than the third evaluation result and the second evaluation result is better than the fourth evaluation result, determine that the second model is correct in the optimization direction of the current iteration, and end the current training and evaluation, and enter the next round of model training and evaluation, wherein the third evaluation result is obtained by evaluating the first model based on the regression test set, and the fourth evaluation result is obtained by evaluating the first model based on the general test set; in the case that the first evaluation result is better than the third evaluation result and the second evaluation result is worse than the fourth evaluation result, determine that the second model is incorrect in the optimization direction of the current iteration, and reconstruct the training data set and train the first model again; in the case that the first evaluation result is worse than the third evaluation result, determine that the second model is incorrect in the optimization direction of the current iteration, and reconstruct the regression test set and evaluate the second model again.

[0014] In a possible implementation, when constructing the regression test set required for the current iteration evaluation, the evaluation module is specifically configured to: construct a first test set based on the online data; evaluate the data in the first test set to obtain a misexample evaluation report, the misexample evaluation report including the scores of the data in the first test set in at least one dimension and the error descriptions of the data in the first test set; and select the regression test set and a first misexample data set from the error data in the first test set based on the misexample evaluation report, wherein the first misexample data set is used to construct the training data set required for the next iteration training.

[0015] In a possible implementation, when constructing the first test set based on the online data, the evaluation module is specifically configured to: sample the second data set from the online data based on the first data set in the regression test set constructed in the last iteration, wherein the first data set is the failed example in the regression test set constructed in the last iteration indicated by the third evaluation result, and the third evaluation result is obtained by evaluating the model trained in the last iteration based on the regression test set constructed in the last iteration; and take the first data set and the second data set as the first test set.

[0016] In a possible implementation, when constructing the training data set required for the current iteration training, the training module is specifically configured to: perform error root cause analysis on the data in the second misexample data set obtained by the evaluation in the last iteration to obtain an error root cause analysis result; correct the data in the second misexample data set based on the error root cause analysis result to obtain a third data set; generalize the error root cause analysis result and the second misexample data set to obtain a misexample generalized data set; sample a fourth data set from the general test set; and take the third data set, the fourth data set, and the misexample generalized data set as the training data set.

[0017] In a possible implementation, before performing error root cause analysis on the data in the second error example data set obtained by evaluating the previous round of iteration, the training module is further configured to: perform clustering on the data in the second error example data set to obtain error example clustering clusters and / or discrete error examples, and the error root cause analysis result is an error root cause summary of the error example clustering clusters and the discrete error examples.

[0018] In a third aspect, the present application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect.

[0019] In a fourth aspect, the present application provides a computer readable storage medium, including computer program instructions, when the computer program instructions are executed by a computing device, the computing device executes the method described in the first aspect or any possible implementation of the first aspect; or, when the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect. Illustratively, the computing device cluster can include one or more computing devices.

[0020] In a fifth aspect, the present application provides a computer program product including instructions, when the instructions are executed by a computing device, the computing device executes the method described in the first aspect or any possible implementation of the first aspect, or, when the instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect. Illustratively, the computing device cluster can include one or more computing devices.

[0021] It can be understood that the beneficial effects of the above-mentioned second aspect to the fifth aspect can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0022] FIG. 1 is an architecture schematic diagram of a model training system according to an embodiment of the present application;

[0023] FIG. 2 is a working process schematic diagram of an evaluation studio in FIG. 1;

[0024] FIG. 3 is a working process schematic diagram of a development studio in FIG. 1;

[0025] FIG. 4 is a schematic diagram of a tenant interacting with a cloud computing platform according to an embodiment of the present application;

[0026] FIG. 5 is a flow schematic diagram of a model training method according to an embodiment of the present application;

[0027] FIG. 6 is a schematic diagram of a step of constructing a training data set required for the current iteration according to an embodiment of the present application;

[0028] FIG. 7 is a schematic diagram of a structure of a model training apparatus according to an embodiment of the present application;

[0029] FIG. 8 is a schematic diagram of a structure of a computing device according to an embodiment of the present application;

[0030] FIG. 9 is a schematic diagram of a structure of a computing device cluster according to an embodiment of the present application;

[0031] FIG. 10 is a schematic diagram of a structure of another computing device cluster according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] The term “and / or” in the present document is used to describe an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. The symbol “ / ” in the present document means an or relationship of associated objects, for example, A / B means A or B.

[0033] The terms “first” and “second” and the like in the description and claims of the present document are used to distinguish different objects, and are not used to describe a specific order of the objects. For example, the first response message and the second response message are used to distinguish different response messages, and are not used to describe a specific order of the response messages.

[0034] In the embodiments of the present application, the words “exemplary” or “for example” are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as “exemplary” or “for example” in the embodiments of the present application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of “exemplary” or “for example” is intended to present concepts in a concrete manner.

[0035] In the description of the embodiments of the present application, unless otherwise specified, “a plurality of” means two or more, for example, a plurality of processing units means two or more processing units, and the like; a plurality of elements means two or more elements, and the like.

[0036] First, the related technical terms involved in the technical solutions provided by the present application are introduced.

[0037] (1) Large model

[0038] The large model refers specifically to a large language model, which is a deep learning model trained based on massive text data. It learns the structure, rules, and semantics of language through training, and can generate text with natural language style or answer natural language questions.

[0039] (2) Automatic evaluation

[0040] Automatic evaluation of large language models refers to the method of systematically evaluating the performance of large-scale pre-trained language models using automated techniques. This process typically involves comprehensive analysis of multiple indicators, through automated tools, algorithms or large language models as evaluators, which can efficiently and objectively measure the model's performance in natural language processing tasks, thereby providing the basis for further optimizing the model. This evaluation method not only improves the efficiency of evaluation, but also reduces the interference of human subjective factors, improves the reliability and repeatability of evaluation results.

[0041] (3) Data augmentation and generalization

[0042] Data augmentation and generalization refer to methods in the field of machine learning and artificial intelligence that expand and transform the original data set to improve the generalization ability and robustness of the model. Data generalization involves generating new data points to cover a wider input space, thereby making the model more universal and adaptable, reducing overfitting and improving the model's performance under different tasks and data distributions. The two work together to generate richer and more diverse training data, thereby improving model performance and practical application effects.

[0043] (4) Large language model self-improvement

[0044] Large language model self-improvement refers to the process of large-scale artificial intelligence models continuously improving their performance and intelligence level through self-analysis, feedback, and optimization mechanisms. It usually relies on self-supervised learning, meta-learning, and feedback loop mechanisms in deep learning technology, aiming to enable the model to fully utilize existing data and experience for self-optimization without relying on external intervention. This process helps the large model maintain high-level intelligent performance in a dynamic environment.

[0045] The technical solutions provided by the present application are described below.

[0046] For example, Figure 1 shows an architecture diagram of a model training system provided by an embodiment of the present application. As shown in Figure 1, the model training system can include an evaluation studio and a development studio.

[0047] Among them, the main tasks of the evaluation studio include automatic sampling to build a test set through live network data, automatic evaluation, regression testing, etc. The evaluation studio mainly includes three processes: evaluation planning, evaluation execution and summary report, and regression testing. Among them, the evaluation planning process is mainly used to build a test set. In the process of building a test set, random sampling can be performed based on live user backflow data (i.e., live network data) in the cold start phase; in other phases, failed test cases of regression testing and resampling based on problem types, etc. can be added. The evaluation execution and summary report process is mainly used to automatically evaluate the built test set through a large language model, and obtain an evaluation report based on error analysis. The regression testing process is mainly used to perform two-level regression testing (this round of regression test set + general evaluation dataset), and both levels pass to enter the next round. Among them, the this round of regression test set can be obtained through the test set built in the evaluation planning process. In addition, the test failed in the last round of regression test set can also be added to the this round of regression test set. In this embodiment, the general evaluation dataset can be, but is not limited to, a long-fixed, artificially prepared test set for evaluating the capabilities of a large language model, which can ensure the basic capabilities of the model. For example, the general evaluation dataset can also be referred to as a "general test set". The regression test set is variable, and the regression test set of each round can be different. The regression test set can be used to find problems existing in the model, and can better guide the model to optimize in the correct direction.

[0048] The main tasks of the development studio include error example root cause analysis, difficult example generalization, training data preparation, model training, etc. based on the test results of the evaluation studio. The development studio mainly includes two stages: training preparation and model training. Among them, in the training preparation stage, the training data set required for the next round of iteration is mainly constructed. In this process, based on the error analysis in the evaluation report obtained in the evaluation studio, the large language model is used to repair the error examples and add them to the training data set; and further error root cause analysis is performed through clustering and the large language model, and instances with similar error root causes are generalized and added to the training data set. In the model training stage, the model is automatically trained based on the new round of training data.

[0049] The specific implementation schemes of the evaluation studio and the development studio are introduced below.

[0050] (1) Evaluation studio

[0051] In each iteration of the round, the evaluation studio can take online data, the large language model to be tested, the last round of regression test report as input, and output the current round of error example evaluation report and regression test report. In the evaluation studio, automated evaluation and error analysis can be realized to improve efficiency and reduce cost. Among them, the error example evaluation report provides guidance for the preparation of training data for the subsequent development studio of the current round of iteration, while the regression test report can be used as a basis for judging the correctness of the optimization direction of the current round, and guiding the preparation of the evaluation set (also known as "test set") of the next round of iteration. For example, as shown in FIG. 2, the specific steps involved in the evaluation studio can include:

[0052] In S21, the test set of the current round of iteration is prepared. In this step, a certain amount of online backflow data (i.e., online data) of the large language model to be tested needs to be accumulated, each data including a user question, a reply to the user question, and a user question type label (optional). Among them, the user question type label can be automatically obtained based on a pre-trained deep learning classification model. When cold starting (i.e., when the first round of iteration), a uniform random sampling can be directly performed according to the user question type to obtain the test set of the first round of iteration. For example, the current round can also be referred to as "this round".

[0053] In S22, the large language model is used to automatically evaluate each data in the test set determined in S21, and an evaluation report is output. For example, a multi-dimensional (factual, information amount, relevance, fluency, safety, etc.) evaluation template can be designed to realize automatic evaluation of the large model, and the output evaluation report can include the score of each data and the error description of each data in each evaluation dimension.

[0054] In S23, the current round of regression test set is prepared. In this step, based on the evaluation results obtained in S22, the regression test set required for the current round and part of the training data required for the development studio are sampled from the test set determined in S21. For example, data with a score below the qualified line in the automatic evaluation result can be selected, and the selected data can be divided into two parts, one of which is used as the regression test set required for the current round, and the other part can be provided to the development studio for model optimization of the current round (details are described later for the development studio). Among them, the specific scheme of sampling can be: dividing the data with a score below the qualified line according to the user question type, and sampling a fixed proportion (such as 20% or the like) of data in each user question type to add to the regression test set required for the current round, and the remaining part is provided to the development studio.

[0055] At S24, regression testing. In this step, based on the two evaluation results of the general test set and the regression test set prepared in S23, it is determined whether the optimization direction of this round is biased, and the next round of planning is made. Specifically, first, the responses of the large language model after the optimization of the development studio in this round on the general test set and the current round of regression test set are obtained. Then, the responses of each piece of data are scored based on the automatic evaluation method in S22. Finally, the average score change of the large language model before and after optimization on the two data sets is calculated, and the next step is planned according to the score change. Among them, according to the score change to make the next step planning, there are three cases: a) Case one: the average score of the large language model after optimization on the current round of regression test set decreases, and the current round of S23 is rolled back, and the current round of regression test set and the data provided to the development studio are resampled. b) Case two: the average score on the current round of regression test set increases, but the average score of the general test set decreases, and the model optimization of the current round of development studio is rolled back. c) Case three: the average scores of the general test set and the current round of regression test set both increase, and the current round of iteration ends. Exemplarily, a regression test report can be output at S24, which can include the identification of failed examples in the current round of test set, etc.

[0056] When the next step planning made in S24 is the end of the current round of iteration, the next round of iteration can be started from S21 again. In addition, when preparing the test set of the next round, the failed examples (i.e. the data whose scores do not increase) in the regression test set of the last round can be added to the test set of the next round, and the sampling proportion of the problem type of the failed examples of the last round can be increased in the sampling of the test set of the next round, such as sampling more data of the same type.

[0057] In this way, in the evaluation studio, based on the general test set and the regression test set automatically constructed in each round of iteration, two-level testing is performed to determine the correctness of the optimization direction of the large language model, and the next step is automatically planned based on the evaluation result, which can prevent the optimization direction from being biased and the model effect from being degraded.

[0058] (2) Development studio

[0059] In each round of iteration, the development studio mainly takes the large language model to be optimized and the error example evaluation report provided by the evaluation studio of the current round as input, and outputs the large language model after optimization of this round. In the development studio, automatic error example root cause analysis, training data preparation and model training can be realized. Exemplarily, as shown in FIG. 3, the specific steps involved in the development studio can include:

[0060] At S31, error case clustering. For each error case in the error case evaluation report provided by the evaluation studio, a vector can be converted based on the evaluation dimensions and corresponding scores. Then, using a density-based clustering algorithm or the like, the error cases are clustered to obtain a series of error case clustering clusters, and individual difficult-to-cluster discrete error cases.

[0061] At S32, root cause analysis, error case correction, and generalization. A root cause analysis template can be designed, and the error root causes of each clustering cluster and each discrete error case can be automatically summarized by a large language model to obtain error root cause analysis text. The input of the large language model can include one or more of the error case question, the error case answer, and the error description. After obtaining the error root cause analysis text, the original error reply can be automatically corrected and added to the current round training set based on the error root cause analysis text. Then, the error root cause analysis text of each clustering cluster (or discrete error case) and the sampled error cases (or discrete error cases) in the corresponding clustering cluster are input into the large language model for generalization to obtain an error case generalization dataset.

[0062] At S33, training set preparation and model training. In addition to the error case generalization dataset obtained at S32, a basic training set containing general evaluation set evaluation capability data can also be sampled to join the current round model training.

[0063] In this way, in the development studio, through clustering of error cases in the evaluation results of the online data, error root cause analysis, and error case generalization based on the root cause, training data preparation and model training can be completed, automatic improvement of the large language model can be realized, efficiency can be improved, and cost can be reduced.

[0064] The above is an introduction to the model training system provided by the embodiments of the present application. As can be seen from the above description, the model training system simulates the software development process, builds an automated test and development studio, and jointly completes the automatic iterative improvement of the large language model. In the model training system, there are mainly three key technical points: (1) the regression test set constructed based on the sampling of online data in each iteration and the general evaluation set containing various basic ability evaluations are used for two-level testing to fully evaluate the correctness of the optimization direction and guide the next step planning to prevent the optimization direction from being biased and the model effect from being degraded. (2) Based on the large language model, the multi-dimensional automatic evaluation is performed, the multi-dimensional evaluation score results are converted into vectors for clustering, and the large language model is used to automatically summarize the error root causes of each clustering category, providing high-quality analysis results for training set construction. (3) Based on the large language model, the error cases found in the evaluation are automatically corrected, and new data with similar error root causes are automatically generalized using the error root cause analysis results, realizing efficient and targeted training data construction and large model iteration.

[0065] In the model training system described above, after each round of model training in the development studio, the trained model can be evaluated in the evaluation studio. Of course, evaluation can be performed after several rounds. The specific number of rounds can be determined according to actual conditions, which is not limited here.

[0066] It should be understood that the model training system described above can be configured on a cloud computing platform, for example, deployed on at least one virtual machine or container instance, so that the cloud computing platform can provide model training services. Of course, the model training system can also be configured on nodes other than the cloud computing platform, for example, can be deployed in at least one data center, or deployed on at least one server. The specific number of rounds can be determined according to actual conditions, which is not limited here. The cloud computing platform can provide a page related to a public cloud service for a tenant to remotely access the public cloud service. In this embodiment, the tenant (also referred to as "user") can purchase the model training service provided by the model training system in advance on the cloud computing platform. For ease of understanding, the interaction between the tenant and the cloud computing platform is described below. As shown in FIG. 4, the interaction between the tenant and the cloud computing platform mainly includes: the tenant logs in to the cloud computing platform 400 through a client web page, selects and purchases a cloud service (i.e., a model training service) related to the model training system in the cloud computing platform 400, and after the purchase, the tenant can perform model training on the cloud computing platform 400 based on the functions provided by the model training service. The cloud computing platform 400 is mainly used to manage the infrastructure for running the model training service. For example, the infrastructure for running the model training service can include multiple data centers arranged in different regions, and each data center includes multiple servers. The data center can provide basic resources such as computing resources and storage resources for the model training service. Therefore, when the tenant purchases and uses the model training service, the tenant mainly pays for the resources used.

[0067] Next, based on the above content, a model training method provided by an embodiment of the present application is introduced.

[0068] For example, FIG. 5 shows a flowchart of a model training method provided by an embodiment of the present application. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. For example, the method can be executed by a model training device, which can be implemented by software and / or hardware, and can be configured in an electronic device or a server, for example, and typically can be configured on a cloud computing platform. For ease of description, the cloud computing platform will be taken as the execution subject for introduction. As shown in FIG. 5, the model training method can include the following steps:

[0069] S501, construct a training data set required for the current round of iterative training, and train the first model based on the training data set to obtain a second model.

[0070] In this embodiment, at each round of training, the training data set required for the current round of iterative training can be constructed first. Then, the first model is trained based on the training data set to obtain a second model. It should be understood that the first model refers to the model required for the current round of training, which can be the original model or the model that passed the last round of iterative evaluation; the second model refers to the model obtained after training the first model.

[0071] When constructing the training data set required for the current round of iterative training, if the current round of iteration is the first round of iteration, the training data set can be directly sampled from the general test set. If the current round of iteration is not the first round of iteration, the training data set can also be directly sampled from the general test set. In some embodiments, when the current round of iteration is not the first round of iteration, in order to achieve efficient and targeted training data construction and model iteration, the training data set required for the current round of iteration can also be constructed based on the error example data set obtained in the last round of evaluation. Specifically, as shown in FIG. 6, in S601, the data in the error example data set obtained in the last round of iterative evaluation is subjected to error root cause analysis to obtain error root cause analysis results. The obtained error example data set can be, but is not limited to, the data used to provide the development studio for model optimization in the last round of iteration evaluation using the foregoing S23 of FIG. 2. For example, the root cause analysis can be performed by a large language model. In addition, in order to improve the efficiency of error root cause analysis, before performing error root cause analysis, the data in the error example data set can be clustered by a clustering algorithm to obtain error example clustering clusters and / or discrete error examples. The error root cause analysis results are a summary of the error root causes of the error example clustering clusters and the discrete error examples. In S602, the data in the error example data set is corrected based on the error root cause analysis results to obtain a third data set. For example, the error root cause analysis results can be input into a large language model to correct the data in the error example data set, such as correcting the original error reply. In S603, the error example data set is generalized based on the error root cause analysis results and the error example data set to obtain an error example generalized data set. For example, the error root cause analysis results and the error example data set can be input into a large language model to generalize the error example generalized data set. In S604, the fourth data set is sampled from the general test set. In S605, the third data set, the fourth data set, and the error example generalized data set are used as the training data set. In this way, the training data set required for the current round of iteration is constructed based on the large language model, the automatic correction of the error examples found in the evaluation, and the automatic generalization of new data with similar error root causes using the results of error root cause analysis, thereby achieving efficient and targeted training data construction and large model iteration.

[0072] S502, construct a regression test set required for evaluation of the current iteration, and evaluate the second model based on the regression test set to obtain a first evaluation result.

[0073] In this embodiment, after training the model in the current iteration, the trained model can be evaluated. When evaluating, a regression test set required for evaluation of the current iteration can be constructed first. Among them, first sampling can be performed from the live data to construct a first test set. Then, the data in the first test set is evaluated by a large language model to obtain a mistake evaluation report. The mistake evaluation report includes the score of the data in the first test set in at least one dimension (for example: one or more of factuality, information amount, relevance, fluency, security, etc.), and the error description of the data in the first test set. Finally, based on the mistake evaluation report, the regression test set and the mistake data set are filtered from the error data (such as data with a score below the pass line) in the first test set. Among them, the mistake data set filtered this time can be used to construct the training data set required for the next iteration training. In some embodiments, when constructing the first test set, if the current iteration is not the first iteration, the failure examples with poor evaluation results in the regression test set constructed in the last iteration can also be sampled from the live data, such as sampling more data of the same type as the failure examples, to obtain a second data set. Then, the failure examples with poor evaluation results in the regression test set constructed in the last iteration and the second data set are used as the first test set. Among them, the failure example indicates that the optimization of the model on this example is poor, so the model can continue to be evaluated on this example to optimize the model in a better direction.

[0074] After constructing the regression test, the second model can be evaluated based on the regression test set to obtain a first evaluation result. Among them, the regression test set can be processed by the second model first to obtain the corresponding reply. Then, the reply is evaluated in at least one dimension to obtain the first evaluation result.

[0075] S503, based on the first evaluation result and the second evaluation result, evaluate the correctness of the second model in the optimization direction of the current iteration, wherein the second evaluation result is obtained by evaluating the second model based on the general test set.

[0076] In this embodiment, the second model can also be evaluated based on the general test set to obtain a second evaluation result. Further, the correctness of the second model in the optimization direction of the current iteration can be evaluated based on the first evaluation result and the second evaluation result.

[0077] In some embodiments, in the case that the first evaluation result is better than the third evaluation result, and the second evaluation result is better than the fourth evaluation result, it is determined that the second model is correct in the direction of the current iteration optimization, and the current training and evaluation is ended, and the next round of model training and evaluation is entered. The third evaluation result is obtained by evaluating the first model based on the regression test set constructed in S502, and the fourth evaluation result is obtained by evaluating the first model based on the general test set.

[0078] In the case that the first evaluation result is better than the third evaluation result, and the second evaluation result is worse than the fourth evaluation result, it indicates that the performance of the second model on the general test set is poor, and there may be a problem with the training data set, so it can be determined that the second model is incorrect in the direction of the current iteration optimization, and the training data set is reconstructed, and the first model is trained.

[0079] In the case that the first evaluation result is worse than the third evaluation result, it indicates that the performance of the second model on the regression test set constructed in the current iteration is poor, and there may be a problem with the regression test set constructed in the current iteration, so it can be determined that the second model is incorrect in the direction of the current iteration optimization, and the regression test set is reconstructed, and the second model is evaluated.

[0080] In this way, in the model training process, based on the regression test set constructed by sampling the online data in each iteration and the general evaluation set containing various basic capability evaluations, two-level testing is performed to fully evaluate the correctness of the optimization direction, and guide the next step planning, which can prevent the optimization direction from being biased and the model effect from being degraded.

[0081] It can be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, the various embodiments described above or the technical features involved in the embodiments can be combined according to the actual situation, and the combined scheme is still within the protection scope of the present application.

[0082] Based on the method in the above embodiments, the present embodiment further provides a model training device.

[0083] Exemplarily, FIG. 7 shows a structural schematic diagram of a model training apparatus provided by an embodiment of the present application. As shown in FIG. 7, the model training apparatus 700 includes a training module 701 and an evaluation module 702. The training module 701 is configured to construct a training data set required for the current iteration training, and train a first model based on the training data set to obtain a second model. The evaluation module 702 is configured to construct a regression test set required for the current iteration evaluation, and evaluate the second model based on the regression test set to obtain a first evaluation result; and evaluate the correctness of the second model in the optimization direction of the current iteration based on the first evaluation result and a second evaluation result, wherein the second evaluation result is obtained by evaluating the second model based on a general test set.

[0084] In some embodiments, when the evaluation module 702 evaluates the correctness of the second model in the optimization direction of the current iteration based on the first evaluation result and the second evaluation result, it is specifically configured to: in the case that the first evaluation result is better than a third evaluation result and the second evaluation result is better than a fourth evaluation result, determine that the second model is correct in the optimization direction of the current iteration, and end the current training and evaluation, and enter the next round of model training and evaluation, wherein the third evaluation result is obtained by evaluating the first model based on the regression test set, and the fourth evaluation result is obtained by evaluating the first model based on the general test set; in the case that the first evaluation result is better than the third evaluation result and the second evaluation result is worse than the fourth evaluation result, determine that the second model is incorrect in the optimization direction of the current iteration, and reconstruct the training data set and train the first model again; in the case that the first evaluation result is worse than the third evaluation result, determine that the second model is incorrect in the optimization direction of the current iteration, and reconstruct the regression test set and evaluate the second model again.

[0085] In some embodiments, when the evaluation module 702 constructs the regression test set required for the current iteration evaluation, it is specifically configured to: construct a first test set based on the online data; evaluate the data in the first test set to obtain a wrong case evaluation report, wherein the wrong case evaluation report includes the scores of the data in the first test set in at least one dimension, and the error description of the data in the first test set; and select the regression test set and a first wrong case data set from the error data in the first test set based on the wrong case evaluation report, wherein the first wrong case data set is used to construct the training data set required for the next iteration training.

[0086] In some embodiments, the evaluation module 702, when constructing the first test set based on the live data, is specifically configured to: sample from the live data to obtain a second data set based on a first data set in the regression test set constructed in the last iteration, wherein the first data set is a failed example in the regression test set constructed in the last iteration indicated by a third evaluation result, and the third evaluation result is obtained by evaluating the model trained in the last iteration based on the regression test set constructed in the last iteration; and take the first data set and the second data set as the first test set.

[0087] In some embodiments, the training module 701, when constructing the training data set required for the current iteration training, is specifically configured to: perform error root cause analysis on the data in the second error example data set obtained by the evaluation of the last iteration to obtain an error root cause analysis result; correct the data in the second error example data set based on the error root cause analysis result to obtain a third data set; perform generalization based on the error root cause analysis result and the second error example data set to obtain an error example generalization data set; sample from the general test set to obtain a fourth data set; and take the third data set, the fourth data set, and the error example generalization data set as the training data set.

[0088] In some embodiments, before performing error root cause analysis on the data in the second error example data set obtained by the evaluation of the last iteration, the training module 701 is further configured to: perform clustering on the data in the second error example data set to obtain error example clustering clusters and / or discrete error examples, wherein the error root cause analysis result is a summary of error root causes of the error example clustering clusters and the discrete error examples.

[0089] In some embodiments, the training module 701 and the evaluation module 702 shown in FIG. 7 can be implemented by software or by hardware. For example, the implementation of the training module 701 is described below. Similarly, the implementation of the evaluation module 702 can refer to the implementation of the training module 701.

[0090] As an example of a software functional unit, the training module 701 can include code running on a compute instance. The compute instance can include at least one of a physical host (computing device), a virtual machine, a container. Further, the compute instance can be one or more. For example, the training module 701 can include code running on multiple hosts / virtual machines / containers. It is noted that the multiple hosts / virtual machines / containers running the code can be distributed in the same region, or in different regions. Further, the multiple hosts / virtual machines / containers running the code can be distributed in the same availability zone (AZ), or in different AZs, each of which includes one data center or multiple data centers in close geographical proximity. Typically, a region can include multiple AZs.

[0091] Similarly, the multiple hosts / virtual machines / containers running the code can be distributed in the same virtual private cloud (VPC), or in multiple VPCs. Typically, a VPC is set up within a region, and communication between two VPCs in the same region, or between VPCs in different regions, requires a communication gateway in each VPC to enable interconnection between the VPCs.

[0092] As an example of a hardware functional unit, the training module 701 can include at least one computing device, such as a server, etc. Alternatively, the training module 701 can be a device implemented with an application-specific integrated circuit (ASIC), or a programmable logic device (PLD), etc. The PLD can be implemented with a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0093] The plurality of computing devices included in the training module 701 can be distributed in the same region or in different regions. The plurality of computing devices included in the training module 701 can be distributed in the same AZ or in different AZs. Similarly, the plurality of computing devices included in the training module 701 can be distributed in the same VPC or in multiple VPCs. The plurality of computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0094] It should be noted that, in other embodiments, the training module 701 can be configured to perform any of the steps of the model training method described in the above embodiments, and the evaluation module 702 can be configured to perform any of the steps of the model training method described in the above embodiments. In addition, the steps to be implemented by the training module 701 and the evaluation module 702 can be specified as needed, and the entire function of the model training apparatus 700 shown in FIG. 7 can be implemented by the training module 701 and the evaluation module 702 implementing different steps of the model training method described in the above embodiments.

[0095] The present application also provides a computing device 800. As shown in FIG. 8, the computing device 800 includes a bus 802, a processor 804, a memory 806, and a communication interface 808. The processor 804, the memory 806, and the communication interface 808 communicate with each other through the bus 802. The computing device 800 can be a server or an electronic device. It should be understood that the present application does not limit the number of processors and memories in the computing device 800.

[0096] The bus 802 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is shown in FIG. 8, but it does not mean that there is only one bus or only one type of bus. The bus 804 can include a path for transmitting information between various components (e.g., the memory 806, the processor 804, the communication interface 808) of the computing device 800.

[0097] The processor 804 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0098] The memory 806 can include volatile memory, such as random access memory (RAM), and non-volatile memory, such as read-only memory (ROM), floppy disks, or hard disks.

[0099] The memory 806 stores executable program code that the processor 804 executes to respectively implement the functions of the training module 701 and the evaluation module 702 shown in FIG. 7, thereby implementing the model training method described in the above embodiments. That is, the memory 806 stores instructions for implementing the model training method described in the above embodiments.

[0100] Alternatively, the memory 806 stores executable program code that the processor 804 executes to respectively implement the functions of the model training apparatus 700 shown in FIG. 7, thereby implementing the model training method described in the above embodiments. That is, the memory 806 stores instructions for implementing the model training method described in the above embodiments.

[0101] The communication interface 803 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to enable communication between the computing device 800 and other devices or communication networks.

[0102] The embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center.

[0103] As shown in FIG. 9, the computing device cluster includes at least one computing device 800. The memory 806 in one or more computing devices 800 in the computing device cluster can store the same instructions for implementing the model training method described in the above embodiments.

[0104] In some possible implementations, the memory 806 of one or more computing devices 800 in the computing device cluster can also respectively store partial instructions for implementing the model training method described in the above embodiments. In other words, the combination of one or more computing devices 800 can collectively execute the instructions for implementing the model training method described in the above embodiments.

[0105] It should be noted that the memories 806 in different computing devices 800 in the computing device cluster can store different instructions for performing the functions of the aforementioned model training apparatus 700 shown in FIG. 7 respectively. That is, the memories 806 in different computing devices 800 store instructions for implementing the functions of one or more of the training module 701 and the evaluation module 702.

[0106] In some possible implementations, one or more computing devices in the computing device cluster can be connected through a network. The network can be a wide area network, a local area network, or the like. FIG. 10 shows one possible implementation. As shown in FIG. 10, two computing devices 800A and 800B are connected through a network. Specifically, the computing devices are connected to the network through the communication interfaces in the computing devices. In this type of possible implementation, the memory 806 in the computing device 800A stores instructions for performing the functions of the training module 701. Meanwhile, the memory 806 in the computing device 800B stores instructions for performing the functions of the evaluation module 702.

[0107] It should be understood that the functions of the computing device 800A shown in FIG. 10 can also be performed by multiple computing devices 800. Similarly, the functions of the computing device 800B can also be performed by multiple computing devices 800.

[0108] Embodiments of the present application also provide another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to the connection manners of the computing device clusters described with reference to FIG. 9 and FIG. 10. The difference is that the memories 806 in one or more computing devices 800 in the computing device cluster can store the same instructions for performing the methods in the aforementioned embodiments.

[0109] In some possible implementations, the memories 806 in one or more computing devices 800 in the computing device cluster can also respectively store part of the instructions for performing the aforementioned methods. In other words, the combination of one or more computing devices 800 can collectively execute the instructions for performing the aforementioned methods.

[0110] Based on the method in the above embodiments, the embodiments of the present application provide a computer readable storage medium including computer program instructions, when the computer program instructions are executed by a computing device, the computing device executes the method in the above embodiments; or, when the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method in the above embodiments. Illustratively, the computer readable storage medium can be any available medium or data storage device that can be accessed by the computing device or a data center including one or more available media or data storage devices. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk), etc.

[0111] Based on the method in the above embodiments, the embodiments of the present application provide a computer program product including instructions, when the instructions are run by a computing device, the computing device executes the method in the above embodiments, or, when the instructions are run by a computing device cluster, the computing device cluster executes the method in the above embodiments.

[0112] It can be understood that the processor in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0113] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0114] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, cloud computing platform, or data center to another website, computer, cloud computing platform, or data center through a wired (such as a coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a cloud computing platform, data center, etc. that includes one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0115] It can be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of differentiation, and do not limit the scope of the embodiments of the present application.

[0116] Finally, it should be noted that: the above examples are used to illustrate the technical solutions of the present application, but not limited to them; although the present application is described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

A model training method, characterized in that, The method comprises: constructing a training data set required for the current iteration training, and training the first model based on the training data set to obtain a second model; constructing a regression test set required for the current iteration evaluation, and evaluating the second model based on the regression test set to obtain a first evaluation result; based on the first evaluation result and the second evaluation result, evaluating the correctness of the second model in the optimization direction of the current iteration, wherein the second evaluation result is obtained by evaluating the second model based on a general test set. The method of claim 1, wherein The evaluation of the correctness of the second model in the optimization direction of the current iteration based on the first evaluation result and the second evaluation result comprises: in the case that the first evaluation result is better than a third evaluation result and the second evaluation result is better than a fourth evaluation result, it is determined that the second model is correct in the optimization direction of the current iteration, and the current training and evaluation are ended, and the next round of model training and evaluation is entered, wherein the third evaluation result is obtained by evaluating the first model based on the regression test set, and the fourth evaluation result is obtained by evaluating the first model based on the general test set; in the case that the first evaluation result is better than the third evaluation result and the second evaluation result is worse than the fourth evaluation result, it is determined that the second model is incorrect in the optimization direction of the current iteration, and the training data set is reconstructed, and the first model is trained; in the case that the first evaluation result is worse than the third evaluation result, it is determined that the second model is incorrect in the optimization direction of the current iteration, and the regression test set is reconstructed, and the second model is evaluated. The method according to claim 1 or 2, characterized in that The construction of the regression test set required for the current iteration evaluation comprises: constructing a first test set based on live network data; evaluating the data in the first test set to obtain a mistake example evaluation report, wherein the mistake example evaluation report comprises the scores of the data in the first test set in at least one dimension, and the error description of the data in the first test set; based on the mistake example evaluation report, filtering the regression test set and a first mistake example data set from the error data in the first test set, wherein the first mistake example data set is used to construct a training data set required for the next iteration training. The method according to claim 3, characterized in that The construction of the first test set based on live network data comprises: sampling the second data set from the live network data based on a first data set in the regression test set constructed in the last iteration, wherein the first data set is a failed example in the regression test set constructed in the last iteration indicated by a third evaluation result, and the third evaluation result is obtained by evaluating the model trained in the last iteration based on the regression test set constructed in the last iteration; the first data set and the second data set are taken as the first test set. The method according to any one of claims 1 to 4, characterized in that The construction of the training data set required for the current iteration training comprises: performing error root cause analysis on the data in the second mistake example data set obtained by the evaluation of the last iteration to obtain an error root cause analysis result; correct the data in the second erroneous example data set based on the error root cause analysis result to obtain a third data set; generalize based on the error root cause analysis result and the second erroneous example data set to obtain an erroneous example generalized data set; sample from the general test set to obtain a fourth data set; use the third data set, the fourth data set, and the erroneous example generalized data set as the training data set. The method according to claim 5, characterized in that Before the error root cause analysis on the data in the second erroneous example data set obtained in the evaluation of the previous iteration, further comprising: cluster the data in the second erroneous example data set to obtain erroneous example clusters and / or discrete erroneous examples, wherein the error root cause analysis result is a summary of error root causes of the erroneous example clusters and the discrete erroneous examples. A model training device characterized by comprising: comprising: a training module configured to construct a training data set required for training in the current iteration, and train a first model based on the training data set to obtain a second model; an evaluation module configured to construct a regression test set required for evaluation in the current iteration, and evaluate the second model based on the regression test set to obtain a first evaluation result; the evaluation module is further configured to evaluate the correctness of the second model in the optimization direction of the current iteration based on the first evaluation result and a second evaluation result, wherein the second evaluation result is obtained by evaluating the second model based on a general test set. A cluster of computing devices, characterized in that, at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform the method of any one of claims 1-6. A computer-readable storage medium, characterized by computer program instructions that, when executed by a computing device cluster comprising at least one computing device, cause the computing device cluster to perform the method of any one of claims 1-6. A computer program product comprising instructions, characterized in that computer program instructions that, when executed by a computing device cluster comprising at least one computing device, cause the computing device cluster to perform the method of any one of claims 1-6.

Images