Model training method and device and computing device cluster

Abstract

The model training method comprises the following steps: constructing a training data set required by the round of iterative training, and training a first model based on the training data set to obtain a second model; constructing a regression test set required by the current round of iterative evaluation, and evaluating the second model based on the regression test set to obtain a first evaluation result; and on the basis of the first evaluation result and a second evaluation result, the correctness of the second model in the current iterative optimization direction is evaluated, and the second evaluation result is obtained by evaluating the second model on the basis of the general test set. Therefore, in the model training process, two-stage testing is carried out based on a regression test set constructed by current network data of each round of iterative sampling and a general evaluation set containing various basic capability evaluations to fully evaluate the correctness of the optimization direction and guide the next step of planning, so that the optimization direction can be prevented from deviating and the model effect can be prevented from being degraded.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to a model training method, apparatus, and computing device cluster. Background Technology

[0002] Currently, generative large language models (LLMs), represented by the chat generative pre-trained transformer (ChatGPT), have made significant progress in natural language understanding and generation. These large language models possess excellent language understanding and generation capabilities, rich knowledge reserves, and high instruction compliance, enabling them to effectively perform complex analysis and reasoning tasks. The training of large language models typically involves two steps: pretraining and fine-tuning. In some practical applications, due to requirements such as knowledge updates and error correction, it is necessary to continuously train and iterate existing large language models. This usually involves two key steps: optimization direction exploration (evaluation and updating of the evaluation set, evaluation execution, and problem analysis) and solution formulation (training set construction and model training). However, in traditional approaches, manually customizing the evaluation set is costly and inefficient, leading to prolonged fixation of the same evaluation set or slow updates, and a tendency to overfit the evaluation set, thus biasing the optimization direction of the large language model training. Summary of the Invention

[0003] This application provides a model training method, apparatus, computing device cluster, computer storage medium, and computer product that can ensure the correctness of the optimization direction in each iteration during model training.

[0004] In a first aspect, this application provides a model training method, comprising: constructing a training dataset required for the current iteration training, and training a first model based on the training dataset to obtain a second model; constructing a regression test set required for the current iteration evaluation, and evaluating the second model based on the regression test set to obtain a first evaluation result; and evaluating the correctness of the second model in the current iteration optimization direction based on the first evaluation result and the second evaluation result, wherein the second evaluation result is obtained by evaluating the second model based on a general test set.

[0005] In this way, during the model training process, the correctness of the optimization direction can be fully evaluated by two levels of testing: a regression test set built based on the live network data sampled in each iteration and a general evaluation set containing various basic capability assessments. This will guide the next step of planning and prevent the optimization direction from being biased and the model performance from deteriorating.

[0006] In one possible implementation, the correctness of the second model in the current iteration optimization direction is evaluated based on the first and second evaluation results, including: if the first evaluation result is better than the third evaluation result and the second evaluation result is better than the fourth evaluation result, the second model is determined to be correct in the current iteration optimization direction, and the current training and evaluation ends, and the next round of model training and evaluation begins, wherein the third evaluation result is obtained by evaluating the first model based on a regression test set, and the fourth evaluation result is obtained by evaluating the first model based on a general test set; if the first evaluation result is better than the third evaluation result and the second evaluation result is worse than the fourth evaluation result, the second model is determined to be incorrect in the current iteration optimization direction, and the training dataset is reconstructed, and the first model is trained; if the first evaluation result is worse than the third evaluation result, the second model is determined to be incorrect in the current iteration optimization direction, and the regression test set is reconstructed, and the second model is evaluated. In this way, by comparing the two types of evaluation results, the correctness of the model in the direction of optimization in this round of iteration can be determined, and the next step can be planned automatically based on the evaluation results, thereby preventing the optimization direction from being biased and the model performance from deteriorating.

[0007] In one possible implementation, constructing the regression test set required for this round of iterative evaluation includes: constructing a first test set based on live network data; evaluating the data in the first test set to obtain a misclassification report, which includes the scores of the data in the first test set on at least one dimension, and a description of the errors in the data in the first test set; and based on the misclassification report, selecting a regression test set and a first misclassification dataset from the erroneous data in the first test set, wherein the first misclassification dataset is used to construct the training dataset required for the next round of iterative training. This allows for the construction of different regression test sets in different iterations, thereby better identifying problems in the model and effectively preventing biased optimization and model performance degradation.

[0008] In one possible implementation, a first test set is constructed based on existing network data. This includes: sampling from existing network data to obtain a second dataset based on a first dataset from the regression test set constructed in the previous iteration. The first dataset consists of failed examples from the regression test set constructed in the previous iteration, as indicated by the third evaluation result, which evaluates the model trained in the previous iteration based on the regression test set constructed in the previous iteration. The first and second datasets are used as the first test set. This allows for targeted construction of regression test sets, improving the accuracy of the tests.

[0009] In one possible implementation, constructing the training dataset required for this round of iterative training includes: analyzing the root causes of errors in the second error dataset obtained from the previous iteration to obtain the root cause analysis results; correcting the data in the second error dataset based on the root cause analysis results to obtain the third dataset; generalizing based on the root cause analysis results and the second error dataset to obtain the error generalization dataset; sampling from the generalized test set to obtain the fourth dataset; and using the third dataset, the fourth dataset, and the error generalization dataset as the training dataset. In this way, the automatic correction of errors discovered during evaluation based on the large language model, and the automatic generalization of new data with similar error root causes using the results of the root cause analysis, construct the training dataset required for this round of iteration, thereby achieving efficient and targeted training data construction and large model iteration.

[0010] In one possible implementation, before analyzing the root causes of errors in the second set of error samples obtained from the previous iteration, the method further includes: clustering the data in the second set of error samples to obtain error sample clusters and / or discrete error samples. The root cause analysis results are a summary of the root causes of errors in both the clusters and the discrete error samples. This clustering approach allows for unified root cause analysis of similar error samples, improving the efficiency of root cause analysis.

[0011] Secondly, this application provides a model training apparatus, including a training module and an evaluation module. The training module is used to construct the training dataset required for the current iteration of training, and to train a first model based on the training dataset to obtain a second model. The evaluation module is used to construct a regression test set required for the current iteration of evaluation, and to evaluate the second model based on the regression test set to obtain a first evaluation result; and, based on the first evaluation result and the second evaluation result, to evaluate the correctness of the second model in the current iteration of optimization direction, wherein the second evaluation result is obtained by evaluating the second model based on a general test set.

[0012] In one possible implementation, when the evaluation module assesses the correctness of the second model in the current iteration optimization direction based on the first and second evaluation results, it specifically performs the following: if the first evaluation result is better than the third evaluation result and the second evaluation result is better than the fourth evaluation result, it determines that the second model is correct in the current iteration optimization direction, ends the current training and evaluation, and proceeds to the next round of model training and evaluation. The third evaluation result is obtained by evaluating the first model based on a regression test set, and the fourth evaluation result is obtained by evaluating the first model based on a general test set. If the first evaluation result is better than the third evaluation result and the second evaluation result is worse than the fourth evaluation result, it determines that the second model is incorrect in the current iteration optimization direction, reconstructs the training dataset, and trains the first model. If the first evaluation result is worse than the third evaluation result, it determines that the second model is incorrect in the current iteration optimization direction, reconstructs the regression test set, and evaluates the second model.

[0013] In one possible implementation, the evaluation module, when constructing the regression test set required for the current iteration evaluation, specifically performs the following: constructing a first test set based on live network data; evaluating the data in the first test set to obtain a misclassification evaluation report, which includes the scores of the data in the first test set on at least one dimension, and a description of the errors in the data in the first test set; and, based on the misclassification evaluation report, selecting a regression test set and a first misclassification dataset from the erroneous data in the first test set, wherein the first misclassification dataset is used to construct the training dataset required for the next iteration training.

[0014] In one possible implementation, when constructing the first test set based on live network data, the evaluation module specifically performs the following: based on the first dataset in the regression test set constructed in the previous iteration, it samples from the live network data to obtain the second dataset, wherein the first dataset consists of the failed examples in the regression test set constructed in the previous iteration as indicated by the third evaluation result, and the third evaluation result is obtained by evaluating the model trained in the previous iteration based on the regression test set constructed in the previous iteration; the first dataset and the second dataset are used as the first test set.

[0015] In one possible implementation, the training module, when constructing the training dataset required for the current iteration of training, specifically performs the following: root cause analysis on the data in the second misclassified dataset obtained from the previous iteration, to obtain the root cause analysis results; based on the root cause analysis results, correct the data in the second misclassified dataset to obtain the third dataset; generalize based on the root cause analysis results and the second misclassified dataset to obtain the misclassified dataset; sample from the general test set to obtain the fourth dataset; and use the third dataset, the fourth dataset, and the misclassified dataset as the training dataset.

[0016] In one possible implementation, before performing error root cause analysis on the data in the second faulty sample dataset obtained from the previous iteration, the training module is also used to: cluster the data in the second faulty sample dataset to obtain faulty sample clusters and / or discrete faulty samples, wherein the error root cause analysis result is a summary of the error root causes of the faulty sample clusters and discrete faulty samples.

[0017] Thirdly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

[0018] Fourthly, this application provides a computer-readable storage medium including computer program instructions, which, when executed by a computing device, perform the method described in the first aspect or any possible implementation thereof; or, when executed by a cluster of computing devices, the cluster of computing devices performs the method described in the first aspect or any possible implementation thereof. Exemplarily, the cluster of computing devices may include one or more computing devices.

[0019] Fifthly, this application provides a computer program product containing instructions that, when executed by a computing device, cause the computing device to perform the method described in the first aspect or any possible implementation thereof; or, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in the first aspect or any possible implementation thereof. Exemplarily, a cluster of computing devices may include one or more computing devices.

[0020] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the architecture of a model training system provided in an embodiment of this application;

[0022] Figure 2 yes Figure 1 A diagram illustrating the workflow in the testing studio.

[0023] Figure 3 yes Figure 1 A diagram illustrating the workflow in the development studio;

[0024] Figure 4 This is a schematic diagram illustrating the interaction between a tenant and a cloud computing platform, provided in an embodiment of this application.

[0025] Figure 5 This is a schematic flowchart of a model training method provided in an embodiment of this application;

[0026] Figure 6 This is a schematic diagram illustrating the steps involved in constructing the training dataset required for this iteration, as provided in an embodiment of this application.

[0027] Figure 7 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0030] Figure 10 This is a schematic diagram of another computing device cluster structure provided in an embodiment of this application. Detailed Implementation

[0031] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0032] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0033] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0034] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0035] First, the relevant technical terms involved in the technical solution provided in this application will be introduced.

[0036] (1) Large Model

[0037] Large models specifically refer to large language models, which are deep learning models trained on massive amounts of text data. By learning the structure, rules, and semantics of language through training, they can generate text with a natural language style or answer questions in natural language.

[0038] (2) Automatic evaluation

[0039] Automated evaluation of large language models refers to a method for systematically evaluating the performance of large-scale pre-trained language models using automated techniques. This process typically involves the comprehensive analysis of multiple metrics. By using automated tools, algorithms, or large language models as evaluators, the performance of the model in natural language processing tasks can be measured efficiently and objectively, thus providing a basis for further model optimization. This evaluation method not only improves evaluation efficiency but also reduces the interference of subjective human factors, enhancing the reliability and repeatability of the evaluation results.

[0040] (3) Data augmentation and generalization

[0041] Data augmentation and generalization, in the fields of machine learning and artificial intelligence, refer to methods that enhance the generalization ability and robustness of models by expanding and transforming the original dataset. Data generalization involves generating new data points to cover a wider input space, thereby making the model more universal and adaptive, reducing overfitting, and improving the model's performance across different tasks and data distributions. Together, these two methods help generate richer and more diverse training data, ultimately improving model performance and practical application effectiveness.

[0042] (4) Self-improvement of large language models

[0043] Large language model self-improvement refers to the process by which large-scale artificial intelligence models continuously improve their performance and intelligence through self-analysis, feedback, and optimization mechanisms. It typically relies on self-supervised learning, meta-learning, and feedback loop mechanisms in deep learning techniques, aiming to enable the model to fully utilize existing data and experience for self-optimization without external intervention. This process helps large models maintain a high level of intelligent performance in dynamically changing environments.

[0044] The technical solution provided in this application will be described below.

[0045] For example, Figure 1 A schematic diagram of the architecture of a model training system provided in an embodiment of this application is shown. Figure 1 As shown, the model training system may include an evaluation studio and a development studio.

[0046] The main tasks of the evaluation studio include automatically sampling and constructing test sets from live network data, automated evaluation, and regression testing. The evaluation studio primarily comprises three processes: evaluation planning, evaluation execution and summary reporting, and regression testing. The evaluation planning process mainly involves constructing the test set. During the test set construction process, random sampling can be performed based on live network user feedback data (i.e., live network data) during the cold start phase; in other phases, failed test cases from regression testing and resampling based on problem types can be added. The evaluation execution and summary reporting process mainly involves automatically evaluating the constructed test set using a large language model and obtaining an evaluation report based on error analysis. The regression testing process mainly involves performing two levels of regression testing (the current round of regression test set + a general evaluation dataset). If both levels pass, the next round proceeds. The current round of regression test set can be obtained from the test set constructed during the evaluation planning process. Additionally, failed test cases from the previous round of regression test set can be added to the current round of regression test set. In this embodiment, the general evaluation dataset can be, but is not limited to, a long-term, fixed, manually designed test set used to evaluate the various capabilities of a large language model, ensuring the model's fundamental capabilities. For example, the general evaluation dataset can also be called a "general test set." The regression test set is variable; the regression test set can be different for each round. The regression test set can be used to discover problems in the model, thereby better guiding the model towards optimization in the right direction.

[0047] The main tasks of the development studio include root cause analysis of errors, generalization of difficult examples, preparation of training data, and model training based on the test results from the evaluation studio. The development studio primarily comprises two phases: training preparation and model training. The training preparation phase mainly involves building the training dataset needed for the next iteration. During this process, based on the error analysis in the evaluation report obtained from the evaluation studio, errors can be corrected using a large language model and added to the training dataset; further root cause analysis can be performed using clustering and a large language model, and instances with similar error root causes can be generalized and added to the training dataset. The model training phase mainly involves automatically starting model training based on the new round of training data.

[0048] The specific implementation plans for the evaluation studio and the development studio will be introduced below.

[0049] (1) Evaluation Studio

[0050] In each iteration, the evaluation studio takes live network data, the large language model to be tested, and the previous regression test report as input, and outputs the current round's error evaluation report and regression test report. The evaluation studio enables automated evaluation and error analysis, improving efficiency and reducing costs. The error evaluation report guides the training data preparation for subsequent development studios in the current iteration, while the regression test report serves as a basis for judging the correctness of the current round's optimization direction and guides the preparation of the evaluation set (also known as the "test set") for the next iteration. For example, as shown... Figure 2 As shown, the specific steps involved in the evaluation studio may include:

[0051] In S21, prepare the test set for the current iteration. This step requires accumulating a certain amount of live network feedback data (i.e., live network data) of the large language model to be tested. Each data point includes a user question, a response to the user question, and a user question type label (optional). The user question type label can be automatically obtained based on a pre-trained deep learning classification model. During the cold start (i.e., the first iteration), user question types can be uniformly and randomly sampled as the test set for the first iteration. For example, the current round can also be referred to as "this round".

[0052] In S22, each data point in the test set determined in S21 is automatically evaluated based on the large language model, and an evaluation report is output. For example, multi-dimensional evaluation templates (factuality, information content, relevance, fluency, security, etc.) can be designed to achieve automated evaluation of the large model. The output evaluation report can include the score of each data point, as well as the error description of each data point in each evaluation dimension.

[0053] In S23, prepare the regression test set for the current round. This step primarily relies on the evaluation results obtained in S22. From the tests determined in S21, sample the regression test set required for the current round and a portion of the training data needed by the development team. For example, data with scores below the passing threshold in the automated evaluation results can be filtered out and divided into two parts through sampling. One part serves as the regression test set required for the current round, while the other part can be provided to the development team for model optimization in this round (see the subsequent description of the development team for details). Specifically, the sampling scheme could be as follows: divide the data with scores below the passing threshold according to user question type, sample a fixed proportion (e.g., 20%) within each user question type to add to the regression test set required for the current round, and provide the remaining portion to the development team.

[0054] In S24, regression testing. This step primarily involves using the evaluation results from the general test set and the regression test set prepared in S23 to determine if the optimization direction of this round is biased and to plan for the next round. Specifically, first, the responses of the optimized large language model from the development studio in this round are obtained on the general test set and the current round's regression test set. Then, the responses of each data point are scored based on the automatic evaluation method in S22. Finally, the average score change of the large language model before and after optimization on the two datasets is calculated, and the next step is planned based on the score change. There are three possible scenarios for planning the next step based on the score change: a) Scenario 1: The average score of the optimized large language model on the current round's regression test set decreases; revert to S23 in the current round and resample and divide the current round's regression test set and the data provided to the development studio. b) Scenario 2: The average score on the current round's regression test set increases, but the average score on the general test set decreases; revert to the current round's model optimization for the development studio. c) Scenario 3: The average scores on both the general test set and the current round's regression test set increase; the current round of iteration ends. For example, in S24, a regression test report can be output, which may include identifiers of the failed test cases in the current test set.

[0055] If the next step plan made in S24 is to start the next iteration from S21 when the current iteration ends, then the next iteration can begin again. Furthermore, when preparing the test set for the next iteration, failed samples from the previous regression test set (i.e., data whose scores did not increase) can be added to the next test set. Also, in the sampling of the next test set, the sampling ratio of the problem types of failed samples from the previous iteration can be increased, such as sampling more data of the same type.

[0056] In this way, in the evaluation studio, based on the general test set and the regression test set automatically constructed in each iteration, two levels of testing are conducted to determine the correctness of the optimization direction of the large language model, and the next step is automatically planned based on the evaluation results, which can prevent the optimization direction from being biased and the model performance from deteriorating.

[0057] (2) Development Studio

[0058] In each iteration, the development studio primarily takes the large language model to be optimized and the error report provided by the current evaluation studio as input, and outputs the optimized large language model for that round. The development studio can automate error root cause analysis, training data preparation, and model training. For example, ... Figure 3 As shown, the specific steps involved in the development studio may include:

[0059] In S31, error clustering is performed. For each error in the error evaluation report provided by the evaluation studio, it can be transformed into a vector based on the evaluation dimension and the corresponding score. Then, density-based clustering algorithms are used to cluster the errors, resulting in a series of error clusters, as well as a few discrete errors that are difficult to cluster.

[0060] In S32, root cause analysis, error correction, and generalization are performed. Root cause analysis templates can be designed, and a large language model can automatically summarize the root causes of errors in each cluster and each discrete error, resulting in error root cause analysis text. The input to the large language model can include one or more of the following: the error question, the error answer, and the error description. After obtaining the error root cause analysis text, the original error responses can be automatically corrected based on the text and added to the current round of training. Next, the error root cause analysis texts for each cluster (or discrete error) and the sampled error examples (or discrete error examples) from the corresponding clusters are input into the large language model for generalization, resulting in an error generalization dataset.

[0061] In S33, training set preparation and model training take place. In addition to the misclassified dataset obtained in S32, a basic training set containing evaluation data of various evaluation capabilities of the general evaluation set can be sampled and added to the current round of model training.

[0062] In this way, in the development studio, by clustering, analyzing the root causes of errors in the evaluation results of live network data, and generalizing the errors based on the root causes, training data preparation and model training can be completed, thereby achieving automated improvement of large language models, increasing efficiency, and reducing costs.

[0063] The above is an introduction to the model training system provided in the embodiments of this application. As can be seen from the above description, the model training system simulates the software development process, builds an automated testing and development studio, and jointly completes the automatic iterative improvement of the large language model. The model training system mainly includes three key technical points: (1) Based on the regression test set and the general evaluation set containing various basic ability assessments constructed by sampling the live network data in each iteration, two-level testing is carried out to fully evaluate the correctness of the optimization direction and guide the next step of planning, preventing the optimization direction from being biased and the model effect from deteriorating. (2) Based on the large language model, multi-dimensional automated evaluation is carried out, and the multi-dimensional evaluation score results are converted into vectors for clustering. The large language model is used to automatically and efficiently summarize the root causes of errors in each cluster category, providing high-quality analysis results for the construction of the training set. (3) Based on the large language model, the errors found in the evaluation are automatically corrected, and the results of the error root cause analysis are used to automatically generalize new data with similar error root causes, so as to achieve efficient and targeted training data construction and large model iteration.

[0064] In the model training system described above, after each round of model training is completed in the development studio, the trained model can be evaluated in the evaluation studio. Of course, evaluation can be performed at intervals of several rounds, depending on the actual situation, and no limit is imposed here.

[0065] It should be understood that the model training system described above can be configured on, but is not limited to, a cloud computing platform. For example, it can be deployed on at least one instance such as a virtual machine or container, enabling the cloud computing platform to provide model training services. Of course, the model training system can also be configured on nodes other than the cloud computing platform. For example, it can be deployed in at least one data center or on at least one server, depending on the actual situation, and is not limited here. The cloud computing platform can provide pages related to public cloud services for tenants to remotely access. In this embodiment, the tenant (also referred to as a "user") can pre-purchase the model training services provided by the model training system on the cloud computing platform. For ease of understanding, the interaction between the tenant and the cloud computing platform is described below. Figure 4 As shown, the interaction between the tenant and the cloud computing platform mainly includes: the tenant logs into the cloud computing platform 400 through a client webpage, selects and purchases cloud services related to the model training system (i.e., model training services) on the cloud computing platform 400, and after purchase, the tenant can perform model training on the cloud computing platform 400 based on the functions provided by the model training services. The cloud computing platform 400 is primarily used to manage the infrastructure for running the model training services. For example, the infrastructure for running the model training services may include multiple data centers located in different regions, each data center including multiple servers. Data centers can provide basic resources for the model training services, such as computing resources and storage resources. Therefore, when purchasing and using the model training services, the tenant mainly pays for the resources used.

[0066] Next, based on the above content, a model training method provided by an embodiment of this application will be introduced.

[0067] For example, Figure 5 This diagram illustrates a flowchart of a model training method provided in an embodiment of this application. It is understood that this method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities. For example, the method can be executed by a model training device, which can be implemented in software and / or hardware, and can be configured in, but is not limited to, electronic devices or servers; typically, it can be configured on a cloud computing platform. For ease of description, a cloud computing platform will be used as the execution subject in the following description. Figure 5 As shown, the model training method may include the following steps:

[0068] S501. Construct the training dataset required for this round of iterative training, and train the first model based on the training dataset to obtain the second model.

[0069] In this embodiment, during each training round, a training dataset required for the current iteration can be constructed first. Then, based on the training dataset, the first model is trained to obtain the second model. It should be understood that the first model refers to the model to be trained in this round, which can be the original model or a model that passed the evaluation in the previous iteration; the second model refers to the model obtained after training the first model.

[0070] When constructing the training dataset required for this round of iteration, if this is the first iteration, sampling can be directly performed from the general test set to obtain the training dataset. If this is not the first iteration, sampling can also be directly performed from the general test set to obtain the training dataset. In some embodiments, when this is not the first iteration, in order to achieve efficient and targeted training data construction and model iteration, the training dataset required for this iteration can also be constructed based on the error sample dataset obtained from the previous evaluation. Specifically, as shown... Figure 6 As shown, in S601, the root causes of errors are analyzed in the dataset of incorrect examples obtained from the previous iteration to obtain the root cause analysis results. The obtained dataset of incorrect examples may, but is not limited to, the dataset used in the previous iteration. Figure 2 In S23, data is provided to the development studio for this round of model optimization. For example, root cause analysis can be performed using a large language model, but is not limited to this. Additionally, to improve the efficiency of error root cause analysis, before performing the analysis, a clustering algorithm can be used to cluster the data in the error dataset to obtain error clusters and / or discrete errors. The root cause analysis result is a summary of the root causes of errors in the clusters and discrete errors. In S602, based on the error root cause analysis results, the data in the error dataset is corrected to obtain a third dataset. For example, the error root cause analysis results can be input into a large language model to correct the data in the error dataset, such as correcting the original error responses. In S603, generalization is performed based on the error root cause analysis results and the error dataset to obtain an error generalization dataset. For example, the error root cause analysis results and the error dataset can be input into a large language model for generalization to obtain an error generalization dataset. In S604, sampling is performed from a general test set to obtain a fourth dataset. In S605, the third dataset, the fourth dataset, and the error generalization dataset are used as training datasets. In this way, the large language model is used to automatically correct the errors found in the evaluation, and the results of error root cause analysis are used to automatically generalize new data with similar error root causes. This constructs the training dataset required for this round of iteration, thereby achieving efficient and targeted training data construction and large model iteration.

[0071] S502. Construct the regression test set required for this round of iterative evaluation, and evaluate the second model based on the regression test set to obtain the first evaluation result.

[0072] In this embodiment, after the model is trained in this iteration, it can be evaluated. During evaluation, a regression test set required for this iteration can be constructed first. This can be done by sampling from existing network data to construct a first test set. Then, the data in the first test set is evaluated using a large language model to obtain a misclassification report. This report includes scores for the data in the first test set on at least one dimension (e.g., factuality, informativeness, relevance, fluency, security, etc.) and a description of the errors in the data. Finally, based on the misclassification report, a regression test set and a misclassification dataset are selected from the erroneous data in the first test set (e.g., data with scores below the passing threshold). The selected misclassification dataset can be used to construct the training dataset required for the next iteration. In some embodiments, when constructing the first test set, if this iteration is not the first iteration, a second dataset can be obtained by sampling from live data based on the failed examples that performed poorly in the regression test set constructed in the previous iteration. This sampling could involve multiple sampling of data of the same type as the failed examples. Then, the failed examples that performed poorly in the regression test set constructed in the previous iteration, along with the second dataset, are used as the first test set. A failed example indicates that the model's optimization is poor on that example; therefore, the model can continue to be evaluated on that example to allow it to optimize in a better direction.

[0073] After constructing the regression test, the second model can be evaluated based on the regression test set to obtain the first evaluation result. This can be achieved by first processing the regression test set using the second model to obtain corresponding responses. Then, the responses are evaluated on at least one dimension to obtain the first evaluation result.

[0074] S503. Based on the first evaluation result and the second evaluation result, evaluate the correctness of the second model in the direction of this round of iterative optimization. The second evaluation result is obtained by evaluating the second model based on a general test set.

[0075] In this embodiment, the second model can also be evaluated based on a general test set to obtain a second evaluation result. Furthermore, the correctness of the second model in the current iteration optimization direction can be assessed based on the first and second evaluation results.

[0076] In some embodiments, if the first evaluation result is better than the third evaluation result, and the second evaluation result is better than the fourth evaluation result, the second model is determined to be correct in the direction of optimization in this round of iteration, and the current round of training and evaluation ends, and the next round of model training and evaluation begins. The third evaluation result is obtained by evaluating the first model based on the regression test set constructed in S502, and the fourth evaluation result is obtained by evaluating the first model based on a general test set.

[0077] If the first evaluation result is better than the third evaluation result, and the second evaluation result is worse than the fourth evaluation result, it indicates that the second model performs poorly on the general test set, which may be due to problems with the training dataset. Therefore, it can be determined that the second model is in the wrong direction of optimization in this round of iteration, and the training dataset should be reconstructed and the first model should be trained.

[0078] If the first evaluation result is worse than the third evaluation result, it indicates that the second model performs poorly on the regression test set constructed in this iteration. This may be due to a problem with the regression test set constructed in this iteration. Therefore, it can be determined that the second model is in the wrong direction of optimization in this iteration, and the regression test set should be reconstructed and the second model evaluated.

[0079] In this way, during the model training process, the correctness of the optimization direction can be fully evaluated by two levels of testing: a regression test set built based on the live network data sampled in each iteration and a general evaluation set containing various basic capability assessments. This will guide the next step of planning and prevent the optimization direction from being biased and the model performance from deteriorating.

[0080] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In addition, the various embodiments or technical features involved in the embodiments described above can be combined according to the actual situation, and the combined solution is still within the protection scope of this application.

[0081] Based on the methods in the above embodiments, this application also provides a model training device.

[0082] For example, Figure 7 A schematic diagram of the structure of a model training device provided in an embodiment of this application is shown. Figure 7As shown, the model training device 700 includes a training module 701 and an evaluation module 702. The training module 701 is used to construct the training dataset required for this round of iterative training, and to train the first model based on the training dataset to obtain the second model. The evaluation module 702 is used to construct the regression test set required for this round of iterative evaluation, and to evaluate the second model based on the regression test set to obtain a first evaluation result; and, based on the first and second evaluation results, to evaluate the correctness of the second model in the optimization direction of this round of iterative training, wherein the second evaluation result is obtained by evaluating the second model based on a general test set.

[0083] In some embodiments, when evaluating the correctness of the second model in the current iteration optimization direction based on the first evaluation result and the second evaluation result, the evaluation module 702 is specifically configured to: determine that the second model is correct in the current iteration optimization direction when the first evaluation result is better than the third evaluation result and the second evaluation result is better than the fourth evaluation result, and end the current training and evaluation, and proceed to the next round of model training and evaluation, wherein the third evaluation result is obtained by evaluating the first model based on the regression test set and the fourth evaluation result is obtained by evaluating the first model based on the general test set; determine that the second model is incorrect in the current iteration optimization direction when the first evaluation result is better than the third evaluation result and the second evaluation result is worse than the fourth evaluation result, and reconstruct the training dataset and train the first model; determine that the second model is incorrect in the current iteration optimization direction when the first evaluation result is worse than the third evaluation result, and reconstruct the regression test set and evaluate the second model.

[0084] In some embodiments, when constructing the regression test set required for the current iteration evaluation, the evaluation module 702 is specifically used to: construct a first test set based on existing network data; evaluate the data in the first test set to obtain a misclassification evaluation report, which includes the scores of the data in the first test set on at least one dimension, and the error descriptions of the data in the first test set; and based on the misclassification evaluation report, select a regression test set and a first misclassification dataset from the error data in the first test set, wherein the first misclassification dataset is used to construct the training dataset required for the next iteration training.

[0085] In some embodiments, when constructing the first test set based on existing network data, the evaluation module 702 is specifically used to: sample from existing network data based on the first dataset in the regression test set constructed in the previous iteration to obtain the second dataset, wherein the first dataset is the failed examples in the regression test set constructed in the previous iteration as indicated by the third evaluation result, and the third evaluation result is obtained by evaluating the model trained in the previous iteration based on the regression test set constructed in the previous iteration; and use the first dataset and the second dataset as the first test set.

[0086] In some embodiments, when constructing the training dataset required for the current iteration training, the training module 701 is specifically used to: perform root cause analysis on the data in the second misclassified dataset obtained from the previous iteration evaluation to obtain the root cause analysis results; correct the data in the second misclassified dataset based on the root cause analysis results to obtain the third dataset; generalize based on the root cause analysis results and the second misclassified dataset to obtain the misclassified dataset; sample from the general test set to obtain the fourth dataset; and use the third dataset, the fourth dataset, and the misclassified dataset as the training dataset.

[0087] In some embodiments, before performing error root cause analysis on the data in the second faulty sample dataset obtained from the previous iteration, the training module 701 is further configured to: cluster the data in the second faulty sample dataset to obtain faulty sample clusters and / or discrete faulty samples, wherein the error root cause analysis result is a summary of the error root causes of the faulty sample clusters and discrete faulty samples.

[0088] In some embodiments, Figure 7 The training module 701 and evaluation module 702 shown can both be implemented in software or in hardware. For example, the implementation of training module 701 will be described below. Similarly, the implementation of evaluation module 702 can refer to the implementation of training module 701.

[0089] As an example of a software functional unit, the training module 701 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, the training module 701 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0090] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0091] As an example of a hardware functional unit, the training module 701 may include at least one computing device, such as a server. Alternatively, the training module 701 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0092] The training module 701 includes multiple computing devices that can be distributed within the same region or in different regions. Similarly, the training module 701 can be distributed within the same Availability Zone (AZ) or in different AZs. Likewise, the training module 701 can be distributed within the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0093] It should be noted that, in other embodiments, the training module 701 can be used to execute any step in the model training method described in the above embodiments, and the evaluation module 702 can be used to execute any step in the model training method described in the above embodiments. Furthermore, the steps implemented by the training module 701 and the evaluation module 702 can be specified as needed, and different steps in the model training method described in the above embodiments can be implemented by the training module 701 and the evaluation module 702 respectively. Figure 7 The model training device 700 shown has all the functions.

[0094] This application also provides a computing device 800. For example... Figure 8As shown, the computing device 800 includes a bus 802, a processor 804, a memory 806, and a communication interface 808. The processor 804, the memory 806, and the communication interface 808 communicate with each other via the bus 802. The computing device 800 can be a server or an electronic device. It should be understood that this application does not limit the number of processors and memories in the computing device 800.

[0095] The 802 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus 804 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 804 may include a path for transmitting information between various components of the computing device 800 (e.g., memory 806, processor 804, communication interface 808).

[0096] Processor 804 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0097] The memory 806 may include volatile memory, such as random access memory (RAM). The processor 804 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0098] The memory 806 stores executable program code, and the processor 804 executes the executable program code to implement the aforementioned functions respectively. Figure 7 The training module 701 and evaluation module 702 shown herein perform their functions to implement the model training method described in the above embodiments. That is, the memory 806 stores instructions for executing the model training method described in the above embodiments.

[0099] Alternatively, the memory 806 stores executable code, and the processor 804 executes the executable code to implement the aforementioned functions respectively. Figure 7 The model training apparatus 700 shown in the diagram performs the functions of the model training method described in the above embodiments. That is, the memory 806 stores instructions for executing the model training method described in the above embodiments.

[0100] The communication interface 803 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 800 and other devices or communication networks.

[0101] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. This computing device can be a server, such as a central server, an edge server, or a local server in a local data center.

[0102] like Figure 9 As shown, the computing device cluster includes at least one computing device 800. The memory 806 of one or more computing devices 800 in the computing device cluster may store the same instructions for executing the model training method described in the above embodiments.

[0103] In some possible implementations, the memory 806 of one or more computing devices 800 in the computing device cluster may also store partial instructions for executing the model training method described in the above embodiments. In other words, a combination of one or more computing devices 800 can jointly execute instructions for executing the model training method described in the above embodiments.

[0104] It should be noted that the memory 806 in different computing devices 800 within the computing device cluster can store different instructions, each used to execute the aforementioned instructions. Figure 7 The model training device 700 shown contains some of the functions. That is, the instructions stored in the memory 806 in different computing devices 800 can implement the functions of one or more modules in the training module 701 and the evaluation module 702.

[0105] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 10 One possible implementation is shown. For example... Figure 10As shown, the two computing devices 800A and 800B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this possible implementation, the memory 806 in computing device 800A stores instructions for executing the functions of training module 701. Simultaneously, the memory 806 in computing device 800B stores instructions for executing the functions of evaluation module 702.

[0106] It should be understood that Figure 10 The functions of the computing device 800A shown can also be performed by multiple computing devices 800. Similarly, the functions of the computing device 800B can also be performed by multiple computing devices 800.

[0107] This application also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 9 and Figure 10 The connection method of the computing device cluster is different in that the memory 806 of one or more computing devices 800 in the computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0108] In some possible implementations, the memory 806 of one or more computing devices 800 in the computing device cluster may also store partial instructions for executing the aforementioned methods. In other words, a combination of one or more computing devices 800 can jointly execute the instructions for executing the aforementioned methods.

[0109] Based on the methods in the above embodiments, this application provides a computer-readable storage medium including computer program instructions. When the computer program instructions are executed by a computing device, the computing device performs the methods in the above embodiments; or, when the computer program instructions are executed by a cluster of computing devices, the cluster of computing devices performs the methods in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium that the computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0110] Based on the methods in the above embodiments, this application provides a computer program product containing instructions that, when executed by a computing device, cause the computing device to perform the methods in the above embodiments, or, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the methods in the above embodiments.

[0111] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0112] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0113] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, cloud computing platform, or data center to another website, computer, cloud computing platform, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a cloud computing platform or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0114] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A model training method, characterized in that, The method comprises: constructing a training data set required for the current iteration training, and training the first model based on the training data set to obtain a second model; constructing a regression test set required for the current iteration evaluation, and evaluating the second model based on the regression test set to obtain a first evaluation result; based on the first evaluation result and the second evaluation result, evaluating the correctness of the second model in the optimization direction of the current iteration, wherein the second evaluation result is obtained by evaluating the second model based on a general test set.

2. The method of claim 1, wherein, The evaluation of the correctness of the second model in the optimization direction of the current iteration based on the first evaluation result and the second evaluation result comprises: in the case that the first evaluation result is better than a third evaluation result and the second evaluation result is better than a fourth evaluation result, it is determined that the second model is correct in the optimization direction of the current iteration, and the current training and evaluation are ended, and the next round of model training and evaluation is entered, wherein the third evaluation result is obtained by evaluating the first model based on the regression test set, and the fourth evaluation result is obtained by evaluating the first model based on the general test set; in the case that the first evaluation result is better than the third evaluation result and the second evaluation result is worse than the fourth evaluation result, it is determined that the second model is incorrect in the optimization direction of the current iteration, and the training data set is reconstructed, and the first model is trained; in the case that the first evaluation result is worse than the third evaluation result, it is determined that the second model is incorrect in the optimization direction of the current iteration, and the regression test set is reconstructed, and the second model is evaluated.

3. The method according to claim 1 or 2, characterized in that, The construction of the regression test set required for the current iteration evaluation comprises: constructing a first test set based on live network data; evaluating the data in the first test set to obtain a mistake example evaluation report, wherein the mistake example evaluation report comprises the scores of the data in the first test set in at least one dimension, and the error description of the data in the first test set; based on the mistake example evaluation report, filtering the regression test set and a first mistake example data set from the error data in the first test set, wherein the first mistake example data set is used to construct a training data set required for the next iteration training.

4. The method of claim 3, wherein, The construction of the first test set based on live network data comprises: sampling a second data set from the live network data based on a first data set in the regression test set constructed in the last iteration, wherein the first data set is a failed example in the regression test set constructed in the last iteration indicated by a third evaluation result, and the third evaluation result is obtained by evaluating the model trained in the last iteration based on the regression test set constructed in the last iteration; the first data set and the second data set are taken as the first test set.

5. The method according to any of claims 1 to 4, characterized in that The construction of the training data set required for the current iteration training comprises: performing error root cause analysis on the data in the second mistake example data set obtained by the evaluation of the last iteration to obtain an error root cause analysis result; correct the data in the second erroneous example data set based on the error root cause analysis result to obtain a third data set; generalize based on the error root cause analysis result and the second erroneous example data set to obtain an erroneous example generalized data set; sample from the general test set to obtain a fourth data set; use the third data set, the fourth data set, and the erroneous example generalized data set as the training data set.

6. The method of claim 5, wherein, Before the error root cause analysis on the data in the second erroneous example data set obtained in the evaluation of the previous iteration, further comprising: cluster the data in the second erroneous example data set to obtain erroneous example clusters and / or discrete erroneous examples, wherein the error root cause analysis result is a summary of error root causes of the erroneous example clusters and the discrete erroneous examples.

7. A model training apparatus characterized by comprising: comprising: a training module configured to construct a training data set required for training in the current iteration, and train a first model based on the training data set to obtain a second model; an evaluation module configured to construct a regression test set required for evaluation in the current iteration, and evaluate the second model based on the regression test set to obtain a first evaluation result; the evaluation module is further configured to evaluate the correctness of the second model in the optimization direction of the current iteration based on the first evaluation result and a second evaluation result, wherein the second evaluation result is obtained by evaluating the second model based on a general test set.

8. A cluster of computing devices, characterized in that, at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, computer program instructions that, when executed by a computing device cluster comprising at least one computing device, cause the computing device cluster to perform the method of any one of claims 1-6.

10. A computer program product comprising instructions, characterized in that, computer program instructions that, when executed by a computing device cluster comprising at least one computing device, cause the computing device cluster to perform the method of any one of claims 1-6.

Images