Large model automated training method, apparatus and device

Abstract

This specification provides an automated training method, apparatus, and device for large language models. The scheme includes: acquiring global state information; the global state information includes historical data construction strategies and historical evaluation data; the historical evaluation data are the results obtained from evaluating the large language model to be trained on the target evaluation set in previous rounds; invoking the large language model inference engine to generate a data construction strategy for the current round based on the global state information; determining the training dataset based on the data construction strategy; incrementally training the large language model to be trained using the training dataset to obtain an updated large language model; evaluating the updated large language model on the target evaluation set to obtain current evaluation data; updating the global state information with the current evaluation data; if the current evaluation data meets the stopping condition, outputting the updated large language model as the completed training model; otherwise, continuing iterative training based on the updated global state information.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a method for automated training of large models. This specification also relates to an apparatus for automated training of large models, a computing device. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models are increasingly being used in code generation, structured data synthesis, and other fields. To improve the generation quality and robustness of these models in specific vertical domains, continuous fine-tuning or incremental training of pre-trained large language models is typically required. This process often involves the selection, cleaning, and construction of massive amounts of training data, as well as multiple rounds of model training and evaluation. The aim is to continuously optimize data distribution and training strategies to gradually bring the model performance closer to or to the preset target metrics.

[0003] In existing automated machine learning workflows, iterative optimization of models typically relies on fixed data pipelines or simple rule-based hyperparameter search mechanisms. However, faced with diverse error patterns and dynamically changing data requirements in complex structured data generation tasks, traditional static data processing methods struggle to flexibly adapt to the changing characteristics of models at different training stages. As the scale of model training increases and accuracy requirements rise, providing more efficient and targeted automated model training solutions has become a pressing issue in the current technological field. Summary of the Invention

[0004] In view of this, one or more embodiments of this specification provide a method, apparatus, or device for automated training of large models, in order to provide a more efficient and targeted automated model training solution.

[0005] According to a first aspect of one or more embodiments of this specification, a method for automated training of large models is provided, comprising: Obtain global state information; the global state information includes historical data construction strategy and historical evaluation data; the historical evaluation data is the result obtained by the large language model to be trained in previous rounds on the target evaluation set; The large language model inference engine is invoked to generate the data construction strategy for the current round based on the global state information. The training dataset is determined based on the data construction strategy described above; The training dataset is used to incrementally train the large language model to be trained, resulting in an updated large language model. The updated large language model is evaluated on the target evaluation set to obtain the current evaluation data; Update the current evaluation data to the global state information; If the current evaluation data meets the stopping condition, the updated large language model is output as the training completed model; otherwise, iterative training continues based on the updated global state information.

[0006] According to a second aspect of one or more embodiments of this specification, a large model automated training apparatus is provided, comprising: The state information acquisition module is used to acquire global state information; the global state information includes historical data construction strategies and historical evaluation data; the historical evaluation data is the result obtained by the large language model to be trained in previous rounds on the target evaluation set; The data construction strategy generation module is used to call the large language model inference engine to generate the data construction strategy for the current round based on the global state information. A training dataset construction module is used to determine the training dataset according to the data construction strategy; The model training module is used to incrementally train the large language model to be trained using the training dataset to obtain an updated large language model. The evaluation module is used to evaluate the updated large language model on the target evaluation set to obtain the current evaluation data; The status information update module is used to update the current evaluation data to the global status information; The iteration control module is used to output the updated large language model as the training completed model if the current evaluation data meets the stopping condition; otherwise, iterative training continues based on the updated global state information.

[0007] According to a third aspect of one or more embodiments of this specification, a computing device is provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor executes the computer instructions to implement the steps of the large model automated training method.

[0008] One embodiment of this specification can achieve at least the following beneficial effects: By first obtaining global state information containing historical data construction strategies and historical evaluation data, then calling the large language model inference engine to generate the data construction strategy for the current round based on the state information, subsequently determining the training dataset based on the strategy and performing incremental training, then evaluating the updated model on the target evaluation set to obtain the current evaluation data, and writing the evaluation data back to the global state information, finally deciding whether to output the model or continue iteration based on whether the stopping condition is met. Thus, by using historical evaluation data as the direct basis for strategy generation, the large language model inference engine can leverage the model's past performance in the current round. The evaluation results on the target evaluation set automatically adjust the data construction direction, thus avoiding the blind spots caused by relying on static rules or human experience. Simultaneously, by continuously updating the current evaluation data to the global state and storing it in association with the policy, a closed loop of "evaluation - evaluation result analysis - policy generation - data construction - training - re-evaluation" is formed. This allows each iteration to specifically address model weaknesses based on the previous iteration, thereby gradually improving performance metrics on the target evaluation set. Furthermore, this closed-loop mechanism allows the system to automatically accumulate effective policies and eliminate ineffective ones, improving the utilization efficiency of training resources and the reproducibility of model optimization without significantly increasing human intervention. In summary, this method can achieve automated data optimization and model iteration oriented towards target evaluation metrics, improving training efficiency and final model performance. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of the overall process of an automated training method for large models provided in one embodiment of this specification; Figure 2 A flowchart illustrating an automated training method for large models provided in the embodiments of this specification; Figure 3 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the structure of a large-scale automated training device; Figure 4 A structural block diagram of a computing device provided according to an embodiment of this specification is shown. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0012] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0013] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “an,” “an,” “the,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification includes any or all possible combinations of one or more associated listed items.

[0014] The terms “comprising,” “including,” or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded.

[0015] Although the terms "first," "second," etc., may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, "first" may also be referred to as "second," and similarly, "second" may also be referred to as "first," without departing from the scope of one or more embodiments of this specification. Ordinal numbers such as "first," "second," etc., do not necessarily indicate order; often they are used to facilitate the distinction of objects. For example, "first server" and "second server" usually refer to two servers. To distinguish these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0016] Depending on the context, the word "if" as used here can be interpreted as "when," "when," or "in response to determination."

[0017] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0018] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0019] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation entry points shall be provided for users to choose to authorize or refuse.

[0020] The following explains the terms and concepts used in one or more embodiments of this specification.

[0021] Large-scale artificial intelligence models (or simply "large models") refer to a class of artificial intelligence models with a large number of parameters built from artificial neural networks. Large models are also known as foundational models. They are pre-trained on large-scale unlabeled corpora, producing pre-trained models with hundreds of millions of parameters. These models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0022] Large Language Models (LLMs) are deep learning models trained on massive amounts of text data, enabling them to generate natural language text or understand the meaning of language text. LLMs can provide in-depth knowledge and language production on a wide range of topics through training on large datasets. They learn patterns and structures of natural language through large-scale unsupervised training, mimicking human language cognition and generation processes to some extent. LLMs employ a similar Transformer architecture and pre-training objectives as smaller models, with the main differences being increased model size, training data, and computational resources. Compared to traditional Natural Language Processing (NLP) models, LLMs can better understand and generate natural text, while also exhibiting some logical thinking and reasoning abilities. LLMs possess context learning capabilities. Context learning (LLM) capabilities enable learners to learn complex patterns in language and perform a wide range of tasks, including text summarization, translation, sentiment analysis, multi-turn dialogue, and more. For example, LLM can include the GPT series, T5 (Text Learning Model), and other related technologies. to Text Transfer Transformer (CT) model, PaLM model, BERT, LLaMA (Large Language Model MetaAI) model, Tongyi Qianwen model, Bailing model, etc.

[0023] Target Evaluation Metrics: Target evaluation criteria used to measure the performance of large language models, including public leaderboard metrics, internal benchmark scores, task-specific sub-metrics, or comprehensive scores.

[0024] Target Evaluation Set: A set of samples corresponding to the target evaluation metric, including publicly available benchmark sets or internal enterprise evaluation sets. In practical applications, the target evaluation set contains several test samples with labeled ground truth or reference standards, and its content remains unchanged throughout the entire automated training iteration process to ensure the comparability of training results in different rounds.

[0025] Automated Research Loop (autoresearch): A large-scale model iterative optimization mechanism that requires no human intervention. It automatically performs training, evaluation, policy generation, and effect feedback, and decides whether to retain, adjust, or eliminate policies based on the feedback results, thus forming a closed-loop iteration.

[0026] Automatic Training Data Construction: The process of selecting high-value samples from the existing data pool and / or automatically generating, rewriting, and enhancing new samples based on the model's performance on the target evaluation set and the identified capability gaps.

[0027] High-value training data refers to training samples that contribute significantly to improving the target evaluation metric, as opposed to high-quality generalization data. In at least some embodiments of this specification, the high-value training data includes a set of samples determined from the original data pool based on historical evaluation feedback and current strategy configuration, which are expected to improve the model's performance on the target evaluation metric.

[0028] Capability Gap: Weaknesses exhibited by the model on the target evaluation set, including but not limited to long sequence understanding, color binding, structural topology recovery, and local alignment. In at least some embodiments of this specification, capability gap refers to a specific capability dimension in which the large language model to be trained performs below a preset threshold or degrades compared to its historical peak performance on the target evaluation set.

[0029] Data Value Score: A score used to quantitatively evaluate the potential of a candidate sample to improve the target evaluation metric.

[0030] Data Action: An operation performed on training data, including at least one of filtering, augmentation, generation, rewriting, upsampling, curriculum-based enabling, retention, and culling. In at least some embodiments of this specification, a data action refers to a specific processing operation performed on the original or intermediate data during the automatic construction of training data.

[0031] In the training of large language models, common technical approaches to improve model performance mainly fall into three categories. The first category involves adjusting the model itself, such as modifying the network structure, loss function, or hyperparameters. The second category involves improving training methods, such as lesson learning, sampling strategies, knowledge distillation, or alignment techniques. The third category involves optimizing training data, such as supplementing samples, data cleaning, or data augmentation.

[0032] Among the three technical approaches mentioned above, optimizing training data is considered to have a crucial impact on model performance. However, current methods for processing training data are generally quite crude. Common practices include: performing only static data cleaning, which makes it difficult to determine which data can truly improve the model's performance on the target evaluation task; performing generalized data augmentation without addressing specific weaknesses exposed by the model in the target evaluation set; relying on manual analysis of error cases to supplement data, resulting in high costs, poor reproducibility, and difficulty in scaling; or using automated machine learning methods, but their optimization targets are usually limited to model structure and hyperparameters, failing to incorporate "data construction strategies" into the research loop. It is evident that current methods for selecting and constructing training data often rely on human experience, lacking systematicity and adaptability.

[0033] To address the aforementioned situation, this application proposes an automated training method for large-scale models based on training data optimization, constructing a closed loop of data construction and model training that is automatically iterated and driven by the target evaluation metric. Specifically, it provides a mechanism that effectively connects the following steps: first, analyzing the model's score loss on the target evaluation set; then, determining what kind of data needs to be supplemented; next, automatically selecting or generating corresponding data from a large-scale data pool; then, automatically executing training and backtesting the results; and finally, deciding whether the data strategy for this round is worth retaining.

[0034] Through the above technical solution, this application effectively solves the problems of data construction and model training separation and reliance on human experience in existing technologies. First, this solution always focuses on the target evaluation metric, rather than generalizing and expanding the data, thus ensuring the targeted nature of data optimization. Second, through an automated research closed loop, the data construction strategy is incorporated into the automatic iteration framework, achieving full automation of the "evaluation → attribution → construction → training → backtesting → strategy retention / elimination" process. Third, using the target evaluation performance as the basis for verifying the true marginal benefit of data actions avoids ineffective data expansion. Finally, this solution continuously iterates, selects, constructs, and optimizes high-value training data during automatic training. Compared to the limitations of traditional Automated Machine Learning (AutoML), which only optimizes the model and hyperparameters, uses static data selection based solely on generalization quality scoring, lacks targeted capabilities in general data augmentation, and suffers from high costs and non-scalability of manual data supplementation, this solution provides a more efficient, scalable, and self-optimizing closed-loop technical path. It can automatically drive data construction research actions around the target evaluation metric, ultimately improving model performance and training resource utilization efficiency.

[0035] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0036] This specification proposes a large-scale automated training method for models, which is driven by target evaluation metrics and organized by an automated research closed loop. Its advantage lies in its ability to automatically identify the model's capability gaps on the target evaluation set, and to dynamically construct training data based on data value scores to obtain high-value training data. Through continuous iteration, the model's performance can be self-evolved.

[0037] The methods described in this specification can be deployed on various computing devices, including but not limited to: single servers, distributed computing clusters, cloud computing nodes, edge computing devices, and dedicated training machines equipped with graphics processing units or tensor processing units. Applicable computing devices range from personal workstations and enterprise data centers to public cloud service platforms, as long as they possess the basic computing resources to perform large language model training and inference.

[0038] In terms of deployment, the methods described in this specification offer good flexibility. They can be deployed as independent training server software, allowing users to initiate training tasks and monitor the iteration process via a command-line interface or graphical user interface; they can also be integrated into existing model training pipelines as client tools, working collaboratively with data management platforms and evaluation systems; and they can employ a cloud-local collaborative model. Furthermore, the methods described in this specification can be embedded into various model development platforms, such as large language model training platforms, multimodal model training platforms, automated experiment orchestration platforms, data engine platforms, and training systems for specific tasks (such as graphical code generation and structured sequence output). A typical application scenario is an automated training optimization process for a specified evaluation set. For example, in an SVG (Scalable Vector Graphics) multimodal training system, target evaluation drives data to construct a closed loop to continuously improve the model's geometric similarity and rendering success rate on the internal SVG evaluation set.

[0039] Figure 1 This is a schematic diagram of the overall process of an automated training method for large models provided in one embodiment of this specification.

[0040] like Figure 1 As shown, this method achieves iterative optimization of the large language model to be trained by constructing a closed-loop feedback mechanism that includes global state information. Figure 1 As shown, the process mainly involves data pooling, strategy generation, dataset determination, training, evaluation, diagnosis, and decision-making and iteration modules based on global state information.

[0041] Specifically, the process begins with obtaining global state information. For example... Figure 1 As shown on the right, the global state information is a continuously updated knowledge base, whose internal structure includes: historical training parameter configurations, historical data construction strategies, and historical evaluation data. The historical evaluation data is further subdivided into historical performance indicator records and historical problem diagnosis records (including capability gap information). This historical information originates from the training and evaluation results of previous rounds, providing contextual basis for strategy generation in the current round.

[0042] While acquiring global state information, the system also reads a predefined rule base (such as...). Figure 1 As shown above, the rule base contains constraints and methodological guidance. Then, the step "Generate the strategy for the current round" is executed. In this step, the large language model inference engine is invoked, combining the global state information (especially historical data construction strategies, historical training parameter configurations, and historical evaluation data) with the rule base to generate the data construction strategy and training parameter configuration for the current round.

[0043] Next, the data processing phase begins. The system retrieves raw data from the data pool on the left and executes the "determine training dataset" step. This step constructs a strategy based on the generated data of the current round, filtering or constructing high-value samples from the data pool. Specifically, the system calculates the data value score of candidate samples and filters out samples that meet a preset threshold; if the number of filtered samples is insufficient, new samples are generated based on capability gap information in historical evaluation data, and finally combined to form the training dataset for this round of training.

[0044] Next, the "training" step is performed. Using the defined training dataset and based on the generated training parameter configuration for the current round, the large language model to be trained is incrementally trained to obtain an updated large language model.

[0045] After training, the "evaluation" and "diagnosis" steps are executed sequentially. In the evaluation phase, samples from the target evaluation set are input into the updated large language model, the model output is obtained and compared with the labeled answers, and a quantized score is calculated to statistically determine the current performance metric. In the diagnosis phase, clustering and attribution analysis are performed on samples with quantized scores below a preset threshold to identify weaknesses in the model across specific dimensions, thereby generating current problem diagnostic information containing capability gap information. The current performance metric and the current problem diagnostic information together constitute the current evaluation data.

[0046] Afterwards, the process enters the decision node "Stop conditions met". Before this node, the system performs the operation of "updating the current evaluation data to the global state information". Specifically, this includes: writing the current performance metric to the performance metric history record and writing the current problem diagnosis information to the problem diagnosis history record. At the same time, based on the comparison results of the current performance metric with the historical peak performance metric or the performance metric of the previous round, the effectiveness of the data construction strategy in the current round is determined: if performance improves, the current strategy is marked as an effective strategy and the successful experience is recorded; if performance deteriorates, the current strategy is marked as an invalid strategy and the failure mode (including the invalid strategy and the corresponding performance degradation) is recorded.

[0047] If the judgment result is "yes" (i.e. the stopping condition is met, such as reaching the target threshold or convergence), the process terminates and the trained model is output.

[0048] If the judgment result is "No" (i.e., the stopping condition is not met), the process enters the "Continue Iterative Training" branch. At this time, based on the updated global state information (including the latest evaluation data of this round, policy effectiveness markers, and failure / success modes), the system may optionally call the large language model data analyzer to generate policy adjustment suggestions before entering the next iteration, and write the suggestions into the global state information. Subsequently, the process returns to the "Generate Policy for the Current Round" step, using the updated global state information to generate the data construction policy and training parameter configuration for the next round, thereby starting a new round of automated training loop.

[0049] To better understand the practical application of the embodiments in this specification, a specific user interaction example is given below. Suppose we need to optimize a large multimodal model for SVG graphics generation. The user can pre-configure the target evaluation set and set stopping conditions based on the optimization target. The system automatically executes the first iteration: calling the large language model inference engine to read the initial global state information (at this time, the historical data construction strategy is empty, and the historical evaluation data is blank), generating a default data construction strategy. Subsequently, the system selects a training dataset from the data pool according to this strategy, performs incremental training on the current model, and evaluates it on the target evaluation set to obtain the current performance indicators and current problem diagnosis information. The system updates the evaluation data to the global state. Assuming the stopping condition is not met, it enters the second iteration—at this time, the large language model inference engine can generate a more targeted data construction strategy based on the historical evaluation data from the first round (including performance indicators and problem diagnoses). In practical applications, during the iterative training process, the user can view the performance indicator curves for each round, the evolution trend of capability gaps, and the list of effective and ineffective strategies automatically marked by the system on the control panel in real time. When the stopping condition is finally met, the system automatically outputs the trained model and generates an iteration report, summarizing the data actions used in each round and comparing their benefits. This process eliminates the need for users to manually analyze losing samples or write data filtering rules, significantly improving the efficiency and automation of model optimization.

[0050] The embodiments of this specification provide a method for automated training of large models. This application also relates to an automated training device for large models and a computing device, which will be described in detail in the following embodiments.

[0051] Figure 2 This is a flowchart illustrating an automated training method for large models provided in an embodiment of this specification.

[0052] From a programming perspective, the entity executing the process can be a program hosted on an application server. It can be understood that this method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities.

[0053] like Figure 2 As shown, the process may include the following steps: Step 202: Obtain global state information; the global state information includes historical data construction strategy and historical evaluation data; the historical evaluation data is the result obtained by the large language model to be trained in previous rounds on the target evaluation set.

[0054] The target evaluation set may include one or more publicly available benchmark sets and / or internal enterprise evaluation sets. The target evaluation set is applicable to evaluating at least one task among image, text, code, structured sequence, or graphical description languages. The large language model to be trained may be a text generation model, a multimodal model, or a graphical code generation model.

[0055] In some embodiments, the computer system executing the large model automated training method includes a processor and memory. In step 202, the processor retrieves a global state identifier from memory.

[0056] Furthermore, the global state information can be stored in the form of a structured state file (e.g., stat_state.json).

[0057] The data construction strategy is used to specify the rules for filtering or generating training data. The data construction strategy may include at least one of the following data actions: data filtering rules, data sampling ratio, data augmentation methods, data generation templates, data bucket selection strategies, data weight allocation strategies, etc. The historical data construction strategy is generated by the large language model inference engine in previous rounds and written into the global state information after automatic verification. In the initial rounds, the historical data construction strategy may be empty or contain preset default data construction rules. The historical data construction strategy is stored in the experience base of the memory in the form of a structured strategy file.

[0058] The historical evaluation data refers to the collection of various result information obtained from evaluating the large language model to be trained on the target evaluation set in previous rounds (i.e., several iterations before the current round). For example, this result information may include performance metrics (such as scores, accuracy, similarity, and other quantitative values) and problem diagnostic information (such as qualitative or categorical descriptions of capability gaps and clusters of samples with lower scores). Historical evaluation data is an important component of the global state information, providing feedback for the generation of data construction strategies in subsequent rounds. The historical evaluation data is stored in a state file in memory in the form of a structured strategy file.

[0059] Based on some embodiments of this specification, a structured state file can be used to store state data and experience from multiple rounds. Subsequently, by obtaining global state information containing historical data construction strategies and historical evaluation data, a complete historical context for generating data strategies in subsequent rounds can be obtained. This design enables the system to make decisions based on past effective experiences and lessons learned, avoiding the drawback of blindly constructing data from scratch in each round, thereby improving the continuity and efficiency of iterative optimization.

[0060] In one or more embodiments of this specification, in order to further clarify which strategies in the historical data construction strategy are worth retaining and which need to be avoided, the historical data construction strategy includes retained effective strategies and eliminated ineffective strategies; wherein, the effective strategy is the strategy that generates positive benefits to the target evaluation index on the target evaluation set, and the ineffective strategy is the strategy that generates negative benefits to the target evaluation index on the target evaluation set.

[0061] Furthermore, the global state information may also include the policy identifier of the effective policy and the corresponding positive reward value, which can be used as a positive example input when calling the large language model inference engine in subsequent rounds; and / or, include the policy identifier of the invalid policy and the corresponding negative reward value, which can be used as a negative example to prompt the large language model inference engine to avoid repeating the same data actions.

[0062] Among them, the retained effective strategies refer to data construction strategies that, after being adopted in a previous round and trained and evaluated, have resulted in an improvement in the target evaluation metric compared to the previous round. These strategies are marked as "effective" and stored in the global state information, serving as positive examples for the large language model inference engine to reference in subsequent rounds, thereby guiding the generation of similar strategies.

[0063] A positive return refers to an improvement in the current performance metric measured on the target evaluation set after adopting a certain data construction strategy, compared to the performance metric before adopting the strategy (such as the performance metric from the previous round or historical peak performance metric), or an improvement exceeding a preset threshold. A positive return can be quantified as a specific numerical value of performance improvement (e.g., an increase of 0.02 in the average crossover ratio).

[0064] Invalid strategies, which have been phased out, refer to data construction strategies that, after being adopted in a previous round and subsequently trained and evaluated, caused the target evaluation metric to decline compared to previous rounds. These strategies are marked as "invalid" and excluded from the set of available strategies in subsequent iterations. They also serve as negative examples to remind the large language model inference engine to avoid generating data actions with similar characteristics.

[0065] Negative returns refer to a situation where, after adopting a certain data construction strategy, the current performance metric measured on the target evaluation set decreases compared to the performance metric before adopting the strategy, or the decrease exceeds a preset tolerance threshold. Negative returns can be quantified as a specific numerical value of performance degradation (e.g., an increase of 5 bad cases).

[0066] For example, suppose that in the second iteration, the system adopted a data construction strategy of "upsampling from complex data buckets." After training and evaluation, the target evaluation metric (such as the mean intersection-over-union ratio) improved by 0.02. This strategy can then be marked as effective, and its strategy identifier and reward value are stored in the global state information. In the subsequent third iteration, when calling the large language model inference engine, this effective strategy can be input as a positive reference example, guiding the engine to prioritize similar data actions. Conversely, if a round uses "blindly upsampling low-quality samples," resulting in performance degradation, this strategy is marked as ineffective. Its identifier and negative reward value will prompt the engine to avoid using similar actions again.

[0067] Based on some embodiments of this specification, by classifying historical data construction strategies into effective and ineffective strategies and recording their corresponding reward values, an accumulative and reusable strategy experience base can be formed. When subsequently calling the large language model inference engine, these positive and negative examples can be used for prompt learning, guiding the engine to generate better data construction strategies, thereby reducing invalid trial and error, accelerating convergence, and improving the overall stability of training and optimization.

[0068] In one or more embodiments of this specification, the evaluation results are decomposed into quantifiable performance metrics and problem diagnostic information that can be used for attribution. The historical evaluation data includes historical performance metrics and historical problem diagnostic information; the current evaluation data includes current performance metrics and current problem diagnostic information.

[0069] The performance metric history refers to the collection of numerical results used to quantify the model performance obtained from evaluating the large language model to be trained on the target evaluation set in one or more previous iterations. The performance metric history is stored in epoch order, and the performance metrics corresponding to each epoch may include, but are not limited to, the overall score, subtask scores, mean IoU, median IoU, number of bad cases, task success rate (such as rendering and parsing success rate), structural similarity metrics (such as SSIM, LPIPS, DINO), and sub-scores on multiple capability dimensions.

[0070] The current performance metric refers to the numerical result obtained in the current iteration round by evaluating the updated large language model on the target evaluation set, which is used to quantify the current performance of the model. This current performance metric may include, but is not limited to, the overall score, subtask scores, average intersection-over-union ratio, median intersection-over-union ratio, number of bad cases, task success rate, structural similarity index, and sub-scores on multiple capability dimensions.

[0071] Problem diagnosis history refers to the data set obtained from evaluating the large language model to be trained on the target evaluation set in one or more previous iterations, used to describe the model's performance defects or shortcomings. Problem diagnosis history is stored sequentially by iteration, and the problem diagnosis information corresponding to each iteration may include, but is not limited to, the distribution of capability gaps, clusters of samples that failed to score, labels of reasons for failure, and the association mapping relationship between samples that failed and candidate samples in the training data pool.

[0072] Current problem diagnostic information refers to the data results obtained in the current iteration round by evaluating the updated large language model on the target evaluation set, which describes the current model's performance defects or shortcomings. This current problem diagnostic information may include, but is not limited to, the distribution of capability gaps, clusters of samples that have failed to achieve their goals, labels for the reasons for failures, and the correlation mapping between samples that have failed to achieve their goals and candidate samples in the training data pool.

[0073] Based on some embodiments of this specification, by decomposing the evaluation data into historical performance metrics and historical problem diagnosis metrics, and storing quantitative scores and qualitative problem descriptions respectively, the system can simultaneously obtain numerical feedback on "how well the model performed" and attributional feedback on "where the model's weaknesses lie" in subsequent iterations. This dual-track recording method provides a comprehensive and accurate basis for capability gap attribution, data strategy generation, and stopping condition determination in subsequent steps, thereby improving the directionality and interpretability of automated training.

[0074] In one or more embodiments of this specification, the problem diagnosis history includes capability gap information of the large language model to be trained on the target evaluation set, wherein the dimension of the capability gap includes at least one of the following: task type dimension, sample length dimension, structural complexity dimension, and semantic mapping dimension.

[0075] Capability gap information refers to data describing the deficiencies or weaknesses of a large language model on the target evaluation set relative to the expected performance standard. This information may include the dimension to which the capability gap belongs, the severity of the gap, the characteristics of the corresponding lost samples, and the priority ranking of the gaps.

[0076] The dimensions of a capability gap can refer to the aspects or attributes used to classify capability gaps, distinguishing different types of inadequacies. For example, capability gaps can be classified according to one or more dimensions such as task type, sample length, structural complexity, and semantic mapping.

[0077] The task type dimension refers to the dimension that categorizes capability gaps based on the task categories to which the samples in the target evaluation set belong. For example, for multimodal models, task types may include image description generation, visual question answering, and graphical code generation; for text models, they may include summarization generation, machine translation, and sentiment analysis.

[0078] The sample length dimension refers to the dimension by which capability gaps are divided according to the sequence length of samples in the target evaluation set. The sequence length can be determined based on the specific data type, such as the number of tokens in text, the number of paths or primitives in a graph, or the number of lines or characters in code. The capability gaps exhibited by the model may differ across different length ranges.

[0079] The structural complexity dimension refers to the dimension by which capability gaps are categorized based on the structural complexity of samples in the target evaluation set. Structural complexity can be reflected in nesting levels, the number of branches, dependencies between elements, and the number of nodes or edges in the topology. For example, in graph generation tasks, structural complexity can be reflected in the number of path segments, the number of primitives, or the complexity of the transformation matrix.

[0080] The semantic mapping dimension refers to the dimension used to categorize capability gaps based on the semantic correspondence between model inputs and outputs. This dimension focuses on whether the model can accurately understand the semantic information in the input and map it to the correct output representation. For example, semantic mapping can include binding colors to shapes, associating attributes with objects, mapping logical relationships to symbolic representations, and mapping spatial relationships to coordinate descriptions.

[0081] For example, for SVG graphics generation tasks, capability gaps can manifest as "insufficient color binding capability" (i.e., the model cannot accurately associate a specified color with a graphic object), "weak ability to recover the topology of long sequence structures" (i.e., for complex SVGs containing multiple sub-paths, the structure output by the model is missing or disordered), or "poor local alignment accuracy" (i.e., the position coordinates of graphic elements deviate significantly from the reference standard). During the evaluation process, the system can identify these capability gaps by analyzing the common characteristics of the samples that failed to score.

[0082] Furthermore, the historical evaluation data may also include a priority ranking of capability gaps generated after each round of evaluation. The priority ranking is calculated based on the degree of negative impact of each capability gap on the target evaluation index, and is used to guide the capability directions to be focused on in the next round of data construction strategy.

[0083] In practical applications, the historical evaluation data can be stored in the experience base in chronological order. Furthermore, the historical evaluation data may include initial round evaluation data, where the large language model to be trained in the initial round is the original model that has not been optimized by this method, and the initial performance metrics and initial problem diagnosis records obtained from the evaluation on the target evaluation set serve as the initial values ​​for the global state information.

[0084] Based on some embodiments in this specification, by refining capability gaps into multiple dimensions (such as task type, sample length, structural complexity, and semantic mapping) and recording the priority of each capability gap, the system can accurately pinpoint the direction in which the current model most needs improvement. This multi-dimensional description of capability gaps allows subsequent data construction strategies to be tailored to specific needs, i.e., targeted selection or generation of training samples for high-priority gaps, thereby avoiding the inefficient investment brought about by generalization enhancement. Simultaneously, the recording of initial rounds of evaluation data provides a baseline reference for subsequent iterations, facilitating the quantification of the real gains brought about by each strategy adjustment.

[0085] In one or more embodiments of this specification, in order to further optimize the hyperparameter adjustment and data construction strategy in coordination during the training process, the global state information also includes historical training parameter configurations; the invocation of the large language model inference engine also generates the training parameter configuration for the current round based on the historical training parameter configurations; during incremental training, training is performed based on the training parameter configuration for the current round.

[0086] In practice, the training parameter configuration for the current round can be generated by the large language model inference engine based on historical evaluation data, historical data construction strategies, and historical training parameter configurations in the global state information, and output together with the data construction strategy in the same strategy file. The training parameter configuration specifies the hyperparameters for model training and may include learning rate, batch size, gradient accumulation steps, training rounds, optimizer type, etc.

[0087] The historical training parameter configuration is the set of training hyperparameters used in previous training rounds of the large language model to be trained. The historical training parameter configuration may include at least one of the following: learning rate, minimum learning rate, training rounds, batch size, gradient accumulation steps, optimizer type, learning rate decay strategy, and model weight initialization method. The historical training parameter configuration can be stored in association with the data construction strategy, performance metrics, and problem diagnosis records of the corresponding rounds, forming the experimental record for that round.

[0088] Based on some embodiments of this specification, by incorporating training parameter configurations (such as learning rate and batch size) into the global state information and generating them together with the large language model inference engine and data construction strategy, joint optimization of "what data to use for training" and "how to train" is achieved. This collaborative design allows training hyperparameters to adapt to changes in data distribution brought about by different data construction strategies. For example, when the sampling strategy introduces more complex samples, the system can automatically reduce the learning rate or increase the gradient accumulation steps to stabilize training. Simultaneously, historical training parameter configurations are associated and stored with the evaluation data of the corresponding rounds, providing a data foundation for subsequent analysis of the relationship between different parameter combinations and performance improvement, further enhancing the decision-making capability of the automated research closed loop.

[0089] Step 204: Invoke the large language model inference engine to generate the data construction strategy for the current round based on the global state information.

[0090] The large language model inference engine can be a collection of one or more large language model instances. Its input is the global state information (including historical data construction strategies, historical evaluation data, etc.), and its output is a structured data construction strategy file. The data construction strategy is used to specify the selection rules, sampling ratio, data source, data generation template, or data weight allocation method for the current round of training data.

[0091] For example, suppose the global state information records the evaluation results of the first two iterations, showing that the model has a low average intersection-union ratio (IU) on "complex structure samples" and that there are multiple samples with low IU. Based on this, the large language model inference engine can generate the following data construction strategy: sample 5000 samples from the "high complexity bucket" in the data pool, assign these samples an upsampling weight of 1.2 times, and adjust the learning rate to 0.8 times the initial value. This strategy is saved in JSON or a similar structured format for use in subsequent steps.

[0092] Based on some embodiments of this specification, by calling the large language model inference engine to automatically generate data construction strategies based on global state information, the data strategy formulation process can be freed from dependence on human experience. Since the engine can integrate performance indicators and problem diagnostic information from historical evaluation data, the generated strategies can more effectively compensate for the current model's capability gaps, thereby improving the rationality and efficiency of data construction and providing a suitable data foundation for subsequent incremental training.

[0093] In one or more embodiments of this specification, in order to further constrain the scope of the large language model inference engine's generation strategy and make its output more in line with the actual training needs, a predefined rule base is also obtained before the large language model inference engine is invoked; when the large language model inference engine is invoked, the large language model inference engine generates the data construction strategy for the current round based on the global state information and the rule base.

[0094] The predefined rule base includes at least one of the following: data cleaning rules, data bucket classification rules, data action constraint rules, data construction rules, and sampling strategy rules.

[0095] Furthermore, the data cleaning rules refer to a set of predefined conditions used to filter samples in the original data pool, removing samples that do not meet preset quality requirements or training objectives. Taking image rendering tasks as an example, the data cleaning rules may include at least one of the following: rules based on data source discarding, rules based on color semantic tags, rules based on geometric coordinate conflicts, rules based on command repetition, and rules based on sequence length exceeding limits. For example, the data cleaning rules can form a progressive filtering chain in the following order: first, execute the rules based on data source discarding and the rules based on color semantic tags; then execute the rules based on geometric coordinate conflicts and the rules based on command repetition; finally, execute the rules based on sequence length exceeding limits. For example, for SVG graphics code, the system can first discard samples from untrusted sources based on data source tags; then check whether color semantics can be correctly parsed, removing samples with abnormal color values ​​or those that cannot be mapped; next, detect whether geometric coordinates have quantization conflicts (such as coordinates exceeding canvas boundaries); then detect whether there are duplicate command fragments; finally, filter out samples whose sequence length exceeds the model's maximum processing capacity.

[0096] Furthermore, the data bucket classification rules refer to a set of classification criteria that pre-divide samples in the data pool into different category sets according to their characteristics, to guide the directional selection during data sampling. The category set may include at least one of the following: medium complexity bucket, high complexity bucket, structure-related bucket, gradient-related bucket, defective bucket, rejected sample bucket, and incomplete sample bucket. For example, samples with moderate complexity and no obvious errors can be classified into the "medium complexity bucket," samples containing a large number of nested paths or complex transformations can be classified into the "high complexity bucket," and samples with geometric topological errors can be classified into the "defective bucket."

[0097] Furthermore, the data action constraint rules refer to a set of restrictions on the types of operations allowed to be performed on the training data, thus defining the permitted data action types. These data action types include at least one of the following: data filtering, data augmentation, data generation, data rewriting, data upsampling, curriculum-based activation, data retention, and data eviction. These actions can process the training data at different stages; for example, in the data filtering stage, only samples with value scores higher than a threshold can be retained, and in the data generation stage, samples can be expanded using rewriting or template generation methods.

[0098] Furthermore, the data construction rules refer to a set of predefined methods for automatically generating new training data samples, which define the way new samples are automatically constructed. These methods may include at least one of the following: rewriting existing samples, template-based generation, generation according to program rules, counterfactual generation, generation with controlled structural complexity, and generation driven by a large language model. For example, when it is necessary to supplement samples related to the "color binding" capability gap, a counterfactual generation method can be used: take an existing sample, randomly replace its color values ​​with other reasonable colors, and adjust the text description accordingly to construct a new training sample.

[0099] Furthermore, the sampling strategy rule refers to a set of predefined strategies regarding the proportion, weight allocation, or selection order of training data samples selected from the data pool or data bucket. The sampling strategy rule can define at least one of the following sampling methods: upsampling, downsampling, curriculum-based sampling, and upsampling of difficult samples based on backtracking of loss values ​​during training. For example, for "long sequence samples" indicating the current capability gap, an upsampling strategy can be used to increase their sampling probability to 1.5 times that of ordinary samples; for simple samples that have already performed well, appropriate downsampling can be used to reduce training redundancy.

[0100] The predefined rule base can be stored in the skill library directory as structured files such as editable script files or configuration files. It is dynamically loaded when the large language model inference engine is invoked to constrain the scope and legality of the data construction strategy generated by the engine. For example, the rule base can be stored as a JSON schema or a Python script. The system reads these rules before invoking the engine and appends them as system prompts or constraints to the engine's input context.

[0101] The data construction strategy generated by the large language model inference engine in step 204 for the current round can be a structured strategy file. This structured strategy file can include data candidate source identifiers, sampling bucket categories, upper limits on the number of samples, data weight allocation strategies, and explanations of the strategy generation basis. For example, a strategy file can contain the following: the data candidate source is "mmsvg cleaned dataset," the sampling bucket category is "high complexity bucket," the upper limit on the number of samples is 3000, the weight allocation strategy is "upsampling weight 1.2," and the strategy generation basis is "Based on the capability gap analysis of the first two rounds, the loss rate of complex structure samples is relatively high, therefore, such samples should be prioritized for supplementation."

[0102] Based on some embodiments in this specification, by introducing a predefined rule base to constrain the scope of the generation strategy of the large language model inference engine, the generated strategy can be both flexible and stay within the feasible operational space. Specifically, data cleaning rules ensure the basic quality of training data, preventing low-quality samples from contaminating the training process; data bucket classification rules allow data sampling to be directed to specific feature sets, improving the purposefulness of sampling; data action constraint rules clarify the allowed operation types, preventing the generation of unexecutable actions; data construction rules provide multiple methods for generating new samples, solving the problem of insufficient data; and sampling strategy rules refine the specific methods of sample selection. The combined effect of these rules makes the strategy generated by the large language model inference engine more reliable and implementable, reducing the workload of subsequent manual verification and correction. Furthermore, storing the rule base in the form of an editable script allows users to customize it according to specific task requirements, enhancing the adaptability and scalability of the method.

[0103] In one or more embodiments of this specification, in order to ensure the stable operation of the large language model inference engine under high availability requirements and to avoid the interruption of the entire iteration process due to the failure of a single model call, the large language model inference engine includes multiple candidate large language models, including a main large language model and at least one backup large language model.

[0104] The invocation of the large language model inference engine specifically includes: invoking the main large language model; if the invocation of the main large language model fails (e.g., network timeout, unexpected return format, service unavailability, or return error code), then the backup large language models are invoked sequentially; if all backup large language models fail to be invoked, then the heuristic rule engine is invoked to generate a default data construction strategy based on the global state information.

[0105] In practical deployments, a set of large language model service endpoints can be pre-configured, sorted by priority from high to low. For example, the primary model could be a high-performance but potentially costly commercial model, while the backup model could be an open-source model or a locally deployed lightweight model. When the primary model becomes unresponsive due to traffic limitations or service failure, the system automatically switches to the first backup model; if the backup model also fails, it continues to try the next backup model. When all models are unavailable, a heuristic rule engine acts as a fallback solution, generating a feasible data construction strategy based on pre-defined simple logic (such as randomly selecting one from historically effective strategies or using the default uniform sampling strategy) to ensure that the training process can continue without complete interruption.

[0106] The heuristic rule engine can contain a set of hand-written rules or templates, such as: "If there is a strategy in the historical effective strategies that targets high-priority capability gaps, then reuse that strategy and reduce the exploration intensity; otherwise, adopt the default uniform sampling strategy by data bucket." Although these rules are not as refined as those generated by large language models, they can maintain the availability of the system under abnormal conditions.

[0107] Based on some embodiments in this specification, by configuring multiple backup large language models in addition to the main large language model and a heuristic rule engine as a fallback mechanism, the robustness and fault tolerance of the automated training process can be significantly improved. When the main model fails to function properly due to network fluctuations, service overload, or incorrect output format, the system can automatically switch to the backup model to continue executing the policy generation task, avoiding iteration interruption caused by a single point of failure. Even in the extreme case where all models are unavailable, the heuristic rule engine can still provide a usable default policy, ensuring the continuity of the training process. This multi-level fallback design allows the entire method to better adapt to various uncertainties in real-world production environments, reduces strong dependence on external services, and improves the overall stability and automation of the system.

[0108] Step 206: Determine the training dataset according to the data construction strategy.

[0109] Specifically, the system can construct a data sample set for incremental training in the current round based on the data generated in the current round, selecting or constructing it from a preset data pool. This training dataset can include samples selected from an existing data pool, automatically generated new samples, or a combination of both.

[0110] For example, if the data construction strategy specifies sampling from a "high-complexity bucket" and requires that the samples be related to a "color binding" capability gap, the system can retrieve samples that meet these conditions from the data pool to form the initial training dataset.

[0111] Based on some embodiments of this specification, by automatically determining the training dataset according to a data construction strategy, the data used in each round of training can be matched with the capability gap of the current model, avoiding the waste of resources caused by blindly expanding the data. Since the dataset selection process is driven by strategy rather than human experience, it has better reproducibility and scalability.

[0112] In order to select the most effective samples for improving the current model from the existing data pool, in one or more embodiments of this specification, the step of determining the training dataset according to the data construction strategy includes: selecting samples whose data value scores meet the preset score threshold from the preset data pool based on the data construction strategy, as the training dataset; the data value score is determined based on one or more of the following factors: the degree of matching with the ability gap of the large language model to be trained on the target evaluation set, the redundancy with the existing training set, the data quality credibility, the data construction cost, and the estimated marginal benefit.

[0113] The data value score refers to a numerical value used to quantitatively evaluate the potential contribution of candidate samples to model training. This score can be calculated based on one or more factors, including the degree of matching between the candidate sample and the current capability gap, the degree of similarity with existing training samples, the quality and reliability of the sample itself, the resource cost required to obtain or generate the sample, and the expected improvement in model performance.

[0114] A preset scoring threshold refers to a pre-defined numerical boundary used to divide data value scores into different intervals. The threshold can be a fixed constant or dynamically adjusted based on the current iteration, available data volume, or training budget. Samples with data value scores higher than the threshold are considered high-value samples, and samples lower than the threshold are considered low-value samples.

[0115] The matching degree with the capability gap refers to the degree of similarity between the knowledge, skills, or task features involved in the candidate sample and the capability shortcomings exposed by the current model. A high matching degree means that the sample may contain training signals that can help the model compensate for specific capability deficiencies. For example, if the capability gap is "insufficient ability to recover topology of long sequence structures", then a graph sample containing multiple complex paths may have a high matching degree.

[0116] Redundancy with the existing training set refers to the degree of repetition or similarity between a candidate sample and the set of samples already used or to be used for training, at the level of feature space, content representation, or semantics. High redundancy indicates that the sample provides less new information and its marginal contribution to model training may be low; low redundancy indicates that the sample can bring more new information.

[0117] Data quality credibility refers to a comprehensive assessment of the accuracy, standardization, completeness, and reliability of candidate samples. Data quality credibility can be determined based on factors such as the credibility of the sample source, whether the format conforms to standards, the correctness of annotations or reference answers, and the presence of obvious errors or noise.

[0118] Data construction cost refers to the resource consumption required to obtain or generate candidate samples. These resources may include computation time, storage space, number of API calls, and manual review workload. Data construction cost can be expressed using standardized units (such as milliseconds, token counts, or point values) or relative levels (such as low, medium, and high).

[0119] The estimated marginal benefit refers to the expected performance improvement the model may achieve on the target evaluation metric after adding the candidate sample to the training dataset. This estimate can be derived based on historical experience (such as the historical benefits of similar samples), the current state of the model (such as its current score on the relevant capability dimension), or heuristic rules.

[0120] In practice, the system can calculate a comprehensive value score for each candidate sample in the data pool, for example, using a weighted summation method: score = w1*matching degree + w2*(1-redundancy) + w3*quality credibility - w4*construction cost + w5*estimated marginal benefit. The preset scoring threshold can be dynamically adjusted based on the current data budget and the expected sample size, or it can be set to a fixed value. Only samples with a value score higher than this threshold will be selected for the training dataset.

[0121] Based on some embodiments of this specification, by introducing a data value scoring mechanism and comprehensively considering multiple factors such as the degree of matching with capability gaps, redundancy with existing training sets, data quality, construction costs, and estimated marginal benefits, fine-grained screening of candidate samples can be achieved. Since the scoring dimensions cover the potential contribution of a sample to model improvement and the cost of acquiring that sample, the selected training dataset can maximize marginal benefits within a limited training budget. Simultaneously, by setting scoring thresholds, the size and quality of the training set can be flexibly controlled, avoiding the introduction of low-value or even negative-return samples, thereby improving training efficiency and model performance.

[0122] When the high-value samples in the existing data pool are insufficient to meet training requirements, new samples need to be automatically constructed to fill the capability gap. Therefore, in one or more embodiments of this specification, determining the training dataset according to the data construction strategy further includes: if there are insufficient samples in the preset data pool whose data value scores meet the preset scoring threshold, then generating new samples based on the capability gap information in the historical evaluation data, and adding the new samples to the training dataset.

[0123] The method of generating new samples may include at least one of the following: rewriting existing samples, generating based on templates, generating according to program rules, generating counterfactually, generating by controlling structural complexity, and generating by calling a large language model.

[0124] Rewriting existing samples refers to selecting an existing sample and modifying some of its content or attributes to generate a new sample. Rewriting operations can include replacing, deleting, inserting, or rearranging elements within the sample. For example, in an SVG graphic generation task, a sample describing a "red circle" can be selected, its color attribute rewritten from "red" to "blue," and the color term in the text description can be modified simultaneously to generate a new "blue circle" sample. Similarly, an SVG sample containing three paths can be taken, and the coordinates of one path can be fine-tuned (e.g., translated or scaled) to generate a new sample with a similar structure but different geometry. Rewriting existing samples is suitable for scenarios where there are sufficient existing samples, but it is necessary to increase sample diversity or expand specific features (such as color, size, or position).

[0125] Template-based generation refers to generating new samples by filling in variable parameters or placeholders within a pre-designed sample template. The template can contain a fixed structural framework and variable slots, each with a preset value range or generation rule. For example, an SVG template can be designed as follows: <svg> <rect x="{x}" y="{y}" width="{w}" height="{h}" fill="{color}" / > < / svg> Where {x}, {y}, {w}, {h}, and {color} are variable parameters. The system can randomly select from preset numerical ranges (e.g., x and y between 0 and 100, w and h between 10 and 50) and a color list, or fill according to rules, thereby generating a large number of rectangular graphic samples in batches. This template-based generation method is suitable for quickly generating a large number of samples with fixed structures and variable parameters, especially when the capability gap manifests as insufficient processing ability for a specific type of structure.

[0126] Generating samples according to program rules refers to automatically generating new samples by writing or calling program code (such as scripts, functions, or generators). Program rules can control the sample generation process based on preset syntax, logic, or statistical distributions, resulting in highly controllable and reproducible results. For example, for SVG graphics tasks, a recursive function can be written to generate tree-structured path combinations at random depths: the root node is... <g>Element, child nodes are <path> 、 <circle>The function generates a large number of new samples with different topological structures by randomly determining the position and style of each basic graphic element according to a certain probability distribution. This method of generation based on program rules is suitable for generating samples with specific mathematical structures or syntactic features, especially when the capability gap involves structural complexity, nesting levels, or combinational logic.

[0127] Counterfactual generation refers to constructing new samples that are semantically similar to the original samples but under different conditions by intervening counterfactually in existing samples (i.e., changing some key attributes of the sample while keeping others unchanged). Counterfactual generation aims to simulate scenarios of "what would happen if a certain condition changed," thereby helping the model learn more robust causal relationships. For example, assuming an SVG sample whose content is "a red circle is above a blue rectangle," counterfactual generation can change one of the colors to generate "a blue circle is above a blue rectangle," or change the positional relationship to generate "a red circle is below a blue rectangle." By comparing the original sample and the counterfactual sample, the model can learn key features of color binding or spatial relationships. Counterfactual generation is suitable for scenarios where the capability gap manifests as insufficient attribute binding (such as color-object association, position-relationship mapping) or causal reasoning.

[0128] Controlling structural complexity generation refers to explicitly controlling the structural complexity parameters (such as nesting depth, number of elements, number of path segments, branching factors, etc.) of new samples during generation, so that the generated samples cover different complexity levels. Progressively difficult sample sequences can be generated by setting complexity tiers (from low to high). For example, in an SVG generation task, complexity levels can be defined: Level 1 (simple) contains 1-2 basic graphic elements, with no nesting; Level 2 (medium) contains 3-5 elements, possibly with single-level nesting; Level 3 (complex) contains more than 6 elements, with multiple levels of nesting or complex transformations. The system can then call the corresponding generation rules to generate samples according to the specified complexity level. For example, for complexity level 3, a sample containing multiple... <g>Nested, each <g>Contains multiple <path>Furthermore, SVGs employ rotation or scaling transformations. The method of controlling structural complexity generation is suitable for capability gaps manifested as performance differences in models at different complexity levels, requiring targeted supplementation of training data within specific complexity ranges.

[0129] Large language model-driven generation refers to using a large language model (which can be the same model used for strategy generation or a dedicated data generation model) to automatically generate new training samples based on the input natural language description, example samples, or generation prompts. Large language models can learn from context to generate outputs with reasonable structure and semantics that match the given description. For example, you could input the following prompt into a large language model: "Please generate an SVG code describing a red square and a blue circle, with the square in the upper left and the circle in the lower right." The large language model might output a code containing... <rect>and <circle>The elements and coordinate positions conform to the described SVG code. For example, a short SVG template can be input as an example, requiring the large language model to generate multiple similar variations with different numbers of elements. This method of using a large language model to drive generation is suitable for generating highly diverse samples with rich linguistic semantics or those difficult to exhaustively enumerate through rules, especially when the capability gap involves complex semantic mappings (such as the conversion from natural language to graphical code).

[0130] In one optional implementation, the process of determining the training dataset according to the data construction strategy may include the following sub-steps: First, obtain the original samples from a preset data pool and calculate the data value score of each original sample relative to the capability gap in the data construction strategy. Second, select samples with data value scores higher than a preset score threshold as first-class samples and include them in the training dataset. Then, for the features in the capability gap information not covered by the first-class samples, invoke a data generation agent to automatically construct a second-class sample as a supplementary part of the training dataset.

[0131] Furthermore, the automatic construction of the second type of sample by calling the data generation agent may include: selecting the corresponding data construction template or generating prompt words based on the common error features in the capability gap information (e.g., structural missingness, coordinate conflict, or color binding errors commonly found in the samples that lost points); controlling the data generation agent to execute at least one of procedural rule generation, counterfactual generation, or large language model-driven generation to generate new samples that conform to the constraints of the common error features; and performing legality verification on the generated new samples, and eliminating samples that failed to parse, had abnormal formats, or had incomplete content.

[0132] For example, suppose capability gap information indicates that the current model performs poorly in "long sequence topology recovery." While some long sequence samples exist in the existing data pool, most have already been selected for the training dataset, resulting in high redundancy among the remaining samples. In this case, the system can invoke a data generation agent to generate a new batch of long sequence SVG samples based on a long sequence template by randomly inserting valid path segments, changing the path order, or adjusting coordinate parameters. After generation, the system will verify whether these samples conform to SVG syntax specifications and whether they will trigger rendering anomalies (such as coordinate out-of-bounds errors or unclosed labels). Only samples that pass the verification will be added to the training dataset.

[0133] In this way, when there are not enough high-value samples in the existing data pool, the system can automatically construct new samples targeting the capability gap features, thereby ensuring the sufficiency and diversity of the training dataset, while controlling the quality of the generated samples through legality verification.

[0134] Furthermore, to facilitate backtracking of anomalies during training, each data item in the training dataset can carry a globally unique identifier. When the training loss increases abnormally or gradient explosion occurs, the system can quickly locate the corresponding sample using this identifier, thereby determining whether the sample is a "hard sample" or "dirty data".

[0135] Based on some embodiments in this specification, by automatically triggering the generation of new samples when there are insufficient high-value screening samples, and employing multiple generation methods, the problems of incomplete data pool coverage or sample redundancy can be effectively solved. Since the generation process targets uncovered capability gap features, the new samples can specifically supplement the training materials most needed by the model. Simultaneously, assigning a globally unique identifier to each sample allows for rapid backtracking to the specific sample in case of abnormal losses during training, facilitating analysis of the cause of the problem (such as sample labeling errors, feature conflicts, etc.), thereby supporting subsequent data cleaning or strategy adjustments. The synergistic effect of the aforementioned multiple generation methods and screening mechanisms enables the training dataset to continuously enrich and optimize itself with each iteration, gradually covering a wider range of capability requirements, and thus continuously improving the model's performance on the target evaluation set.

[0136] Step 208: Use the training dataset to incrementally train the large language model to be trained, and obtain the updated large language model.

[0137] The incremental training can be performed based on the training dataset determined by the data construction strategy of the current round, and optionally based on the training parameter configuration of the current round (such as learning rate, batch size, gradient accumulation steps, etc.). After training is completed, an updated large language model is output, which will be used for subsequent evaluation and iteration.

[0138] To enhance the observability and problem localization capabilities of the training process, in some implementations, the incremental training in step 208 includes: assigning a globally unique identifier to each sample in the training dataset, and recording the loss value of each training batch and the globally unique identifier of the corresponding sample during the training process.

[0139] In practical applications, during the training data preprocessing stage, a globally unique identifier (Global Unique ID) can be assigned to each training sample. During the model forward propagation process, the independent loss value of each sample in the batch is calculated in parallel. The independent loss value is associated with and stored with the corresponding globally unique identifier. Based on the distribution of the independent loss value, the identifiers of abnormal samples whose loss values ​​exceed a preset threshold are identified, thereby achieving accurate backtracking from batch-level loss to sample-level loss.

[0140] Furthermore, the incremental training also includes: when the average loss value during the training process is detected to exceed a preset safety threshold, terminating the current training task, isolating the currently generated model parameter snapshot, and loading the valid model parameter snapshot from the previous historical moment to resume training.

[0141] In practical applications, the incremental training also includes a training stability monitoring step: real-time monitoring of the loss value during the training process; when the average loss value in the current batch or sliding window exceeds a preset safety threshold, the current training state is determined to be abnormal; the current training task is immediately terminated, and the currently generated model parameter snapshot (Checkpoint) is marked as invalid; the valid model parameter snapshot from the previous historical moment is automatically loaded, and training is resumed from the interruption point to prevent performance collapse caused by model parameter divergence.

[0142] Furthermore, the incremental training also includes: using the correspondence between the recorded loss values ​​and sample identifiers to backtrack the specific samples that caused the abnormal increase in loss; marking the samples as negative-return data, and reducing the sampling weight of the data source containing the features of the samples in the data construction strategy of the next iteration.

[0143] Furthermore, the training method also includes: after training, backtracking the abnormal samples corresponding to high loss values ​​based on the globally unique identifier; marking the abnormal samples as negative gain data, and reducing the sampling weight of data sources containing the features of the abnormal samples in the data construction strategy of the next iteration.

[0144] In a specific incremental training embodiment, the following operations may be included: initializing training hyperparameters (e.g., learning rate, optimizer state, etc.) based on the training parameters of the current round; assigning a globally unique identifier (e.g., a string consisting of dataset name, sample index, and timestamp) to each sample in the training dataset; and recording the loss value of each training batch and the globally unique identifier of the corresponding sample during the training process.

[0145] To further improve the robustness of the training process, when the average loss value detected during training exceeds a preset safety threshold (e.g., three standard deviations higher than the historical average loss), the system can terminate the current training task, isolate the currently generated model parameter snapshot, and load the valid model parameter snapshot from the previous historical moment to resume training. This prevents extreme anomalous samples from severely damaging the model parameters. Simultaneously, by utilizing the recorded loss value and sample identifier correspondence, the system can trace back to the specific sample causing the abnormally high loss, facilitating subsequent analysis of whether the sample is a difficult sample or dirty data (e.g., incorrect labeling, formatting errors, or content conflicts). These anomalous samples can be marked as negative-gain data, and in the next iteration's data construction strategy, the sampling weight of the data source containing their characteristics is reduced (e.g., reducing the proportion sampled from the data bucket or data source to which the sample belongs), thereby gradually eliminating low-quality or harmful samples. After training, high-loss samples can also be traced back based on globally unique identifiers, performing the same labeling and sampling weight adjustment operations.

[0146] Furthermore, in actual deployments, hardware resource or environmental anomalies may be encountered. In some embodiments, the method may also include training anomaly handling: when an Out of Memory (OOM) error occurs during incremental training, the batch size is automatically reduced or the gradient accumulation steps are adjusted before retrying training; when a communication port conflict occurs (e.g., NCCL error or port being occupied), the port is automatically switched before retrying training. These anomaly handling mechanisms can improve the automation and fault tolerance of the training process, avoiding the interruption of the entire iteration due to temporary resource problems.

[0147] Based on some embodiments of this specification, by assigning globally unique identifiers to training samples and recording the correspondence between batch loss and samples, specific samples causing abnormal losses can be located during or after training, thus providing a direct basis for identifying difficult samples or dirty data. When the average loss is detected to exceed a safety threshold, the system can automatically terminate the current training and roll back to the previous effective model parameter snapshot. This protection mechanism can prevent the model from being corrupted by abnormal samples and ensure the stability of model weights. Furthermore, by marking abnormal samples as negative-return data and reducing their sampling weight in subsequent strategies, progressive purification of training data can be achieved. That is, as the number of iterations increases, the influence of low-quality samples gradually weakens, while the contribution of high-value samples is strengthened. At the same time, automatic recovery processing for hardware or environmental anomalies such as memory overflow and port conflicts can reduce manual intervention and improve the reliability and continuity of the entire automated training loop. The synergistic effect of these multiple mechanisms enables the incremental training process to learn effectively from the data and maintain system stability when encountering anomalies, thus providing a reliable foundation for subsequent evaluation and iteration.

[0148] Step 210: Evaluate the updated large language model on the target evaluation set to obtain the current evaluation data.

[0149] The evaluation process may include: inputting each sample in the target evaluation set into the updated large language model and obtaining the model output; comparing the model output with the labeled answer and calculating the quantization score of each sample; generating at least some of the current performance metrics in the current evaluation data based on the quantization score of each sample, and obtaining a structured performance metric file (e.g., metrics.json).

[0150] Based on some embodiments of this specification, by evaluating the updated model on the target evaluation set after each iteration, quantitative performance indicators can be obtained, thereby providing an objective basis for the generation of data strategies and the determination of stopping conditions in subsequent iterations. Since the evaluation process uses a fixed target evaluation set and a consistent scoring method, the evaluation results between different rounds are comparable and can clearly reflect the performance change trend brought about by adjustments to the data construction strategy.

[0151] In one or more embodiments of this specification, the current evaluation data includes current performance metrics; the evaluation of the updated large language model on the target evaluation set specifically includes: inputting each sample in the target evaluation set into the updated large language model and obtaining the model output; comparing the model output with the labeled answer and calculating the quantization score of each sample; and statistically obtaining at least a portion of the current performance metrics based on the quantization scores of each sample.

[0152] The labeled answers include both reference answers representing the expected output of manually labeled data and target structures representing the expected format of structured data.

[0153] Furthermore, the reference answer refers to the expected output, pre-annotated manually or obtained through a reliable method, for each sample in the target evaluation set. It is typically a specific, correct output used for comparison with the model's actual output. For example, in text summarization tasks, the reference answer could be a manually written standard summary; in image description tasks, it could be a manually annotated sentence; and in SVG graphics generation tasks, it could be a correct SVG code snippet. The reference answer is primarily used to calculate quantitative metrics such as the accuracy and similarity of the model's output.

[0154] The target structure refers to the format, syntax, or structural specifications that the model output should meet, rather than the specific content. It defines the "skeleton" or "template" of the output, without limiting the specific filler content. For example, in structured data generation tasks, the target structure could be a JSON schema, requiring the model output to contain certain fields with correct data types; in code generation tasks, the target structure could be a valid syntax tree structure; and in SVG generation, the target structure could require the output to contain... <svg>Root element, coordinate values ​​within a specified range, and tags must be closed, etc. The target structure is used to check the output's format validity and structural integrity, and is generally not used for content matching.

[0155] In practical applications, when the evaluation task requires precise content matching (such as calculating the intersection-union ratio or BLEU score), the reference answer should be used; when the evaluation task only needs to check whether the output format or structure is correct (such as verifying whether SVG can be rendered or whether JSON conforms to the schema), the target structure should be used. Either can be used, or they can be used in combination (for example, checking the target structure first, then comparing it with the reference answer).

[0156] In some embodiments, the current performance metric is at least partially obtained based on the quantized scores of each sample, and the performance metric may also include other sources (e.g., capability dimension scores). For example, for an SVG graphics generation task, the Intersection over Union (IoU) of each sample can be calculated, and then the average IoU of all samples can be used to obtain the Mean IoU, or the median IoU can be obtained by taking the median IoU. Simultaneously, the proportion of samples that are successfully rendered and parsed can be counted as the task success rate (Render Parse OK), and the number of samples with scores below a preset threshold can be counted as the number of bad cases (Bad Cases). These metrics together constitute the current performance metric, which can be saved as a metrics.json file.

[0157] In some embodiments, the current performance metric may further include: the change in performance metric compared to the previous round, and / or the change in performance metric compared to historical peak performance metric. Step 210 may include: calculating the change in the current round quantification metric compared to the previous round performance metric and / or historical peak performance metric to determine the degree of performance improvement or degradation.

[0158] Based on some embodiments in this specification, by calculating the quantitative scores of each sample and statistically obtaining various performance indicators, the model's performance on the target evaluation set can be comprehensively quantified. Since these indicators reflect model performance from different dimensions (global average, median, extreme cases, success rate, structural similarity), they can more sensitively capture subtle changes brought about by data construction strategies. Simultaneously, by calculating the change in performance indicators compared to the previous round or historical peak values, it is possible to intuitively determine whether positive progress has been made in the current round, thus providing a basis for subsequent gating judgments and strategy adjustments.

[0159] To deeply analyze the reasons for model errors and guide subsequent data construction, in one or more embodiments of this specification, the current evaluation data further includes current problem diagnosis information; the evaluation of the updated large language model on the target evaluation set includes: evaluating each sample in the target evaluation set to obtain a quantitative score for each sample; marking samples with quantitative scores lower than a preset score threshold as errors (i.e., bad cases); clustering the errors according to the reasons for errors to form error sample clusters; performing attribution analysis on the error sample clusters to obtain capability gap information in the current problem diagnosis information; one capability gap corresponds to a group of error sample clusters with the same reasons for errors, and one error sample cluster belongs to at least one capability gap dimension.

[0160] The clusters of samples that lost points can be classified according to the reasons for the loss, including missing area, low IoU, overfilled, silhouette, and truncation warning.

[0161] Furthermore, the "missing region" category refers to situations where the model's output lacks expected graphical elements, path segments, or structural components. For example, in an SVG graphics generation task, the reference answer might contain three independent paths, while the model's output only contains two; the missing path constitutes a missing region. This type of loss typically reflects the model's insufficient ability to preserve element completeness or topological structure.

[0162] Low IoU (Intersection over Union) score indicates a low degree of overlap between the model's output and the reference answer. IoU is a commonly used metric to measure the similarity between two graphical regions or sets; it is calculated by dividing the area of ​​their intersection by the area of ​​their union. When this ratio is below a preset threshold (e.g., 0.5), it can be considered a low IoU score. This type of score loss typically reflects poor model performance in spatial localization, shape fitting, or contour accuracy.

[0163] Overfilling refers to situations where the model's output includes redundant graphic elements, path segments, or structural components that are not present in the reference answer. For example, the reference answer might only contain a rectangle, while the model's output might generate an additional circle in addition to the rectangle. This type of loss indicates that the model's ability to improve in terms of output compactness, noise suppression, or constraint compliance needs improvement.

[0164] Contour anomalies refer to graphical contours generated by the model that differ significantly from the contours of the reference answer in terms of shape, boundary smoothness, or topological structure. These anomalies may manifest as contour distortion, gaps, self-intersections, jagged boundaries, or non-manifold structures. This type of loss is typically used to identify deficiencies in the model's geometric accuracy, curve fitting, or topological correctness.

[0165] Truncated warnings indicate that the model's output is forcibly truncated because the sequence length exceeds the preset maximum processing capacity, resulting in incomplete output. In sequence-based generation models, the generation process stops when the number of output tokens reaches the model's maximum length limit, and the remaining portion is discarded. Truncated output often lacks crucial suffix information, such as the closing tags of SVG codes or the end segments of paths. This type of penalty reflects insufficient control over output length or sequence planning ability of the model, and may also indicate that the preset length threshold needs adjustment.

[0166] Each cluster of samples that have lost marks may include: an identifier for the sample that lost marks, a label indicating the reason for losing marks, and a distribution statistic for the cluster within the target evaluation set. The distribution statistic may include the number of samples in the cluster, its proportion within the evaluation set, and its score distribution across at least one capability dimension. The identifier for a sample that lost marks can be achieved by assigning a globally unique ID to the sample, for example, by assigning a unique identifier to each evaluation sample during data preprocessing. The label indicating the reason for losing marks can be abstracted from specific error types, such as marking samples that failed to render as "rendering anomaly," samples with IoU below a threshold as "low Intersection over Union," and samples with structural defects as "missing regions," etc. The distribution statistic within the target evaluation set can reflect the cluster's proportion in the entire evaluation set and its distribution across different difficulties or types.

[0167] A capability gap is an abstraction of a common problem among multiple samples that have failed to achieve a score, and there can be a many-to-many mapping relationship between them: one capability gap can correspond to multiple samples that have failed to achieve a score, and one sample that has failed to achieve a score may also reflect multiple capability gaps. Furthermore, the capability gap dimension can include at least one of the following: task type dimension, sample length dimension, structural complexity dimension, and semantic mapping dimension. After determining the capability gap, capability dimension scores can be further determined as part of the current performance indicators. Explanations of each capability gap dimension can be found in the relevant descriptions above, and will not be repeated here.

[0168] In one specific implementation, samples with scores below a preset threshold in the evaluation results are clustered to obtain several clusters of samples with lower scores. Common error features are extracted from each cluster, and clusters with the same or similar common error features are mapped to the same capability gap identifier. A gap weight is calculated for each capability gap identifier, generating a list of capability gap objects sorted by the gap weight. Extracting common error features from each cluster may include: parsing the structured data of the samples with lower scores to extract at least one of syntactic structure features, geometric complexity features, or semantic attribute features; or, calling a large language model interface, inputting representative samples from the clusters with lower scores, and generating natural language labels describing the causes of common errors.

[0169] Based on some embodiments of this specification, by clustering and attribution analysis of the low-scoring samples, the low-scoring samples are transformed into structured clusters of low-scoring samples and capability gap information, which can transform the black-box performance of the model into an interpretable description of weaknesses. Since there is a correlation between capability gaps and clusters of low-scoring samples, the system can clearly know which specific low-scoring samples correspond to each capability gap, thus providing precise targets for subsequent data construction strategies (e.g., for a "color-binding" capability gap, color-related samples can be constructed in a targeted manner). Simultaneously, the division of capability gap dimensions (task type, sample length, structural complexity, semantic mapping) makes the diagnostic information more granular, helping the system to select data construction rules more effectively. This structured storage of diagnostic information provides a data foundation for cross-round capability gap evolution analysis, making iterative optimization no longer blind trial and error, but rather targeted improvement with a clear direction.

[0170] To determine the priority order for handling multiple capability gaps, in one or more embodiments of this specification, the current problem diagnosis information further includes capability gap priority ranking information; the priority ranking is calculated based on at least one of the following factors: the number of samples in the sample clusters corresponding to each capability gap; the degree of score loss of the samples in the sample clusters corresponding to each capability gap; and the negative contribution value of each capability gap to the overall score of the target evaluation index.

[0171] The severity of a sample's score loss can be reflected in the magnitude of the loss, for example, by measuring the average score deviation of the lost samples (i.e., the difference between the labeled answer and the model output). The negative contribution value represents the overall performance decline caused by the capability gap. For instance, if the scores of a cluster of lost samples corresponding to a capability gap are generally low, then its effect on lowering the overall average score is significant. In practical calculations, priority ranking can be achieved as follows: first, count the number of samples in each cluster corresponding to the lost samples; a larger number indicates a wider impact. Then, assess the average score loss within each cluster; a more severe loss indicates a more urgent gap. Finally, calculate the upper limit of performance improvement that eliminating the gap might bring (i.e., the negative contribution value). Considering all these factors, a priority weight can be assigned to each capability gap; a higher weight indicates that the capability gap is more worthy of priority processing in the next round.

[0172] In some embodiments, after generating the current evaluation data in step 210, an evaluation validity check is also performed: checking whether the current performance metric is within a preset reasonable range, and whether the sample clusters of scores lost in the current problem diagnosis information are not empty and consistent with the changing trend of the performance metric; if the check fails, a retry evaluation is triggered or the current round of evaluation is marked as abnormal. For example, when all sample outputs are empty SVGs, resulting in the sample clusters of scores lost covering the entire evaluation set and the performance metric being abnormally low, a retry should be triggered.

[0173] Based on some embodiments of this specification, by calculating a priority ranking for capability gaps and comprehensively considering the size, severity, and negative contribution of the clusters of samples with poor performance to the overall index, the system can identify the most pressing capability shortcomings that need to be addressed. This ranking mechanism ensures that the generation of data construction strategies is no longer an even distribution of effort, but rather prioritizes the allocation of limited training resources to the capability gaps that yield the greatest benefit, thereby improving the efficiency of iterative optimization. Since the priority ranking can be dynamically updated with each round of evaluation results, the system can adapt to changes in capability gaps during the evolution of model performance, automatically adjusting its focus to achieve adaptive and gradual performance improvement. Simultaneously, the priority information is stored in the form of weights, which can be directly used as prompts for the next round of large language model inference engine, guiding it to generate more targeted data construction strategies, forming a complete closed loop from evaluation → diagnosis → priority ranking → strategy generation.

[0174] Step 212: Update the current evaluation data to the global state information.

[0175] In one or more embodiments of this specification, updating the current evaluation data to the global state information specifically includes: writing the current performance metric to the performance metric history record; and writing the current problem diagnosis information to the problem diagnosis history record. Through such a writing operation, the performance metric history record accumulates the quantitative scores of each round by round, and the problem diagnosis history record accumulates diagnostic information such as capability gaps and score-loss sample clusters of each round by round.

[0176] Furthermore, the data construction strategy identifier and training parameter configuration identifier for the current round can be associated and stored with the current evaluation data to form an experimental record for that round. In other words, a clear correspondence is established between the strategy, parameters, and evaluation results for each round, facilitating subsequent traceability and analysis.

[0177] In some optional embodiments, after updating the global state information, the current stage identifier in the global state information can also be updated to the next stage. The stage may include at least one of the following: policy planning stage, training stage, evaluation stage, diagnosis stage, and policy adjustment stage. Furthermore, the global state information may also include the call history of the large language model inference engine, synchronously recording the large language model identifier, input summary, output result, and call status used in the current round of data generation to construct the policy during the update.

[0178] Before writing the global state information, a data validity check can be performed: check whether the current evaluation data is complete and whether it contains the necessary performance indicators and problem diagnosis fields; if the check fails, the global state information can be left unupdated and the evaluation can be retried or the current iteration can be terminated.

[0179] Based on some embodiments of this specification, by updating the current evaluation data to the global state information and storing it in association with the corresponding data construction strategy, subsequent rounds can obtain a complete experimental history, including which strategy was used in each round and what performance results were achieved. This structured state management provides a data foundation for strategy effectiveness analysis, identification of ineffective strategies, and fine-tuning of strategy parameters, enhancing the interpretability and reproducibility of iterative optimization. Simultaneously, data validity verification avoids incomplete or abnormal data contaminating the global state, ensuring data quality during the iterative process.

[0180] To track the model's peak performance during iterations and guide subsequent strategies accordingly, in one or more embodiments of this specification, the historical evaluation data further includes historical peak performance metrics. Updating the global state information with the current evaluation data specifically includes: determining whether the current performance metric is superior to the historical peak performance metric; if the current performance metric is superior to the historical peak performance metric, then updating the historical peak performance metric with the current performance metric. Simultaneously, the model checkpoint corresponding to the historical peak performance metric can be updated with the model checkpoint of the current round corresponding to the current performance metric.

[0181] If the current performance metric is worse than the historical peak performance metric, then the model checkpoint corresponding to the historical peak performance metric is retained, and the model checkpoint of the current round is marked as a candidate model. Simultaneously, the data construction strategy of the current round is recorded as the strategy to be adjusted. This mechanism ensures that the model weights of the current performance peak and their corresponding strategy information are always preserved in the global state.

[0182] Based on some embodiments of this specification, by maintaining and dynamically updating historical peak performance metrics, the system can always retain the peak model state. When performance degrades in subsequent rounds, it can revert to the model checkpoint corresponding to the historical peak performance metric, avoiding irreversible model performance degradation due to a single policy error. Furthermore, when training results are not as expected, the current round's policy is marked as a policy to be adjusted, providing a clear direction for subsequent policy optimization. This combination of peak model tracking and policy adjustment mechanisms allows the iterative process to actively explore new policy spaces while safely reverting when exploration fails, thus balancing exploration and utilization and improving the stability and convergence efficiency of the entire automated training loop.

[0183] In order to accumulate and reuse effective data construction strategies, in one or more embodiments of this specification, if the current performance index is better than the historical peak performance index, the data construction strategy of the current round is marked as an effective strategy, and the data construction strategy of the current round and the corresponding performance improvement are stored in the global state information.

[0184] The performance improvement can be the difference between the current performance metric and the previous round or the historical peak performance metric. After storing the global state information, the effective strategy and its reward value can be referenced as positive examples by the large language model inference engine in subsequent rounds, guiding the engine to generate similar high-yield strategies.

[0185] Based on some embodiments of this specification, a continuously growing "effective strategy library" can be formed by marking strategies that outperform historical peak performance as effective and recording their performance improvements. These validated strategies and their benefit information can serve as experience for subsequent strategy generation, enabling the large language model inference engine to learn which types of data construction strategies are more likely to yield positive returns. As the number of iterations increases, the effective strategy library gradually becomes richer, and the system can generate new strategies with greater reliability, thereby reducing the cost of blind trial and error and accelerating the convergence process towards the optimal strategy.

[0186] In order to identify and avoid data construction strategies that negatively impact model performance, in one or more embodiments of this specification, if the current performance metric is worse than the performance metric of the previous round, the data construction strategy of the current round is marked as an invalid strategy, and the data construction strategy of the current round and the corresponding performance degradation are stored together in the global state information.

[0187] The performance degradation can be the difference (negative value) between the current performance metric and the performance metric of the previous round. After storing the global state information, the invalid strategy and its negative reward value can be referenced as negative examples by the large language model inference engine in subsequent rounds, prompting the engine to avoid generating similar data actions.

[0188] Based on some embodiments of this specification, a "failure mode library" can be constructed by marking strategies that cause performance degradation as invalid and recording the amount of performance degradation. These invalid strategies and their negative benefit information can serve as "negative examples" for subsequent strategy generation, enabling the large language model inference engine to clearly understand which data actions should be avoided. With increasing iterations, the invalid strategy library helps the system gradually eliminate verified harmful data construction methods, thereby purifying the strategy search space and reducing the possibility of repeating errors. This dual accumulation mechanism of positive and negative experience allows the automated research loop to learn from both successes and failures, improving the efficiency and robustness of strategy optimization.

[0189] Step 214: If the current evaluation data meets the stopping condition, output the updated large language model as the training completed model; otherwise, continue iterative training based on the updated global state information.

[0190] The specific way to determine the stopping condition in step 214 can be as follows: if the current performance index meets the stopping condition (e.g., the current performance index reaches a preset threshold, the improvement of the performance index in multiple consecutive rounds is less than a preset amount of change, or the preset maximum number of iterations is reached), then the updated large language model is output as the training completed model; otherwise, the updated global state information is used as the new input, and the process returns to step 204, the large language model inference engine is called again to generate the data construction strategy for the next round, and steps 206 to 214 are repeated.

[0191] In practical applications, the stopping conditions may include at least one of the following: the performance index of the current round reaches a preset performance threshold; the improvement of the performance index of consecutive preset rounds is less than a preset gain threshold; the preset maximum number of iteration rounds is reached; or an external stop command is received.

[0192] In addition, the stopping condition may also include: detecting that the number of irreversible training anomalies exceeds a preset limit.

[0193] In addition, the stopping condition may also include: triggering a rollback mechanism when the performance index fails to improve or degrades in multiple consecutive rounds, restoring the current model to the model checkpoint corresponding to the historical peak performance index in the global state information, and optionally reducing the exploration intensity of the next round of data construction strategy or increasing the strictness of data cleaning.

[0194] In practical applications, optionally, in step 214, when the current evaluation data meets the stopping condition, before outputting the updated large language model as the training completed model, a final verification can also be performed: the updated large language model is evaluated using an independent verification set to confirm that its performance indicators meet the stopping condition; if the verification fails, the model is not output and the process returns to step 204 to continue iterating.

[0195] Furthermore, the global state information may also include a target achievement status identifier. In step 214, when the current evaluation data meets the stopping condition and passes the final verification, the target achievement status identifier can be set to "achieved," and the iteration can be terminated.

[0196] In step 214, in the branch that continues iterative training based on the updated global state information, the updated global state information includes the data construction strategy of the current round and its corresponding current evaluation data. This information can be used as a reference for generating the data construction strategy in the next round.

[0197] Based on some embodiments in this specification, by setting various stopping conditions (performance threshold, gain magnitude, maximum number of rounds, external instructions, number of anomalies, etc.), the timing of iteration termination can be flexibly controlled according to different application scenarios, avoiding ineffective training or overfitting. When performance fails to improve or degrades after multiple consecutive rounds, a rollback mechanism is triggered to restore the model to the model checkpoint corresponding to the historical peak performance metric, preventing performance crashes caused by catastrophic forgetting or policy bias. The final verification step uses an independent validation set to further confirm model performance, helping to avoid over-reliance on the target evaluation set and improving the model's generalization ability. The goal achievement status indicator provides a clear termination signal for the system, facilitating integration into larger automated processes.

[0198] To introduce more refined strategy adjustment suggestions between each iteration, enabling the next round of data construction strategy to more effectively address the shortcomings of the current model, in one or more embodiments of this specification, the iterative training based on the updated global state information specifically includes: before entering the next iteration, calling the large language model data analyzer to generate adjustment suggestions for the next round of data construction strategy based on the current evaluation data and the historical evaluation data in the updated global state information; and writing the adjustment suggestions into the global state information.

[0199] The large language model data analyzer can be a standalone instance of the large language model or the same model shared with the policy generation engine, but its purpose differs: the data analyzer focuses on uncovering the underlying causes and evolutionary trends of capability gaps from the evaluation data, outputting analytical conclusions and specific adjustment suggestions in natural language form. For example, the analyzer can receive the performance metrics and problem diagnosis information of the current round, and, combined with the historical records of previous rounds, output the following suggestion: "In the last two iterations, the 'color binding' capability gap has not improved. It is recommended to increase the upsampling weight of color-object pairing samples from the data pool in the next round and try a counterfactual generation strategy." This adjustment suggestion will be written into the "next round strategy hint" field in the global state information for the large language model inference engine to refer to when generating a new strategy in step 204.

[0200] Based on some embodiments in this specification, by invoking a dedicated large language model data analyzer before continuing iterations, more abstract and macroscopic policy adjustment directions can be extracted from current and historical evaluation data. This analysis is not a simple numerical comparison, but rather, based on the semantic understanding and reasoning capabilities of the large language model, it identifies the root causes of capability gaps and their changing trends over rounds, thereby outputting actionable suggestions (such as increasing the weight of a certain type of sample, adopting a specific generation method, etc.). After writing the adjustment suggestions into the global state information, the policy generation engine in the next round can directly utilize these hints, making policy adjustments more accurate and efficient, avoiding blind trial and error based entirely on numerical indicators. Simultaneously, this mechanism complements the effective / ineffective policy labeling: the latter is a binary evaluation of a single-round policy, while the data analyzer provides qualitative guidance on the policy evolution direction. The synergy of both can significantly improve the intelligence level and convergence speed of the automated research loop.

[0201] While one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is merely one possible execution order among many steps and does not represent the only possible execution order. The order of some steps may be adjusted according to actual needs, or some steps may be omitted. When the claims involve method steps, changes in the order of such steps, or parallel execution between steps, are also within the scope of protection of the claims.

[0202] Figure 2 The method described here acquires global state information containing historical data construction strategies and historical evaluation data. It then calls a large language model inference engine to automatically generate a data construction strategy for the current round based on this global state information. This strategy determines the training dataset and incrementally trains the model. The updated model is evaluated on the target evaluation set to obtain the current evaluation data. This current evaluation data is then written back to the global state information. If a stopping condition is met, the trained model is output; otherwise, the process continues iteratively using the updated global state information. This constructs an end-to-end iterative framework driven by target evaluation metrics and organized using an automated research loop (autoresearch). Its technical effectiveness is reflected in the synergistic effects at multiple levels. First, by using historical evaluation data (including performance metrics and problem diagnoses) as core input, the large language model inference engine can dynamically generate data construction strategies tailored to current capability gaps, rather than relying on static rules or human experience. This directly improves the main and sub-metrics on the target evaluation set, as each round of data construction is optimized around the target evaluation distribution, increasing the matching degree between the training data and the target evaluation task. Secondly, the method incorporates a capability gap attribution mechanism and a data value scoring mechanism: by analyzing the samples that failed to score, a capability gap distribution is formed, and then high-value training data is selected based on factors such as the matching degree with the capability gap and marginal benefits, avoiding the inefficiency caused by generalization expansion. This improves the marginal benefits per sample under a limited training budget, while reducing the proportion of invalid complements and low-return data expansion, thereby improving the efficiency of training resource utilization. Thirdly, the method adopts a unified dual-path approach of data screening and automatic construction: high-value samples are selected first from the data pool, and new samples are automatically generated when insufficient, ensuring the sufficiency and diversity of the dataset in each round. Fourthly, by updating the current evaluation data to the global state and using it as a reference for the next iteration, the system upgrades data optimization from static processing to an automatic research closed loop, supporting continuous iterative optimization of the model around the target evaluation index. Fifthly, the strategy retention / adjustment / elimination mechanism based on automatic training and backtesting allows effective data construction strategies to be automatically accumulated in multiple rounds of experiments, while ineffective strategies are eliminated, thereby improving the automation and reproducibility of model iterative optimization and further ensuring the continuous improvement of overall performance. In summary, this approach significantly improves the performance of large models on target evaluation tasks, the efficiency of training resource utilization, and the efficiency of discovering and constructing high-value training data through a closed-loop collaborative process of "evaluation → attribution → policy generation → data construction → training → re-evaluation".

[0203] The various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they have not been described one by one. Therefore, the arbitrary combination of various technical features in the above embodiments is also within the scope of this specification.

[0204] Based on the same idea, embodiments of this specification also provide apparatus corresponding to the above methods.

[0205] Figure 3 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the structure of a large-scale automated training device.

[0206] like Figure 3 As shown, the device may include: The state information acquisition module 302 is used to acquire global state information; the global state information includes historical data construction strategy and historical evaluation data; the historical evaluation data is the result obtained by the large language model to be trained in previous rounds on the target evaluation set; The data construction strategy generation module 304 is used to call the large language model inference engine to generate the data construction strategy for the current round based on the global state information. Training dataset construction module 306 is used to determine the training dataset according to the data construction strategy; The model training module 308 is used to incrementally train the large language model to be trained using the training dataset to obtain an updated large language model. Evaluation module 310 is used to evaluate the updated large language model on the target evaluation set to obtain the current evaluation data; The status information update module 312 is used to update the current evaluation data to the global status information; The iteration control module 314 is used to output the updated large language model as the training completed model if the current evaluation data meets the stopping condition, otherwise iterative training continues based on the updated global state information.

[0207] based on Figure 3 The embodiments of this specification also provide some specific implementation schemes of the method, which are described below.

[0208] Optionally, the historical data construction strategy includes retained effective strategies and eliminated ineffective strategies; wherein, the effective strategy is a strategy that generates positive returns on the target evaluation index on the target evaluation set, and the ineffective strategy is a strategy that generates negative returns on the target evaluation index on the target evaluation set.

[0209] Optionally, the historical evaluation data includes historical performance metrics and historical problem diagnosis data; the current evaluation data includes current performance metrics and current problem diagnosis information.

[0210] Optionally, the problem diagnosis history includes capability gap information of the large language model to be trained on the target evaluation set, and the dimension of the capability gap includes at least one of the following: task type dimension, sample length dimension, structural complexity dimension, and semantic mapping dimension.

[0211] Optionally, the global state information also includes historical training parameter configurations; the invocation of the large language model inference engine further generates the current round's training parameter configuration based on the historical training parameter configurations; during incremental training, training is performed based on the current round's training parameter configurations.

[0212] Optionally, before invoking the large language model inference engine, a predefined rule base is obtained; when invoking the large language model inference engine, the large language model inference engine generates the data construction strategy for the current round based on the global state information and the rule base; wherein, the predefined rule base includes at least one of the following: data cleaning rules, data bucket classification rules, data action constraint rules, data construction rules, and sampling strategy rules.

[0213] Optionally, the large language model inference engine includes multiple candidate large language models, which include a primary large language model and at least one backup large language model. The invocation of the large language model inference engine specifically includes: invoking the primary large language model; if the invocation of the primary large language model fails, then sequentially invoking the backup large language models; if all backup large language models fail to be invoked, then a heuristic rule engine is invoked to generate a default data construction strategy based on the global state information.

[0214] Optionally, determining the training dataset according to the data construction strategy includes: selecting samples from a preset data pool whose data value scores meet a preset score threshold based on the data construction strategy, as the training dataset; the data value score is determined based on one or more of the following factors: the degree of matching with the capability gap of the large language model to be trained on the target evaluation set, the redundancy with the existing training set, the data quality credibility, the data construction cost, and the estimated marginal benefit.

[0215] Optionally, determining the training dataset according to the data construction strategy further includes: if there are insufficient samples in the preset data pool whose data value scores meet the preset scoring threshold, then generating new samples based on the capability gap information in the historical evaluation data, and adding the new samples to the training dataset.

[0216] Optionally, the current evaluation data includes current performance metrics; the evaluation of the updated large language model on the target evaluation set specifically includes: inputting each sample in the target evaluation set into the updated large language model and obtaining the model output; comparing the model output with the labeled answer and calculating the quantization score of each sample; and statistically obtaining at least a portion of the current performance metrics based on the quantization scores of each sample.

[0217] Optionally, the current evaluation data further includes current problem diagnosis information; the evaluation of the updated large language model on the target evaluation set includes: evaluating each sample in the target evaluation set to obtain a quantitative score for each sample; marking samples with quantitative scores lower than a preset score threshold as samples with poor scores; clustering the samples with poor scores according to the reasons for poor scores to form a cluster of samples with poor scores; performing attribution analysis on the clusters of samples with poor scores to obtain capability gap information in the current problem diagnosis information; one capability gap corresponds to a cluster of samples with the same reasons for poor scores, and one cluster of samples with poor scores belongs to at least one capability gap dimension.

[0218] Optionally, the current problem diagnosis information also includes priority ranking information of capability gaps; the priority ranking is calculated based on at least one of the following factors: the number of samples in the sample clusters corresponding to each capability gap; the degree of loss of samples in the sample clusters corresponding to each capability gap; and the negative contribution value of each capability gap to the overall score of the target evaluation index.

[0219] Optionally, updating the current evaluation data to the global status information specifically includes: writing the current performance metric to the performance metric history record; and writing the current problem diagnosis information to the problem diagnosis history record.

[0220] Optionally, the historical evaluation data further includes historical peak performance metrics; updating the current evaluation data to the global state information specifically includes: determining whether the current performance metric is better than the historical peak performance metric; if the current performance metric is better than the historical peak performance metric, then updating the historical peak performance metric with the current performance metric.

[0221] Optionally, it further includes: if the current performance metric is better than the historical peak performance metric, then the data construction strategy of the current round is marked as a valid strategy, and the data construction strategy of the current round and the corresponding performance improvement are stored in the global state information.

[0222] Optionally, it further includes: if the current performance metric is worse than the performance metric of the previous round, then the data construction strategy of the current round is marked as an invalid strategy, and the data construction strategy of the current round and the corresponding performance degradation amount are stored together in the global state information.

[0223] Optionally, the step of continuing iterative training based on the updated global state information specifically includes: before entering the next iteration, calling the large language model data analyzer to generate adjustment suggestions for the next round of data construction strategy based on the current evaluation data and the historical evaluation data in the updated global state information; The adjustment suggestions are written into the global status information.

[0224] It is understood that the modules mentioned above refer to computer programs or program segments used to perform one or more specific functions. Furthermore, the distinction between these modules does not imply that the actual program code must also be separate.

[0225] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0226] The above is a schematic scheme of a large-scale automated training device according to this embodiment. It should be noted that the technical solution of this large-scale automated training device and the technical solution of the large-scale automated training method described above belong to the same concept. For details not described in detail in the technical solution of the large-scale automated training device, please refer to the description of the technical solution of the large-scale automated training method described above.

[0227] Based on the same idea, this specification also provides devices corresponding to the above methods in its embodiments.

[0228] Figure 4 A structural block diagram of a computing device 400 provided according to an embodiment of this specification is shown.

[0229] The computing device 400 includes: Memory 410 and processor 420; The memory 410 is used to store computer programs / instructions, and the processor 420 is used to execute the computer programs / instructions, which, when executed by the processor 420, implement the steps of the large model automated training method.

[0230] Specifically, the components of the computing device 400 include, but are not limited to, a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and the database 450 is used to store data.

[0231] The computing device 400 also includes an access device 440, which enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 440 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0232] In one embodiment of this specification, the above-described components of the computing device 400 and Figure 4 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 4 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0233] The computing device 400 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 400 can also be a mobile or stationary server.

[0234] The processor 420 executes the computer instructions to implement the steps of the large model automated training method.

[0235] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above-described large model automated training method belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-described large model automated training method.

[0236] An embodiment of this specification also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the large model automated training method described above.

[0237] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the above-described large model automated training method. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the above-described large model automated training method.

[0238] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described large model automated training method.

[0239] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above-described large-scale automated training method belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the above-described large-scale automated training method.

[0240] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the apparatus and device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The apparatus, device and method provided in the embodiments of this specification are corresponding to each other, and therefore the apparatus and device also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the corresponding apparatus and device will not be repeated here.

[0241] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0242] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using a hardware physical module. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0243] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0244] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0245] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0246] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0247] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0248] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0249] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0250] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0251] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0252] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital character versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0253] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0254] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.< / svg> < / circle> < / rect> < / path> < / g> < / g> < / circle> < / path> < / g>

Claims

1. An automated training method for large models, comprising: Obtain global status information; The global state information includes historical data construction strategies and historical evaluation data; The historical evaluation data refers to the results obtained by the large language model to be trained in previous rounds on the target evaluation set; The large language model inference engine is invoked to generate the data construction strategy for the current round based on the global state information. The training dataset is determined based on the data construction strategy described above; The training dataset is used to incrementally train the large language model to be trained, resulting in an updated large language model. The updated large language model is evaluated on the target evaluation set to obtain the current evaluation data; Update the current evaluation data to the global state information; If the current evaluation data meets the stopping condition, the updated large language model is output as the training completed model; otherwise, iterative training continues based on the updated global state information.

2. The method as described in claim 1, wherein the historical data construction strategy includes retained effective strategies and obsolete invalid strategies; wherein, The effective strategy is a strategy that generates positive returns on the target evaluation metrics on the target evaluation set, and the ineffective strategy is a strategy that generates negative returns on the target evaluation metrics on the target evaluation set.

3. The method as described in claim 1, wherein the historical evaluation data includes historical performance index records and historical problem diagnosis records; and the current evaluation data includes current performance indexes and current problem diagnosis information.

4. The method of claim 3, wherein the problem diagnosis history includes capability gap information of the large language model to be trained on the target evaluation set.

5. The method as described in claim 1, wherein the global state information further includes historical training parameter configuration; the invocation of the large language model inference engine further generates the current round's training parameter configuration based on the historical training parameter configuration; and during incremental training, training is performed based on the current round's training parameter configuration.

6. The method as described in claim 1, wherein before invoking the large language model inference engine, a predefined rule base is obtained; when invoking the large language model inference engine, the large language model inference engine generates the data construction strategy for the current round based on the global state information and the rule base.

7. The method of claim 1, wherein determining the training dataset according to the data construction strategy comprises: Based on the data construction strategy, samples whose data value scores meet the preset score threshold are selected from the preset data pool and used as the training dataset. The data value score is determined based on one or more of the following factors: the degree of matching with the capability gap of the large language model to be trained on the target evaluation set, the redundancy with the existing training set, the reliability of data quality, the cost of data construction, and the estimated marginal benefit.

8. The method of claim 7, wherein determining the training dataset according to the data construction strategy further comprises: If there are insufficient samples in the preset data pool whose data value scores meet the preset scoring threshold, then new samples are generated based on the capability gap information in the historical evaluation data, and the new samples are added to the training dataset.

9. The method of claim 1, wherein the current evaluation data further includes current problem diagnosis information; the evaluation of the updated large language model on the target evaluation set includes: Each sample in the target evaluation set is evaluated to obtain a quantitative score for each sample; Samples with quantified scores below a preset score threshold are marked as samples that have lost points; The samples that lost points were clustered according to the reasons for the loss, forming clusters of samples that lost points. Attribution analysis is performed on the cluster of samples that have failed to achieve their scores to obtain capability gap information in the current problem diagnosis information.

10. The method of claim 9, wherein the current problem diagnostic information further includes priority ranking information for capability gaps; the priority ranking is calculated based on at least one of the following factors: The number of samples in each capability gap corresponding to the cluster of samples that lost points; The degree of score loss among samples in the sample clusters corresponding to each capability gap; The negative contribution of each capability gap to the overall score of the target evaluation indicators.

11. The method as described in claim 3, wherein updating the current evaluation data to the global state information specifically includes: Write the current performance metric to the performance metric history record; Write the current problem diagnosis information into the problem diagnosis history.

12. The method of claim 3, wherein the historical evaluation data further includes historical peak performance indicators; and updating the current evaluation data to the global state information specifically includes: Determine whether the current performance metric is better than the historical peak performance metric; If the current performance metric is better than the historical peak performance metric, then the historical peak performance metric is updated with the current performance metric.

13. The method of claim 12, further comprising: If the current performance metric is better than the historical peak performance metric, then the data construction strategy of the current round is marked as a valid strategy, and the data construction strategy of the current round and the corresponding performance improvement are stored in the global state information.

14. The method of claim 3, further comprising: If the current performance metric is worse than the previous round's performance metric, then the data construction strategy for the current round is marked as an invalid strategy, and the data construction strategy for the current round and the corresponding performance degradation are stored in the global state information.

15. The method as described in claim 1, wherein the step of continuing iterative training based on the updated global state information specifically includes: Before proceeding to the next iteration, the large language model data analyzer is invoked to generate adjustment suggestions for the next round of data construction strategy based on the current evaluation data and the historical evaluation data in the updated global state information. The adjustment suggestions are written into the global status information.

16. An automated training device for large models, comprising: The status information acquisition module is used to acquire global status information; The global state information includes historical data construction strategies and historical evaluation data; The historical evaluation data refers to the results obtained by the large language model to be trained in previous rounds on the target evaluation set; The data construction strategy generation module is used to call the large language model inference engine to generate the data construction strategy for the current round based on the global state information. A training dataset construction module is used to determine the training dataset according to the data construction strategy; The model training module is used to incrementally train the large language model to be trained using the training dataset to obtain an updated large language model. The evaluation module is used to evaluate the updated large language model on the target evaluation set to obtain the current evaluation data; The status information update module is used to update the current evaluation data to the global status information; The iteration control module is used to output the updated large language model as the training completed model if the current evaluation data meets the stopping condition; otherwise, iterative training continues based on the updated global state information.

17. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 15.

Images