A synthetic data generation method, system, device and storage medium

Abstract

This invention relates to a method, system, device, and storage medium for generating synthetic data, belonging to the field of big data synthesis technology. It addresses the problem of constructing high-quality training data under conditions of uneven data coverage and complex task hierarchy. The method includes generating an initial synthetic dataset based on an initial classification system; training a target model and testing the trained model in the t-th round using a validation sample set; mapping validation samples to classification nodes to obtain node-level feedback; identifying weak classification nodes based on comparison results; dynamically evolving the classification system for the set of weak classification nodes; generating an incremental synthetic dataset based on the updated classification system; and repeatedly executing the steps of node-level feedback, weak classification node identification, dynamic evolution of the classification system, and incremental synthetic dataset generation until a preset stopping condition is met, at which point the final classification system, final training dataset, and final model are output. This method enables the construction of high-quality training data.

Description

Technical Field

[0001] This invention relates to the field of big data synthesis technology, and in particular to a method, system, device and storage medium for generating synthetic data. Background Technology

[0002] With the rapid development of large-scale model technology, generative artificial intelligence, and automated data construction technology, synthetic data has been widely applied in various scenarios such as large-scale model pre-training, supervised fine-tuning, instruction alignment, domain adaptation, evaluation set construction, and agent task training. Especially in applications such as knowledge-based question answering, complex reasoning, multi-round decision-making, domain task adaptation, and long-tail capability enhancement, real-world high-quality training data often faces challenges such as high acquisition costs, long annotation periods, limited coverage, strong privacy compliance constraints, and a scarcity of long-tail capability samples. How to construct a high-quality synthetic data system that can continuously support model capability improvement under the condition of insufficient high-quality real-world data, while ensuring data coverage, relevance, and effectiveness and improving data construction efficiency, has become a crucial technical problem that urgently needs to be solved in the field of large-scale model training and data engineering.

[0003] Especially in application scenarios such as finance, law, healthcare, cybersecurity, industrial control, government Q&A, complex reasoning, and multi-step decision-making, models need to deal with a large number of task types with significant differences in hierarchy, difficulty, semantic boundaries, and long-tail distribution. These tasks typically have the following characteristics: First, the capability requirements are not uniformly distributed, with significant differences between different knowledge points, task subclasses, and error patterns; second, the learning performance of the model is highly unbalanced across different capability areas, often manifesting as some nodes being well-mastered while others remain weak; third, the data construction process is not a one-time event but requires continuous reinforcement and dynamic adjustment as the model's training status changes. Existing synthetic data generation methods mostly adopt static task partitioning, fixed generation strategies, and offline data augmentation, i.e., generating uniform training data under a pre-given data structure or task template. They lack a dynamic evolution mechanism that continuously adjusts the data organization structure and generation focus based on actual model feedback. This leads to problems such as misalignment between data coverage and model weaknesses, repeated generation of mastered areas, insufficient reinforcement of weak areas, and low overall data utilization efficiency during actual training, making it difficult to balance capability improvement with data generation costs.

[0004] However, existing synthetic data generation methods, when facing complex tasks and continuous training scenarios, often focus on improving the quality of a single generation, expanding the scale, enhancing cue design capabilities, or optimizing sample surface diversity, but lack systematic modeling and joint optimization of "differences in capability structure," "differences in node mastery," "evolutionary requirements of classification granularity," and "dynamic data redistribution mechanisms based on model feedback." This leads to the following prominent problems during model training: (1) Existing synthetic data generation methods lack the ability to model the task capability structure, making it difficult to simultaneously cover global capability coverage and local fine-grained reinforcement. Current data generation technologies for training large models typically construct generation targets based on task descriptions, a small number of examples, manual experience rules, or fixed templates, and generate training samples under preset capability categories. However, in complex real-world tasks, the capabilities that the model needs to learn often have significant differences in hierarchical structure: for example, in financial question answering, there are high-level capabilities such as financial statement analysis, risk identification, and compliance explanation, as well as fine-grained sub-capabilities such as liquidity risk identification, insufficient collateral judgment, and cross-border disclosure explanation; similarly, in legal and medical tasks, there are multi-layered semantic structures from coarse to fine, from commonalities to specific cases. Existing methods usually lack a systematic expression mechanism for these hierarchical capability structures, making it difficult to achieve unified modeling from the global task space to local fine-grained capability nodes. This results in insufficient training data support for long-tailed sub-tasks, fine-grained error patterns, and tasks with ambiguous boundaries, thereby limiting the refined development of the model's overall capabilities and its deep generalization effect.

[0005] (2) Existing methods lack explicit modeling of the mastery level of different capability regions of the model, making it difficult to achieve targeted reinforcement and dynamic scheduling for weak nodes. During the training of large models, different capability nodes contribute differently to the training effect. For some high-frequency, simple, or semantically clear task regions, the model can converge quickly after training with a small number of samples; while capability nodes in complex reasoning, long-tail concepts, easily confused categories, cross-knowledge boundary tasks, and low-resource scenarios are often key bottlenecks affecting the overall quality of the model. However, most existing synthetic data generation methods assume that the importance and generation requirements of each capability region are basically the same, and use a uniform data generation depth, uniform sampling logic, and uniform generation budget for different nodes. They lack a mechanism for detecting, quantifying, and explicitly modeling the model's error situation, mastery level, and reinforcement priority at each capability node, and cannot identify which nodes need more samples and which nodes can reduce repeated generation. As a result, the system consumes a lot of generation resources on the capability regions that the model has already mastered, while it is difficult to allocate sufficient data reinforcement capabilities to the truly weak key nodes, thus affecting the model training efficiency and capability improvement effect.

[0006] (3) Existing synthetic data generation methods lack an adaptive classification system evolution mechanism based on model feedback, making it difficult to dynamically update the classification granularity and data structure during continuous training. Currently, most data generation systems use pre-defined task classification structures, capability categories, or knowledge catalogs to generate samples. That is, the system determines the task boundaries and node levels before generation begins and maintains this structure throughout the training cycle. Although this approach is simple to implement, it has significant limitations in complex tasks: for some coarse-grained nodes, the model may only encounter difficulties in certain subdivision patterns within the node, while having mastered other patterns; for some newly emerging error patterns or boundary problems, the original classification structure may not even have corresponding nodes. Existing methods typically lack an adaptive classification system evolution mechanism that can dynamically adjust node levels, automatically refine weak nodes, expand new sub-nodes, and reorganize the data organization structure based on model feedback. This prevents the system from implementing a dynamic evolution strategy where "the classification system grows where the model is weak" during training. Consequently, data generation systems struggle to continuously optimize data structures in line with changes in model capabilities, especially in long-term training, domain adaptation, and multi-round iterative generation scenarios, where this problem is particularly prominent.

[0007] (4) Existing classification organization methods and data generation processes are often disconnected, lacking a deep collaborative mechanism for addressing capability gaps. For large model training tasks, existing technologies include methods for organizing data space through knowledge catalogs, capability trees, and task labels, as well as methods for generating samples in batches through template generation, instruction expansion, automatic question answering, and model self-generation. However, these methods are mostly independently designed and loosely combined, lacking a unified collaborative optimization framework. In actual training scenarios, the value of the classification system lies not only in statically organizing the task space, but also in providing fine guidance for the data generation process to address capability gaps; and the significance of data generation lies not only in increasing the number of samples, but also in addressing training shortcomings in a targeted manner based on the classification structure and model feedback. Existing methods generally lack a deep two-way collaborative mechanism for "how the classification system evolves in response to model feedback" and "how the evolved classification system drives data generation refocusing in reverse," making it difficult to effectively suppress problems such as overly coarse node definitions, drifting generation direction, misaligned data reinforcement, and redundant repeated generation, affecting the stability and relevance of synthetic data systems in continuous training scenarios.

[0008] (5) Existing synthetic data generation processes lack a closed-loop optimization mechanism that balances capability enhancement, data generation efficiency, and the feasibility of continuous evolution, making it difficult to stably output high-quality training data. Traditional synthetic data methods typically optimize around a single objective, such as improving the quality of a single generation round, enhancing sample surface diversity, expanding data scale, or improving the effect of prompt templates. However, they lack a closed-loop optimization mechanism that can simultaneously balance capability structure expression, node weakness perception, classification system adaptive evolution, and data generation resource control within a unified framework. Due to the lack of a complete feedback chain from initial task structure establishment, sample generation, model training, node feedback extraction, weak capability identification, classification system evolution, incremental sample generation to retraining and stopping determination, existing methods often fail to achieve the comprehensive optimization goals of "more adequate reinforcement of key capability nodes without significantly increasing overall generation redundancy," "more accurate refinement of weak areas without repeatedly wasting budget in already mastered areas," and "maintaining the controllability, interpretability, and engineering feasibility of the data construction process while improving model capabilities." This limits their practical application value in complex scenarios such as finance, law, healthcare, industry, government affairs, and edge intelligence. Summary of the Invention

[0009] Based on the above analysis, the embodiments of the present invention aim to provide a synthetic data generation method, system, device, and storage medium to solve the problem of how to stably, continuously, and controllably construct high-quality training data that is truly effective for the model under conditions such as uneven data coverage, complex task hierarchy, concentrated capability shortcomings, scarcity of real data, expensive manual annotation, and continuous changes in training status.

[0010] On one hand, embodiments of the present invention provide a synthetic data generation method, comprising: generating an initial synthetic dataset based on an initial classification system; training a target model using the initial synthetic dataset to obtain a trained round-t model, and then testing the trained round-t model using a validation sample set, where t is a positive integer; mapping validation samples to classification nodes to obtain node-level feedback, wherein the node-level feedback includes the classification node to which the validation sample belongs and the node-level accuracy; identifying weak classification nodes based on the comparison result between the node-level accuracy and a node-level accuracy threshold; dynamically evolving the initial classification system or the current classification system for the set of weak classification nodes to obtain an updated classification system; generating an incremental synthetic dataset based on the updated classification system, and retraining the round-t model based on the cumulative training dataset to obtain a round-t+1 model, wherein the cumulative training dataset is the union of the current synthetic dataset and the incremental synthetic dataset; and repeatedly executing the node-level feedback step, the weak classification node identification step, the classification system dynamic evolution step, and the incremental synthetic dataset generation step until a preset stopping condition is met, and then outputting the final classification system, the final training dataset, and the final model.

[0011] The beneficial effects of the above technical solution are as follows: This application can finely distinguish and explicitly model different capability nodes and regions with different levels of mastery in complex training tasks. It dynamically evolves the classification system based on the model's real feedback at classification nodes, correcting the data redundancy and insufficient reinforcement problems caused by the traditional static task organization method's "equal treatment" of mastered and weak regions. Furthermore, it introduces a collaborative update mechanism of classification system evolution and incremental generation during data construction, and achieves unified control over model capability improvement and data generation costs through closed-loop iterative optimization. When constructing synthetic data for different domains, task levels, and training stages, this method effectively avoids common problems in traditional solutions such as overly coarse task structures, insufficient reinforcement of weak nodes, excessive repetitive generation of mastered regions, and insufficient long-term iterative controllability. It significantly improves the data utilization efficiency, model reinforcement effect, operational stability, and engineering application value of the synthetic data generation system.

[0012] Further improvements to the above method, mapping validation samples to classification nodes to obtain node-level feedback, further include: defining a mapping function from validation samples to classification nodes; obtaining the corresponding validation sample set for any classification node; calculating the node-level accuracy rate based on the number of correct and incorrect answers at any classification node, wherein the node-level accuracy rate is the ratio between the number of correct answers at any classification node and the total number of samples, and the total number of samples is the sum of the number of correct and incorrect answers at any classification node; and using all classification nodes and their corresponding node-level accuracy rates as the node-level feedback results.

[0013] A further improvement to the above method, identifying weak classification nodes based on the comparison between the node-level accuracy and a preset node-level accuracy threshold, further includes: pre-setting a node-level accuracy threshold; comparing the node-level accuracy of the current classification node among all classification nodes with the node-level accuracy threshold, wherein when the node-level accuracy of the current classification node is less than the node-level accuracy threshold, the current classification node is identified as a weak classification node; otherwise, the current classification node is identified as a non-weak classification node.

[0014] Further improvements to the above method, for the set of weak classification nodes, dynamically evolving the initial classification system or the current classification system to obtain an updated classification system further include: dynamically refining and expanding the weak classification nodes to obtain an expanded child node set of the weak classification nodes, so as to automatically adjust the granularity with node-level feedback; and obtaining an updated classification system by replacing the weak classification nodes in all classification nodes with the expanded child node set.

[0015] Further improvements to the above method, generating an incremental synthetic dataset based on the updated classification system, further include: generating an updated candidate sample set for the expanded child node set in the updated classification system; performing consistency screening on the updated candidate sample set to obtain the incremental synthetic dataset; and merging the incremental synthetic dataset into the current synthetic dataset as the cumulative training dataset.

[0016] Based on further improvements to the above method, the preset stopping conditions include: no weak classification nodes are identified or the classification system no longer changes in two consecutive rounds.

[0017] Further improvements to the above method include generating an initial synthetic dataset based on an initial classification system, which further comprises: establishing an initial classification system based on a target task description received from a client; generating a candidate sample dataset based on the initial classification system; and performing consistency screening on each candidate sample in the candidate sample dataset to obtain the initial synthetic dataset according to the consistency screening.

[0018] Based on a further improvement of the above method, the consistency screening includes: calculating a consistency score for each candidate sample, wherein the consistency score is a weighted sum of the consistency score between the candidate sample and the target node, the semantic matching score between the input and output, and the logical consistency score of the candidate sample content; and retaining the candidate sample when the consistency score is greater than or equal to the sample screening threshold.

[0019] On the other hand, embodiments of the present invention provide a synthetic data generation system, comprising: a synthetic sample generation module for generating an initial synthetic dataset based on an initial classification system; a target model training and validation module for training a target model using the initial synthetic dataset to obtain a trained round-t model, and then testing the trained round-t model using a validation sample set, where t is a positive integer; a node-level model feedback module for mapping validation samples to classification nodes to obtain node-level feedback, wherein the node-level feedback includes the classification node to which the validation sample belongs and the node-level accuracy; a weak node identification module for identifying weak classification nodes based on a comparison between the node-level accuracy and a node-level accuracy threshold; and a dynamic classification system. The system includes an adaptive evolution module for dynamically evolving the initial or current classification system for the set of weak classification nodes to obtain an updated classification system; an incremental synthetic data generation module for generating an incremental synthetic dataset based on the updated classification system and retraining the t-th round model based on the cumulative training dataset to obtain the t+1-th round model, wherein the cumulative training dataset is the union of the current synthetic dataset and the incremental synthetic dataset; and an iterative update and stopping module for repeatedly executing the node-level feedback step, the weak classification node identification step, the classification system dynamic evolution step, and the incremental synthetic dataset generation step until a preset stopping condition is met, at which point the final classification system, the final training dataset, and the final model are output.

[0020] In another aspect, embodiments of the present invention provide a synthetic data generation device, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to execute the method described in the above embodiments.

[0021] In another aspect, embodiments of the present invention provide a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the methods described in the above embodiments.

[0022] Compared with existing technologies, the present invention can achieve at least one of the following beneficial effects in terms of task capability structure expression ability, model weakness perception ability, classification granularity adaptive evolution ability, data generation resource allocation ability, continuous training reinforcement ability, and system engineering deployability: (1) A hierarchical classification system modeling mechanism enables the collaborative expression of the global task space and local fine-grained capability nodes. This invention introduces a hierarchical classification system modeling mechanism to uniformly organize and express target tasks according to capability levels, knowledge structures, error patterns, or business subdomains. This allows the system to represent the domain-level task space while retaining fine-grained node differences and local capability boundary information. This mechanism enables the system to collaboratively model "task-level capabilities / subtask-level capabilities / node-level subdivision patterns," achieving a key shift from "flat task organization" to "hierarchical capability structure expression," significantly improving task decomposability, capability localization, and data organization stability in complex training scenarios.

[0023] (2) Node-level model feedback perception mechanism to achieve fine identification and explicit modeling of key weak capability nodes. This invention introduces a node-level model feedback perception mechanism, which combines the mapping relationship between samples and classification nodes, node accuracy statistics, node error rate statistics, and node weakness analysis to detect, quantify, and model the model's performance on different classification nodes, thereby effectively distinguishing between mastered nodes and high-value weak nodes. This mechanism enables the system to differentiate between "stable nodes / weak nodes / high-risk error-prone nodes," achieving a key improvement from "overall training effect evaluation" to "fine analysis based on node mastery," significantly enhancing the system's ability to locate the model's true shortcomings in complex domains, long-tail tasks, and multi-round training scenarios.

[0024] (3) Based on the model feedback-driven adaptive evolution mechanism of the classification system, the effect of strengthening weak capabilities is improved and static structural mismatch is suppressed. This invention further introduces an adaptive evolution mechanism of the classification system on the basis of node-level model feedback results. It dynamically performs refinement expansion, capability boundary reconstruction and local structural evolution on weak nodes, so that the system can form a more granular capability characterization of difficult nodes, while maintaining stability or reducing redundant expansion of mastered areas. This mechanism realizes the transformation from the "preset static classification system" to the "dynamic classification system that continuously evolves for model capability shortcomings", which significantly improves the matching degree between the classification system and the actual training requirements of the model, while reducing the data organization mismatch problem caused by static task structure.

[0025] (4) A collaborative mechanism between classification system evolution and synthetic data generation improves the targeting, continuity, and effectiveness of the data reinforcement process. This invention constructs a collaborative reinforcement mechanism based on the classification system evolution results and the sample generation process. In each iteration, the system can reorganize the generation target and adjust the generation focus around the weakest classification node according to the model training state of the previous round, and use the newly generated samples to reinforce the training support of the weak node. As a result, the system achieves a key improvement from "static classification structure serving one-time generation" to "dynamic collaboration between classification system evolution and incremental generation". It enables high-risk capability nodes to gradually converge from coarse-grained task areas to fine-grained reinforcement targets, significantly improving the stability, continuity, and interpretability of the reinforcement process for long-tail nodes, boundary nodes, and error-prone nodes in complex training scenarios.

[0026] (5) Closed-loop iterative optimization mechanism to achieve unified enhancement of model capability improvement and data generation efficiency. This invention integrates classification system modeling, sample generation, sample-node mapping, model training, node-level feedback extraction, weak node identification, dynamic evolution of the classification system, and incremental sample generation into a unified data closed loop, enabling the system to dynamically control and provide feedback correction on the reinforcement effect and generation resource usage at each stage of training data construction. On the reinforcement side, the system can improve the data support depth of key capability nodes; on the generation side, the system can reduce the repeated expansion of low-value nodes; on the training side, the system can control the overall data expansion and ineffective generation overhead through an iterative stopping mechanism. This closed-loop mechanism transforms the synthetic data construction process from a single static generation into a dynamic optimization process targeting the real capability shortcomings of the model, thereby significantly improving the relevance, effectiveness, and engineering practical value of the system's output training data.

[0027] (6) A low-intrusion, integrable solution that adapts to existing large-scale model training processes and possesses good scalability. The core mechanism of this invention mainly functions in the organization of classification systems, node-level feedback extraction, weak node identification, classification system evolution, and incremental generation scheduling. It can be integrated as a modular capability into existing pre-training data construction systems, supervised fine-tuning data systems, instruction data generation platforms, domain-adaptive training platforms, automatic evaluation sample construction systems, or agent task data engines. It is compatible with existing data construction systems, including template generation frameworks, bootstrapping expansion frameworks, automatic question answering frameworks, multi-model collaborative generation frameworks, and sample screening modules. The system only needs to add steps such as sample node mapping, node-level feedback extraction, adaptive evolution of classification systems, and incremental generation to the original data generation and training process to achieve a significant improvement in the reinforcement effect of weak capabilities and data utilization efficiency. Meanwhile, this invention supports dynamic adjustment of parameters such as node performance threshold, node expansion granularity, incremental generation budget, iteration round limit, and stopping threshold. It can be flexibly configured according to different domain task complexity, different model capability base, different training cycle requirements, and different engineering budget conditions, and has strong versatility, scalability, and engineering deployment adaptability.

[0028] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0029] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0030] Figure 1 is a flowchart of a synthetic data generation method according to an embodiment of the present invention; Figure 2 The diagram illustrates the steps of a synthetic data generation method according to an embodiment of the present invention. Figure 3 This is a block diagram of a synthetic data generation system according to an embodiment of the present invention. Detailed Implementation

[0031] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0032] refer to Figure 1A specific embodiment of the present invention discloses a method for generating synthetic data, comprising the following steps: In step S101, an initial synthetic dataset is generated based on the initial classification system. Generating the initial synthetic dataset based on the initial classification system further includes: establishing an initial classification system based on the target task description received from the client; generating a candidate sample dataset based on the initial classification system; and performing consistency screening on each candidate sample in the candidate sample dataset to obtain the initial synthetic dataset according to the consistency screening.

[0033] In step S102, the target model is trained using the initial synthetic dataset to obtain the trained round t model, and then the trained round t model is tested using the validation sample set, where t is a positive integer.

[0034] In step S103, validation samples are mapped to classification nodes to obtain node-level feedback, whereby the node-level feedback includes the classification node to which the validation sample belongs and the node-level accuracy. Specifically, mapping validation samples to classification nodes to obtain node-level feedback further includes: defining a mapping function from validation samples to classification nodes; obtaining the corresponding validation sample set for any classification node; calculating the node-level accuracy based on the number of correct and incorrect answers at any classification node, whereby the node-level accuracy is the ratio between the number of correct answers at any classification node and the total number of samples, and the total number of samples is the sum of the number of correct and incorrect answers at any classification node; and using all classification nodes and their corresponding node-level accuracy as the node-level feedback result.

[0035] In step S104, weak classification nodes are identified based on the comparison between node-level accuracy and a node-level accuracy threshold. Specifically, identifying weak classification nodes based on the comparison between node-level accuracy and a preset node-level accuracy threshold further includes: pre-setting a node-level accuracy threshold; comparing the node-level accuracy of the current classification node among all classification nodes with the node-level accuracy threshold, wherein if the node-level accuracy of the current classification node is less than the node-level accuracy threshold, the current classification node is identified as a weak classification node; otherwise, the current classification node is identified as a non-weak classification node.

[0036] In step S105, for the set of weak classification nodes, the initial classification system or the current classification system is dynamically evolved to obtain an updated classification system. Specifically, for the set of weak classification nodes, dynamically evolving the initial classification system or the current classification system to obtain an updated classification system further includes: dynamically performing refinement expansion on the weak classification nodes to obtain an expanded child node set of the weak classification nodes, so as to automatically adjust the granularity with node-level feedback; and obtaining the updated classification system by replacing the weak classification nodes in all classification nodes with the expanded child node set.

[0037] In step S106, an incremental synthetic dataset is generated based on the updated classification system, and the model at round t is retrained based on the cumulative training dataset to obtain the model at round t+1. The cumulative training dataset is the union of the current synthetic dataset and the incremental synthetic dataset. Specifically, generating the incremental synthetic dataset based on the updated classification system further includes: generating an updated candidate sample set for the expanded child node set in the updated classification system; performing consistency screening on the updated candidate sample set to obtain the incremental synthetic dataset; and merging the incremental synthetic dataset into the current synthetic dataset as the cumulative training dataset. The consistency screening includes: calculating a consistency score for each candidate sample in the updated candidate sample set, where the consistency score is a weighted sum of the consistency score between the candidate sample and the target node, the semantic matching score between the input and output, and the logical consistency score of the candidate sample content; and retaining the candidate sample when the consistency score is greater than or equal to the sample screening threshold.

[0038] In step S107, the node-level feedback step, the weak classification node identification step, the classification system dynamic evolution step, and the incremental synthetic dataset generation step are repeatedly executed until a preset stopping condition is met, at which point the final classification system, the final training dataset, and the final model are output. Specifically, the preset stopping conditions include: no weak classification nodes are identified or the classification system no longer changes after two consecutive rounds.

[0039] Compared with existing technologies, the synthetic data generation method provided in this embodiment can finely distinguish and explicitly model different capability nodes and regions with different levels of mastery in complex training tasks. It dynamically evolves the classification system based on the model's real feedback at classification nodes, correcting the data redundancy and insufficient reinforcement problems caused by the traditional static task organization method's "equal treatment" of mastered and weak regions. Furthermore, it introduces a collaborative update mechanism of classification system evolution and incremental generation during data construction, and achieves unified control over model capability improvement and data generation costs through closed-loop iterative optimization. When constructing synthetic data for different domains, task levels, and training stages, this method effectively avoids common problems in traditional solutions such as overly coarse task structures, insufficient reinforcement of weak nodes, excessive repetitive generation of mastered regions, and insufficient long-term iterative controllability. It significantly improves the data utilization efficiency, model reinforcement effect, operational stability, and engineering application value of the synthetic data generation system.

[0040] In the following text, refer to Figure 2This paper provides a detailed explanation of the synthetic data generation method according to embodiments of the present invention, using specific examples. For instance, to train a large-scale financial risk control model, the goal is to enable the model to identify risk types in banking scenarios, particularly credit risk, liquidity risk, and compliance risk. Initially, the system discovers that existing real data is insufficient. Therefore, the method of this invention is used to automatically generate training data and continuously update the classification system based on error feedback after model training, thereby generating more targeted synthetic data. The core steps include: model feedback acquisition → weak point identification → dynamic evolution of the classification system → incremental synthetic data generation → closed-loop iterative optimization.

[0041] Step 1: Initialize the initial classification system: First, obtain the target task description and establish an initial classification system based on the target task description. This initial classification system provides the initial structural framework for subsequent dynamic evolution.

[0042] Let the initial classification system be represented as:

[0043] in: · This represents the initial classification system in round 0; · Indicates the first Each category node; · This represents the total number of nodes in the initial classification system. In this embodiment, the initial classification system can be set as follows: • Primary risk levels: credit risk, liquidity risk, and compliance risk; Second-level nodes: Credit risk includes: borrower default, insufficient collateral, and excessive industry concentration; Liquidity risks include: short-term debt investment in long-term investments, reliance on wholesale financing, and deposit outflows. Compliance risks include: insufficient information disclosure, deficiencies in anti-money laundering measures, and irregularities in the approval process. The purpose of this step is to establish an initial capability partitioning framework for the system, so that subsequent model feedback can be mapped to specific nodes, rather than remaining at the level of overall accuracy.

[0044] Step 1: Real-world case demonstration. The system first establishes three main categories: "Credit Risk," "Liquidity Risk," and "Compliance Risk," then refines them into sub-nodes such as "Reliance on Wholesale Financing," "Insufficient Collateral," and "Deficiencies in Anti-Money Laundering." At this point, this classification system is just the starting point and will continue to evolve based on model feedback.

[0045] Step 2: Generate initial synthetic data based on the initial classification system. The system establishes sample generation templates for each leaf node in the initial classification system and calls the teacher model to generate initial candidate synthetic samples.

[0046] Set nodes The corresponding candidate sample set is:

[0047] Composition of all candidate samples:

[0048] in: · Represents a node The corresponding set of candidate samples; · Represents a node The next generated 10 candidate samples; · Represents a node The number of candidate samples generated; · This represents the set of all candidate samples generated in round 0. To ensure sample quality, each candidate sample... Calculate the consistency score:

[0049] in: · Indicates sample The overall consistency score; · This represents the consistency score between the sample and the target node; · This represents the semantic matching score between the input and output. For example, if the input asks "What is this risk?", but the output becomes "Give suggestions for rectification", or the answer doesn't address the key point of the input, this is a semantic mismatch. · The logical consistency score represents the content of the sample. · , , This represents the weighting coefficients of the three scoring items; And satisfy:

[0050] When the sample satisfies:

[0051] When this happens, it is retained in the initial valid dataset:

[0052] in: · Indicates the sample selection threshold; · This represents the valid synthetic dataset for round 0. The purpose of this step is to generate the first round of initial synthetic data that can be used for training, providing basic samples for subsequent model training and feedback analysis.

[0053] Step Two: Practical Case Demonstration: For example, the following sample is generated under the "Liquidity Risk - Wholesale Funding Dependence" node: Input: "In the past three months, more than 60% of the new liabilities of a certain regional bank came from short-term interbank lending, while the growth of customer deposits continued to slow down." Output: "This scenario mainly reflects the dependence on wholesale funding in liquidity risk, because the bank is overly reliant on short-term market-based financing and lacks stable deposit sources." After a consistency check, the sample was retained in the initial valid synthetic dataset. middle.

[0054] Step 3: Establish the mapping relationship between samples and classification nodes. For effective synthetic datasets... For each sample, record its corresponding classification node and establish a mapping relationship between samples and nodes:

[0055] in: · The function representing the mapping from samples to nodes; · This represents a valid synthetic sample; · Indicates sample The category node to which it belongs. This yields the set of mapping relationships:

[0056] in: · This represents the set of sample-node mapping relationships in round 0. The purpose of this step is to ensure that subsequent model errors can be accurately mapped back to the classification nodes, thereby supporting node-level feedback analysis and dynamic evolution.

[0057] Step 3: Real-world case demonstration: The sample retained in the previous step will be recorded as:

[0058] If the model answers incorrectly on this sample in the future, the system will know that the error comes from the "wholesale financing dependency" node, instead of simply recording it as "financial risk control error".

[0059] Step 4: Train the target model using the initial synthetic data. Utilize the initial, valid synthetic dataset. The target model is trained to obtain the first-round model.

[0060] Let the training process be represented by the training operator. Then we have:

[0061] in: · Represents the initial target model; · This represents the model after training with synthetic data in round 0; · This indicates the model training or fine-tuning process; · This represents the valid synthetic dataset in round 0. The purpose of this step is to allow the model to develop an initial learning capability based on the current classification system and the currently generated data, so that we can then observe which nodes the model is still lacking.

[0062] Step 4: Real-world case demonstration: The system feeds a dataset containing node samples such as "borrower default," "wholesale financing dependence," and "anti-money laundering deficiencies" into the bank risk identification model for fine-tuning, resulting in the first round of training model. .

[0063] Step 5: Extract model feedback based on validation data. Use the validation data to evaluate the trained model. Tests are conducted, and validation samples are mapped to classification nodes to obtain node-level performance feedback.

[0064] Let the validation set be:

[0065] in: · Represents the set of validation samples; · Indicates the first Verification sample; · This represents the total number of validation samples. Define the mapping function from validation samples to classification nodes:

[0066] in: · This represents the mapping function from the validation sample to the node; · Indicates verification sample The category node to which it belongs. For any node The corresponding set of verification samples is:

[0067] in: · Represents all nodes mapped to The set of verification samples. Furthermore, set nodes The number of correct answers is The number of incorrect answers was The node-level accuracy is:

[0068] in: · Represents a node The accuracy rate; · Represents a node The number of validation samples that the model answered correctly; · Represents a node The number of validation samples that the model answered incorrectly. The feedback results from all nodes consist of:

[0069] in: · This represents the set of node-level performance feedback results for the first round of the model. The purpose of this step is to switch from "overall model performance" to "node-level capability profile" to identify the model's true weak areas.

[0070] Step 5: Practical Case Demonstration System testing revealed the following: • The “Credit Risk – Insufficient Collateral” node performed well; • The performance of the “Liquidity Risk – Short-Term Debt for Long-Term Investment” segment was generally mediocre; The “Liquidity Risk – Wholesale Funding Dependence” node performed the worst. For example, the system calculates:

[0071] This indicates that the model's ability to identify the "wholesale financing dependence" node is significantly insufficient.

[0072] Step Six: Identify Vulnerable Nodes. Set Node-Level Performance Thresholds. When the accuracy of a node is lower than the threshold, it is identified as a weak node.

[0073] The set of weak nodes is defined as:

[0074] in: · This represents the set of weak nodes identified in the first round; · Indicates the node performance threshold; · Represents a node The accuracy rate. The purpose of this step is to transform model feedback into actionable evolutionary objectives, that is, to clearly identify "which nodes in the classification system need to be further refined or strengthened".

[0075] Step Six: Practical Case Demonstration: Assume the system sets a threshold:

[0076] Since the "wholesale funding dependence" node is only 0.52, which is below the threshold, the system determines that:

[0077] At this point, the system knows that the next step should not be to replenish all the data evenly, but to prioritize dealing with this weak node.

[0078] Step 7: Perform dynamic evolution on weak nodes. This applies to the set of weak nodes. For the original classification system The classification system is dynamically evolved to obtain an updated classification system. .

[0079] In this embodiment, the dynamic evolution adopts a refined expansion approach. Considering the weak nodes... The evolutionary result is as follows:

[0080] in: · Represents a node The extended results; · Indicates that by node The first obtained through evolution A new child node; · Represents a node The number of new child nodes after splitting. The updated classification system is defined as follows:

[0081] in: · This represents the classification system after the first round of evolution; · This indicates the removal of weak, coarse-grained nodes from the original classification system; · This represents the new set of child nodes after adding all weak nodes. The purpose of this step is to allow the classification system to automatically adjust its granularity based on model feedback, so that the subsequently generated data no longer remains at the coarse-grained category level, but can more accurately compensate for the shortcomings of the model.

[0082] Step 7: Practical Case Demonstration: The system analyzes the error samples under the "Wholesale Financing Dependence" node and finds that the errors mainly come from two situations: • Dependence on short-term interbank financing; • High-cost wholesale financing rollover. The system then undergoes the following evolution:

[0083] This means that the original coarse node "wholesale financing dependence" has been replaced with two new, finer nodes. This allows the system to subsequently supplement data around these two more specific sub-problems.

[0084] The process of dynamically refining and expanding the weak classification nodes includes the following steps: Step 7-1: Obtain the set of erroneous samples and feedback features corresponding to the weak classification nodes.

[0085] For any weak classification node The system extracts samples from validation samples, test samples, or post-training evaluation samples that show errors in model output at a given node, low confidence levels, unstable answers, or significant deviations from the standard answer, forming the set of error samples corresponding to that weak classification node.

[0086] in, Indicates the first Weak classification nodes in the wheel The corresponding set of error samples, Indicates the first node under this node Error sample This indicates the number of incorrect samples under this weak classification node.

[0087] Furthermore, the system extracts error feedback features for each error sample. These error feedback features include at least one or more of the following: task semantic features, error type features, model output bias features, answer confusion object features, missing reasoning step features, node affiliation boundary features, and sample difficulty features. For each error sample... Its feedback characteristics are expressed as:

[0088] in, Indicates erroneous samples The feedback feature vector, This represents the feedback feature extraction function. Indicates standard output. Indicates the model output, This indicates the weak classification node to which the erroneous sample belongs.

[0089] The purpose of this step is to further transform "a node is performing poorly" into "what specific error modes within that node are causing the poor performance", providing a basis for subsequent refinement and expansion.

[0090] Step 7-2: Identify sub-error patterns within weak classification nodes based on error feedback features.

[0091] The system identifies weak classification nodes. The corresponding feedback feature set is then subjected to clustering, rule induction, semantic similarity analysis, or teacher model attribution analysis to identify one or more sub-error patterns within the weak node. Let the feedback feature set be:

[0092] The system divides its execution sub-modes as follows:

[0093] get:

[0094] in, Indicates weak classification nodes The set of sub-error patterns identified internally. Indicates the first Individual error patterns, This indicates the number of sub-error modes that the weak classification node is divided into.

[0095] The sub-error patterns can correspond to different knowledge gaps, reasoning defects, task boundary confusion, changes in input conditions, risk triggering factors, abnormal scenario types, or long-tail business situations. For example, in a financial risk control scenario, if the weak classification node is "liquidity risk - wholesale financing dependence", the system can further identify sub-error patterns such as "short-term interbank financing dependence", "high-cost wholesale financing rollover", "excessive proportion of unstable liabilities", and "failure of rollover under tightening market liquidity" based on the error samples.

[0096] Step 7-3: Generate candidate child nodes based on the sub-error pattern.

[0097] For each sub-error mode The system generates one or more candidate child nodes. Each candidate child node includes not only its name but also its semantic definition, applicable boundaries, exclusion boundaries, typical input features, typical output requirements, and constraints for subsequent sample generation. A candidate child node can be represented as:

[0098] in, Indicates the candidate child node name. This represents the semantic definition of a candidate child node. Indicates the applicable boundaries of this node. This indicates the exclusion boundary between this node and other nodes. This indicates the sample generation template or template constraint corresponding to the candidate child node.

[0099] This leads to the weak classification nodes. The set of candidate child nodes:

[0100] The purpose of this step is to convert abstract error patterns into structured classification nodes that can be carried by the classification system and called by the sample generation module.

[0101] Step 7-4: Verify the validity of candidate child nodes.

[0102] To avoid overexpansion of the classification system, semantic redundancy of sub-nodes, or unclear boundaries, the system performs validity checks on candidate sub-nodes. These validity checks include at least coverage checks, discriminative checks, sample support checks, and hierarchical rationality checks.

[0103] For candidate child nodes Its overall effectiveness score is:

[0104] in, The coverage score measures whether the candidate child node can cover a sufficient number of error samples. The distinguishability score is used to measure whether the semantic boundaries between the candidate child node and its sibling or neighboring nodes are clear. The sample support score is used to measure whether the candidate child node has sufficient sample generation value and training support value. This represents the hierarchical rationality score, used to measure whether the candidate child node is suitable as a subordinate node of the current weak node; Let represent the weight coefficient of each scoring item, and satisfy:

[0105] When the candidate child node satisfies:

[0106] If the condition is met, the candidate child node is determined as a valid child node; otherwise, the candidate child node is merged, renamed, its boundaries corrected, or deleted. This represents the threshold for the validity of child nodes.

[0107] Step 7-5: Determine the refined expansion results of weak classification nodes.

[0108] The candidate child nodes that pass the validity check are summarized to obtain the weak classification nodes. Detailed expansion results:

[0109] in, Indicates weak classification nodes The new set of child nodes obtained after dynamic refinement and expansion. Indicates weak classification nodes The generated first One valid child node, This indicates the number of new child nodes retained after passing the validity check.

[0110] In one alternative implementation, if the number of candidate child nodes is too large, the system selects several top-ranked child nodes based on node reinforcement priority, generation budget, and maximum expansion granularity limit. Node reinforcement priority can be expressed as:

[0111] in, Indicates error concentration. Indicates the frequency of sample occurrence. Indicates the difficulty of the sample. Indicates business value or training value. This represents the weighting coefficient.

[0112] Step 7-6: Update the hierarchical structure and node paths of the classification system.

[0113] The system will classify the original weak nodes. The leaf node is adjusted to become the parent node, and the effective child nodes are attached to the weak classification node to form the updated local classification structure. The updated classification system is represented as follows:

[0114] In another implementation, the system may also retain the original weak classification nodes. As an intermediate node, it is not removed from the classification system, that is:

[0115] At this point, the original weak classification node is retained as the parent node, and the newly generated refined child node is used as its lower-level leaf node for subsequent sample generation, model feedback statistics, and incremental training.

[0116] For each newly generated child node The system generates the corresponding node path:

[0117] in, This represents the node path of the original weakly classified node. This indicates the complete path of the newly added child node in the updated classification system.

[0118] Step 7-7: Generate sample templates and mapping rules for newly added child nodes.

[0119] For each newly added child node The system generates corresponding sample generation templates based on their node definitions, applicable boundaries, and typical error samples:

[0120] in, This indicates the sample generation template corresponding to the newly added child node. This represents the template generation function. Indicates node definition, Indicates the applicable boundaries, Indicates exclusion of boundaries, This represents the set of erroneous samples that were assigned to this child node.

[0121] Simultaneously, the system updates the sample-node mapping rules, enabling newly generated samples, validation samples, and error samples to be mapped to newly added child nodes, instead of only mapping to the original coarse-grained weak nodes. The updated mapping function is expressed as:

[0122] in, This indicates that it belongs to a newly added child node. The sample, Indicates the first The updated sample-node mapping function.

[0123] Steps 7-8: Output the refined and expanded classification system for incremental synthetic data generation.

[0124] After completing the above processing, the system outputs the updated dynamic classification system. The system adds a set of child nodes, node paths, node definitions, sample generation templates, and mapping rules, which are then used as inputs for the next step of generating incremental synthetic data. Subsequently, the system no longer generates samples in a general manner around the original weak classification nodes, but instead generates incremental synthetic data separately around the newly added fine-grained child nodes, thereby improving the targeting of data reinforcement and the effectiveness of training.

[0125] Step Seven: Supplementary Case Study Taking the "Liquidity Risk - Wholesale Funding Dependence" node as an example, the system detected that the accuracy rate for this node was below the threshold in the model feedback, and further analyzed the erroneous samples under this node. The analysis results show that the model's errors are mainly concentrated in the following categories: First, it is unable to identify the insufficient financing stability caused by an excessively high proportion of short-term interbank lending; Second, it is impossible to distinguish between wholesale financing dependence and the growth in general liabilities; Third, it is impossible to determine the liquidity pressure caused by the continuous rollover of high-cost market financing; Fourth, there is insufficient identification of the risks associated with refinancing under tight market liquidity conditions.

[0126] Therefore, the system further refines the original "Liquidity Risk - Wholesale Funding Dependence" node into: "Liquidity risk - reliance on wholesale funding - reliance on short-term interbank funding"; "Liquidity risk - reliance on wholesale funding - high-cost wholesale funding rollover"; "Liquidity risk - reliance on wholesale financing - excessively high proportion of unstable liabilities"; "Liquidity risk - reliance on wholesale funding - risk of refinancing under market tightening"

[0127] For each new sub-node, the system generates a corresponding data generation template. For example, for the "short-term interbank financing dependence" sub-node, the template requires that the sample input include characteristics such as short term, interbank lending, concentrated financing terms, and insufficient customer deposits. The output should clearly indicate that the scenario belongs to short-term interbank financing dependence under liquidity risk, and explain that the source of its risk lies in the short financing term, weak stability, and high pressure for renewal.

[0128] Through the above refinement and expansion, the classification system has changed from the original coarse-grained nodes to fine-grained nodes that can reflect the specific sources of errors in the model, enabling subsequent incremental synthetic data generation to more accurately strengthen the model's true weaknesses.

[0129] Step 8: Generate incremental synthetic data based on the evolved classification system. The system generates incremental synthetic data based on the updated classification system. The newly added fine-grained nodes establish new sample generation templates and generate a new round of candidate synthetic samples.

[0130] Let the set of newly added candidate samples in the first round be:

[0131] in: · This represents the set of candidate samples added in the first round; · This represents the set of nodes newly added to the updated classification system compared to the previous round. · Indicates the addition of a new node The corresponding set of candidate samples. These candidate samples were also subjected to consistency screening to obtain an incremental effective dataset:

[0132] Then add it to the cumulative training dataset:

[0133] in: · This indicates the newly added valid synthetic dataset in the first round; · This represents the updated cumulative training dataset. The purpose of this step is to allow the newly generated data to directly serve the evolved fine-grained nodes, thereby achieving targeted incremental generation that "fills in the gaps where they are weak".

[0134] Step 8: Real-world case demonstration: The system no longer generates general "liquidity risk" data, but instead generates data with a specific focus: • Sample of “reliance on short-term interbank financing”; • Sample of “high-cost wholesale financing rollover”. For example, the system generates a new sample: Input: "A certain small and medium-sized bank has long relied on overnight interbank funding to maintain its cash balance, with funding terms generally less than seven days."

[0135] Output: "This scenario reflects the reliance on short-term interbank financing in liquidity risk. The problem is that the institution is overly dependent on short-term market financing, and it will be difficult to continue financing once liquidity tightens."

[0136] It can be seen that the data generated in this round is significantly more focused and detailed than that in the previous round.

[0137] Step 9: Retrain the model based on the cumulative data. Utilize the updated cumulative training dataset. Retrain the model to obtain the next model:

[0138] in, This represents the second round of the model; This represents the previous model; This represents the updated cumulative training dataset. The purpose of this step is to allow the model to absorb new data generated targeting weaknesses, thereby gradually improving its performance at fine-grained risk points.

[0139] Step Nine: Practical Case Demonstration The system fine-tuned the model again using a dataset that included new samples of "short-term interbank financing dependence" and "high-cost wholesale financing rollover," resulting in the second-round model. .

[0140] Step 10: Repeat the feedback, evolution, and incremental generation process. For the first... The process of "node-level feedback extraction - weak node identification - dynamic evolution of classification system - incremental synthetic data generation - model retraining" is repeated in each round.

[0141] No. The general form of a wheel is:

[0142] in: · Indicates the first Round-time node The accuracy rate; · Indicates the first Round-time node The number of samples that answered correctly; · Indicates the first Round-time node The number of samples that answered incorrectly. No. The set of weak nodes of the wheel is:

[0143] in: · Indicates the first The set of weak nodes identified by the round; · This indicates the previous classification system. The classification system continued to evolve as follows:

[0144] The cumulative training dataset has been updated to:

[0145] The model is updated as follows:

[0146] in: · Indicates the first Add new valid synthetic datasets in each round; · Indicates as of the date The cumulative training dataset of rounds; · Indicates the first Wheel model. The purpose of this step is to form a complete closed loop, so that the classification system and training data evolve synchronously with the model performance, rather than remaining in a single-round static generation mode.

[0147] Step 10: Practical Case Demonstration: After the second round of training, the system found that the accuracy of the "short-term interbank financing dependence" node had improved, while the accuracy of the "high-cost wholesale financing rollover" node remained low. The system then continues to generate more detailed and targeted samples around the concept of "high-cost wholesale financing rollover" and retrains the model. This process is repeated until all key nodes are raised to the target level.

[0148] Step 11: Output the final result after the stopping condition is met. The iteration stops when the system meets the preset stopping condition. The stopping condition can be defined as:

[0149] in: Indicates the first The minimum accuracy of all nodes in the round classification system; This indicates the preset performance threshold. Alternatively, the classification system can be stopped when it no longer changes after two consecutive rounds:

[0150] in: This indicates that the classification system is stable and no new or refined nodes will be added.

[0151] The final output is:

[0152]

[0153]

[0154] This represents the final dynamic classification system; This represents the final synthesized dataset; This represents the target model obtained after final training. The purpose of this step is to ensure that the entire dynamic evolution process has a convergence endpoint, ensuring that the system stops iterating after reaching the expected performance and outputs a final result that can be directly applied.

[0155] Step Eleven: Real-world case demonstration: After multiple iterations, the system found that the accuracy of all major nodes was higher than 0.80, and no new sub-nodes were generated in the last two rounds. Therefore, the iteration stopped, and the three outputs are as follows: A more detailed risk classification system that more closely reflects the real-world shortcomings of the model; A set of high-quality training data that is progressively reinforced around weak points; A large-scale financial risk control model that performs more stably in bank risk identification tasks.

[0156] With the continuous development of large-scale models, generative artificial intelligence, automated data construction, and continuous training optimization technologies, synthetic data systems are increasingly widely used in scenarios such as pre-training data augmentation, supervised fine-tuning, instruction data construction, domain adaptation, low-resource task reinforcement, and agent training. When facing complex task data construction scenarios, the system not only needs to generate a sufficient number of training samples based on the target task, but also needs to balance data quality, reinforcement relevance, generation efficiency, and engineering deployment feasibility under conditions such as complex task hierarchies, uneven distribution of capability weaknesses, scarcity of long-tail nodes, insufficient real data, limited generation budgets, and continuously changing training states. However, existing synthetic data construction schemes typically employ static task classification, fixed sample templates, and uniform budget generation methods. This easily leads to the system repeatedly investing significant generation resources in nodes already mastered by the model, while lacking sufficient data reinforcement depth for truly weak and complex nodes. This affects the system's ability improvement and application value in real training environments.

[0157] To address the aforementioned issues, this invention proposes a "method, system, device, and storage medium for adaptive evolution of dynamic classification systems and generation of synthetic data based on model feedback." By constructing a closed-loop mechanism encompassing "initial classification system establishment—classification node sample generation—sample-node mapping—target model training—node-level performance feedback extraction—weak node identification—dynamic evolution of the classification system—incremental synthetic data generation—iterational stop determination," this approach achieves a combined consideration of differences in capability structure, differences in node mastery, evolutionary requirements for classification granularity, and resource allocation requirements for generation. This improves the effectiveness of strengthening weak capabilities and reduces the generation of invalid data.

[0158] Regarding the question of whether there are other alternative solutions that can also achieve the invention's objective, the following can be stated: Without deviating from the invention's objective of "improving the ability of synthetic data to compensate for the shortcomings of the model's real-world capabilities and reducing the overhead of constructing invalid or duplicate data," several alternative technical routes do exist. However, these alternative solutions typically only cover a portion of the invention's objective, or have significant limitations in terms of task-level expressiveness, node-level feedback perception, dynamic evolution of the classification system, accuracy of resource allocation, and stability of continuous training. Existing alternative solutions mainly include the following categories.

[0159] (1) Alternatives based on larger generative models or stronger prompts. Some existing solutions improve the language quality, logical integrity, and task coverage of synthetic data by using generative models with larger parameter scales, more complex prompt word designs, longer contextual inputs, or more refined example construction methods. For example, by adopting a more powerful large model, introducing more few-shot examples, adding task background descriptions, designing more complex role prompts, or optimizing the generation process through multi-round prompts, the system can have a stronger overall generation capability. The advantage of this type of solution is that the implementation path is relatively direct. It usually does not require explicit design of node-level feedback extraction and classification system evolution mechanisms. It only needs to improve sample quality and a certain degree of task adaptability by enhancing model capabilities. However, this type of approach also has significant drawbacks: First, stronger generative models typically generate data with approximately uniform intensity across the entire task space, lacking the ability to specifically reinforce weak nodes. Second, the model inference and sample generation costs increase significantly, making them unsuitable for deployment in continuous training, large-scale data construction, and budget-sensitive scenarios. Third, even with improved overall generation quality, insufficient reinforcement may still occur when dealing with long-tail nodes, nodes with ambiguous boundaries, and complex sub-tasks due to the lack of explicit node feedback awareness and dynamic adjustment mechanisms for the classification system. Therefore, while this type of approach can improve data quality to some extent, it is difficult to achieve the comprehensive goal of "focused reinforcement of key weak nodes + overall optimization of generation resources" pursued by this invention.

[0160] (2) Data organization alternatives based on static hierarchical knowledge trees or fixed capability graphs. Another type of existing solution uses static knowledge trees, capability trees, task graphs, or manually defined hierarchical labeling systems to organize the target task in layers and generate training samples under each level node. This type of method can usually better balance the semantic scope of the task level and the capability boundaries of the sub-task level, and has certain effects in professional domain question answering, rule-based tasks, and complex knowledge construction scenarios. Its advantages are that the task structure is relatively clear, the logical relationship between nodes is relatively stable, and the system implementation method is relatively mature, making it easy to combine with existing template generation frameworks. However, its limitations are as follows: First, static hierarchical structures typically employ fixed classification paths and fixed granularity designs, lacking the ability to dynamically adjust classification boundaries based on model training results and node mastery. Second, while hierarchical structures can cover task organization at multiple scales, they primarily emphasize "how tasks are broken down," lacking explicit modeling of "which nodes need reinforcement in the current training phase." Third, when the classification system is used statically, complex and simple nodes often still share similar generation budgets, resulting in insufficiently refined data allocation and difficulty in "focusing on generating only truly weak nodes." Therefore, such schemes can improve task organization capabilities to some extent, but they cannot simultaneously achieve the dynamic classification system evolution and node-level generation resource optimization goals required by this invention.

[0161] (3) Data resampling alternatives based on overall validation error or sample-level hard example mining. Some existing technologies introduce validation set error analysis, hard example mining, sample-level loss ranking, sample resampling, or active learning mechanisms to determine whether to add a certain type of sample based on the overall performance of the model during the training or validation phase. For example, when the model's overall accuracy decreases on a certain type of problem or the confidence level of the output for some samples is low, the system can resample similar samples or add more similar training data to reduce the overall error. The advantage of this type of approach is that it can more easily reduce the omission of some explicit hard examples and improve the average training effect. However, its main shortcomings are as follows: First, most of these schemes assess the difficulty of the entire sample set or a single sample, determining "which samples need more training," rather than differentiating the classification nodes and capability levels within the task. Second, when a task contains both high-frequency simple nodes and a small number of complex long-tail nodes, sample-level resampling often fails to accurately reflect the true weak areas in the capability structure, easily leading to insufficient identification of local key nodes. Third, sample-level data adjustment lacks the ability to be combined with the dynamic evolution of the classification system, failing to achieve the "node-level structural reconstruction + node-level incremental reinforcement" goal emphasized in this invention. Therefore, while these schemes can improve training results to some extent, they cannot achieve the comprehensive effect of this invention in terms of node-level capability reinforcement and adaptive structural adjustment.

[0162] (4) Alternative solutions based on template expansion, question rewriting, or instruction variant generation. Another alternative approach involves constructing more training samples based on existing data through template expansion, question rewriting, answer reconstruction, expression variant generation, or semantic perturbation to improve data diversity and surface coverage. For example, multiple rounds of rewriting of existing questions, style transformation of existing answers, or expansion of various expression forms under fixed nodes through generative models. The advantage of this approach is that it can directly increase the number of samples and enhance the model's adaptability to different expression methods to a certain extent. However, its limitations are also quite prominent: First, the main goal of existing question rewriting or instruction expansion methods is usually to enhance sample diversity, rather than to refine the classification system and reconstruct capabilities at the weak nodes, thus providing limited support for complex fine-grained capability enhancement; Second, if the rewriting direction mainly revolves around existing high-frequency tasks, it may further amplify the sample size of the already mastered area, rather than accurately supplementing the real long-tail nodes; Third, this type of approach usually lacks a unified node-level feedback perception and classification system evolution mechanism, making it difficult to achieve gradual enhancement from global task organization to local weak nodes. Therefore, this type of approach can be seen as a way to increase data richness, but it usually leans more towards "expanding samples" rather than "dynamically evolving around weak nodes and focusing on strengthening them" as achieved in this invention.

[0163] (5) Data construction alternatives based on multi-round bootstrapping, recursive generation, or iterative self-enhancement. Some existing technologies may also employ multi-round bootstrapping generation, shared hint recursive expansion, model self-instruction growth, or iterative automatic data augmentation mechanisms to update existing sample sets or task sets in multiple rounds to improve the model's adaptability to complex tasks. The advantage of this type of approach is that it can gradually expand training data through multiple rounds of generation, making it more suitable for handling complex task expansion, low-resource scenario supplementation, and continuous training problems compared to single static generation. However, this type of approach still has the following shortcomings: First, many recursive or iterative generation methods operate on a unified task set or a unified sample pool, lacking explicit modeling of the differences in the classification nodes within the task, which easily leads to repeated expansion of simple and difficult nodes indiscriminately; Second, the recursive generation process is usually not deeply coordinated with the dynamic classification system, lacking a mechanism of "classification system evolution supporting data growth"; Third, when the generation rounds are fixed, simple nodes will generate obvious redundant data, while extremely difficult nodes may still be insufficiently reinforced. Therefore, although such solutions can enhance the data supply capability during continuous training to some extent, they are difficult to achieve the comprehensive technical effect of "node feedback-driven + classification system adaptive evolution + differentiated incremental reinforcement" emphasized in this invention.

[0164] (6) Two-stage alternative based on explicit task splitting and local append construction. In some application scenarios, a two-stage approach can also be adopted, in which tasks are first split coarsely by humans or rule-based systems, and then data is added separately for low-performance sub-tasks. For example, the system first establishes a fixed task directory, completes the first round of data generation and model training, and then selects several poorly performing sub-tasks based on overall error analysis or human judgment, and supplements samples separately for these sub-tasks. The advantage of this approach is that the implementation logic is relatively clear, and the generation budget can be concentrated on key task areas to a certain extent, avoiding high-intensity repeated generation of the entire task space. However, its shortcomings are: firstly, the two-stage process usually relies on the manual task division in the previous stage and subsequent manual judgment. If the initial classification design is too coarse or the error attribution is inaccurate, it is difficult to fix it later; secondly, the stages are often serial splitting rather than unified closed-loop optimization, lacking continuous coordination between sample node mapping, node-level feedback perception and dynamic evolution of the classification system; thirdly, for scenarios with ambiguous boundaries, complex error patterns and large differences in fine-grained capabilities, a single local append construction may still be insufficient to fully characterize and make up for the shortcomings of the model. Therefore, this type of solution is close to the idea of ​​"focusing on analyzing key weak points" in this invention to a certain extent, but it usually cannot achieve the unified dynamic classification evolution closed-loop optimization effect achieved by this invention.

[0165] In summary, although existing technologies offer alternative approaches such as strengthening generative models, static hierarchical organization, sample-level resampling, template expansion and expression variant generation, multi-round bootstrapping recursive generation, and two-stage local append construction, which can alleviate data shortages or efficiency issues in complex training scenarios to some extent, these solutions generally suffer from shortcomings such as "lack of explicit quantification of differences in node-level mastery," "lack of deep collaboration between the classification system and the generation process," "insufficiently fine-grained allocation of generation resources," "difficulty in achieving a unified balance between strengthening weak nodes and overall generation efficiency," and "still relying on additional manual adjustments, fixed processes, or high budgets during engineering deployment." They often only address a portion of the objectives of this invention or heavily rely on model expansion, static design, and subsequent manual remediation.

[0166] In contrast, this invention employs a collaborative mechanism of "hierarchical classification system modeling + node-level model feedback perception + feedback-driven adaptive evolution of the classification system + evolution-driven incremental synthetic data generation + iterative stopping and closed-loop optimization." This mechanism not only fully utilizes global task semantics and local fine-grained capability boundary information at different levels on the task organization side, but also identifies and focuses on truly weak key nodes on the generation side, allocating more data construction budget to the parts that need to be strengthened. Simultaneously, the iterative stopping mechanism suppresses redundant expansion of already mastered nodes. Therefore, this invention effectively maintains overall data construction efficiency, engineering feasibility, and long-term evolutionary stability while improving model capability enhancement. It combines node targeting, controllable generation process, and system integrability, achieving comprehensive technical effects that are difficult to achieve simultaneously with existing alternatives, demonstrating significant innovation and practical value.

[0167] refer to Figure 3 Another specific embodiment of the present invention discloses a synthetic data generation system, comprising: a synthetic sample generation module 301, used to generate an initial synthetic dataset based on an initial classification system; a target model training and validation module 302, used to train a target model using the initial synthetic dataset to obtain a trained round-t model, and then test the trained round-t model using a validation sample set, where t is a positive integer; a node-level model feedback module 303, used to map validation samples to classification nodes to obtain node-level feedback, wherein the node-level feedback includes the classification node to which the validation sample belongs and the node-level accuracy; and a weak node identification module 304, used to identify weak classification nodes based on the comparison result between the node-level accuracy and the node-level accuracy threshold; and dynamic... The classification system adaptive evolution module 305 is used to dynamically evolve the initial classification system or the current classification system for the set of weak classification nodes to obtain an updated classification system; the incremental synthetic data generation module 306 is used to generate an incremental synthetic dataset based on the updated classification system, and retrain the t-th round model based on the cumulative training dataset to obtain the t+1-th round model, wherein the cumulative training dataset is the union of the current synthetic dataset and the incremental synthetic dataset; and the iterative update and stopping module 307 is used to repeatedly execute the node-level feedback step, the weak classification node identification step, the classification system dynamic evolution step, and the incremental synthetic dataset generation step until a preset stopping condition is met, and then output the final classification system, the final training dataset, and the final model.

[0168] This invention belongs to the fields of artificial intelligence, large model training, automated data construction, machine learning, generative artificial intelligence, knowledge organization and task modeling technology, specifically involving task-level modeling, node-level model feedback extraction, adaptive evolution of classification systems, synthetic data generation and scheduling, continuous training reinforcement and closed-loop data optimization.

[0169] This invention is particularly applicable to synthetic data construction scenarios under conditions of complex tasks, long-tail capabilities, low-resource scenarios, professional domain training, and continuous learning. Without relying on simply expanding the scale of the generative model or blindly increasing the number of samples, it constructs a closed-loop data generation mechanism based on the initial classification system of the target task, the mapping relationship between samples and classification nodes, the performance feedback of the model at each node, and the dynamic evolution mechanism of the classification system. This mechanism consists of "initial classification system establishment—classification node sample generation—sample and node mapping—target model training—node-level performance feedback extraction—weak node identification—dynamic evolution of the classification system—incremental synthetic data generation—iteration stop determination." This improves the reinforcement effect of key weak capability nodes, effectively reduces redundant sample construction in the already mastered areas, and enhances the system's relevance, controllability, interpretability, and engineering deployment adaptability in complex real-world training scenarios.

[0170] This invention can be further categorized into the following sub-technical directions: first, hierarchical capability structure modeling technology for complex task spaces; second, node-level model feedback perception technology for continuous training optimization; third, adaptive evolution technology of classification system for dynamic adjustment of task structure; fourth, dynamic allocation technology of data generation resources for model capability enhancement; and fifth, closed-loop data construction system implementation technology for large model training, domain adaptation, and continuous learning.

[0171] The synthetic data generation system according to embodiments of the present invention will be described in detail below with specific examples.

[0172] (1) Initial Task Analysis and Classification System Establishment Module. This module is responsible for obtaining the target task description and performing preliminary analysis and structured representation of the target task. This module maps the original task objectives, domain scope, capability requirements, sample format, and output requirements into an organizeable initial classification system, which is used to form the first-level structural representation of the target task space. This module is the starting point of the entire data generation process, and its role is to provide the basic task skeleton for subsequent sample generation, model feedback mapping, and classification system evolution.

[0173] (2) Classification Node Sample Generation Template Construction Module. This module is responsible for building corresponding sample generation templates for different nodes in the initial classification system. This module can generate suitable template descriptions based on node semantics, capability boundaries, output format, and task type, enabling different nodes to correspond to different types of training sample generation strategies. The role of this module is to transform abstract classification nodes into executable data generation units, and it is a key component in realizing "organizing sample generation by node".

[0174] (3) Synthetic Sample Generation Module. This module is responsible for calling the generative model, teacher model, or sample construction engine to generate candidate synthetic samples based on the classification nodes and their sample generation templates. This module can output question-answer pairs, instruction-response pairs, case analysis samples, classification judgment samples, inference samples, or other forms of training data. The role of this module is to transform the capability requirements corresponding to each node in the classification system into specific and usable training data. It is the core entry point for the entire system to achieve automated data supply.

[0175] (4) Sample Consistency Check and Quality Screening Module. This module is responsible for performing consistency checks, semantic correctness checks, input-output matching checks, and format standardization checks on the generated candidate synthetic samples, and filtering out invalid samples that do not meet the requirements. This module can be implemented in conjunction with rule validation, model validation, or multi-round review. The core function of this module is to ensure that the samples entering the training set have high consistency in node affiliation, semantic expression, and label structure, thereby providing a reliable foundation for subsequent training and node feedback analysis.

[0176] (5) Sample-Classification Node Mapping Module. This module is responsible for establishing a mapping relationship between the selected valid samples and the nodes of the classification system. Preferably, this module can record the first-level node, second-level node, leaf node, and their complete path of the sample, so that the output results after subsequent model training can be reprojected into the classification system. The role of this module is to realize the connection from "sample-level data" to "node-level capability feedback," which is the key foundation for the dynamic evolution of the classification system in this invention.

[0177] (6) Target Model Training and Validation Module. This module is responsible for training, fine-tuning, or incrementally updating the target model using the currently available synthetic dataset, and obtaining the performance results of the target model on different task nodes through validation samples, test samples, or evaluation samples. The role of this module is to truly transform synthetic data into changes in model capabilities and provide raw performance data for subsequent node-level feedback extraction. It is the training execution unit that connects the preceding and following steps in the entire closed-loop process.

[0178] (7) Node-level model feedback extraction module. This module is responsible for statistically analyzing the performance metrics of the target model at different classification nodes based on the validation results. Preferably, this module can combine sample accuracy, error rate, recall rate, rejection rate, confusion rate, and complexity-related performance to quantitatively model the mastery level of each node. The core function of this module is to identify which nodes have been stably mastered by the model and which nodes remain weak points, thereby achieving a shift from overall training effect evaluation to node-level capability perception.

[0179] (8) Weak Node Identification Module. This module is responsible for identifying weak nodes in the current classification system based on node-level model feedback results. It can determine which nodes need priority reinforcement based on node-level accuracy thresholds, error rate thresholds, gain change rate, or other performance indicators. The role of this module is to further transform model training feedback into a direct target for subsequent classification system evolution and incremental data generation; it is a key judgment unit for achieving "organizing data around model weaknesses."

[0180] (9) Adaptive Evolution Module for Dynamic Classification System. This module is responsible for dynamically evolving the classification system for weak nodes. Preferably, this module can perform operations such as refinement expansion, boundary reconstruction, addition of child nodes, node reorganization, or node level drill-down on weak nodes, allowing the original static classification system to be updated round by round with model feedback. This module is one of the core modules of this invention. Its function is to make the classification system no longer fixed, but dynamically evolve with the model's capabilities, thereby achieving a closed-loop optimization process where "the classification system is refined wherever the model is weak."

[0181] (10) Incremental Synthetic Data Generation Module. This module is responsible for reorganizing the generation targets based on the evolved classification system and generating a new round of synthetic samples around newly added nodes or refined weak nodes. This module can increase the generation budget, enhance sample density, or increase sample difficulty for key weak nodes, thereby forming more targeted incremental training data. The role of this module is to truly transform the evolution results of the classification system into new data reinforcement capabilities, and it is an important computational unit for achieving dynamic data enhancement.

[0182] (11) Cumulative Dataset Maintenance and Iterative Update Module. This module is responsible for incorporating newly generated and filtered incremental samples into the existing training dataset to form an updated cumulative training dataset, and maintaining the distribution of data from different rounds, different nodes, and different capability regions. This module ensures that the system can continuously accumulate effective samples in multiple iterations, while avoiding disorganization and structural instability of the training data.

[0183] (12) Iteration Stop Determination Module. This module dynamically determines whether to continue subsequent classification system evolution and data generation during the closed-loop iteration process. It can determine whether the system has reached the preset data construction goal based on whether node performance reaches a threshold, whether the classification system tends to stabilize, whether the gains over multiple rounds are sufficiently significant, or other stopping conditions. When the stopping conditions are met, the system can terminate subsequent iterations early. This module is used to reduce invalid data generation, control overall construction costs, and improve the manageability and deployment feasibility of the system in practical engineering.

[0184] (13) Training parameter scheduling and generation budget control module. This module is responsible for dynamically configuring and adjusting parameters such as node threshold, incremental generation budget, node expansion granularity, upper limit of iteration count, sample selection threshold, and stopping decision threshold during system operation. This module ensures that the present invention can be flexibly configured according to different model capabilities, different training stages, different domain task complexity, and different computing power budget conditions, so as to achieve better data generation resource utilization efficiency while ensuring the capability enhancement effect.

[0185] (14) Output Evaluation and System Monitoring Module. This module outputs key indicators of the synthetic data generation process and results, including node-level accuracy, node-level error rate, changes in the number of weak nodes, number of classification system evolutions, average node expansion granularity, incremental sample utilization rate, proportion of duplicate samples, cumulative sample size, improvement in training performance, and changes in generation costs. These indicators are used to evaluate the quality of system data reinforcement, the effectiveness of dynamic classification system evolution, and engineering deployment performance. This module provides interpretable data for model optimization, parameter configuration, and system diagnosis in practical applications.

[0186] Compared with existing technologies, the innovative aspects of this application are as follows: (1) Dynamic matching between classification system and real capability shortcomings. This invention does not simply establish a static classification system, but rather introduces node-level feedback and an adaptive evolution mechanism of the classification system based on the model's performance in the real training process. This allows the classification structure to gradually approach the model's real weak areas, thereby improving the matching degree between the task structure and the model's capabilities.

[0187] (2) Node-level feedback explicit modeling and priority of weak nodes. In view of the problem that existing methods generally lack the ability to quantify the mastery of the model at different capability nodes, this invention introduces a node-level model feedback extraction mechanism, which enables the system to explicitly distinguish between mastered nodes and high-risk weak nodes, thereby upgrading the data construction process from "overall average expansion" to "prioritizing the reinforcement of key weak nodes".

[0188] (3) Feedback-driven adaptive evolution of the classification system. This invention does not passively generate data under a fixed task directory, but dynamically refines, expands and adjusts the classification system based on training feedback, so that the classification structure is upgraded from a static preset to a dynamic structure that continuously evolves to address the shortcomings of the model's capabilities. This mechanism enables the system to maintain continuous task discrimination and node representation capabilities during long-term training and adaptation to complex domains.

[0189] (4) Two-way synergy between classification system evolution and incremental data generation. After the classification system evolves, this invention does not simply add more general samples, but regenerates more granular and targeted incremental data around the evolved nodes, and uses the new data to enhance the training support of weak nodes, forming a two-way closed-loop synergy between classification system evolution and data generation.

[0190] (5) Unified control of node-level reinforcement and overall generation efficiency. This invention integrates node feedback identification, classification system evolution, incremental sample generation and stopping determination into a unified closed loop, realizing full-process quality control and budget control of the data construction process, so that the system can "reinforce more accurately" on weak nodes and "expand less" on mastered nodes, thereby taking into account both the ability improvement effect and the overall generation efficiency.

[0191] (6) Low-intrusive and integrable engineering implementation. The core mechanism of this invention mainly functions in task organization, feedback perception, structural evolution, and incremental generation scheduling. It is compatible with existing pre-training systems, supervised fine-tuning platforms, automatic data construction systems, prompt generation systems, and sample screening systems. It does not require complete reconstruction of the original data link and is easy to integrate and deploy quickly in large model training platforms, industry knowledge systems, intelligent agent training engines, and continuous learning systems.

[0192] The application prospects of this application are as follows: This invention can be widely applied to various fields such as large model pre-training, instruction fine-tuning, domain question answering construction, complex reasoning task reinforcement, low-resource task adaptation, continuous learning and agent training. It is suitable for data construction tasks such as long-tail knowledge reinforcement, complex node reinforcement, task boundary refinement, error pattern correction, and multi-round data optimization.

[0193] By employing hierarchical classification system modeling, node-level model feedback perception, dynamic classification system adaptive evolution, incremental synthetic data generation, and closed-loop stopping control mechanisms, this invention can significantly reduce the generation of redundant samples and the overall data construction cost in the mastered area while improving the reinforcement effect of key weak capability nodes and the adaptability to complex tasks. This reduces the reliance on ultra-large-scale static template systems and large-scale manual data completion processes, thereby improving the practicality and economy of synthetic data systems in real training environments.

[0194] This invention is particularly applicable to the following typical application scenarios: First, continuous reinforcement of fine-grained sub-tasks such as liquidity risk identification, compliance interpretation, and complex product analysis in the training of large financial models; Second, targeted data generation for tasks such as boundary clause interpretation, complex rule application, and fine-grained case classification in legal and government Q&A systems; Third, incremental data supplementation for low-frequency abnormal patterns, complex diagnostic nodes, and high-risk misjudgment nodes in medical and industrial models; Fourth, continuous training support for long-tail problems, complex failure modes, and high-risk decision nodes in cybersecurity, industrial control, and intelligent operation and maintenance scenarios; Fifth, structured data supply for task chain nodes, decision nodes, and complex action patterns in intelligent agent, multi-step reasoning, and continuous learning systems.

[0195] Based on the "model feedback-driven dynamic classification system evolution + incremental synthetic data generation" mechanism constructed by this invention, the system can better meet the comprehensive requirements of real training environment for "more accurate reinforcement, more economical generation, evolvable structure, and controllable process". It has broad prospects for implementation and engineering application value in fields such as large model training automation, intelligent data engineering, domain knowledge systems and continuous learning intelligence.

[0196] Another specific embodiment of the present invention discloses a synthetic data generation device, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to execute the method described in the above embodiments.

[0197] Another specific embodiment of the present invention discloses a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the method described in the embodiment.

[0198] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0199] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A synthetic data generation method, characterized by, include: An initial synthetic dataset is generated based on the initial classification system; The target model is trained using the initial synthetic dataset to obtain the trained model for round t. Then, the trained model for round t is tested using the validation sample set, where t is a positive integer. The validation samples are mapped to classification nodes to obtain node-level feedback, wherein the node-level feedback includes the classification node to which the validation sample belongs and the node-level accuracy. Weak classification nodes are identified based on the comparison between the node-level accuracy and the node-level accuracy threshold. For the set of weak classification nodes, the initial classification system or the current classification system is dynamically evolved to obtain an updated classification system; An incremental synthetic dataset is generated based on the updated classification system, and the model at round t is retrained based on the cumulative training dataset to obtain the model at round t+1, wherein the cumulative training dataset is the union of the current synthetic dataset and the incremental synthetic dataset; and Repeat the node-level feedback step, weak classification node identification step, classification system dynamic evolution step, and incremental synthetic dataset generation step until the preset stopping condition is met, and then output the final classification system, final training dataset, and final model.

2. The synthetic data generation method of claim 1, wherein, Mapping validation samples to classification nodes to obtain node-level feedback further includes: Define the mapping function from the verification sample to the classification node; For any given classification node, obtain the corresponding validation sample set; The node-level accuracy rate is calculated based on the number of correct and incorrect answers at any given classification node, wherein the node-level accuracy rate is the ratio between the number of correct answers at any given classification node and the total number of samples, and the total number of samples is the sum of the number of correct and incorrect answers at any given classification node; and All classification nodes and their corresponding node-level accuracy are used as node-level feedback results.

3. The synthetic data generation method of claim 2, wherein, Identifying weak classification nodes based on the comparison between the node-level accuracy and a preset node-level accuracy threshold further includes: Pre-set node-level accuracy thresholds; The node-level accuracy of the current classification node among all classification nodes is compared with the node-level accuracy threshold, wherein, When the node-level accuracy of the current classification node is less than the node-level accuracy threshold, the current classification node is identified as a weak classification node; otherwise, the current classification node is identified as a non-weak classification node.

4. The synthetic data generation method of claim 2, wherein, For the set of weak classification nodes, dynamically evolving the initial or current classification system to obtain an updated classification system further includes: The weak classification node is dynamically refined and expanded to obtain the set of expanded child nodes of the weak classification node, so that the granularity is automatically adjusted as node-level feedback occurs; The updated classification system is obtained by replacing the weak classification nodes in all classification nodes with an expanded set of child nodes.

5. The method for generating synthetic data according to claim 2, characterized in that, The generation of incremental synthetic dataset based on the updated classification system further includes: generating an updated candidate sample set for the expanded child node set in the updated classification system; performing consistency screening on the updated candidate sample set to obtain the incremental synthetic dataset; and merging the incremental synthetic dataset into the current synthetic dataset as the cumulative training dataset. The preset stopping conditions include: no weak classification nodes are identified or the classification system no longer changes after two consecutive rounds.

6. The synthetic data generation method of claim 5, wherein, The generation of the initial synthetic dataset based on the initial classification system further includes: An initial classification system is established based on the target task description received from the client; A candidate sample dataset is generated based on the initial classification system; A consistency screening is performed on each candidate sample in the candidate sample dataset to obtain the initial synthetic dataset based on the consistency screening.

7. The synthetic data generation method of claim 5, wherein, The consistency screening includes: A consistency score is calculated for each candidate sample, wherein the consistency score is a weighted sum of the consistency score between the candidate sample and the target node, the semantic matching score between the input and output, and the logical consistency score of the candidate sample content; When the consistency score is greater than or equal to the sample screening threshold, the candidate sample is retained.

8. A synthetic data generation system, characterized in that, include: The synthetic sample generation module is used to generate an initial synthetic dataset based on the initial classification system; The target model training and validation module is used to train the target model using the initial synthetic dataset to obtain the trained round t model, and then use the validation sample set to test the trained round t model, where t is a positive integer; A node-level model feedback module is used to map validation samples to classification nodes to obtain node-level feedback, wherein the node-level feedback includes the classification node to which the validation sample belongs and the node-level accuracy. The weak node identification module is used to identify weak classification nodes based on the comparison result between the node-level accuracy and the node-level accuracy threshold. The dynamic classification system adaptive evolution module is used to dynamically evolve the initial classification system or the current classification system for the set of weak classification nodes to obtain an updated classification system; An incremental synthetic data generation module is used to generate an incremental synthetic dataset based on the updated classification system, and to retrain the t-th round model based on the cumulative training dataset to obtain the t+1-th round model, wherein the cumulative training dataset is the union of the current synthetic dataset and the incremental synthetic dataset; and The iterative update and stopping module is used to repeatedly execute the node-level feedback step, the weak classification node identification step, the classification system dynamic evolution step, and the incremental synthetic dataset generation step until the preset stopping condition is met, and then output the final classification system, the final training dataset, and the final model.

9. A synthetic data generation device, characterized in that, include: One or more processors; Memory used to store executable instructions; The one or more processors are configured to invoke executable instructions stored in the memory to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon computer program instructions, wherein, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 1.

Images