Synthetic data generation method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention discloses an agent-oriented synthetic data generation method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining an initial sample data source and data synthesis pipeline configuration information of a synthetic data generation task, the assembly line configuration information comprises at least one atomic execution step arranged according to an execution dependency relationship, taking an initial sample data source of the synthetic data generation task as a reference, scheduling and executing each atomic execution step based on the execution dependency relationship to perform data synthesis processing to obtain multiple pieces of target synthetic data, a target synthetic data set carrying metadata labels is output, the target synthetic data set is used for large model training and / or large model evaluation of a target agent corresponding to the synthetic data generation task, and when data synthesis processing of the atomic execution step is executed, current task configuration context information is obtained and used for processing the target agent according to the current task configuration context information. And taking the task configuration context information as a metadata label and writing the task configuration context information into each piece of target synthetic data.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a method, apparatus, storage medium and electronic device for generating synthetic data for intelligent agents. Background Technology

[0002] With the deepening application of artificial intelligence (AI) technology, industry transaction processing agents driven by large language models (LLM) for vertical fields (such as those for government affairs processing, financial investment consulting, and travel planning) are becoming the core form of AI industry implementation. To improve the intent understanding and task planning capabilities of these LLM-driven agents in specific service scenarios, it is typically necessary to construct large-scale, vertical industry scenario data covering complex business logic for targeted supervised fine-tuning (SFT) training or capability evaluation. Since real-world industry data often involves privacy-sensitive restrictions or insufficient coverage of long-tail scenarios, synthetic data generation technology based on large models has become a key means to fill data gaps and ensure agent performance. Summary of the Invention

[0003] This specification provides a method, apparatus, storage medium, and electronic device for generating synthetic data for intelligent agents, the technical solutions of which are as follows: In a first aspect, embodiments of this specification provide a method for generating synthetic data for intelligent agents, the method comprising: Obtain the initial sample data source and data synthesis pipeline configuration information for the synthetic data generation task, wherein the pipeline configuration information includes at least one atomic execution step arranged according to execution dependencies; Taking the initial sample data source of the synthetic data generation task as a reference, the atomic execution steps are scheduled and executed based on the execution dependency to perform data synthesis processing to obtain multiple target synthetic data, and output the target synthetic dataset carrying metadata tags. The target synthetic dataset is used for large model training and / or large model evaluation of the target agent corresponding to the synthetic data generation task. Specifically, during the data synthesis process of the atomic execution step, the current task configuration context information is obtained, and the task configuration context information is written as a metadata tag into each piece of target synthesis data.

[0004] Secondly, embodiments of this specification provide a synthetic data generation apparatus for intelligent agents, the apparatus comprising: The information acquisition module is used to acquire the initial sample data source and data synthesis pipeline configuration information of the synthetic data generation task. The pipeline configuration information includes at least one atomic execution step arranged according to execution dependencies. The data synthesis module is used to take the initial sample data source of the synthetic data generation task as a reference, schedule and execute each of the atomic execution steps based on the execution dependency to perform data synthesis processing to obtain multiple target synthetic data, and output the target synthetic dataset carrying metadata tags. The target synthetic dataset is used for large model training and / or large model evaluation of the target agent corresponding to the synthetic data generation task. Specifically, during the data synthesis process of the atomic execution step, the current task configuration context information is obtained, and the task configuration context information is written as a metadata tag into each piece of target synthesis data.

[0005] Thirdly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the above-described method steps.

[0006] Fourthly, embodiments of this specification provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.

[0007] Fifthly, embodiments of this specification provide a computer program product that stores at least one instruction, which is loaded by the processor and executed as described above in the method steps.

[0008] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following: In one or more embodiments of this specification, a modular pipeline scheduling mechanism based on execution dependencies is constructed. During the data synthesis process of atomic execution steps, the current task configuration context information is captured in real time, and the context information is structured and written into each target synthesized data as metadata tags. This solves the limitations of non-standardized workflows, difficulty in large-scale reuse and collaboration caused by discrete script processing in data synthesis. It realizes automated orchestration of data synthesis tasks, improving the efficiency and flexibility of large-scale data production and development for intelligent agents. Furthermore, a rigorous data traceability system is established through metadata tags, ensuring that each synthesized data carries complete production traceability information, achieving full-process traceability and experimental reproducibility. Finally, it guarantees high-quality synthesized data output, reduces labor costs, and improves the engineering collaboration capabilities of data synthesis. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating a synthetic data generation method for intelligent agents provided in the embodiments of this specification; Figure 2 This is a schematic diagram of a scenario architecture for a data synthesis framework provided in the embodiments of this specification; Figure 3 This is a schematic diagram of a data synthesis processing flow provided in the embodiments of this specification; Figure 4 This is a flowchart illustrating a data synthesis processing step based on a large model, as provided in the embodiments of this specification. Figure 5 This is a schematic diagram of a data governance process provided in the embodiments of this specification; Figure 6 This is a schematic diagram of a data evaluation process provided in the embodiments of this specification; Figure 7 This is a schematic diagram of a data synthesis pipeline configuration information determination process provided in the embodiments of this specification; Figure 8 This is a schematic diagram illustrating data flow and asset synchronization under a lightweight Git collaboration mode, as provided in the embodiments of this specification. Figure 9 This is a schematic diagram of the structure of a synthetic data generation device provided in the embodiments of this specification; Figure 10 This is a schematic diagram of the structure of an electronic device provided in the embodiments of this specification. Detailed Implementation

[0011] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0012] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0013] In related technologies, existing data synthesis methods mostly rely on a discrete pattern of manually written prompts combined with temporary scripts. When facing the high standards required for training and evaluating industry-specific intelligent agents, this non-standardized operating mode has shortcomings: on the one hand, scripted processing logic is difficult to reuse and coordinate, making it difficult to support the construction of complex industry scenario data across large scales and multiple teams; on the other hand, existing synthesis processes generally lack a rigorous end-to-end traceability mechanism. In the scenario of evaluating industry transaction processing intelligent agents, due to the lack of a strong correlation mapping between data products and generated configurations and model versions, when intelligent agent evaluation indicators show abnormalities, technicians find it difficult to distinguish whether the problem stems from inherent limitations in the intelligent agent's capabilities or from distributional drift in the synthesized data. This leads to difficulties in attribution analysis and severely restricts the iteration efficiency and deployment reliability of industry-specific intelligent agents.

[0014] The present specification will now be described in detail with reference to specific embodiments.

[0015] In one embodiment, such as Figure 1As shown, a synthetic data generation method for intelligent agents is proposed. This method can be implemented using a computer program and can run on an intelligent agent-oriented synthetic data generation device based on the von Neumann architecture. This computer program can be integrated into applications or run as a standalone tool application. The intelligent agent-oriented synthetic data generation device can be an electronic device, including but not limited to: personal computers, tablets, handheld devices, server devices, computing devices, or other processing devices connected to a wireless modem. In different networks, terminal devices can be called by different names, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user equipment, cellular phone, cordless phone, device in 5G network or future evolved network, etc.

[0016] Specifically, this agent-oriented synthetic data generation method includes: The following is an explanation of the intelligent agents addressed in this specification: An intelligent agent, also known as an agent service object (or GVA), is a computer program that relies on electronic devices to perceive, reason, and make decisions. The intelligent agent service object is the service carrier of the intelligent agent. It relies on a transaction processing model configured for a specific transaction processing scenario to drive the intelligent agent service object to provide users with task processing functions for transaction tasks. Transaction processing can be user tasks in transaction scenarios such as application transactions, web automation transactions, robotic process automation transactions, and intelligent virtual assistant transactions. In this specification, the intelligent agent service object is the user-oriented object generated by the synthetic data generation method to produce the target synthetic dataset. The target synthetic dataset is used for large-scale model training and / or large-scale model evaluation of the target intelligent agent (an intelligent agent service object) corresponding to the synthetic data generation task. For example, an intelligent agent service object can be a service control that carries the intelligent agent service object. Users can trigger the service control corresponding to the intelligent agent service object, and input the target task for the target transaction in the interface of the intelligent agent service object corresponding to the service control. Based on the intelligent agent service object, the transaction task input by the user is implemented. The intelligent agent service object autonomously interacts with the target transaction service indicated by the transaction task and navigates the task, thereby completing the autonomous processing of the transaction task and saving the user the operation path to complete the target task, thus improving the task processing efficiency. Intelligent agents can be applied to various scenarios such as AI dialogue, robotics, virtual reality, games, automatic control, information retrieval, recommendation systems, and natural language processing.

[0017] In related technologies, the large language model upon which the intelligent agent service object (hereinafter referred to as the intelligent agent) relies can be a multimodal large language model adapted to transaction scenarios using a basic large language model. Basic large language models include the Tongyi Qianwen large model, the GPT series of large models, and the DeepSeek series of large models. Alternatively, the intelligent agent can be directly driven by a basic large language model; no restrictions are placed on the implementation method of the intelligent agent in these related technologies.

[0018] S102: Obtain the initial sample data source and data synthesis pipeline configuration information for the synthetic data generation task, wherein the pipeline configuration information includes at least one atomic execution step arranged according to execution dependencies; Synthetic data generation tasks refer to job requests that augment training data or construct capability evaluation data for specific Large Language Model (LLM)-driven agents (such as agents in industry verticals). These tasks typically include explicit business objectives for data generation, such as generating 100,000 medical consultation question-and-answer pairs for a medical processing agent, or generating an adversarial evaluation set for a medical processing agent based on historical erroneous medical processing cases.

[0019] The initial sample data source refers to the reference basis used by the data synthesis task for data generation. It can be seen as seed data or context anchors that play a role in data synthesis. The initial sample data source includes, but is not limited to, the following types: high-quality small samples with manual annotation (Few-shot), user interaction failure cases (Badcase) returned from the online production environment, publicly available benchmark evaluation sets (such as general dialogue or instruction data in the database, used for migration generation), and structured meta-information templates used to define the data structure (such as user profile templates).

[0020] Data synthesis pipeline configuration information refers to a digital blueprint describing the entire data synthesis process, typically existing in the form of a declarative language or code objects. It defines the processing steps required from input to output, and the execution dependencies between these steps. The pipeline configuration information includes at least one atomic execution step arranged according to execution dependencies.

[0021] An atomic execution step refers to the smallest schedulable unit in a pipeline, encapsulating specific data processing functions. In this specification, atomic execution steps include, but are not limited to: data generation steps (such as calling a large model), data governance steps (such as cleaning and deduplication), and data evaluation steps (such as scoring).

[0022] Execution dependencies refer to the sequential order or data flow constraints between atomic execution steps. For example, the data governance step can only begin execution after the data generation step is completed. This relationship typically forms a directed acyclic graph (DAG).

[0023] In a specific implementation scenario, agent-oriented synthetic data generation methods can be implemented through a data synthesis framework, such as... Figure 2 As shown, Figure 2 This is a schematic diagram of a scenario architecture for a data synthesis framework. The data synthesis framework adopts a "Pipeline + Atomic Execution Step" architecture. This data synthesis framework abstracts the data synthesis process corresponding to the synthetic data generation method for intelligent agents into a configurable, composable, and iterative pipeline, realizing the upgrade from temporary manual workshops to framework-level infrastructure.

[0024] The data synthesis framework architecture for agent-oriented synthetic data generation methods consists of four layers: input, configuration and execution, core processing pipeline, and output.

[0025] 1) Input layer: Responsible for receiving the initial sample data sources for diverse synthetic data generation tasks, serving as the "introduction" or "context anchor" of the synthesis process.

[0026] 2) Configuration and Execution Layer: Users (through Python / YAML, etc.) define the pipeline to define the configuration information of the entire synthesis task to generate data synthesis pipeline, including but not limited to the atomic execution steps to be executed, the parameters of each atomic execution step, and the dependencies between atomic execution steps.

[0027] The configuration information of this data synthesis pipeline serves as the "blueprint" for the synthesis task, and is submitted to the (distributed) execution engine for execution S104, enabling large-scale parallel generation. Each run automatically records the configuration version, providing a basis for subsequent experimental reproduction and comparative analysis.

[0028] 3) Core Processing Pipeline Layer: This layer is the core engine of the data synthesis framework, consisting of a pipeline scheduler and modular atomic execution step components.

[0029] (Pipeline) Scheduler: Dynamically resolves configurations and schedules the execution order of atomic execution steps corresponding to atomic execution step components according to dependencies. The atomic execution step components are pluggable and reusable. 4) Output layer: The final output is a standardized, high-quality synthetic dataset of the target, which can be directly connected to HuggingFace, internal data platforms, and training systems.

[0030] Each piece of target synthesis data in the target synthesis dataset carries a structured metadata tag, which may include: the pipeline version used, the prompt template ID, the quality score, the timestamp, etc.

[0031] Regarding S102: In one feasible implementation, an interactive platform that provides users with visual operations is provided. The backend receives the initial sample file (e.g., CSV or JSONL format file) uploaded by the user through the frontend interface, parses the initial sample file into a list of objects in memory, or directly obtains the storage index address of the file in a distributed file system (such as OSS or HDFS) and marks it as the initial sample data source.

[0032] Further configuration information for the data synthesis pipeline: The system receives user configuration instructions from the front-end user to configure the data synthesis pipeline. These instructions can be generated by the user dragging and dropping atomic execution step components and connecting them on the pipeline configuration canvas in the front-end interface. The graphical pipeline configuration from the front-end is converted into a structured configuration file. The parser verifies the validity of the structured configuration file, identifying multiple nodes (i.e., atomic execution steps) and the directions of connections between nodes (i.e., execution dependencies). Based on the nodes and connection directions, a directed acyclic graph (DAG) structure of the task is constructed in memory as the data synthesis pipeline configuration information.

[0033] In one feasible implementation, this can also be achieved by reading a declarative configuration file from the local file system or version control system, suitable for code-based collaborative scenarios. That is, the execution engine reads a seed data file from the local disk or remote repository based on the file path specified in the startup parameters. If the data source is a database record, a query statement (SQL) is executed through a pre-defined database connector to retrieve the target record from the online database as the initial sample data source. The execution engine reads a predefined declarative configuration file. This declarative configuration file explicitly defines the sequence of atomic execution steps and the step component parameters for each atomic execution step using structures such as lists and nested dictionaries. The electronic device deserializes this configuration file into a program-recognizable configuration object class instance through a parser. It then iterates through the step component parameter definitions in the configuration object, and based on the input and output fields declared in the step component parameters, as well as the explicit dependency fields, parses the topological order between each atomic execution step to determine the execution dependencies. Finally, it performs synthesis to obtain the data synthesis pipeline configuration information.

[0034] S104: Taking the initial sample data source of the synthetic data generation task as a reference, the atomic execution steps are scheduled and executed based on the execution dependency relationship to perform data synthesis processing to obtain multiple target synthetic data, and the target synthetic dataset carrying metadata tags is output. The target synthetic dataset is used for large model training and / or large model evaluation of the target agent corresponding to the synthetic data generation task; wherein, when performing the data synthesis processing of the atomic execution steps, the current task configuration context information is obtained, and the task configuration context information is written as metadata tags into each of the target synthetic data.

[0035] Task configuration context information: refers to the set of parameters that characterize the current execution environment and logic during the execution of atomic execution steps. This includes, but is not limited to: a globally unique pipeline execution trace identifier (TraceID), the currently active configuration file version fingerprint, the algorithm model version called in the current step, the data synthesis prompt template identifier (PromptID), and the execution timestamp.

[0036] Target synthetic data refers to the final state of synthetic data records produced after processing through atomic execution steps. Examples include text generated from a large model, cleaned text, and structured data with accompanying quality scores.

[0037] In some embodiments, a distributed computing approach can be employed. Specifically, a directed acyclic graph (DAG) structure for the data synthesis task is constructed based on the data synthesis pipeline configuration information. The electronic device reads the DAG structure constructed in step S102 through the pipeline scheduler and calculates the execution priority of each atomic execution step component based on the execution dependencies in the DAG. The scheduler divides the initial sample data source into multiple data shards and sequentially distributes the task instances of each atomic execution step (such as data generation steps and data governance steps) to multiple worker nodes in the computing cluster for parallel execution of each atomic execution step.

[0038] Metadata injection is achieved using techniques such as Aspect-Oriented Programming (AOP). When the execution function of each atomic execution step is called, the electronic device automatically captures the current task configuration context information. Specifically, when a worker node generates or processes each piece of target synthetic data, the electronic device's interceptor automatically reads the current task's global configuration object and extracts task configuration context information, such as fields like `pipeline_version`, `step_name`, and `model_signature`. The interceptor encapsulates this extracted task configuration context information into a metadata dictionary and merges it into the metadata tag retention field of that target synthetic data.

[0039] Finally, after the terminal atoms in the directed acyclic graph (DAG) have completed their processing steps, the system aggregates the data records scattered across the nodes, each carrying complete metadata tags, and writes them to the distributed file system in a specified format to form the target synthetic dataset.

[0040] In one feasible implementation, a streaming pipeline pattern can also be used for real-time data stream processing. Specifically, the execution engine instantiates each atomic execution step component based on execution dependencies and chains them together into a data stream processing pipeline. Data items from the initial sample data source enter the pipeline head one by one. The data items flow sequentially through each atomic execution step component in the data stream processing pipeline. Each atomic execution step component processes the input data and outputs the processed target composite data to the next stage.

[0041] Within the internal logic of each atomic execution step component, context acquisition and writing operations are performed. When a component initializes, it receives a context object containing configuration information. When processing each piece of data, the component generates target synthetic data based on business logic. Following the generation of the target synthetic data, the component calls the context object's interface to obtain the current task configuration context information and appends this information as key-value pairs directly to the dictionary structure of the current data item. Subsequent steps (such as data governance steps) will retain the existing metadata tags when processing this data item and can continue to append new processing metadata, forming a chain-like accumulation of metadata.

[0042] Finally, the output component at the end of the pipeline receives data items carrying metadata tags in real time to generate the target synthetic dataset, and appends it to a local disk file or pushes it to a message queue for downstream model training tasks to consume in real time.

[0043] In one or more embodiments of this specification, a modular pipeline scheduling mechanism based on execution dependencies is constructed. During the data synthesis process of atomic execution steps, the current task configuration context information is captured in real time, and the context information is structured and written into each target synthesized data as metadata tags. This solves the limitations of non-standardized workflows, difficulty in large-scale reuse and collaboration caused by discrete script processing in data synthesis. It realizes automated orchestration of data synthesis tasks, improving the efficiency and flexibility of large-scale data production and development for intelligent agents. Furthermore, a rigorous data traceability system is established through metadata tags, ensuring that each synthesized data carries complete production traceability information, achieving full-process traceability and experimental reproducibility. Finally, it guarantees high-quality synthesized data output, reduces labor costs, and improves the engineering collaboration capabilities of data synthesis.

[0044] In one feasible implementation, the specific execution of writing the task configuration context information as metadata tags into each piece of target synthesis data can refer to the following method: Step A2: Check whether the current data synthesis process has generated the target synthesized data; As an example, an event-stream-based monitoring and verification method can be used, invoking atomic execution steps to output results through a message queue or event bus.

[0045] Electronic devices can deploy listeners to subscribe to the output channels of atomic execution steps. When a listener captures a specified event (such as DataGenerated or ItemEmitted), it parses the event payload and verifies whether the data fields in the event payload meet preset minimum length or integrity requirements. If they do, it determines that the target synthetic data should be generated, triggering step A4.

[0046] Step A4: If the current data synthesis process generates target synthesized data, then obtain pipeline configuration information, prompt word template information used in the synthesized data, data evaluation quality information, and data synthesis metadata; Pipeline configuration information refers to static descriptive data that defines the overall architecture and dependencies of the current synthesis task, including but not limited to the version hash of the configuration file, the configuration file name, and the version of the currently effective parameter set.

[0047] The prompt word template information refers to the identifier of the specific instruction template used to drive the large language model to generate the current synthetic data, including but not limited to the template ID, template version number, and specific content summary of the template.

[0048] Data assessment quality information refers to the quality metrics calculated for the currently generated synthetic data. If the current step is a generation step, this information may be empty or marked as pending assessment; if the current step is an assessment step, this information includes specific quality scores or compliance verification results.

[0049] Data synthesis metadata refers to dynamic attribute information that is strongly related to the runtime environment, including but not limited to globally unique task tracking identifiers, the name of the current atomic execution step, the name and version of the model instance being called, and the precise timestamp of data generation.

[0050] In one implementation, the system centrally acquires information by accessing the global singleton's context manager. Specifically, when target synthetic data is detected, the execution engine first calls the global configuration interface to read the pipeline configuration information of the currently running task, including the version number of the configuration file and the dependency graph identifier. Next, the engine accesses the input parameter area of ​​the current atomic execution step to extract the prompt word template information used to drive model inference, including the template ID and its instantiated hash value. Subsequently, the engine queries the evaluation status register in the current execution context. If a calculated quality score exists, it is directly read as data evaluation quality information; otherwise, the field is set to null. Finally, the engine calls the system kernel's time service and thread-local storage interface to obtain the current timestamp and the global tracking identifier of the current request chain as data synthesis metadata, thereby completing the collection of information from all dimensions.

[0051] Step A6: Perform tagged structuring on the pipeline configuration information, prompt word template information used in the synthesized data, data evaluation quality information, and data synthesis metadata to obtain metadata tags, and write the metadata tags into the target synthesized data.

[0052] In one implementation, key-value pair mapping and serialization techniques can be used to achieve structuring and writing. Specifically, the electronic device first creates a standard dictionary object and a hash map table as containers for metadata tags. The electronic device can traverse pipeline configuration information, prompt word template information, data evaluation quality information, and data synthesis metadata, mapping various types of information to key-value pairs in the dictionary. During the mapping process, data types are standardized, and the result is obtained after standardization.

[0053] After the dictionary is constructed, the write operation is performed. If the target synthesized data is typically a structured object, a new reserved field is added to the object, and the dictionary is assigned to this field. If the target synthesized data is unstructured text, the dictionary is serialized into a JSON string and appended to the end of the text data according to preset delimiter rules (such as newline characters or special tags), thereby achieving data isomorphism between metadata tags and content.

[0054] In this specification, steps A2-A6 are performed to construct and inject metadata into synthetic data assets. This not only enables downstream intelligent agent training or evaluation tasks to perform refined data reuse based on the information in the metadata tags, improving data utilization, but also provides the smallest granular index key for anomaly attribution in practical applications. This ensures that in complex engineering iterations, any dirty data can be accurately traced back to its source configuration, guaranteeing the interpretability and controllability of intelligent agent training data.

[0055] Furthermore, in practical applications, the following steps can also be performed: Step B2: In response to the anomaly analysis request for the evaluation metrics of the target intelligent agent, locate the batch of abnormal synthetic data and the task configuration context information based on the metadata tags; An evaluation metric anomaly analysis request refers to an instruction automatically triggered by the electronic device monitoring component or manually initiated by technicians through a terminal when key performance indicators (such as accuracy, recall, illusion rate, security score, etc.) are generated after training or evaluating the target intelligent agent using synthetic data, and the key performance indicators show an unexpected decline, drastic fluctuation, or failure to meet the standards.

[0056] An anomalous synthetic data batch refers to a specific subset of anomalous synthetic data that causes abnormal metrics during the training or evaluation of a target agent using synthetic data. This anomalous synthetic data subset typically consists of data records with quality evaluation scores below a certain threshold, or data records that the evaluation model judges to have given incorrect answers.

[0057] In one implementation, a conditional retrieval technique based on an inverted index can be used for location. Specifically, when an electronic device receives a request for an anomaly analysis of evaluation metrics, it first parses the filtering condition parameters contained in the request, such as the evaluation task identifier, the anomaly time window, and the specific metric threshold. Then, based on the parsed parameters, it constructs a query statement containing Boolean logic and executes the query in a database that supports inverted indexes to retrieve all data records belonging to the specified evaluation task and whose quality scores meet the anomaly threshold conditions. The retrieved record set is then marked as an anomaly synthetic data batch. Next, the metadata tag fields stored in the batch of data are aggregated and statistically analyzed or deduplicated to extract key attributes common to the batch of data, such as pipeline configuration version, prompt word template identifier, and model version. Finally, these attributes are combined to determine the task configuration context information.

[0058] Step B4: Based on the metadata tags, locate the abnormal synthetic data batch and the task configuration context information to start the abnormal attribution analysis process.

[0059] Anomaly attribution analysis process: This refers to the process of using existing metadata information and pre-defined techniques such as variable control, statistical comparison, or lineage tracing to determine the root cause of anomalies in evaluation metrics. Common root causes of anomalies include: fluctuations in data source quality, changes in pipeline configuration (such as iterative prompt word templates), model version degradation, or random noise. The anomaly attribution analysis process is customizable based on the actual application environment and is not limited here.

[0060] In one feasible implementation, the anomaly attribution analysis process can be initiated by automated rerun tests based on the controlled variable method. Specifically, it parses and locates the task configuration context information, identifies key variable parameters in the current synthetic task, such as the prompt word template version or the model version called, and searches the historical task library for historical tasks similar to the current task but with normal evaluation metrics as benchmark tasks. Then, it automatically generates one or more diagnostic control tasks. The diagnostic control tasks inherit most of the configuration of the current abnormal task, but force the identified key variable parameters to be rolled back to the values ​​corresponding to the benchmark task, such as replacing the prompt word template from the current V2 version back to the V1 version. By scheduling and executing the control tasks and collecting new evaluation metrics, if the metrics of a certain control task return to normal, the system determines that the replaced parameter is the root cause of the anomaly and generates an attribution report containing rollback recommendations.

[0061] In one or more embodiments of this specification, by executing steps B2-B4, an automated attribution analysis process based on pre-injected metadata tags is implemented for the actual data application stage, so as to guide developers to perform targeted configuration rollback or parameter tuning, thereby constructing a quality feedback closed loop between data synthesis and intelligent agent model iteration.

[0062] Optional, please see Figure 3 , Figure 3 This is a schematic diagram of a data synthesis process proposed in this specification. Specifically, the process of using the initial sample data source of the synthesized data generation task as a reference, and scheduling and executing each of the atomic execution steps based on the execution dependencies to perform data synthesis processing and obtain multiple target synthesized data can be carried out in the following manner: S202: Based on the execution dependencies, determine the timing of the target data synthesis steps, including data generation steps, data governance steps, and data evaluation steps; Execution dependencies refer to the logical constraints between atomic execution steps defined in the pipeline configuration information. Execution dependencies can manifest as "pre-dependent" constraints, such as when the input of step B depends on the output of step A, then step A is a pre-dependent of step B.

[0063] The timing of the target data synthesis steps refers to a linear sequence of instructions or a queue of tasks that have a sequential execution order, as indicated by dependencies.

[0064] Data generation steps: These are usually located upstream in the chain, such as steps responsible for diverging, expanding, or rewriting based on the initial samples.

[0065] Data governance steps: These are in the middle of the chain, such as steps responsible for cleaning, deduplicating, de-identifying, or formatting the generated content.

[0066] Data evaluation steps: These are downstream steps in the process, such as those responsible for scoring, verifying, or filtering data quality.

[0067] Intuitively, the data synthesis pipeline configuration information is read, each atomic execution step is instantiated as a logical node in the graph structure, and the relationship between the preceding and succeeding steps is instantiated as directed edges based on the input-output reference relationships between steps, thereby constructing a directed acyclic graph representing the execution dependencies in memory. Then, the directed acyclic graph is traversed, and the in-degree of each node is calculated. All nodes with an in-degree of zero are added to the queue to be executed as the starting node. Then, nodes are sequentially taken out from the queue and added to the sequence list when the target data synthesis step is executed. Each time a node is taken out, the node and its emanating directed edges are logically removed, and the in-degree of the remaining downstream nodes is updated. If the in-degree of a downstream node becomes zero, it is added to the queue. This process is repeated until all nodes are added to the sequence list, thereby generating a linear execution sequence that strictly satisfies the dependency order.

[0068] S204: Based on the timing of the target data synthesis steps and referring to the initial sample data source of the synthesis data generation task, the data generation steps, the data governance steps, and the data evaluation steps are scheduled and executed sequentially to perform data synthesis processing.

[0069] In one implementation, a chain-like passing pattern based on memory objects is used to perform data synthesis. Specifically, the task execution context container is initialized, and the initial sample data source is loaded into memory. The control scheduler extracts the first data generation step from the target data synthesis step sequence, calls the large model interface configured for that step, and transforms the initial sample into the first batch of intermediate data objects. The control scheduler passes the first batch of intermediate data objects as input parameters to the next step in the sequence, namely the data governance step. This step performs data governance operations such as regular expression cleaning and hash deduplication on the data objects in memory, producing the second batch of intermediate data objects. Then, the scheduler passes the second batch of intermediate data objects to the data evaluation step. This step performs data evaluation operations such as calculating the quality score of the data objects and performing threshold filtering, retaining only the data objects that pass the data evaluation as the final target synthesized data.

[0070] In the embodiments described in this specification, execution steps S202-S204 employ a pipeline scheduling execution mechanism based on a three-order logical closed loop of "generation-governance-evaluation". By parsing execution dependencies, a strict step sequence is determined, and data generation steps are scheduled sequentially for sample expansion, data governance steps for cleaning and desensitization, and data evaluation steps for quality scoring and threshold filtering. This establishes a standardized data production operation mode, improving the usability and security of the synthesized data.

[0071] For further details, please see Figure 4 , Figure 4 This is a flowchart illustrating a data synthesis processing step based on a large model, as proposed in this specification. When the atomic execution step is a data generation step, the data synthesis processing is specifically performed, including: S3002: Obtain the preset prompt word template and the context data corresponding to the initial sample data source, and fill the context data into the prompt word template to construct synthetic data and generate prompts; Contextual data can be specific business information extracted from the initial sample data source and used to populate the parameter slots in the prompt word template. For example, if the sample source is a user complaint record, the content of that record would be the contextual data.

[0072] Synthetic data generation prompts refer to complete instruction texts that can be directly understood and executed by the large model formed after context data is filled into the template.

[0073] Batch synthetic data inference refers to the computational process of sending multiple generation requests to a large model through concurrent calls or batch processing interfaces to improve throughput and reduce latency.

[0074] S3004: Based on the synthetic data generation prompt, the preset data generation large language model interface is used to call the data generation large language model to perform batch synthetic data reasoning to generate a synthetic data set, and the synthetic dataset is used to output the target synthetic dataset.

[0075] The pre-defined data generation large language model interface refers to the standardized call entry point for the large language model service provider or locally deployed model.

[0076] Batch synthetic data inference refers to the process of sending a large number of generation requests to the model and obtaining results within a unit of time through concurrency or batch processing mechanisms.

[0077] Indicatively, the synthetic data generation prompts and the initial sample data source preset data generation big language model interface are called to perform batch synthetic data inference to generate synthetic data sets. When there is no next atomic execution step, the synthetic dataset can be directly used as the target synthetic dataset. If there is a next atomic execution step (such as a data governance step), the next atomic execution step is executed to further process the synthetic data set.

[0078] In one feasible implementation, an offline batch processing mode based on asynchronous task submission and polling mechanism can be used. Specifically, multiple synthetic data prompts in a preset batch processing format are aggregated and written into a batch processing file, which is then uploaded to the cloud object storage space. The index ID of this file is submitted by calling the interface of the large model to create an offline inference task. The large model server queues and processes the prompts in this file in the background. During this period, a polling state is entered, and the execution progress status of the task ID is checked periodically. When the task status changes to completed, the download interface is called to obtain the output file containing all inference results. This file is then streamed and deserialized, and each line of inference result is matched and mapped with the original request ID, thereby constructing a complete synthetic data set at once.

[0079] The following is an illustrative, optional example of a model training process for a large data generation model: In some embodiments, a pre-trained basic large language model can be obtained and adapted to the data generation scenario to obtain a large data generation model. However, directly applying the basic large language model to the data generation scenario is often difficult to adapt to new data generation scenarios. Therefore, the basic large language model is first obtained to create an initial large data generation model, and sample data for the new data generation scenario is obtained. This sample data includes synthetic data generation prompts and initial training sample data sources. Since the basic large language model is usually a pre-trained open-source AIGC model with content generation capabilities, this specification only needs to adapt it to the data generation scenario. Specifically, the sample data can be used to fine-tune the initial large data generation model. After the model fine-tuning training is completed, a large data generation model adapted to the data generation scenario is obtained.

[0080] Model creation: Obtain the basic large language model, create an initial data generation scenario plugin model for the data generation scenario, and form an initial data generation large model based on the basic large language model and the initial data generation scenario plugin model; the basic large language model (LLM) includes, but is not limited to, the DeepSeek large model, the GPT series large models, etc.

[0081] Sample data acquisition: Acquire sample data for new data generation scenarios. This sample data is the "synthetic data generation hints and initial training sample data source" for the data generation scenario.

[0082] Sample data annotation: Labels for the "key features of synthetic data" corresponding to the data generation requirements of the data generation scenario.

[0083] Model training process: Input sample data into initial data to generate a large model and perform at least one round of model training. During the forward training of the model: Based on the sample data, use the initial data to generate a large model and process it to obtain the predicted synthetic data. During the reverse training of the model, key features of the predicted synthetic data are extracted based on the predicted synthetic data, and the model loss value is determined by the model loss function using the key feature labels of the synthetic data. Based on the model loss value, the model parameters are adjusted in the initial data to generate a large model, and the trained data generation large model is obtained.

[0084] As an illustration, the initial data generation scenario plugin model can be created based on a machine learning model.

[0085] Optionally, the model's training termination conditions may include, for example, the loss function value being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold. Specific training termination conditions can be determined based on actual circumstances and are not specifically limited here.

[0086] It should be noted that the machine learning models involved in one or more embodiments of this specification include, but are not limited to, fitting of one or more of the following machine learning models: Convolutional Neural Network (CNN) model, Deep Neural Network (DNN) model, Recurrent Neural Networks (RNN) model, embedding model, Gradient Boosting Decision Tree (GBDT) model, Logistic Regression (LR) model, etc.

[0087] This specification describes how standardized generation prompts are constructed using dynamic variable interpolation technology based on preset prompt word templates and context data, executed in steps S3002-S3004. Furthermore, a high-concurrency or asynchronous batch processing mechanism using a preset data generation large language model interface is employed for batch synthetic data inference. This achieves intelligent production from a small amount of seed data to high-quality synthetic data. The batch inference mechanism shortens the data preparation cycle, enabling the construction of large amounts of synthetic data in a short time and at low cost.

[0088] For further details, please see Figure 5 , Figure 5 This is a schematic diagram of a data governance process proposed in this specification. When the atomic execution step is a data generation step, the specific data synthesis processing is performed, including: S4002: Obtain the governance configuration parameters of the data governance steps and the synthetic data generated for the synthetic data generation task; Governance configuration parameters refer to the specific set of rules and threshold settings that define the data governance logic. They can be understood as determining how data is cleaned and what content is retained. Examples include regular expression patterns for inference trace cleaning, similarity thresholds for semantic deduplication, and target field paths for structured expansion.

[0089] The synthetic data generated for the aforementioned synthetic data generation task refers to the original data output produced in the preceding data generation steps. Synthetic data typically contains redundant information, potential formatting errors, or duplicate content specific to large models.

[0090] In one feasible implementation, a query request is initiated to the distributed configuration center or local configuration file based on the unique identifier of the current atomic execution step to obtain the corresponding governance strategy description file, and then deserializes it into a governance rule object in memory. This governance rule object contains a list of enabled governance operators and specific parameters for each operator, such as whether the mind chain cleaning function is enabled and the corresponding regular expression matching rules. In parallel, the list of synthetic data objects to be processed is obtained directly by receiving memory references passed by upstream data generation steps or message carriers in message queues.

[0091] S4004: Perform at least one of the following synthetic data governance logics on the synthetic data according to the governance configuration parameters: inference trace cleaning logic, structured expansion processing logic, field concatenation processing logic, semantic deduplication logic, and field retention processing logic; The inference trace cleaning logic is used to detect large model inference trace information and large model inference content in the synthetic data, and remove the large model inference trace information and the large model inference content from the synthetic data. For example, the data governance step corresponding to a certain inference trace cleaning logic is used to remove labels from the output of a large model LLM, and is usually used immediately after TextGeneration.

[0092] The structured expansion processing logic is used to identify nested structure list fields in the synthetic data, and to parse the key-value content of the nested structure list fields to obtain the target synthetic data fields. For example, the data governance step corresponding to a certain structured expansion processing logic is used to expand fields containing JSON lists into multiple rows and parse multiple target fields in nested structures, which is suitable for structured information extraction. The field concatenation processing logic is used to concatenate the contents of multiple source fields in the composite data according to a specified delimiter according to a preset field concatenation rule to obtain a composite composite data field; it can be understood as merging the contents of multiple source fields in the same composite data record at the character level according to a preset format template or delimiter to generate a new composite field.

[0093] For example, the data governance step corresponding to the field concatenation processing logic is used to concatenate the contents of multiple fields into a new field according to a specified delimiter, which is used to construct compound queries or contexts.

[0094] The semantic deduplication processing logic is used to calculate the semantic similarity of the synthesized data, and based on the semantic similarity, remove semantically duplicated synthesized data of semantic repetition type; For example, the data governance step corresponding to a certain semantic deduplication processing logic is used in a MinHash-based semantic deduplication component, which supports mixed Chinese and English text and effectively removes redundant samples with highly similar expressions.

[0095] The field retention processing logic is used to filter out non-preset key fields from the synthesized data. For example, it can retain only key business fields based on a preset whitelist, while filtering out temporary variables or irrelevant metadata generated during the production process.

[0096] For example, the data governance steps corresponding to the field retention processing logic are used to retain only the key fields specified by the user and discard irrelevant information, thereby achieving data lightweighting and privacy desensitization.

[0097] In one feasible implementation, a data link-based streaming pipeline technology can be used to execute a combination of various governance logics. Specifically, the system parses governance configuration parameters, instantiates corresponding processor objects based on the enabled logical type, and links these processors into a data processing pipeline in a preset order such as cleaning, unfolding, splicing, deduplication, and retention. The synthetic data to be processed enters the pipeline head as a streaming object, passing through the inference trace cleaning node, which uses regular expressions to match and remove thought chain tags and their content from the text. The data then flows through the structured unfolding node; if the target field is identified as an array structure, the current data object is cloned and split into multiple independent records using an iterator to continue downstream transmission. Next, the data flows through the field splicing node, where the system formats and merges the values ​​of multiple source fields according to a predefined string template. Next, at the semantic deduplication node, the system calculates the semantic fingerprint of the current data in real time and queries the global Bloom filter or vector index; if it is determined to be duplicated, the data packet is directly discarded. Finally, at the field retention node, the system trims the attributes of the data object according to the field whitelist, outputting only compliant target synthetic data. The entire process can be completed in memory with a single traversal of the data.

[0098] In this specification, by executing S4002 - S4004, a configuration - driven multi - dimensional data governance mechanism is adopted. By obtaining governance configuration parameters, combinatorial governance logic including inference trace cleaning to remove chain - of - thought noise, structured unfolding to flatten nested records, field concatenation to standardize input formats, semantic deduplication to eliminate similar samples, and field retention to filter redundant metadata is flexibly executed on the original synthetic data. This enhances the training value of the synthetic data. Among them, inference trace cleaning ensures the purity of the data, preventing the model from learning to output irrelevant thinking processes; the structured unfolding logic disassembles multiple pieces of data produced by a single inference, improving the utilization rate of the generated data; the semantic deduplication logic ensures the sample diversity and distribution balance of the data set, thus effectively enhancing the generalization ability and robustness of the target intelligent agent model in downstream tasks.

[0099] Furthermore, please refer to Figure 6 , Figure 6 which is a schematic diagram of a data evaluation process proposed in this specification. When the atomic execution step is a data evaluation - type step, the data synthesis process is as follows: S5002: When the atomic execution step is a data evaluation - type step, determine the synthetic data generated for the synthetic data generation task; Data evaluation - type step: Refers to the quality - control link in the pipeline. Its input is, for example, the synthetic data after governance, and the output is a subset of qualified data after screening.

[0100] The data evaluation - type step can, for example, be to call a review model to perform multi - dimensional quality scoring on each piece of data, including multiple dimensions such as specificity, domain knowledge, cognitive challenge, etc.; for example, it can be to perform basic verification and filtering on fields based on input parameter rules (such as non - null, length range, regular expression matching) to eliminate obviously invalid samples; S5004: Call a data evaluation large - language model to perform data quality evaluation on the synthetic data to obtain the first quality evaluation score of the synthetic data, and remove the low - quality data in the synthetic data whose first quality score is less than the quality score threshold; and / or, obtain field input parameter verification rules, use the field input parameter verification rules to perform field verification and screening on the synthetic data to obtain field quality evaluation information, and based on the field quality evaluation information, eliminate invalid data in the synthetic data that does not meet the field verification rule type.

[0101] The data evaluation large - language model (which can be called JudgeLLM) can refer to a large - language model that has been fine - tuned by instructions or configured as a judge role through prompt engineering. It has the ability to understand complex instructions, judge logical coherence, and factual accuracy, and is used to output a quantitative quality score. For example, as Figure 2As shown, the corresponding synthetic data is sent to the annotator. The annotator can call the data evaluation big language model to perform multi-dimensional quality scoring on the synthetic data, including multiple dimensions such as specificity, domain knowledge, and cognitive challenge, to obtain the first quality evaluation score of the synthetic data.

[0102] The first quality assessment score refers to quantitative indicators given by the data evaluation big language model, such as reflecting the quality assessment dimensions of data in terms of semantics, logic, or usefulness.

[0103] Field parameter validation rules refer to hard constraints based on data structure definitions or regular expressions. For example, checking if a JSON field exists, if the string length exceeds the limit, or if the code conforms to syntax rules.

[0104] In one feasible implementation, a pre-set evaluation prompt template for a specific task scenario is loaded. The evaluation prompt template can define the role setting of the evaluation language model (such as a senior code audit expert) and specific scoring dimension standards (such as factual accuracy, logical coherence, security, and instruction compliance). The synthetic data text to be evaluated is filled into the variable slots of the template to construct an evaluation input instruction containing a complete context. The evaluation input instruction is sent to a pre-set data evaluation language model for inference via an API interface. The evaluation model performs multi-dimensional analysis of the quality of the synthetic data based on the instruction and outputs text response information containing specific numerical scores and evaluation reasons. The numerical score is accurately extracted from the text response information as the first quality evaluation score of the synthetic data. A pre-set quality score threshold is read, and the first quality evaluation score of each piece of data is compared with the threshold. Data with a first quality evaluation score lower than the threshold is determined to be low-quality data that does not meet the semantic or logical standards, and physical deletion or logical filtering operations are performed. Only data with a score higher than or equal to the threshold is retained for subsequent processes, thereby ensuring that the final output dataset has high availability at the content level.

[0105] In one feasible implementation, a static validation technique based on hard rule constraints can be used to preemptively intercept invalid data that does not conform to structured specifications. Specifically, preset field input parameter validation rules are read from the configuration center or local rule base. These rules define the integrity and compliance conditions of the target synthetic data based on data attributes, including existence checks of required fields, correctness checks of field data types, boundary checks of field lengths, and regular expression matching checks for specific formats. The validation engine is then started, and the aforementioned field input parameter validation rules are applied one by one to each piece of synthetic data to be processed in memory. Field-level scanning and validation are performed on the synthetic data, and corresponding field quality evaluation information is generated for each piece of synthetic data. Finally, decision logic is executed based on the field quality evaluation information. Data records marked as "validation failed" are determined to be invalid data that does not meet the field validation rule type and are removed from the data stream to quickly filter out structurally poor data caused by truncation, incorrect formatting, or missing key information.

[0106] Optionally, the above two methods can be executed in parallel or sequentially. You can choose one to execute, depending on the actual application scenario; no specific restrictions are imposed here. The following is an illustrative, optional example of the model training process for evaluating a large language model using data: In some embodiments, a pre-trained basic large language model can be obtained, and the basic large language model can be adapted to the data evaluation scenario to obtain a data evaluation large language model. However, it is usually difficult to adapt the basic large language model directly to the new data evaluation scenario. Therefore, the basic large language model is first obtained to create an initial data evaluation large language model, and sample data for the new data evaluation scenario is obtained. This sample data includes prompt words and synthesized data. Since the basic large language model is usually a pre-trained open-source AIGC model with content generation capabilities, this specification only needs to adapt it to the data evaluation scenario. Specifically, the sample data can be used to fine-tune the initial data evaluation large language model. After the model fine-tuning training is completed, the data evaluation large language model adapted to the data evaluation scenario is obtained.

[0107] Model creation: Obtain the basic large language model, create an initial data evaluation scenario plugin model for the data evaluation scenario, and form an initial data evaluation large language model based on the basic large language model and the initial data evaluation scenario plugin model; the basic large language model (LLM) includes, but is not limited to, the DeepSeek large language model, the GPT series of large language models, etc.

[0108] Sample data acquisition: Acquire sample data for new data evaluation scenarios. This sample data consists of "preset data evaluation prompts and sample synthesis data" for the data evaluation scenario.

[0109] Sample data labeling: Label the corresponding quality assessment score tags based on the data assessment requirements of the data assessment scenario.

[0110] Model training process: Input the sample data into the initial data evaluation large language model for at least one round of model training. During the forward training of the model: Based on the sample data, the initial data evaluation large language model is used to process the data to obtain the prediction quality score. During the reverse training of the model, the model loss value is determined by the model loss function based on the prediction quality assessment score and the key feature labels of the synthetic data. Based on the model loss value, the model parameters are adjusted in the initial data evaluation large language model to obtain the trained data evaluation large language model.

[0111] As an illustration, the initial data evaluation scenario plugin model can be created based on a machine learning model.

[0112] Optionally, the model's training termination conditions may include, for example, the loss function value being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold. Specific training termination conditions can be determined based on actual circumstances and are not specifically limited here.

[0113] It should be noted that the machine learning models involved in one or more embodiments of this specification include, but are not limited to, fitting of one or more of the following machine learning models: Convolutional Neural Network (CNN) model, Deep Neural Network (DNN) model, Recurrent Neural Networks (RNN) model, embedding model, Gradient Boosting Decision Tree (GBDT) model, Logistic Regression (LR) model, etc.

[0114] In this specification, by executing S5002-S5004, a quality firewall is constructed from formal compliance to content usability, improving the quality of the target synthetic dataset. Rule validation, as a coarse-screening method, significantly reduces the marginal cost of data cleaning, while model evaluation, as a fine-screening method, ensures high semantic fidelity of the data. This hierarchical evaluation strategy not only guarantees the convergence stability of downstream agent model training (avoiding being misled by dirty data), but also...

[0115] For further details, please see Figure 7 , Figure 7This is a flowchart illustrating the process of determining data synthesis pipeline configuration information as described in this specification. The specific steps for obtaining the data synthesis pipeline configuration information can be found below: S602: Access the preset atomic execution step library, which includes atomic execution step components that encapsulate a unified input / output interface, and the atomic execution step components include data generation components, data governance components, and data evaluation components; Atomic execution step library (also known as Step template library): refers to a repository for storing and managing predefined functional components. Each component in the library is an independent logical unit, such as data generation components, data governance components, and data evaluation components.

[0116] In some embodiments, this Step template library systematically categorizes atomic execution steps based on "data generation, data governance, and data evaluation," and further expands input / output (I / O) operations to form a set of capability components covering the entire lifecycle of synthesis. All Steps adhere to a unified data contract (e.g., inputs / outputs are structured dictionary lists), supporting flexible orchestration and cross-project reuse.

[0117] Optionally, you can set the Step component in the atomic execution step library corresponding to input / output (I / O) operations. For example, the LoadDataFromDicts component is used to load a dataset from a dictionary list and generate it in batches; the LoadDataFromDataFrame component is used to load a dataset from a df variable and generate it in batches; the SaveToCSV component saves the data to a CSV file; and the SaveToJSONL component saves the data to a JSONL file. Optionally, you can set the Step component corresponding to the data generation class operation in the atomic execution step library. Please refer to the explanation of the data generation class steps mentioned above. Optionally, you can set the Step component corresponding to the data governance class steps in the atomic execution step library. Please refer to the above explanation of data governance class steps. Optionally, you can customize the Step component corresponding to the data evaluation step in the atomic execution step library. Please refer to the explanation of the data evaluation steps mentioned above. In one implementation, a low-code coding mode based on a graphical interface can be adopted, providing a low-code orchestration environment based on a graphical interface to execute the above steps. Specifically, a visualization panel of the atomic execution step library is loaded and displayed. This visualization panel encapsulates components such as data generation, governance, and evaluation components with unified input / output interfaces and displays them in icon form. The user process is summarized, responding to the user's component orchestration instructions within the canvas area, such as dragging specific component icons onto the canvas and drawing connecting lines between component icons, thereby instantiating multiple target atomic execution step components in memory and establishing a visual execution dependency relationship. When the user selects a component node, the electronic device pops up a parameter configuration sidebar, receiving the component running parameters input by the user for that node, such as prompt word template content, API key, or filtering threshold. Finally, when the user triggers a save or publish operation, the electronic device traverses all nodes on the canvas and their connection relationships, verifies the integrity and logical closure of the configuration, and after successful verification, serializes the graphical topology structure into standard JSON format pipeline configuration information and persists it to the database.

[0118] In one feasible implementation, a declarative definition pattern based on a domain-specific language (DSL) can be adopted. Specifically, a standardized configuration syntax is provided that defines how to reference component classpaths in the atomic execution step library and how to declare input-output mappings. In practice, the electronic device receives a configuration file (such as a .yaml script) containing component orchestration instructions. This configuration file can explicitly declare identifiers of multiple target atomic execution step components required, key-value pairs of runtime parameters for each component, and execution dependencies defined by the `needs` or `depends_on` keywords. Then, a parsing engine reads the configuration file, verifies whether the referenced components exist in the library and follow a unified input-output interface, and constructs a directed acyclic graph (DAG) to detect circular dependencies. After parsing and verification, the electronic device compiles the script into an internally executable pipeline configuration information object, thereby enabling the management of the data synthesis process through a code version control system (Git).

[0119] S604: In response to the component orchestration instructions for the atomic execution step library, select multiple target atomic execution step components from the atomic execution step library and configure component runtime parameters and execution dependencies for the target atomic execution step components; Component orchestration instructions refer to control signals triggered by the user to define the structure of the data processing flow. For example, in a graphical interface, this is manifested as drag-and-drop or connection operations; in code mode, it is manifested as referencing component IDs and submitting flow definitions.

[0120] The target atomic execution step component refers to the component instance selected by the user from the atomic execution step library to participate in this specific data synthesis task.

[0121] Component runtime parameters refer to static configuration items that control the behavior of individual components. Examples include prompt word templates for generating components, regular expressions for managing components, and qualification thresholds for evaluating components.

[0122] Execution dependency: refers to the constraints on the direction of data flow and the order of execution between components.

[0123] In one feasible implementation, the above steps are achieved through dynamic orchestration using a visual interactive canvas. Specifically, the system monitors user actions on the graphical orchestration interface in real time. When the system detects that the user drags a specific component icon from the sidebar of the atomic execution step library to the central canvas area, it responds to the component orchestration instruction by instantiating the corresponding target atomic execution step component object in memory and assigning it a globally unique node identifier. The electronic device captures the user's click on the component node, loads the parameters supported by the component in the attribute configuration panel, receives the component's input execution parameters (such as temperature values ​​for large model calls or regular expressions for cleaning rules), and performs real-time validation of the parameter format. Simultaneously, the system monitors the user's actions of drawing connecting lines between different component nodes, establishes directed edges based on the starting direction of the connecting lines, thereby defining the execution dependencies from upstream to downstream nodes, and automatically checks for loop dependencies or data type mismatches to ensure that the constructed process topology is logically valid and closed-loop.

[0124] In one feasible implementation, a parsing and loading method based on declarative configuration scripts is adopted. Specifically, a configuration file (such as YAML or JSON format) containing component orchestration instructions is received through an API interface or file system listening mechanism. This configuration file typically describes the list of components and topology required for the task in declarative syntax. The electronic device starts the parsing engine to read the configuration file, retrieves and loads the corresponding class definitions from the atomic execution step library according to the component name index referenced in the file, thereby identifying multiple target atomic execution step components. Then, the component parameter fields defined for each component node in the file are parsed, mapped and injected into the component runtime parameters, and the fields corresponding to the dependency relationships are parsed to construct the adjacency matrix between components to establish execution dependencies. During this process, the electronic device performs dependency integrity checks to ensure that all components declared as prerequisites have been correctly defined in the configuration and have complete parameter configuration, thereby transforming the static text description into a logical object network that can be recognized by the scheduling engine.

[0125] S606: Generate pipeline configuration information based on the target atomic execution step component, the component running parameters, and the execution dependencies.

[0126] In one feasible implementation, the component node graph already constructed in memory is traversed, and the globally unique identifier, path reference of the underlying implementation class, and specific component running parameters configured by the user in the preceding steps are extracted one by one for each target atomic execution step component. At the same time, the connection pointers between components are parsed to construct an adjacency list or edge list to accurately represent the execution dependencies. Then, the serialization engine is called to aggregate and convert the above-mentioned scattered metadata into a hierarchical structured text object according to the predefined pipeline definition language and standard JSON specification. During this process, the electronic device automatically injects global metadata fields into the root node, including pipeline name, creator ID, creation timestamp, and default global timeout policy, and immediately performs a final logical verification on the generated structured object to check for the existence of circular dependencies, whether the parameter types match the component interface definition, and whether required fields are missing. Finally, the text object that passes the verification is marked as ready and persistently stored in the configuration management database, thus completing the generation of pipeline configuration information.

[0127] This specification establishes a library of pre-built atomic execution steps, including at least data generation, governance, and evaluation components. All components are encapsulated with unified input / output interfaces. Responding to user orchestration commands, the library selects target components and configures their runtime parameters and dependencies, ultimately generating structured pipeline configuration information from the orchestration results. This achieves standardized and modular construction of the data synthesis workflow. On one hand, the unified interface enables plug-and-play functionality, allowing validated, high-quality components (such as specific cleaning rules or prompt templates) to be reused across different tasks, reducing development costs. On the other hand, decoupling the process definition from the underlying code and serializing it into configuration information enables versioned management and rapid iteration of data production tasks, allowing users to flexibly adjust data production strategies without modifying the code.

[0128] Optionally, in some embodiments, the agent-oriented synthetic data generation method can be implemented using a data synthesis framework, such as... Figure 2 As shown, Figure 2 This is a schematic diagram of a data synthesis framework's scenario architecture. The framework employs a "Pipeline + Atomic Execution Step" architecture, abstracting the data synthesis process of agent-oriented synthetic data generation methods into a configurable, composable, and iterative pipeline, thus upgrading from a temporary, manual process to a framework-level infrastructure. The data synthesis framework architecture for agent-oriented synthetic data generation methods consists of four layers: input, configuration and execution, core processing pipeline, and output.

[0129] Furthermore, the following lists one or more illustrative application methods using the data synthesis framework. The agent-oriented synthetic data generation method can support a dual-mode deployment strategy through the data synthesis framework, aiming to balance the centralized management needs of enterprise-level large-scale production with the lightweight R&D needs of algorithm teams for rapid iteration. Specifically, the system includes a platform-based integration mode and a lightweight version control (Git) collaboration mode.

[0130] (1) Platform-based integration model In embodiments corresponding to the platform-based integration model, the data synthesis framework is deeply integrated into the enterprise's internal AI data platform or model evaluation platform in the form of microservices or SDKs. This model is suitable for high-frequency, large-scale, multi-team collaborative data production and governance scenarios, enabling centralized governance and closed-loop management of data assets. Specifically, it includes: It provides a graphical user interface that allows users to configure the pipeline using a declarative syntax by dragging and dropping atomic execution step components (Steps). The interface supports visual parameter settings, shielding users from the complexity of the underlying code implementation.

[0131] The distributed execution engine schedules tasks in response to large-scale parallel synthesis requests. The backend interfaces with a distributed computing cluster (such as a Kubernetes cluster), automatically splits pipeline tasks and schedules the execution engine, and supports concurrent data synthesis and governance.

[0132] A unified storage and metadata indexing approach is adopted, establishing a centralized data warehouse to uniformly store the generated synthetic data and its associated metadata tags. The system builds an index based on metadata, supporting fine-grained management of dataset versions.

[0133] It supports bidirectional traceability and attribution analysis by establishing a bidirectional mapping relationship between "dataset ID" and "model experiment ID" at the database level. When the evaluation platform detects abnormal model metrics, it can directly trace back to the pipeline configuration snapshot that produced the data through this mapping relationship, thereby supporting automated evaluation attribution analysis (i.e., refer to steps B2-B4 above).

[0134] This model is suitable for high-frequency, large-scale, multi-team collaborative synthesis scenarios, achieving "out-of-the-box usability, centralized governance, and asset closure".

[0135] (2) Lightweight Git Collaboration Mode In the implementation of the lightweight Git collaboration mode, a distributed collaboration mode based on a version control system (such as Git) is provided for resource-constrained experimental environments or initial verification scenarios. This mode allows developers to flexibly define processes in their local environment and synchronize assets through code repositories.

[0136] 1) Users first copy the project code, including the system's core framework and the atomic execution step template library (StepLibrary), from the remote code repository to their local computing environment. In the local environment, users use YAML or Python scripts to write configuration files, define personalized synthesis pipelines, and trigger data generation task scripts locally.

[0137] 2) Further combine with the appendix Figure 8 Detailed process instructions Figure 8 This diagram illustrates a lightweight Git collaboration model for data flow and asset synchronization. As shown, the collaboration process mainly includes three core interaction domains: the input assembly domain, the code repository domain (AntCode), and the local execution domain.

[0138] Step 1: Assembly of Input Resources: In the local execution domain, the user first gathers basic data resources, including but not limited to benchmark datasets, industry knowledge bases, and preset enumeration values. The user then uses these resources to populate and assemble the prompt word template. Figure 8 The image shows the prompt jinja (a template file based on the Jinja2 engine), which constructs the initial context for data generation.

[0139] Step 2 Asset Pull and Environment Initialization: The user executes the pull command (git pull) of the version control system to synchronize the latest project dependencies from the remote code repository (AntCode).

[0140] Step library: Contains encapsulated atomic function modules, divided into input / output modules, data generation modules, data governance modules, and data evaluation modules.

[0141] Team assets: Includes shared historical Prompt templates and pipeline configuration files (pipelineyaml).

[0142] Step 3: Execution of the local pipeline: The user starts the pipeline engine locally. The engine reads the locally written or pulled pipeline configuration file (pipeline yaml) and the assembled prompt word template, loads the functional modules in the Step library, and performs data synthesis processing. This process produces two types of results: Data synthesis result: the final generated structured data file - the target synthesized dataset.

[0143] Configuration file snapshot: A YAML file used to record the parameters for this run.

[0144] Step 4 Asset Accumulation and Push: After task verification is completed, the user executes the push command ( Figure 8The `git push` command, as shown, submits the validated, valid pipeline configuration file (`pipeline yaml`) and the optimized prompt word template (`prompt jinja`) to the "team assets" directory of the remote code repository.

[0145] In this specification, the configuration items for data production are managed as code, ensuring version traceability of the Prompt and Pipeline. This allows different developers to reuse high-quality assets accumulated by the team, while supporting the development and debugging of data synthesis tasks in offline or private environments.

[0146] The following will combine Figure 9 This specification provides a detailed description of the agent-oriented synthetic data generation apparatus provided in the embodiments. It should be noted that... Figure 9 The agent-oriented synthetic data generation apparatus shown is used to execute this specification. Figures 1-8 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figures 1-8 The example shown.

[0147] Please see Figure 9 This diagram illustrates the structure of an agent-oriented synthetic data generation device according to an embodiment of this specification. This agent-oriented synthetic data generation device 1 can be implemented as all or part of a device through software, hardware, or a combination of both. According to some embodiments, the agent-oriented synthetic data generation device 1 includes an agent-oriented information acquisition module 11 and a data synthesis module 12, specifically used for: Information acquisition module 11 is used to acquire the initial sample data source and data synthesis pipeline configuration information of the synthetic data generation task. The pipeline configuration information includes at least one atomic execution step arranged according to execution dependencies. The data synthesis module 12 is used to take the initial sample data source of the synthetic data generation task as a reference, schedule and execute each of the atomic execution steps based on the execution dependency relationship to perform data synthesis processing to obtain multiple target synthetic data, and output the target synthetic dataset carrying metadata tags. The target synthetic dataset is used for large model training and / or large model evaluation of the target agent corresponding to the synthetic data generation task. Specifically, during the data synthesis process of the atomic execution step, the current task configuration context information is obtained, and the task configuration context information is written as a metadata tag into each piece of target synthesis data.

[0148] It should be noted that the above embodiments of the agent-oriented synthetic data generation apparatus, when executing the agent-oriented synthetic data generation method, are only illustrative examples of the above-described functional module divisions. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the agent-oriented synthetic data generation apparatus and the agent-oriented synthetic data generation method embodiments belong to the same concept, and their implementation process is detailed in the method embodiments, which will not be repeated here.

[0149] The example numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the examples.

[0150] This specification also provides a computer storage medium that can store multiple instructions adapted to be loaded and executed by a processor as described above. Figures 1-8 The agent-oriented synthetic data generation method described in the illustrated embodiment can be found in the following documentation for its specific execution process. Figures 1-8 The specific details of the illustrated embodiments will not be elaborated here.

[0151] This specification also provides a computer program product that stores at least one instruction, said at least one instruction being loaded and executed by the processor as described above. Figures 1-8 The agent-oriented synthetic data generation method described in the illustrated embodiment can be found in the following documentation for its specific execution process. Figures 1-8 The specific details of the illustrated embodiments will not be elaborated here.

[0152] Please refer to Figure 10 This is a structural block diagram of an electronic device provided in an embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 1010, a memory 1020, an input device 1030, an output device 1040, and a bus 1050. The processor 1010, memory 1020, input device 1030, and output device 1040 may be connected to each other via the bus 1050.

[0153] Processor 1010 may include one or more processing cores. Processor 1010 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 1020, and by calling data stored in memory 1020. Optionally, processor 1010 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 1010 may integrate one or more of a central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 1010 and may be implemented separately through a communication chip.

[0154] The memory 1020 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1020 may include non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, code, code sets, or instruction sets.

[0155] The input device 1030 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 1040 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In this embodiment, the input device 1030 can be a temperature sensor for acquiring the operating temperature of the electronic device. The output device 1040 can be a speaker for outputting audio signals.

[0156] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WIFI) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0157] In the embodiments of this specification, the executing entity for each step can be the electronic device described above. Optionally, the executing entity for each step can be the operating system of the electronic device. The operating system can be Android, iOS, or other operating systems; this specification does not limit this.

[0158] exist Figure 10 In the electronic device, the processor 1010 can be used to call a program stored in the memory 1020 and execute it to implement the agent-oriented synthetic data generation method as described in the various method embodiments of this specification.

[0159] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0160] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

Claims

1. A method for generating synthetic data for intelligent agents, the method comprising: Obtain the initial sample data source for the synthetic data generation task and obtain the data synthesis pipeline configuration information, wherein the pipeline configuration information includes at least one atomic execution step arranged according to execution dependencies; Taking the initial sample data source of the synthetic data generation task as a reference, the atomic execution steps are scheduled and executed based on the execution dependency to perform data synthesis processing to obtain multiple target synthetic data, and output the target synthetic dataset carrying metadata tags. The target synthetic dataset is used for large model training and / or large model evaluation of the target agent corresponding to the synthetic data generation task. Specifically, during the data synthesis process of the atomic execution step, the current task configuration context information is obtained, and the task configuration context information is written as a metadata tag into each piece of target synthesis data.

2. The method according to claim 1, wherein the step of scheduling and executing each of the atomic execution steps based on the execution dependency relationship to perform data synthesis processing to obtain multiple target synthetic data, with reference to the initial sample data source of the synthetic data generation task, includes: Based on the execution dependencies, the timing of the target data synthesis steps, including data generation steps, data governance steps, and data evaluation steps, is determined; Based on the timing of the target data synthesis steps and with reference to the initial sample data source of the data synthesis generation task, the data generation steps, the data governance steps, and the data evaluation steps are scheduled and executed sequentially to perform data synthesis processing.

3. The method according to claim 1 or 2, when the atomic execution step is a data generation step, the data synthesis processing includes: Obtain the preset prompt word template and the context data corresponding to the initial sample data source, and fill the context data into the prompt word template to construct synthetic data and generate prompts; Based on the synthetic data generation prompt, a preset data generation large language model interface is used to call the data generation large language model to perform batch synthetic data inference and generate a synthetic data set. The synthetic dataset is used to output the target synthetic dataset.

4. The method according to claim 1 or 2, when the atomic execution step is a data governance step, the data synthesis processing includes: Obtain the governance configuration parameters of the data governance steps and the synthetic data generated for the synthetic data generation task, and execute at least one of the following synthetic data governance logics on the synthetic data according to the governance configuration parameters: inference trace cleaning logic, structured expansion processing logic, field concatenation processing logic, semantic deduplication logic, and field retention processing logic; The inference trace cleaning logic is used to detect large model inference trace information and large model inference content in the synthetic data, and remove the large model inference trace information and the large model inference content from the synthetic data. The structured expansion processing logic is used to identify nested structure list fields in the synthetic data, and to parse the key-value content of the nested structure list fields to obtain the target synthetic data fields. The field concatenation processing logic is used to concatenate the contents of multiple source fields in the composite data according to a specified delimiter according to a preset field concatenation rule to obtain a composite data field. The semantic deduplication processing logic is used to calculate the semantic similarity of the synthesized data, and based on the semantic similarity, remove semantically duplicated synthesized data of semantic repetition type; The field retention processing logic is used to filter non-preset key fields in the synthesized data.

5. The method according to claim 1 or 2, when the atomic execution step is a data evaluation step, the data synthesis processing includes: Determine the synthetic data generated for the synthetic data generation task; The data quality of the synthesized data is evaluated by calling a large language model for data evaluation to obtain a first quality evaluation score. Low-quality data with a first quality score less than a quality score threshold is removed from the synthesized data. And / or, field input parameter verification rules are obtained, and the synthesized data is filtered for field verification using the field input parameter verification rules to obtain field quality evaluation information. Invalid data that does not meet the field verification rule type is removed from the synthesized data based on the field quality evaluation information.

6. The method according to claim 1, wherein writing the task configuration context information as metadata tags into each piece of target synthesis data includes: Detect whether the current data synthesis process has generated the target synthesized data; If the current data synthesis process generates target synthesized data, then obtain pipeline configuration information, prompt word template information used in the synthesized data, data evaluation quality information, and data synthesis metadata. The pipeline configuration information, the prompt word template information used in the synthesis data, the data evaluation quality information, and the data synthesis metadata are tagged and structured to obtain metadata tags, and the metadata tags are written into the target synthesis data.

7. The method according to claim 1 or 6, further comprising: In response to an anomaly analysis request for the evaluation metrics of the target intelligent agent, the batch of abnormal synthesized data and task configuration context information are located based on the metadata tags; Based on the metadata tags, the abnormal synthetic data batch and task configuration context information are used to initiate the abnormal attribution analysis process.

8. The method according to claim 1, wherein obtaining the data synthesis pipeline configuration information includes: Access a pre-built atomic execution step library, which includes atomic execution step components that encapsulate a unified input / output interface. These atomic execution step components include data generation components, data governance components, and data evaluation components. In response to component orchestration instructions for the atomic execution step library, multiple target atomic execution step components are selected from the atomic execution step library, and component runtime parameters and execution dependencies are configured for the target atomic execution step components; Based on the target atomic execution step component, the component's running parameters, and the execution dependencies, pipeline configuration information is generated.

9. A synthetic data generation apparatus for intelligent agents, the apparatus comprising: The information acquisition module is used to acquire the initial sample data source for the synthetic data generation task and acquire the data synthesis pipeline configuration information, wherein the pipeline configuration information includes at least one atomic execution step arranged according to execution dependencies. The data synthesis module is used to take the initial sample data source of the synthetic data generation task as a reference, schedule and execute each of the atomic execution steps based on the execution dependency to perform data synthesis processing to obtain multiple target synthetic data, and output the target synthetic dataset carrying metadata tags. The target synthetic dataset is used for large model training and / or large model evaluation of the target agent corresponding to the synthetic data generation task. Specifically, during the data synthesis process of the atomic execution step, the current task configuration context information is obtained, and the task configuration context information is written as a metadata tag into each piece of target synthesis data.

10. A computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method steps of any one of claims 1 to 8.

11. A computer program product storing at least one instruction, the at least one instruction being loaded by a processor and executing the method steps of any one of claims 1 to 8.

12. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed the method steps as claimed in any one of claims 1 to 8.

Images