Artificial intelligence-based test data generation system and method

Abstract

This application discloses an AI-based test data generation system and method. The system includes: an atomic tool library storing N atomic tools; a tool invocation rule library storing the execution order and parameter passing dependencies of multiple atomic tools under various business scenarios; and a test data generation agent for performing the following processes: semantic parsing of test data generation requests to identify user intent and extract target parameters; querying the target business scenario corresponding to the user intent from the tool invocation rule library; constructing toolchain execution instances based on the target parameters and the tool invocation rules corresponding to the target business scenario; sequentially scheduling multiple atomic tools to execute test data generation tasks according to the toolchain execution instances, and passing the output results of upstream atomic tools to downstream atomic tools; and integrating the test parameters generated by multiple atomic tools to obtain test data. This application can improve the quality and efficiency of test data generation.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and software testing technology, and in particular to an artificial intelligence-based test data generation system and method. Background Technology

[0002] As the business logic of software systems becomes increasingly complex and the iteration speed continues to accelerate, test data, as the core foundation of software testing, directly affects the testing progress and the reliability of software products through its generation efficiency, quality, and adaptability.

[0003] Traditional test data generation methods typically rely on manually writing SQL scripts and using fixed templates. This is not only time-consuming and labor-intensive, but also inefficient. In complex business scenarios, it often requires significant manpower and has a long data generation cycle. Furthermore, manual data generation is prone to inconsistencies, non-compliance with business rules, and insufficient boundary value coverage, which greatly reduces the effectiveness of the tests.

[0004] Existing test data generation tools are mostly single-function tools with high technical barriers, requiring users to master professional skills such as SQL and scripting languages. This makes it difficult for non-technical personnel to participate in test data generation, resulting in unreasonable resource allocation. Moreover, existing test data generation tools usually rely on fixed rules, lacking flexible collaborative scheduling mechanisms between tools. They are unable to cover complex business scenarios involving multiple steps and cross-modules, and have poor adaptability to new or changing business requirements. In addition, existing solutions lack effective parameter passing and process control mechanisms. Parameters require manual intervention during tool calls, which is prone to parameter omissions or errors. Furthermore, the test data generation process lacks end-to-end monitoring, making it difficult to track task execution status and promptly identify and resolve anomalies during data generation. Summary of the Invention

[0005] In view of this, embodiments of this application provide an artificial intelligence-based test data generation system and method to solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, embodiments of this application provide an artificial intelligence-based test data generation system, comprising: an atomic tool library storing N atomic tools that perform different preset basic operations, where N is an integer greater than 1; a tool invocation rule library storing tool invocation rules corresponding to various business scenarios, the tool invocation rules including the execution order and parameter passing dependencies of multiple atomic tools under the business scenario; and a test data generation intelligent agent, which performs the following processing: semantic parsing of a user-inputted test data generation request to identify the user intent and extract target parameters for generating test data; querying the target business scenario corresponding to the user intent from the tool invocation rule library; constructing a toolchain execution instance based on the target parameters and the tool invocation rules corresponding to the target business scenario; wherein, the toolchain execution instance includes an atomic tool sequence with an execution order, parameter passing dependencies of the atomic tool sequence, and initial input parameters bound to at least one atomic tool; sequentially scheduling multiple atomic tools in the atomic tool sequence to execute test data generation tasks according to the toolchain execution instance; during execution, passing the output results of upstream atomic tools as input parameters to downstream atomic tools according to parameter passing dependencies; and integrating the test parameters generated by multiple atomic tools to obtain test data.

[0007] According to some embodiments of this application, optionally, the tool invocation rule includes a directed acyclic graph, wherein nodes in the directed acyclic graph represent atomic tools, and edges represent execution dependencies and parameter passing dependencies between different atomic tools; based on the tool invocation rule corresponding to the target parameters and the target business scenario, constructing a toolchain execution instance includes: performing topological sorting on the directed acyclic graph corresponding to the target business scenario to obtain a sequence of atomic tools with execution order; extracting the parameter passing dependencies of the atomic tool sequence from the edge information of the directed acyclic graph corresponding to the target business scenario; mapping the target parameters to the initial input parameters of at least one atomic tool in the atomic tool sequence to obtain the toolchain execution instance.

[0008] According to some embodiments of this application, optionally, the output of the upstream atomic tool is passed as an input parameter to the downstream atomic tool according to the parameter passing dependency relationship, including: creating a task context object according to the toolchain execution instance; during execution, storing the initial input parameters and the execution output of the upstream atomic tool in the task context object, and retrieving the initial input parameters and / or the execution output of the upstream atomic tool from the task context object according to the parameter passing dependency relationship in the toolchain execution instance, and providing them to the currently scheduled atomic tool.

[0009] According to some embodiments of this application, optionally, the atomic tool library stores atomic tools for data generation, atomic tools for data verification, atomic tools for business rule configuration, atomic tools for data anonymization, and / or atomic tools for external service calls, each type of atomic tool including at least one atomic tool; wherein, the atomic tools for data generation are used to construct basic test parameters with a preset data structure, the atomic tools for data verification are used to verify the validity of data or business compliance, the atomic tools for business rule configuration are used to configure the business rule parameters that the test data must meet, the atomic tools for data anonymization are used to anonymize data containing sensitive information, and the atomic tools for external service calls are used to call external system interfaces to obtain data.

[0010] According to some embodiments of this application, optionally, the tool invocation rule base stores tool invocation rules corresponding to various business scenarios of the recruitment platform. These various business scenarios include at least two of the following: generating job description data, simulating resume submission process, evaluating candidate matching degree, generating interview arrangement data, and generating product order test data.

[0011] According to some embodiments of this application, optionally, the test data generation agent is further configured to perform the following processing: when an anomaly is detected during the execution of an atomic tool, a corresponding hierarchical processing strategy is executed according to the anomaly type; wherein, the hierarchical processing strategy includes at least one of the following: in response to a temporary anomaly in which the atomic tool fails to execute due to network connection or service response timeout, the execution of the atomic tool is rescheduled until the atomic tool executes successfully or reaches a preset maximum number of retries; in response to parameter anomalies in which input parameters are missing or incorrectly formatted, automatic correction or supplementation is performed based on preset parameter rules, and the corresponding atomic tool is rescheduled to execute using the corrected parameters; in response to rule anomalies in which there is a conflict with the tool invocation rules in the tool invocation rule base, the execution of the current toolchain is interrupted, and error information and correction suggestions are returned to the user.

[0012] Optionally, according to some embodiments of this application, the system further includes an atomic tool capability library, which stores capability description information for each atomic tool. The capability description information includes tool function descriptions, applicable business scenarios, input parameter modes, and output parameter modes. Semantic parsing is performed on the user-inputted test data generation request to identify the user intent. This includes: calling a large language model to perform preliminary semantic parsing on the test data generation request to obtain a preliminary user intent; based on the preliminary user intent, retrieving capability description information of atomic tools matching the preliminary user intent from the atomic tool capability library; and combining the retrieved capability description information of atomic tools to perform in-depth semantic parsing on the test data generation request to identify the final user intent and extract target parameters.

[0013] According to some embodiments of this application, optionally, the test data generating agent is also used to perform the following processing: after receiving a test data generation request input by a user, rate limiting of test data generation requests sent to the large language model is performed based on the leaky bucket algorithm.

[0014] According to some embodiments of this application, optionally, rate limiting of test data generation requests sent to a large language model is performed based on the leaky bucket algorithm, including: determining the maximum request capacity of the leaky bucket based on the estimated peak traffic; calculating the fixed leakage rate of the leaky bucket based on the preset access limit rate and security attenuation coefficient of the large language model; queuing the received test data generation requests in the leaky bucket; and extracting test data generation requests from the leaky bucket and sending them to the large language model at the fixed leakage rate, so as to achieve smooth control of access traffic.

[0015] Optionally, according to some embodiments of this application, the system further includes a storage module for storing the execution results of the test data generation task and its corresponding cache key; the test data generation agent is also used to perform the following processing: after recognizing the user intent and extracting the target parameters, performing a hash calculation on the user intent and / or target parameters to obtain the cache key; before executing the test data generation task, querying the storage module based on the cache key; if the query is successful, returning the execution result of the test data generation task; if the query is unsuccessful, executing subsequent test data generation tasks based on the user intent and target parameters, and caching the cache key and the execution result of the test data generation task to the storage module.

[0016] According to some embodiments of this application, optionally, the test data generation agent is also used to perform the following processes: after obtaining the test data, performing multi-dimensional data verification on the test data, including format verification, business rule verification and data integrity verification; converting the format of the verified test data into the target format; and generating a test data generation report based on the format-converted test data, the execution logs of each atomic tool and the verification results of the multi-dimensional data verification.

[0017] Secondly, embodiments of this application provide an artificial intelligence-based test data generation method. This method is implemented based on an artificial intelligence-based test data generation system as described in any embodiment of the first aspect. The method includes: semantically parsing a user-inputted test data generation request to identify the user's intent and extracting target parameters for generating test data; querying a target business scenario corresponding to the user's intent from a tool invocation rule base; constructing a toolchain execution instance based on the target parameters and the tool invocation rules corresponding to the target business scenario; wherein the toolchain execution instance includes an atomic tool sequence with execution order, parameter passing dependencies of the atomic tool sequence, and initial input parameters bound to at least one atomic tool; sequentially scheduling multiple atomic tools in the atomic tool sequence to execute test data generation tasks according to the toolchain execution instance; during execution, passing the output results of upstream atomic tools as input parameters to downstream atomic tools according to parameter passing dependencies; and integrating the test parameters generated by multiple atomic tools to obtain test data.

[0018] The AI-based test data generation system and method provided in this application, on the one hand, by configuring and storing an atomic tool library of N preset basic operation atomic tools and a tool call rule library covering various business scenarios, can automatically generate test data that meets user needs through a test data generation intelligent agent. This eliminates the need for manual script writing or configuring fixed templates, lowering the technical threshold. Non-technical personnel can complete test data generation simply by inputting a test data generation request, reducing manpower and time costs and improving data generation efficiency. On the other hand, by semantically parsing the user's input request, the test data generation intelligent agent can accurately identify user intent and extract target parameters. Then, it strictly constructs toolchain execution instances according to preset rules in the tool call rule library. This ensures that the invocation of atomic tools and parameter passing follow business logic throughout the process, fundamentally reducing the probability of the test data generation intelligent agent outputting test data that does not conform to business logic or outputting false test data. This effectively solves the problem of the intelligent agent randomly fabricating test data or outputting inaccurate test data, thus improving the quality of test data. On the other hand, by leveraging the explicit parameter passing dependencies within the toolchain execution instances, the output results of upstream atomic tools are automatically passed to downstream atomic tools, reducing parameter omissions or errors caused by manual intervention. Simultaneously, the structured toolchain design provides a clear tracing path for end-to-end monitoring of the test data generation process, facilitating real-time control of task execution status and timely detection and resolution of anomalies. Furthermore, when adding or changing business scenarios, only some atomic tools or the corresponding rules in the tool call rule base need to be updated, enhancing the system's adaptability to business requirements and reducing maintenance costs. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings of the embodiments of this application will be briefly described below.

[0020] Figure 1 This is a structural block diagram of an artificial intelligence-based test data generation system provided in an embodiment of this application.

[0021] Figure 2 This is a schematic diagram of an execution flow for generating an intelligent agent based on test data provided in an embodiment of this application.

[0022] Figure 3 A flowchart illustrating step S203 of the process of generating an intelligent agent from test data provided in the embodiments of this application.

[0023] Figure 4 This is a schematic diagram of an architecture for an artificial intelligence-based test data generation system provided in an embodiment of this application.

[0024] Figure 5 This is a schematic diagram of an operation of an artificial intelligence-based test data generation system provided in an embodiment of this application.

[0025] Figure 6 This is a schematic diagram of another execution flow for generating an intelligent agent based on the test data provided in the embodiments of this application. Detailed Implementation

[0026] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0027] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0028] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0029] Various modifications and variations can be made to this application without departing from its spirit or scope, which will be apparent to those skilled in the art. Therefore, this application is intended to cover modifications and variations falling within the scope of the corresponding claims (the claimed technical solutions) and their equivalents. It should be noted that the implementation methods provided in the embodiments of this application can be combined with each other without contradiction.

[0030] Before describing the technical solutions provided in the embodiments of this application, in order to facilitate understanding of the embodiments of this application, this application first specifically explains the problems existing in the related technologies: As the business logic of software systems becomes increasingly complex and the iteration speed continues to accelerate, test data, as the core foundation of software testing, directly affects the testing progress and the reliability of software products through its generation efficiency, quality, and adaptability.

[0031] Traditional test data generation methods typically rely on manually writing SQL scripts and using fixed templates. This is not only time-consuming and labor-intensive, but also inefficient. In complex business scenarios, it often requires significant manpower and has a long data generation cycle. Furthermore, manual data generation is prone to inconsistencies, non-compliance with business rules, and insufficient boundary value coverage, which greatly reduces the effectiveness of the tests.

[0032] Existing test data generation tools are mostly single-function tools with high technical barriers, requiring users to master professional skills such as SQL and scripting languages. This makes it difficult for non-technical personnel to participate in test data generation, resulting in unreasonable resource allocation. Moreover, existing test data generation tools usually rely on fixed rules, lacking flexible collaborative scheduling mechanisms between tools. They are unable to cover complex business scenarios involving multiple steps and cross-modules, and have poor adaptability to new or changing business requirements. In addition, existing solutions lack effective parameter passing and process control mechanisms. Parameters require manual intervention during tool calls, which is prone to parameter omissions or errors. Furthermore, the test data generation process lacks end-to-end monitoring, making it difficult to track task execution status and promptly identify and resolve anomalies during data generation.

[0033] To address at least one of the aforementioned technical problems, this application provides an artificial intelligence-based test data generation system and method. By encapsulating various basic operations required for test data generation (such as data verification, data construction, and parameter configuration) into independent atomic tools, a flexible atomic tool library is formed, providing fundamental support for diverse testing needs. Secondly, for different business scenarios, the execution order and parameter passing logic of the atomic tools are set and stored in a tool call rule library in a standardized rule format, clarifying the implementation path of complex business processes. The test data generation agent understands the user's test data generation request described in natural language, identifies user intent and extracts parameters, and then matches the corresponding business scenario based on the tool call rule library. It automatically constructs a toolchain execution instance containing atomic tool sequences and parameter dependencies. Finally, through automated scheduling of the toolchain and parameter pass-through, it integrates and generates test data that meets business requirements.

[0034] The embodiments of this application do not require manual script writing or configuration of fixed templates, which not only lowers the technical threshold but also improves the efficiency and quality of test data generation. At the same time, the scalable design of atomic tools and rule bases enhances the system's adaptability to different business scenarios.

[0035] The test data generation system based on artificial intelligence provided in the embodiments of this application will be introduced first.

[0036] Figure 1 This is a structural block diagram of an artificial intelligence-based test data generation system provided in an embodiment of this application. Figure 1 As shown, the AI-based test data generation system 10 provided in this application embodiment may include an atomic tool library 101, a tool invocation rule library 102, and a test data generation agent 103.

[0037] Atomic Tool Library 101 stores N atomic tools that perform different preset basic operations, where N is an integer greater than 1. Specifically, Atomic Tool Library 101 stores N (N>1) atomic tools that perform different preset basic operations. These atomic tools can cover the basic operations of the entire test data generation process and can be flexibly combined. In some examples, Atomic Tool Library 101 can store multiple types of atomic tools, such as data generation atomic tools, data validation atomic tools, business rule configuration atomic tools, data desensitization atomic tools, and / or external service call atomic tools. Each type of atomic tool can include at least one atomic tool, and each atomic tool focuses on a single-dimensional basic operation, ensuring the independence and composability of functions.

[0038] Among them, data generation atomic tools can be used to construct basic test parameters with preset data structures. Data validation atomic tools can be used to verify the validity of data or business compliance. Business rule configuration atomic tools can be used to configure business rule parameters that test data must meet. Data anonymization atomic tools can be used to anonymize data containing sensitive information. External service call atomic tools can be used to call external system interfaces to obtain data.

[0039] For example, taking recruitment platforms / scenarios as an example, atomic tools for data generation may include user information generation tools, job description (JD) generation tools, interview assessment result generation tools, order parameter generation tools, and order creation tools. For example, atomic tools for data verification may include job seeker information validity verification tools, job requirement matching verification tools, job compliance verification tools, and product information verification tools. For example, atomic tools for business rule configuration may include job salary range configuration tools, resume screening rule configuration tools, super exposure parameter configuration tools, and super exposure service activation tools. For example, atomic tools for data anonymization may include mobile phone number anonymization tools, ID card number anonymization tools, name anonymization tools, and email address anonymization tools. For example, atomic tools for external service calls may include recruitment channel data synchronization tools, educational background information verification tools, and enterprise qualification verification tools. The above atomic tools are merely illustrative examples; the type, quantity, and functional details of atomic tools can be flexibly adjusted according to actual circumstances, and this application does not impose any limitations on them.

[0040] The tool invocation rule base 102 stores tool invocation rules corresponding to various business scenarios. Each business scenario's tool invocation rule can include the execution order and parameter passing dependencies of multiple atomic tools within that scenario. For various business scenarios, tool invocation rules corresponding to each scenario are set and stored in the tool invocation rule base 102. Each tool invocation rule can include the execution order of the required atomic tools in that business scenario, such as verification tool → generation tool → configuration tool → de-identification tool, as well as parameter passing dependencies between different atomic tools, such as the product ID output by the product information verification tool serving as input parameters for the user information generation tool and the order parameter configuration tool. This ensures that the invocation of atomic tools and parameter passing follow business logic throughout the process, fundamentally reducing the probability that the test data generation agent outputs data that does not conform to business logic or outputs false test data, effectively solving the problem of agents fabricating test data or outputting inaccurate test data.

[0041] Figure 2 This is a schematic diagram of an execution flow for generating an intelligent agent based on test data provided in an embodiment of this application. (Combined with...) Figure 1 and Figure 2As shown, the test data generating agent 103 can be used to perform the following steps S201 to S204.

[0042] S201: Perform semantic parsing on the user-inputted test data generation request, identify the user's intent, and extract the target parameters used to generate the test data.

[0043] The test data generation agent 103 receives a test data generation request from the user. This request can be in natural language, such as the user inputting: "Create a super exposure order, product ID: 123456". The test data generation agent 103 performs semantic parsing of the user's input request by calling a fine-tuned large language model. On one hand, it accurately identifies the user's intent, such as identifying the user's intent as generating test data for a super exposure type product order. On the other hand, it extracts the target parameters required to generate the test data from the request, such as core business parameters (e.g., product ID: 123456), scenario attribute parameters (e.g., order type: super exposure order), and quantity parameters (e.g., order quantity: 1). These target parameters are mandatory for subsequent toolchain execution instances and scheduling atomic tools to execute test data generation tasks. When target parameters are missing, the agent can guide the user to input them through multiple rounds of questioning. For example, if the product ID is missing, the agent automatically asks: "Please supplement the product ID of the required super exposure order." By obtaining complete and accurate target parameters, the integrity and validity of input parameters when calling atomic tools can be ensured, reducing interruptions in the data generation process or data quality issues caused by missing or incorrect parameters, and ensuring that the generated test data is highly consistent with user needs.

[0044] S202: Query the target business scenario corresponding to the user intent from the tool invocation rule base; construct a toolchain execution instance based on the target parameters and the tool invocation rules corresponding to the target business scenario.

[0045] Based on the identified user intent, the test data generation agent 103 can query the tool invocation rule base 102 to find the target business scenario corresponding to the user intent, such as the test data generation scenario for super-exposure product orders. Then, the test data generation agent 103 combines the extracted target parameters and the tool invocation rules corresponding to the target business scenario to automatically construct a toolchain execution instance.

[0046] Specifically, the required atomic tools can be selected and sorted according to the execution order specified in the tool invocation rules corresponding to the target business scenario. Taking the super exposure product order test data generation scenario as an example, the atomic tool sequence can be obtained as follows: product information verification tool → user information generation tool → order parameter generation tool → super exposure parameter configuration tool → order creation tool → super exposure service activation tool.

[0047] Based on the parameter passing dependencies in the tool invocation rules corresponding to the target business scenario, the parameter passing dependencies or parameter passing paths of the atomic tool sequence can be determined. For example, the product ID is passed from the product information verification tool to the user information generation tool and the order parameter generation tool, the user ID is passed from the user information generation tool to the order parameter generation tool, and the basic order information is passed from the order parameter generation tool to the super exposure parameter configuration tool, etc.

[0048] The target parameters extracted in step S201 can be bound to at least one atomic tool in the atomic tool sequence according to parameter mapping rules. For example, the product information verification tool is bound to the initial input parameters {product_id:123456, validate_stock:true}, etc., ultimately forming a complete toolchain execution instance. The toolchain execution instance may include an atomic tool sequence with execution order, parameter passing dependencies of the atomic tool sequence, and initial input parameters bound to at least one atomic tool.

[0049] S203: According to the toolchain execution instance, multiple atomic tools in the atomic tool sequence are scheduled to execute the test data generation task in sequence; during the execution process, the output results of the upstream atomic tools are passed to the downstream atomic tools as input parameters according to the parameter passing dependency relationship.

[0050] The test data generation agent 103 can sequentially schedule multiple corresponding atomic tools in the atomic tool library 101 to execute test data generation tasks according to the execution order of the atomic tool sequence determined in the toolchain execution instance.

[0051] During execution, based on the parameter passing dependencies in the toolchain execution instances, the first atomic tool in the scheduling sequence (such as the product information verification tool) performs its operation using the bound initial input parameters and outputs the execution result, such as the product validity verification result or inventory status. Furthermore, for downstream atomic tools that depend on the output results of upstream atomic tools, the output results of the upstream atomic tools are automatically passed to the downstream atomic tools as input parameters based on the parameter passing dependencies in the toolchain execution instances, without manual intervention. For example, the user information generation tool automatically obtains the product ID output by the product information verification tool, and the order parameter generation tool automatically obtains the product ID output by the product information verification tool and the user ID output by the user information generation tool.

[0052] During the execution of atomic tools, the test data generation agent 103 can also monitor the execution status of each atomic tool in real time and record the execution logs of each atomic tool, such as execution time and execution results.

[0053] S204: Integrate test parameters generated by multiple atomic tools to obtain test data.

[0054] Once all atomic tools in the toolchain execution instance have completed execution, the test data generation agent 103 can integrate the test parameters generated by each atomic tool to obtain the test data. Taking the target business scenario of generating super-exposure product order test data as an example, the test data may include basic order information (e.g., order number: ORD20240520123456789, order time: 2024-05-20 15:30:45), product information (e.g., product ID: 123456, product name: smart test product, inventory status: sufficient), user information (e.g., user ID: USER87654321, username: test_user_001, mobile number: ...), super-exposure parameters (e.g., exposure level: high, exposure duration: 24 hours), order status (e.g., payment status: pending payment, order status: created), and super-exposure service information (e.g., service period: 1 year, usage limit: maximum 100 job super-exposures, number of uses: 0, service status: activated). The final output test data can be viewed by users or directly used in the software testing process, meeting the needs of business scenarios for test data.

[0055] The AI-based test data generation system provided in this application, on the one hand, by configuring and storing an atomic tool library of N preset basic operation atomic tools, and a tool call rule library covering various business scenarios, can automatically generate test data that meets user needs through a test data generation intelligent agent, without the need for manual script writing or configuration of fixed templates, thus lowering the technical threshold. Non-technical personnel can complete the generation of test data simply by inputting a test data generation request, reducing the human input and time cost of test data generation and improving data generation efficiency.

[0056] On the other hand, by generating intelligent agents from test data to perform semantic parsing of user input requests, it is possible to accurately identify user intent and extract target parameters. Then, by strictly following the preset rules in the tool call rule base to build toolchain execution instances, the invocation of atomic tools and parameter passing can follow business logic throughout the entire process. This fundamentally reduces the probability that the test data generating intelligent agents will output test data that does not conform to business logic or output false test data. It effectively solves the problem of intelligent agents randomly compiling test data or outputting inaccurate test data, and improves the quality of test data.

[0057] On the other hand, by leveraging the explicit parameter passing dependencies within the toolchain execution instances, the output results of upstream atomic tools are automatically passed to downstream atomic tools, reducing parameter omissions or errors caused by manual intervention. Simultaneously, the structured toolchain design provides a clear tracing path for end-to-end monitoring of the test data generation process, facilitating real-time control of task execution status and timely detection and resolution of anomalies. Furthermore, when adding or changing business scenarios, only some atomic tools or the corresponding rules in the tool call rule base need to be updated, enhancing the system's adaptability to business requirements and reducing maintenance costs.

[0058] According to some embodiments of this application, optionally, the tool invocation rules may include a directed acyclic graph (DAG). That is, the tool invocation rules corresponding to each business scenario stored in the tool invocation rule base are all presented in the form of a directed acyclic graph (DAG). Each node in the DAG corresponds to an atomic tool, and the directed edges between nodes not only represent the execution dependencies between atomic tools but also indicate parameter passing dependencies, such as the product ID output by the product information verification tool needing to be passed to the user information generation tool as an input parameter. Using a directed acyclic graph to define tool invocation rules can clearly and intuitively display the execution order and parameter dependencies of atomic tools, avoiding circular invocation problems and ensuring the logic and feasibility of toolchain execution.

[0059] Accordingly, S202: Based on the target parameters and the tool invocation rules corresponding to the target business scenario, construct a toolchain execution instance, which may include the following steps: Perform topological sorting on the directed acyclic graph corresponding to the target business scenario to obtain a sequence of atomic tools with execution order; extract the parameter transit dependencies of the atomic tool sequence from the edge information of the directed acyclic graph corresponding to the target business scenario; map the target parameters to the initial input parameters of at least one atomic tool in the atomic tool sequence to obtain a toolchain execution instance.

[0060] Specifically, taking the target business scenario of generating test data for super exposure product orders as an example, after obtaining the directed acyclic graph corresponding to the target business scenario, the directed acyclic graph is topologically sorted to obtain an atomic tool sequence with execution order, such as the product information verification tool → user information generation tool → order parameter generation tool → super exposure parameter configuration tool → order creation tool → super exposure service activation tool.

[0061] The annotation information of each directed edge in the directed acyclic graph is analyzed to extract the parameter transit dependency relationship of multiple atomic tools in the atomic tool sequence. For example, the output parameter product_id of the product information verification tool is used as the input parameter of the user information generation tool, and the output parameter user_id of the user information generation tool is used as the input parameter of the order parameter generation tool.

[0062] The target parameters extracted from the test data generation request (such as order type: super exposure order, product ID: 123456, quantity: 1) are mapped to the initial input parameters of at least one atomic tool in the atomic tool sequence (such as product information verification tool and order parameter generation tool) according to the parameter mapping rules.

[0063] In this way, the atomic tool sequence is automatically generated through topological sorting, eliminating the need for manual arrangement of tool execution order, reducing human error, and improving the efficiency and accuracy of toolchain construction. Furthermore, extracting parameter propagation dependencies based on edge information from a directed acyclic graph ensures the clarity and consistency of parameter propagation paths, reducing the possibility of parameter omissions or errors, and further improving the reliability of generated test data.

[0064] Figure 3 A flowchart illustrating step S203 of the intelligent agent execution process for generating test data provided in the embodiments of this application is shown. Figure 3 As shown, according to some embodiments of this application, optionally, S203: passing the output result of the upstream atomic tool as an input parameter to the downstream atomic tool according to the parameter passing dependency may include the following steps S301 and S302.

[0065] S301: Create a task context object based on the toolchain execution instance.

[0066] After building the toolchain execution instance, the test data generation agent can create an independent task context object based on the Model Context Protocol (MCP). This task context object can be used to store and manage the parameter information of the current test data generation task, such as initial input parameters, execution output results of each atomic tool, and tool execution status.

[0067] S302: During execution, the initial input parameters and the execution output of the upstream atomic tools are stored in the task context object. Based on the parameter passing dependencies in the toolchain execution instance, the initial input parameters and / or the execution output of the upstream atomic tools are retrieved from the task context object and provided to the currently scheduled atomic tool.

[0068] Specifically, during toolchain execution, the initial input parameters requested by the user can be stored in the task context object. When an upstream atomic tool (such as a product information verification tool) completes its execution, its output is automatically written to the task context object. Before each subsequent atomic tool executes, it can retrieve its required initial input parameters and / or the output of the upstream atomic tool from the task context object, and after execution, it stores its own output in the task context object.

[0069] If a breakpoint or retry occurs during execution, the task context object can retain the current state of all parameters. When resuming execution, the relevant parameters can be retrieved directly from the task context object without re-executing the completed atomic tools.

[0070] In this way, by uniformly managing the parameter information of the current test data generation task through the task context object, the parameter passing process requires no manual intervention. Instead, it is automatically retrieved from the context object based on parameter passing dependencies, reducing human error and improving the accuracy and efficiency of parameter passing. In addition, the task context object supports breakpoint resumption, which can retain the execution state and parameter information, improving the reliability of complex data generation tasks and reducing the cost of repetitive work caused by abnormal interruptions.

[0071] Optionally, according to some embodiments of this application, the AI-based test data generation system of this application can be applied to the generation of test data for various business scenarios of a recruitment platform. The tool invocation rule base can store tool invocation rules corresponding to various different business scenarios of the recruitment platform. For example, the various business scenarios may include at least two of the following: generating job description data, simulating resume submission processes, evaluating candidate matching, generating interview arrangement data, and generating product order test data.

[0072] The tool call rule library stores corresponding tool call rules for various business scenarios of the recruitment platform.

[0073] For example, in the scenario of generating job description data, the tool invocation rules can include the execution order of multiple atomic tools in the scenario of generating job description data, such as job category confirmation tool → job responsibility generation tool → job requirement configuration tool → salary and benefits setting tool → job posting information integration tool, as well as the parameter passing dependency relationship between job category and job responsibilities, job requirements, etc.

[0074] For example, in a simulated resume submission process scenario, the tool invocation rules can include the execution order of multiple atomic tools in the simulated resume submission process scenario, such as job seeker information generation tool → resume content construction tool → job matching preliminary verification tool → resume submission record generation tool → submission status feedback tool, as well as the parameter passing dependencies between each atomic tool.

[0075] For example, in a scenario of assessing candidate fit, the tool invocation rules can include the execution order of multiple atomic tools in the candidate fit assessment scenario, such as job requirement extraction tool → candidate resume parsing tool → core skills matching tool → work experience benchmarking tool → fit scoring tool, as well as the parameter passing dependencies between each atomic tool.

[0076] In practical use, users can input test data generation requests related to the recruitment platform, such as generating a job description for a technical position or simulating the submission of three resumes to a product manager position. The test data generation agent can match the corresponding target business scenario rules from the tool's rule library, build a toolchain execution instance, and schedule relevant atomic tools to generate test data that conforms to various business scenarios of the recruitment platform.

[0077] In this way, the tool invocation rule library stores tool invocation rules corresponding to various business scenarios of the recruitment platform, meeting the testing data needs of the recruitment platform in multiple stages such as job posting, resume submission, candidate evaluation, and interview arrangement. The scenario-based rule design means that when adding or changing recruitment business scenarios, only some atomic tools or tool invocation rules need to be updated, without refactoring the entire system, thus reducing system maintenance costs.

[0078] According to some embodiments of this application, optionally, the test data generating agent is also used to perform the following processes: When an exception is detected during the execution of an atomic tool, the corresponding hierarchical handling strategy is executed according to the exception type.

[0079] Specifically, the test data generation agent can monitor the execution status of atomic tools in real time during the scheduling of atomic tool execution. Based on log records and status feedback, it can identify anomaly types. For example, anomaly types can include temporary anomalies, parameter anomalies, or rule anomalies. Corresponding tiered processing strategies can be executed based on the anomaly type.

[0080] Accordingly, a tiered processing strategy may include at least one of the following: In response to temporary exceptions, such as when an atomic tool fails to execute due to network connection or service response timeout, the execution of the atomic tool is rescheduled until the atomic tool executes successfully or the preset maximum number of retries is reached. In response to parameter anomalies, such as missing or incorrectly formatted input parameters, the system automatically corrects or supplements the parameters based on preset rules and reschedules the execution of the corresponding atomic tools using the corrected parameters. In response to rule exceptions that conflict with tool invocation rules in the tool invocation rule base, the execution of the current toolchain is interrupted, and error information and correction suggestions are returned to the user.

[0081] Specifically, when an atomic tool fails to execute due to network connection timeouts, external service response timeouts, or other reasons, it is considered a temporary anomaly. The test data generation agent automatically triggers a retry mechanism, rescheduling the atomic tool according to an exponential backoff strategy. For example, the maximum number of retries is set to n. If it still fails after n retries, execution is paused and a breakpoint is recorded. n is a positive integer. For instance, if a product information verification tool fails to execute due to a database connection timeout, the test data generation agent automatically retryes once and executes successfully.

[0082] When abnormal input parameters are detected, such as missing parameters or incorrect formatting (e.g., entering a numeric product ID as a string), the test data generation agent automatically corrects or supplements the parameters based on preset parameter rules. For example, if the user does not enter the order quantity, the default quantity is 1; if the parameter format is incorrect, it is automatically converted to the target format, corrected, and then the corresponding atomic tool is rescheduled for execution.

[0083] If a conflict arises during tool execution with a rule in the tool's rule base (e.g., the generated order amount exceeds the product's pricing range), it is considered a rule anomaly. In this case, the test data generation agent interrupts the current toolchain execution, returns an error message to the user, and provides correction suggestions, such as adjusting the order amount to the XX-YY yuan range if it exceeds the product's pricing range.

[0084] Thus, by employing a tiered handling strategy to address different types of anomalies, the impact of anomalies on test data generation tasks is reduced, improving system stability and fault tolerance. The automatic retry mechanism for temporary anomalies allows task execution to resume without manual intervention, improving data generation efficiency. The automatic correction function for parameter anomalies reduces task failures caused by parameter issues, lowering user operating costs. Timely feedback and correction suggestions for rule anomalies help users quickly locate problems and adjust requirements, improving user experience while reducing the risk of generating test data that does not conform to business rules.

[0085] Figure 4 This is a schematic diagram of an architecture for an artificial intelligence-based test data generation system provided in an embodiment of this application. Figure 4 As shown in some embodiments of this application, optionally, the AI-based test data generation system 10 further includes an atomic tool capability library 104, which can store capability description information for each atomic tool. The capability description information may include tool function descriptions, applicable business scenarios, input parameter modes, and output parameter modes.

[0086] Taking the product information verification tool as an example, the tool's function description is to verify the validity of the product ID and the inventory status. The applicable business scenarios are product order generation and product query. The input parameter mode can be "product_id: string type, required", and the output parameter mode can be "validate_result: boolean type, stock_status: string type".

[0087] Accordingly, S201: Semantic parsing of the test data generated by the user input to identify the user's intent may include the following steps one to three.

[0088] Step 1: Call the large language model to perform preliminary semantic parsing on the test data to obtain preliminary user intent.

[0089] Receive user-inputted test data generation requests, such as creating a super exposure order with product ID: 123456. Call the fine-tuned large language model to perform preliminary semantic parsing on the test data generation request, for example, to obtain the preliminary user intent for generating product order test data.

[0090] Step 2: Based on the initial user intent, retrieve the capability description information of atomic tools that match the initial user intent from the atomic tool capability library.

[0091] For example, based on preliminary user intent, the capability description information of atomic tools related to product order generation can be retrieved from the atomic tool capability library, such as the capability description information of atomic tools like product information verification tools and user information generation tools.

[0092] Step 3: Combine the capability description information of the retrieved atomic tools to perform in-depth semantic analysis on the test data generation request, identify the final user intent and extract target parameters.

[0093] The user's original request and the capability description information of the retrieved atomic tools are input into the large language model for in-depth semantic parsing. For example, the final user intent of generating super exposure type product order test data can be accurately identified, and target parameters such as order type, product ID, and quantity can be extracted, thereby improving the accuracy of intent recognition and parameter extraction.

[0094] In this way, the atomic tool capability library provides rich tool-related knowledge support for semantic parsing, enabling large language models to more accurately understand the matching relationship between user needs and atomic tools, thereby improving the accuracy of intent recognition and the completeness of parameter extraction. Deepening the semantic parsing process reduces intent misjudgment caused by relying solely on user input, especially when user needs are vague or incomplete. It can combine tool capability information to complete the understanding of needs, reducing data generation failures due to intent recognition errors.

[0095] Given the limitations on the number of tokens and queries per minute (QPM) of the large language model, if the upstream platform calls the test data generation system in batches, it may cause the frequency of requests sent to the large language model to be too high, triggering server-side rate limiting and affecting system stability.

[0096] In view of this, according to some embodiments of this application, optionally, the test data generating agent is also used to perform the following processing: after receiving a test data generation request input by a user, rate limiting of test data generation requests sent to the large language model based on the leaky bucket algorithm.

[0097] Specifically, after receiving a test data generation request from the user, the test data generation agent does not directly send the request to the large language model. Instead, it first uses a leaky bucket algorithm for flow control. Each test data generation request is treated as a "flow" and placed into a pre-set "leaky bucket." The outlet of the leaky bucket sends the request to the large language model at a fixed rate, ensuring that the request rate to the large language model remains stable within its capacity range and avoiding triggering rate limiting.

[0098] In this way, by using the leaky bucket algorithm to rate-limit requests to the large language model, the request frequency is effectively controlled, preventing the large language model service from being throttled or crashing due to traffic spikes, thus ensuring the stable operation of the system. Furthermore, smooth traffic control allows the large language model to handle requests more evenly, reducing semantic parsing latency caused by request congestion and improving the overall system response speed.

[0099] According to some embodiments of this application, optionally, rate limiting of test data generation requests sent to a large language model based on the leaky bucket algorithm may include steps four to seven.

[0100] Step 4: Determine the maximum request capacity of the leaky bucket based on the estimated peak traffic.

[0101] The system's access traffic is generated based on historical data statistics, and the estimated peak traffic is calculated. Then, the maximum request capacity of the leaky bucket is set to "estimated peak traffic × preset multiple". The preset multiple can be greater than 1, such as 1.2, to ensure that the leaky bucket can accommodate requests during peak traffic and avoid request loss.

[0102] Step 5: Based on the preset large language model's limited access rate and security attenuation coefficient, calculate the fixed leakage rate of the leaky bucket.

[0103] The maximum number of accesses per minute for the large language model is obtained (e.g., 300 times per minute), divided by 60 to get the base rate per second, such as 5 times / second. Then, the base rate per second is multiplied by a safety factor to determine the fixed leakage rate of the leaky bucket, ensuring that the leakage rate does not exceed the carrying capacity limit of the large language model. The size of the safety factor can be flexibly adjusted according to the actual situation, and this application does not limit it. For example, in some examples, the safety factor can be 0.8.

[0104] Step 6: Place the received test data generation requests into a leaky bucket for queuing.

[0105] All received test data generation requests are first queued in a leaky bucket for processing.

[0106] Step 7: Extract test data from the leak bucket at a fixed leakage rate, generate requests, and send them to the large language model to achieve smooth control of access traffic.

[0107] At a fixed leakage rate, test data is extracted from the leak bucket to generate requests and sent to the large language model for semantic parsing. If the number of test data generation requests exceeds the maximum request capacity, the excess test data generation requests will be temporarily cached and processed when there is free capacity in the leak bucket.

[0108] In this way, by configuring the maximum request capacity of the leaky bucket and a fixed leakage rate, fine-grained control of access traffic is achieved. This ensures that requests are not lost during traffic peaks while preventing the request rate from exceeding the capacity of the large language model. The fixed leakage rate ensures that requests sent to the large language model are evenly distributed, avoiding a sudden surge in service pressure caused by a concentrated burst of requests, and further improving the stability and reliability of the system.

[0109] Optionally, according to some embodiments of this application, the AI-based test data generation system 10 may further include a storage module, which can be used to store the execution results of the test data generation task and its corresponding cache key. For example, the execution results of the test data generation task may include test data or a test data generation report, etc.

[0110] Accordingly, the test data generating agent can also be used to perform steps eight and nine.

[0111] Step 8: After identifying the user intent and extracting the target parameters, perform a hash calculation on the user intent and / or target parameters to obtain the cache key.

[0112] After the test data generating agent completes user intent recognition and target parameter extraction, it performs hash calculations on the clearly defined user intent (such as generating resumes for recent graduates) and / or extracted target parameters (such as data type: resumes for recent graduates, quantity: 2) to generate a unique corresponding cache key. For example, if a user enters "Help me prepare 2 resumes for recent graduates" or "Help me find 2 resumes for recent graduates," although the two statements are different, after intent recognition, both user intents are the same: to generate 2 resumes for recent graduates, and / or the target parameters are also the same, therefore the same cache key will be generated.

[0113] Step 9: Before executing the test data generation task, query the storage module based on the cache key. If the query is successful, return the execution result of the test data generation task. If the query is unsuccessful, execute the subsequent test data generation task based on the user intent and target parameters, and cache the cache key and the execution result of the test data generation task in the storage module.

[0114] Figure 5 This is a schematic diagram illustrating the operation of an artificial intelligence-based test data generation system provided in an embodiment of this application. Figure 5 As shown, the cache key obtained in step eight can be queried in the storage module. If the query is successful, meaning the test data generation task corresponding to the user intent has been processed and the cache has not expired, the corresponding execution result is directly retrieved from the storage module and returned to the user, without repeating the toolchain construction and tool scheduling processes. If the query is unsuccessful, the subsequent test data generation task is executed based on the identified user intent and target parameters. After generating the test data, the cache key and the corresponding execution result are stored in the storage module.

[0115] In some examples, if the test data generation task executes successfully, the result can be cached for a first preset duration, such as 30 minutes. If the test data generation task fails, the result can be cached for a second preset duration, such as 2 minutes. The first preset duration can be longer than the second preset duration to avoid cache penetration.

[0116] Thus, the caching mechanism reuses results based on user intent rather than the original request text, allowing different expressions of the same intent to match the same cached result, further improving the cache hit rate. Simultaneously, the introduction of the caching mechanism reduces subsequent processes such as toolchain construction and tool scheduling for duplicate requests, lowering system resource consumption. Duplicate requests can directly return cached results, improving the efficiency of batch test case execution, shortening response time, and enhancing user experience. Furthermore, successful and failed results employ different caching duration strategies, ensuring the reuse of valid results while avoiding erroneous feedback caused by long-term caching of failed results, thus balancing caching efficiency and accuracy.

[0117] Figure 6 This is a schematic diagram illustrating another execution flow for generating an intelligent agent from the test data provided in the embodiments of this application. For example... Figure 6 As shown, according to some embodiments of this application, optionally, the test data generating agent is also used to perform the following steps S601 to S603.

[0118] S601: After obtaining the test data, perform multi-dimensional data verification on the test data. Multi-dimensional data verification includes format verification, business rule verification, and data integrity verification.

[0119] Format validation can include checking whether the field types, number of fields, and format of the test data meet preset requirements according to the preset JSON Schema specification. For example, whether the order number is a string type and whether it contains required fields.

[0120] Business rule validation can include verifying whether test data conforms to business logic. For example, whether the product ID is valid, whether the order type matches the super exposure parameters, and whether the order amount is within a reasonable range.

[0121] Data integrity verification can include checking whether all required fields are filled and whether there are any missing or empty values.

[0122] S602: Convert the format of the validated test data to the target format.

[0123] For the validated test data, the format can be converted to the target format, such as JSON or Excel, according to the user's requirements or the requirements of the testing tool.

[0124] S603: Generate a test data generation report based on the format-converted test data, the execution logs of each atomic tool, and the verification results of multi-dimensional data verification.

[0125] The execution logs of each atomic tool, the verification results of multi-dimensional data validation, and the format-converted test data are combined to generate a structured test data generation report. This report displays test data details, execution status, and verification results.

[0126] In this way, multi-dimensional data verification can promptly identify formatting errors, business logic conflicts, and missing data in the test data, further improving the quality of the test data and reducing test invalidation caused by data issues. The test data generation report provides users with a clear traceability of the data creation process and a display of results, facilitating user verification of data validity.

[0127] According to some embodiments of this application, optionally, the test data generating agent can also be used to perform the following processes: Before outputting the test data, the data masking tool in the Atomic Tool Library is used to mask the test data. Masking can include mask replacement, character replacement, or encryption conversion. Different masking rules, or the same rules, can be applied to different types of data or parameters in the test data.

[0128] For example, for mobile phone numbers, a masking algorithm can be used to retain the first 3 and last 4 digits, while replacing the middle 4 digits with asterisks. For desensitizing ID card numbers, the middle 8 digits of the birth date can be hidden, retaining the first 6 digits of the administrative division code and the last 4 digits of the check digit, or an encryption algorithm can be used to convert the real ID card number into an irreversible virtual identifier. For information such as names and email addresses, character replacement or anonymization can be used, such as hiding the middle characters of the name or the middle characters of the username in the email address.

[0129] In this way, by calling the data desensitization tools in the atomic tool library to desensitize the test data, we can achieve precise protection of various sensitive information, reduce the risk of sensitive information leakage, and ensure that the generated test data not only meets the business testing needs, but also does not contain real privacy information.

[0130] According to some embodiments of this application, optionally, the test data generating agent can also be used to perform the following processes: Record all operations related to test data generation, such as user identity information, data generation request content, tool call records, parameter configuration details, data generation results, and operation timestamps. Each audit log is bound to a unique trace identifier (traceId), supporting full-process traceability from user request initiation to data output completion. The retention period of audit logs can be flexibly set according to actual needs to meet relevant compliance traceability requirements.

[0131] According to some embodiments of this application, optionally, the test data generating agent can also be used to perform the following processes: Establish a monitoring system covering the entire process of test data generation, which includes at least the following stages: request reception, semantic parsing, toolchain construction, tool scheduling, and result output; collect and statistically analyze performance, quality, and resource metrics throughout the entire process.

[0132] Specifically, a monitoring system can be built that covers the entire test data generation process, including "request reception - semantic parsing - toolchain construction - tool scheduling - data generation - result output". Performance metrics, quality metrics, and resource metrics throughout the entire process can be collected and statistically analyzed.

[0133] Performance metrics can include statistics on response time at each stage (such as semantic parsing time, tool call response time, and total data generation time), system throughput, and concurrent request count, enabling real-time monitoring of system processing capacity. Quality metrics can include tracking data generation success rate, intent recognition accuracy, parameter extraction completeness rate, and data validation pass rate, evaluating data generation quality. Resource metrics can include monitoring server CPU utilization, memory usage, disk storage utilization, network bandwidth usage, and the resource consumption status of large language models and atomic tools, ensuring a stable supply of system resources.

[0134] According to some embodiments of this application, optionally, the test data generating agent can also be used to perform the following processes: Configure an alarm mechanism and preset alarm thresholds for monitoring indicators; when any monitoring indicator triggers its alarm threshold, an alarm message is automatically generated and sent to the operation and maintenance personnel via at least one of the following channels: email, SMS, or instant messaging tools.

[0135] Specifically, alarm thresholds can be preset for monitoring metrics, such as response time exceeding 5 seconds, data generation success rate below 90%, and CPU utilization exceeding 85%. When a monitoring metric triggers the corresponding alarm threshold, the test data generation agent automatically triggers an alarm, pushing alarm information to operations and maintenance personnel via at least one channel, such as email, SMS, or instant messaging tools. Alarm information can include alarm type, triggering metric, abnormal data, occurrence time, and preliminary handling suggestions. Alarm classification is also supported, such as urgent, important, and general. For high-priority alarms, such as system downtime or large-scale data generation failures, multi-channel synchronous notifications will be triggered to ensure timely response from operations and maintenance personnel.

[0136] In this way, the combination of preset alarm thresholds and a tiered notification mechanism enables rapid early warning of abnormal situations. Multiple channels for pushing alarm information (email, SMS, instant messaging tools, etc.) ensure that operations and maintenance personnel receive anomaly notifications promptly, avoiding delays in handling.

[0137] According to some embodiments of this application, optionally, the test data generating agent can also be used to perform the following processes: A standardized error code system is constructed, which includes multiple error codes categorized according to error type. Each error code is associated with an error cause and a solution. When an error occurs during system operation, the original error information is mapped to the corresponding standardized error code and descriptive information for output.

[0138] Specifically, a standardized error code system covering all scenarios can be constructed. This system can be categorized by error type, such as semantic parsing errors, tool invocation errors, parameter errors, compliance errors, and resource insufficiency errors. Each error code is associated with its cause, scope of impact, and solution. For example, error code E101 corresponds to semantic parsing failure, caused by an unclear user requirement or a scenario beyond the system's support; the solution is to prompt the user to provide more details. Error code E203 corresponds to tool invocation timeout, caused by network anomalies or a lack of response from the tool service; the solution is to trigger automatic retry or check the tool service status. When an error occurs, the system automatically converts the original error information into standardized error codes and descriptions, returning them to users and operations personnel.

[0139] In this way, by standardizing the output format of error messages, each error code clearly associates with the error cause, scope of impact, and solution, reducing the difficulty for users and maintenance personnel to understand and locate errors, and reducing the cost of problem communication.

[0140] According to some embodiments of this application, optionally, the AI-based test data generation system is configured with a visual monitoring interface to dynamically display the historical trends and real-time status of performance indicators, quality indicators, and resource indicators in the form of charts; it also supports querying and retrieving the corresponding detailed execution logs through end-to-end tracing identifiers.

[0141] For example, in some examples, the visual monitoring interface can present the historical trends and real-time fluctuations of performance metrics such as response time and throughput using line charts. It can compare quality metrics such as data generation success rate and data validation pass rate across different time periods and business scenarios using bar charts. Pie charts can visually display the allocation percentages of resource metrics such as CPU utilization and memory usage, helping operations and maintenance personnel quickly grasp the system's operational status.

[0142] Furthermore, it supports operations and maintenance personnel to perform combined queries using multiple dimensions such as traceId, user ID, operation time range, and business scenario type to query the full-link execution log of the target test data generation task. The full-link execution log can include information such as request parameters, tool call records, parameter passing details, exception triggering nodes, and processing results.

[0143] In addition, it supports custom monitoring panel configuration. Operation and maintenance personnel can choose the type of indicators, chart style and data refresh frequency to be displayed according to different scenarios such as daily operation and maintenance, troubleshooting and performance optimization, and flexibly build a special monitoring view to adapt to diverse operation and maintenance needs.

[0144] Thus, the constructed visual monitoring interface presents the real-time status and historical trends of various indicators in an intuitive chart format, reducing the understanding cost of system operation data for maintenance personnel and realizing visualized management of system status. The end-to-end log query function supports precise retrieval of execution logs through multi-dimensional conditions, helping maintenance personnel quickly locate abnormal nodes and root causes in the data generation process, shortening troubleshooting time. Customizable monitoring panels meet the personalized needs of different maintenance scenarios, improving the flexibility and efficiency of maintenance operations.

[0145] Based on the AI-based test data generation system 10 provided in any of the above embodiments, this application also provides an AI-based test data generation method. This method can be implemented based on the AI-based test data generation system 10 provided in any of the above embodiments.

[0146] The test data generation method based on artificial intelligence provided in this application embodiment may include the following steps: Semantic parsing is performed on the user-inputted test data generation request to identify the user's intent and extract the target parameters used to generate the test data. From the tool's call rule base, query the target business scenario corresponding to the user's intent; Based on the tool invocation rules corresponding to the target parameters and target business scenarios, a toolchain execution instance is constructed; wherein, the toolchain execution instance includes an atomic tool sequence with execution order, the parameter passing dependencies of the atomic tool sequence, and the initial input parameters bound to at least one atomic tool; According to the toolchain execution instance, multiple atomic tools in the atomic tool sequence are scheduled to execute the test data generation task in sequence; during the execution process, the output results of the upstream atomic tools are passed to the downstream atomic tools as input parameters according to the parameter passing dependencies; Test parameters generated by multiple atomic tools are integrated to obtain test data.

[0147] The specific processes of each of the above steps have been described in detail above and will not be repeated here.

[0148] The test data generation method based on artificial intelligence provided in this application embodiment, on the one hand, by configuring and storing an atomic tool library of N preset basic operation atomic tools, and a tool call rule library covering various business scenarios, the test data generation intelligent agent can automatically generate test data that meets user needs without the need for manual script writing or configuration of fixed templates, thus lowering the technical threshold. Non-technical personnel can complete the generation of test data simply by inputting a test data generation request, reducing the human input and time cost of test data generation and improving data generation efficiency.

[0149] On the other hand, by generating intelligent agents from test data to perform semantic parsing of user input requests, it is possible to accurately identify user intent and extract target parameters. Then, by strictly following the preset rules in the tool call rule base to build toolchain execution instances, the invocation of atomic tools and parameter passing can follow business logic throughout the entire process. This fundamentally reduces the probability that the test data generating intelligent agents will output test data that does not conform to business logic or output false test data. It effectively solves the problem of intelligent agents randomly compiling test data or outputting inaccurate test data, and improves the quality of test data.

[0150] On the other hand, by leveraging the explicit parameter passing dependencies within the toolchain execution instances, the output results of upstream atomic tools are automatically passed to downstream atomic tools, reducing parameter omissions or errors caused by manual intervention. Simultaneously, the structured toolchain design provides a clear tracing path for end-to-end monitoring of the test data generation process, facilitating real-time control of task execution status and timely detection and resolution of anomalies. Furthermore, when adding or changing business scenarios, only some atomic tools or the corresponding rules in the tool call rule base need to be updated, enhancing the system's adaptability to business requirements and reducing maintenance costs.

[0151] It should be noted that the AI-based test data generation method provided in this application has the same or corresponding technical features as the AI-based test data generation system 10 provided in any of the above embodiments, and produces the same technical effects. For the sake of brevity, further details are omitted here.

[0152] The flowcharts and / or block diagrams of methods and systems according to embodiments of this application have been described above by way of example, and related aspects have been described. It should be understood that each block or combination thereof in the flowcharts and / or block diagrams may be implemented by computer program instructions, by dedicated hardware performing a specified function or action, or by a combination of dedicated hardware and computer instructions. For example, these computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to form a machine such that these instructions, which execute via such processor, enable the implementation of the function / action specified in each block or combination thereof in the flowcharts and / or block diagrams. Such a processor may be a general-purpose processor, a dedicated processor, a special-purpose application processor, or a field-programmable logic circuit.

[0153] The functional blocks shown in the structural block diagrams of this application can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc.; when implemented in software, they are programs or code segments used to perform the required tasks. Programs or code segments can be stored in memory or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. Code segments can be downloaded via computer networks such as the Internet or intranets.

[0154] It should be noted that this application is not limited to the specific configurations and processes described above or shown in the figures. The above descriptions are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the described systems, devices, modules, or units can be referred to the corresponding processes in the method embodiments, and need not be repeated here. It should be understood that the scope of protection of this application is not limited thereto. Any person skilled in the art can conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application.

Claims

1. A test data generation system based on artificial intelligence, characterized in that, include: The atomic tool library stores N atomic tools that perform different preset basic operations, where N is an integer greater than 1; The tool invocation rule base stores tool invocation rules corresponding to various business scenarios. The tool invocation rules include the execution order and parameter passing dependencies of multiple atomic tools in the business scenario. A test data generation agent is used to perform the following processes: Semantic parsing is performed on the user-inputted test data generation request to identify the user's intent and extract the target parameters used to generate the test data. From the tool invocation rule base, query the target business scenario corresponding to the user intent; based on the target parameters and the tool invocation rules corresponding to the target business scenario, construct a toolchain execution instance; wherein, the toolchain execution instance includes an atomic tool sequence with execution order, the parameter passing dependency relationship of the atomic tool sequence, and the initial input parameters bound to at least one atomic tool; According to the toolchain execution instance, multiple atomic tools in the atomic tool sequence are scheduled to execute the test data generation task in sequence; during the execution process, the output results of the upstream atomic tools are passed to the downstream atomic tools as input parameters according to the parameter passing dependencies; Test parameters generated by multiple atomic tools are integrated to obtain test data.

2. The system according to claim 1, characterized in that, The tool invocation rules include a directed acyclic graph, where nodes in the directed acyclic graph represent atomic tools, and edges represent execution dependencies and parameter passing dependencies between different atomic tools; Based on the target parameters and the tool invocation rules corresponding to the target business scenario, construct toolchain execution instances, including: Perform topological sorting on the directed acyclic graph corresponding to the target business scenario to obtain a sequence of atomic tools with execution order; extract the parameter transit dependencies of the atomic tool sequence from the edge information of the directed acyclic graph corresponding to the target business scenario; map the target parameters to the initial input parameters of at least one atomic tool in the atomic tool sequence to obtain a toolchain execution instance.

3. The system according to claim 1, characterized in that, Based on parameter transitivity, the output of the upstream atomic tool is passed through as input parameters to the downstream atomic tool, including: Based on the toolchain execution instance, create a task context object; During execution, the initial input parameters and the execution output of the upstream atomic tools are stored in the task context object. Based on the parameter passing dependencies in the toolchain execution instance, the initial input parameters and / or the execution output of the upstream atomic tools are retrieved from the task context object and provided to the currently scheduled atomic tool.

4. The system according to claim 1, characterized in that, The atomic tool library stores atomic tools for data generation, data verification, business rule configuration, data desensitization, and / or external service call, with each type of atomic tool including at least one atomic tool. Among them, the data generation atomic tools are used to construct basic test parameters with preset data structures, the data verification atomic tools are used to verify the validity of data or business compliance, the business rule configuration atomic tools are used to configure the business rule parameters that the test data must meet, the data desensitization atomic tools are used to desensitize data containing sensitive information, and the external service call atomic tools are used to call external system interfaces to obtain data.

5. The system according to claim 1, characterized in that, The tool invocation rule library stores tool invocation rules corresponding to various business scenarios of the recruitment platform. These various business scenarios include at least two of the following: generating job description data, simulating resume submission process, evaluating candidate matching degree, generating interview arrangement data, and generating product order test data.

6. The system according to claim 1, characterized in that, The test data generating agent is also used to perform the following processes: When an exception is detected during the execution of an atomic tool, the corresponding hierarchical handling strategy is executed according to the exception type. The hierarchical processing strategy includes at least one of the following: In response to a temporary exception where an atomic tool fails to execute due to network connectivity or service response timeout, the execution of the atomic tool is rescheduled until the atomic tool executes successfully or the preset maximum number of retries is reached. In response to parameter anomalies such as missing or incorrectly formatted input parameters, the system automatically corrects or supplements the parameters based on preset rules and reschedules the execution of the corresponding atomic tools using the corrected parameters. In response to a rule exception that conflicts with a tool invocation rule in the tool invocation rule base, the execution of the current toolchain is interrupted, and an error message and correction suggestions are returned to the user.

7. The system according to claim 1, characterized in that, The system also includes an atomic tool capability library, which stores capability description information for each atomic tool. The capability description information includes tool function description, applicable business scenarios, input parameter mode, and output parameter mode. Semantic parsing is performed on user-inputted test data to generate requests and identify user intent, including: The large language model is invoked to generate a request based on the test data and perform preliminary semantic parsing to obtain the initial user intent. Based on the initial user intent, retrieve the capability description information of the atomic tools that match the initial user intent from the atomic tool capability library; By combining the capability description information of the retrieved atomic tools, the test data generation request is subjected to in-depth semantic analysis to identify the final user intent and extract target parameters.

8. The system according to claim 7, characterized in that, The test data generating agent is also used to perform the following processes: After receiving a test data generation request from the user, the test data generation requests sent to the large language model are rate-limited based on the leaky bucket algorithm.

9. The system according to claim 8, characterized in that, Rate limiting is applied to test data generation requests sent to the large language model based on the leaky bucket algorithm, including: Determine the maximum request capacity of the leaky bucket based on the estimated peak traffic. Based on the preset large language model, the access rate limit and security attenuation coefficient are used to calculate the fixed leakage rate of the leaky bucket. The received test data generation requests are placed into a leaky bucket for queuing; At the fixed leakage rate, test data is extracted from the leak bucket to generate a request and sent to the large language model to achieve smooth control of access traffic.

10. The system according to claim 1, characterized in that, The system also includes a storage module, which is used to store the execution results of the test data generation task and its corresponding cache key; The test data generating agent is also used to perform the following processes: After identifying the user intent and extracting the target parameters, a hash calculation is performed on the user intent and / or target parameters to obtain the cache key; Before executing the test data generation task, a query is performed in the storage module based on the cache key. If the query finds a match, the execution result of the test data generation task is returned. If the query fails, the subsequent test data generation task is executed based on the user intent and target parameters, and the cache key and the execution result of the test data generation task are cached in the storage module.

11. The system according to claim 1, characterized in that, The test data generating agent is also used to perform the following processes: After obtaining the test data, the test data is subjected to multi-dimensional data verification, which includes format verification, business rule verification and data integrity verification. Convert the format of the validated test data to the target format; Based on the format-converted test data, the execution logs of each atomic tool, and the verification results of the multi-dimensional data verification, a test data generation report is generated.

12. A test data generation method based on artificial intelligence, characterized in that, The method is implemented based on the artificial intelligence-based test data generation system as described in any one of claims 1-11, and the method includes: Semantic parsing is performed on the user-inputted test data generation request to identify the user's intent and extract the target parameters used to generate the test data. From the tool's call rule base, query the target business scenario corresponding to the user's intent; Based on the tool invocation rules corresponding to the target parameters and target business scenarios, a toolchain execution instance is constructed; wherein, the toolchain execution instance includes an atomic tool sequence with execution order, the parameter passing dependencies of the atomic tool sequence, and the initial input parameters bound to at least one atomic tool; According to the toolchain execution instance, multiple atomic tools in the atomic tool sequence are scheduled to execute the test data generation task in sequence; during the execution process, the output results of the upstream atomic tools are passed to the downstream atomic tools as input parameters according to the parameter passing dependencies; Test parameters generated by multiple atomic tools are integrated to obtain test data.

Images