Test method and device based on intelligent agent and storage medium
By using an agent-based testing method, test context information is generated and actions are executed iteratively, solving the problems of unmanned and adaptive automation in existing technologies. This achieves efficient and accurate intelligent testing, improving the flexibility and adaptability of the testing system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-09
- Publication Date
- 2026-04-03
AI Technical Summary
Existing automated testing technologies cannot achieve fully unmanned, adaptive, and semantically driven intelligent testing, resulting in difficulties in parsing test requirements, high maintenance costs, incomplete coverage or redundant execution, and a lack of autonomous decision-making capabilities.
The test method adopts an agent-based approach. By generating test context information corresponding to test requirements, the pre-trained agent performs autonomous parsing and dynamic decision-making, driving end-to-end automation of the entire test process, generating the current action and iteratively executing it until the test ends.
It achieves fully automated, adaptive, and high-precision intelligent testing from natural language requirements to test reports, improving testing efficiency, coverage, and maintainability. It also has dynamic adaptive and context-aware capabilities, enhancing the robustness and cross-scenario migration capabilities of the testing system.
Smart Images
Figure CN121785908A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of automated testing technology, and more specifically, to a test method, device, and storage medium based on intelligent agents. Background Technology
[0002] With the increasing complexity of software systems, especially the widespread application of large-scale information systems such as enterprise-level business management systems and industrial IoT platforms, testing requirements are becoming increasingly multi-dimensional, highly dynamic, and strongly interconnected. Particularly in the field of automated testing, achieving end-to-end automation from original requirements documents to test execution has become a key challenge for improving testing efficiency and coverage.
[0003] Currently, mainstream automated testing technologies typically rely on manually written or tool-generated static test cases, driven by pre-set scripts. In practical applications, testers must first transform the requirements described in natural language into structured test cases, clarifying the input conditions, operation steps, and expected results, before configuring the corresponding test data and execution environment.
[0004] However, current technologies are not yet able to achieve truly unmanned, adaptive, and semantically driven intelligent testing throughout the entire process. Summary of the Invention
[0005] The purpose of this application is to address the shortcomings of the prior art by providing an agent-based testing method, device, and storage medium, thereby solving the problem that the prior art cannot achieve truly unmanned, adaptive, and semantically driven intelligent testing throughout the entire process.
[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, one embodiment of this application provides a test method based on an intelligent agent, applied to a test system, the test system including at least one execution tool and at least one result extraction tool, the method comprising: Generate test context information corresponding to the test requirements based on the test requirements; The test context information is input into the pre-trained agent, which generates the current action based on the test context information, calls at least one of the target execution tools, and the target execution tool performs the test of the current action to obtain and output the current test result. The result extraction tool is invoked based on the current test results, and the result extraction tool extracts the results to be verified from the current test results; The result to be verified is input into the agent, which determines whether to end the test based on the result. If so, a test report corresponding to the test requirement is generated. Otherwise, the agent generates the next action of the current action based on the test context information and the result to be verified, and calls the execution tool and the result extraction tool to iterate until the agent determines to end the test.
[0007] Optionally, generating the test context information corresponding to the test requirements based on the test requirements includes: Based on the test requirements, structured test information corresponding to the test requirements is generated. The test requirements are represented by natural language, and the test information includes: test actions, test objects, and key parameters. Based on the test information, generate test context information corresponding to the test requirements.
[0008] Optionally, generating test context information corresponding to the test requirement based on the test information includes: Based on the test information, at least one reference test sample corresponding to the test requirement is retrieved from a preset historical test sample library; Based on the test information, at least one test dataset corresponding to the test requirement is retrieved from a preset test database; Based on the test information, the reference test examples, and the test datasets, test context information corresponding to the test requirements is generated.
[0009] Optionally, retrieving at least one reference test sample corresponding to the test requirement from a preset historical test sample library based on the test information includes: Based on the test information, generate a first semantic vector corresponding to the test requirement; Based on the first semantic vector, at least one reference test sample is retrieved from the preset historical test sample library.
[0010] Optionally, retrieving at least one test dataset corresponding to the test requirement from a preset test database based on the test information includes: Based on the test information, a second semantic vector corresponding to the test requirement is generated; Based on the second semantic vector, at least one candidate dataset is retrieved from the preset test database; Based on each of the candidate datasets, at least one test dataset corresponding to the test requirement is determined.
[0011] Optionally, determining at least one test dataset corresponding to the test requirement based on each of the candidate datasets includes: From the multiple pre-constructed parameter inheritance trees, determine the target parameter inheritance tree corresponding to the test requirement; Based on the target parameter inheritance tree, each of the candidate datasets is filtered to obtain the target candidate dataset; Based on the target candidate dataset, at least one of the test datasets is determined.
[0012] Optionally, determining at least one of the test datasets based on the target candidate datasets includes: Based on the target candidate dataset, the current node is found from the target parameter inheritance tree; At least one test dataset is determined based on the current node, at least one parent node of the current node, and / or at least one child node of the current node.
[0013] Optionally, the step of invoking the result extraction tool based on the current test result includes: Determine the test type corresponding to the current test result. The test type includes at least one of the following: content comparison, path verification, visual feature, and status code. Based on the test type, a call request is sent to at least one of the target result extraction tools in the result extraction tools, the call request including the test type and the current test result; The target result extraction tool extracts the result to be verified from the current test results based on the call request.
[0014] Secondly, another embodiment of this application provides an agent-based testing device applied to a testing system, the testing system including at least one execution tool and at least one result extraction tool, the device comprising: The first generation module is used to generate test context information corresponding to the test requirements based on the test requirements. The second generation module is used to input the test context information into a pre-trained agent, which generates the current action based on the test context information, calls at least one target execution tool in the execution tools, and the target execution tool performs the test of the current action to obtain and output the current test result. The extraction module is used to call the result extraction tool based on the current test result, and the result extraction tool extracts the result to be verified from the current test result; The determination module is used to input the result to be verified into the agent, and the agent determines whether to end the test based on the result to be verified. If so, a test report corresponding to the test requirement is generated. Otherwise, the agent generates the next action of the current action based on the test context information and the result to be verified, and calls the execution tool and the result extraction tool to iterate until the agent determines to end the test.
[0015] Optionally, the first generation module is specifically used for: Based on the test requirements, structured test information corresponding to the test requirements is generated. The test requirements are represented by natural language, and the test information includes: test actions, test objects, and key parameters. Based on the test information, generate test context information corresponding to the test requirements.
[0016] Optionally, the first generation module is specifically used for: Based on the test information, at least one reference test sample corresponding to the test requirement is retrieved from a preset historical test sample library; Based on the test information, at least one test dataset corresponding to the test requirement is retrieved from a preset test database; Based on the test information, the reference test examples, and the test datasets, test context information corresponding to the test requirements is generated.
[0017] Optionally, the first generation module is specifically used for: Based on the test information, generate a first semantic vector corresponding to the test requirement; Based on the first semantic vector, at least one reference test sample is retrieved from the preset historical test sample library.
[0018] Optionally, the first generation module is specifically used for: Based on the test information, a second semantic vector corresponding to the test requirement is generated; Based on the second semantic vector, at least one candidate dataset is retrieved from the preset test database; Based on each of the candidate datasets, at least one test dataset corresponding to the test requirement is determined.
[0019] Optionally, the first generation module is specifically used for: From the multiple pre-constructed parameter inheritance trees, determine the target parameter inheritance tree corresponding to the test requirement; Based on the target parameter inheritance tree, each of the candidate datasets is filtered to obtain the target candidate dataset; Based on the target candidate dataset, at least one of the test datasets is determined.
[0020] Optionally, the first generation module is specifically used for: Based on the target candidate dataset, the current node is found from the target parameter inheritance tree; At least one test dataset is determined based on the current node, at least one parent node of the current node, and / or at least one child node of the current node.
[0021] Optionally, the extraction module is specifically used for: Determine the test type corresponding to the current test result. The test type includes at least one of the following: content comparison, path verification, visual feature, and status code. Based on the test type, a call request is sent to at least one of the target result extraction tools in the result extraction tools, the call request including the test type and the current test result; The target result extraction tool extracts the result to be verified from the current test results based on the call request.
[0022] Thirdly, another embodiment of this application provides an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of any of the methods described in the first aspect above.
[0023] Fourthly, another embodiment of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of any of the methods described in the first aspect above.
[0024] The beneficial effects of this application are as follows: By generating test context information corresponding to test requirements through test requirements, natural language requirements can be transformed into structured input. The test context information is then input into a pre-trained agent, which generates the current action based on the test context information, calls at least one target execution tool in the execution tool, and the target execution tool executes the test of the current action, obtains and outputs the current test result, calls the result extraction tool based on the current test result, extracts the result to be verified from the current test result, inputs the result to be verified into the agent, and the agent determines whether to end the test based on the result to be verified. If so, a test report corresponding to the test requirements is generated; otherwise, the agent generates the next action of the current action based on the test context information and the result to be verified, calls the execution tool and the result extraction tool, and iterates until the agent determines to end the test. This achieves fully unmanned, adaptive, and high-precision intelligent testing from natural language requirements to test report generation, significantly improving test efficiency, coverage, and maintainability.
[0025] Furthermore, during testing, the agent generates only the current action at a time, rather than planning a complete chain of test actions all at once. This allows the agent to adjust the generation of the current action based on the results of the previous action, thus avoiding the rigidity of pre-defined paths and achieving dynamic adaptation and context awareness. It also enhances the agent's ability to conduct exploratory testing, significantly improving test coverage and completeness. In addition, it improves the robustness and fault tolerance of the testing system, and possesses good scalability and cross-scenario migration capabilities, making it suitable for intelligent testing of various types of systems under test. Attached Figure Description
[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 A system architecture diagram of a testing system involved in the agent-based testing method provided in the embodiments of this application; Figure 2 A flowchart illustrating an agent-based testing method provided in an embodiment of this application; Figure 3 This is a flowchart illustrating the process of generating test context information corresponding to test requirements in the agent-based testing method provided in this application embodiment; Figure 4This is another flowchart illustrating the generation of test context information corresponding to test requirements in the agent-based testing method provided in this application embodiment; Figure 5 This is a schematic diagram of a process for obtaining at least one reference test example corresponding to the test requirements in the agent-based testing method provided in the embodiments of this application. Figure 6 A schematic diagram of a process for retrieving at least one test dataset corresponding to a test requirement in the agent-based testing method provided in the embodiments of this application; Figure 7 This is a flowchart illustrating the process of determining at least one test dataset corresponding to test requirements in the agent-based testing method provided in this application embodiment; Figure 8 A flowchart illustrating the invocation of a result extraction tool in an agent-based testing method provided in this application embodiment; Figure 9 A schematic diagram of a test device based on an intelligent agent provided in an embodiment of this application; Figure 10 This is a schematic diagram of the electronic device structure provided in an embodiment of this application. Detailed Implementation
[0028] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.
[0029] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0030] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.
[0031] Currently, mainstream automated testing technologies typically rely on manually written or tool-generated static test cases, driven by pre-set scripts. In practical applications, testers must first transform the requirements described in natural language into structured test cases, clarifying the input conditions, operation steps, and expected results, before configuring the corresponding test data and execution environment.
[0032] However, existing technologies have the following drawbacks: First, test requirements are mostly in the form of unstructured text, lacking unified standard fields and semantic annotations, which makes requirement parsing difficult and prone to ambiguity. Second, the generation and maintenance of test cases rely heavily on human experience, making it difficult to cope with frequently changing requirement scenarios, resulting in low reusability and high maintenance costs. Third, the matching of test tools and datasets is mainly based on parameter configuration or simple keyword matching, which cannot dynamically adjust the execution strategy according to the context semantics, leading to incomplete test coverage or redundant execution. Finally, the test execution and result verification stages are disconnected, and after a problem is discovered, manual intervention is still required to locate the cause and adjust the test plan, lacking the autonomous decision-making ability of a closed loop from "execution-judgment-optimization".
[0033] Therefore, existing technologies are not yet able to achieve truly unmanned, adaptive, and semantically driven intelligent testing throughout the entire process.
[0034] Based on the above-mentioned problems, this application proposes an agent-based testing method. By generating test context information corresponding to the test requirements during the testing process, the pre-trained agent performs autonomous parsing and dynamic decision-making based on the test context information, driving end-to-end automation of the entire testing process and achieving high-precision, adaptive, and iterative intelligent testing without human intervention.
[0035] It is understood that the agent-based testing method provided in this application embodiment can be deployed in any electronic device with processing capabilities to perform automated testing of the system under test.
[0036] For example, Figure 1 A system architecture diagram of a testing system involved in the agent-based testing method provided in the embodiments of this application is shown below. Figure 1 As shown, the testing system provided in this application embodiment can be a Turing machine-like automated testing closed-loop system. The testing system includes: a requirement understanding unit, an intelligent agent, a vector database, at least one execution tool, and at least one result extraction tool.
[0037] Among them, the execution tool is a callable interface module used to perform actual interactive operations on the system under test (SUT). Each execution tool corresponds to an atomic test action.
[0038] For example, the execution tool is a secondary encapsulation based on browser automation frameworks such as Selenium, Playwright, or Puppeteer, providing standardized and semantic method names.
[0039] Among them, the result extraction tool is used to read specific information or status from the system under test.
[0040] The vector database is a structured database built on embedding vectors, including a historical test sample library and a test data library.
[0041] For example, the historical test case library includes multiple historical test records, each of which includes: original requirement description, sequence of tools used, parameter configuration, result judgment logic, and repair suggestions. The original requirement description is stored in the form of an encoded semantic vector. The test data library includes multiple test datasets.
[0042] Among them, the intelligent agent is a closed-loop autonomous decision-making system with a large language model (LLM) as the core engine, combined with a vector knowledge base, result extraction tools, execution tools, and a demand understanding unit.
[0043] The following describes in detail the agent-based testing method provided in this application with reference to several embodiments.
[0044] Figure 2 A flowchart illustrating an agent-based testing method provided in this application is shown below. Figure 2 As shown, the execution subject of this method can be the aforementioned test system, and the method includes: S201. Generate test context information corresponding to the test requirements based on the test requirements.
[0045] Optionally, the testing system can also provide a graphical user interface (GUI) where users can input test requirements. After the testing system obtains the test requirements, the requirement understanding unit in the testing system generates test context information corresponding to the test requirements.
[0046] Among them, test requirements can be natural language requirements entered by users on the graphical user interface. Test requirements can also include expected results. Test context information refers to the collection of all real-time, variable, and structured data that the agent depends on.
[0047] In one example, after obtaining the test requirements, the test requirements can be extracted according to the preset structured field extraction rules to obtain at least one structured field. The extracted structured fields can then be filled into a preset test context information template to obtain the test context information.
[0048] In another example, after obtaining the test requirements, the test requirements can be input into a pre-trained agent. The agent then rewrites and optimizes the test requirements using a pre-obtained dictionary and terminology database to obtain the rewritten test requirements. This allows the generation of test context information corresponding to the test requirements, thereby unifying the description of the test requirements and reducing ambiguity.
[0049] In another example, test requirements can be extracted according to preset structured field extraction rules to obtain at least one structured field. Based on the test requirements, relevant information can be retrieved from the vector database, thereby generating test context information based on each structured field and related information.
[0050] S202. Input the test context information into the pre-trained agent. The agent generates the current action based on the test context information, calls the target execution tool in at least one execution tool, and the target execution tool performs the test of the current action to obtain and output the current test result.
[0051] Optionally, after obtaining the test context information, the test context information can be input into the pre-trained agent, which will then perform reasoning and prediction based on the test context information to generate the current action.
[0052] Optionally, after obtaining the current action, the testing system can determine and call at least one target execution tool based on the current action and the pre-stored mapping relationship between actions and execution tools, and the target execution tool can perform the test to obtain and output the current test result.
[0053] The current action refers to the action that is being processed or about to be executed during the execution of the test action chain, and the current test result refers to the result obtained after the current action test is completed.
[0054] For example, taking the current action as clicking the login button, the current test results include at least one of the following: page screenshot, HTTP response body, console logs, DOM tree structure, etc.
[0055] S203. Based on the current test results, call the result extraction tool, which will extract the results to be verified from the current test results.
[0056] Optionally, after obtaining the current test result, the result extraction tool is invoked, and the result extraction tool extracts the result to be verified from the current test result through methods such as field reading and error information capture.
[0057] Among them, the results to be verified refer to the key facts or indicators extracted from the current test results that can be logically compared with the expected results.
[0058] S204. Input the result to be verified into the agent. The agent determines whether to end the test based on the result to be verified. If yes, generate a test report corresponding to the test requirements. Otherwise, the agent generates the next action of the current action based on the test context information and the result to be verified, and calls the execution tool and the result extraction tool to iterate until the agent determines to end the test.
[0059] Optionally, the result to be verified is input into the agent, which matches the result to be verified with the expected result in the test requirements, and determines whether to end the test based on the matching result. If so, a test report corresponding to the test requirements is generated.
[0060] Optionally, if not, the agent generates the next action of the current action based on the test context information and the result to be verified, and calls the next target execution tool in at least one execution tool, which performs the test, obtains and outputs the next test result, and calls the result extraction tool to extract the result to be verified from the next test result. This process is repeated iteratively until the agent determines to end the test.
[0061] For example, the agent can initiate the test report generation process when it determines that "the test is complete" or "a conclusion has been reached." Examples include: successful completion of all actions, encountering a fatal error, reaching a preset termination condition, and detecting an abnormal pattern.
[0062] For example, the agent can dynamically maintain the execution context record during the test. The execution context record includes: test context information, test action record, execution action record, current test result and result to be verified for each action, decision logic between actions and final judgment conclusion.
[0063] It is understandable that the testing process is executed iteratively, and there may be multiple tests. Therefore, during this test, the agent can summarize the previous test processes to generate a more concise and clear context record, thereby improving testing efficiency.
[0064] For example, after the test is completed, the agent generates a test report by performing context recording.
[0065] In this embodiment, test context information corresponding to the test requirements is generated through test requirements. This transforms natural language requirements into structured input, and the test context information is input into a pre-trained agent. The agent generates the current action based on the test context information, calls at least one target execution tool, and the target execution tool performs the test of the current action, obtaining and outputting the current test result. Based on the current test result, a result extraction tool is called, which extracts the result to be verified from the current test result. The result to be verified is input into the agent, which determines whether to end the test based on the result to be verified. If so, a test report corresponding to the test requirements is generated; otherwise, the agent generates the next action of the current action based on the test context information and the result to be verified, and calls the execution tool and the result extraction tool, iteratively executing until the agent determines to end the test. This achieves fully unmanned, adaptive, and high-precision intelligent testing from natural language requirements to test report generation, significantly improving test efficiency, coverage, and maintainability.
[0066] Furthermore, during testing, the agent generates only the current action at a time, rather than planning a complete chain of test actions all at once. This allows the agent to adjust the generation of the current action based on the results of the previous action, thus avoiding the rigidity of pre-defined paths and achieving dynamic adaptation and context awareness. It also enhances the agent's ability to conduct exploratory testing, significantly improving test coverage and completeness. In addition, it improves the robustness and fault tolerance of the testing system, and possesses good scalability and cross-scenario migration capabilities, making it suitable for intelligent testing of various types of systems under test.
[0067] In one possible implementation, Figure 3 This is a flowchart illustrating the process of generating test context information corresponding to test requirements in the agent-based testing method provided in this application embodiment, with reference to... Figure 3 As shown, in S202 above, test context information corresponding to the test requirements is generated based on the test requirements, including: S301. Generate structured test information corresponding to the test requirements based on the test requirements.
[0068] Optionally, test requirements can be obtained and structured extracted to generate structured test information corresponding to the test requirements.
[0069] For example, test requirements can be extracted according to preset structured field extraction rules to obtain at least one structured field, and the extracted structured fields can be used as test information.
[0070] The test requirements are represented using natural language, and the test information includes: test actions, test objects, and key parameters. The key parameters include: scene parameters, device parameters, and numerical parameters.
[0071] For example, taking the test requirement of "verifying whether the recognition response time of mobile face payment in low-light environment is less than 500ms" as an example, the extracted test information is as follows: {"Test Action": "Performance Verification",} "Test Target": "Face Recognition Engine" "Key parameters": { "Scenario": "Payment Process" "Device": "Mobile device" "Light intensity": "200 lux" Threshold: 500ms } } S302. Based on the test information, generate the test context information corresponding to the test requirements.
[0072] In one example, after obtaining the test information, the test information can be filled into a preset test context information template to obtain the test context information.
[0073] In another example, relevant information can be retrieved from a vector database based on the test information, thereby generating test context information based on each structured field and related information.
[0074] By generating structured test information corresponding to the test requirements, and by generating test context information corresponding to the test requirements, the system can transform ambiguous human language into the smallest semantic units that machines can process. This bridges the semantic gap between natural language and automated execution, providing accurate query basis for subsequent knowledge retrieval.
[0075] In one possible implementation, Figure 4 This is another flowchart illustrating the generation of test context information corresponding to test requirements in the agent-based testing method provided in this application embodiment, referring to... Figure 4 As shown, in S302 above, test context information corresponding to the test requirements is generated based on the test information, including: S401. Based on the test information, retrieve at least one reference test sample corresponding to the test requirement from the preset historical test sample library.
[0076] Optionally, after obtaining the test information, at least one reference test sample corresponding to the test requirement can be retrieved from a preset historical test sample library based on the test information.
[0077] For example, the test information can be processed into a first query vector, and at least one reference test sample corresponding to the test requirement can be obtained from a preset historical test sample library by calculating the cosine similarity of the vectors.
[0078] Among them, reference test cases are used to provide the agent with reference information during the testing process, thereby improving the accuracy of the test.
[0079] S402. Based on the test information, retrieve at least one test dataset corresponding to the test requirements from the preset test database.
[0080] Optionally, the test information is encoded into a second query vector, and at least one test dataset corresponding to the test requirement is obtained from a preset data database by calculating the cosine similarity of the vectors.
[0081] S403. Based on the test information, each reference test case, and each test dataset, generate the test context information corresponding to the test requirements.
[0082] Optionally, test information, reference test cases, and test datasets can be populated into a preset test context template to obtain the test context information corresponding to the test requirements.
[0083] By generating test context information corresponding to the test requirements through test information, reference test examples, and test datasets, a complete execution environment containing task definitions, historical experience, and available resources can be built for the intelligent agent, significantly improving the accuracy, reusability, and overall adaptability of the test action chain generation.
[0084] In one possible implementation, Figure 5 This is a flowchart illustrating the process of obtaining at least one reference test case corresponding to the test requirements in the agent-based testing method provided in this application embodiment. Figure 5 As shown, in step S401 above, at least one reference test sample corresponding to the test requirement is retrieved from a preset historical test sample library based on the test information, including: S501. Based on the test information, generate the first semantic vector corresponding to the test requirements.
[0085] Optionally, the test actions, test objects, and key parameters in the test information are concatenated and then input into a pre-trained encoding model to encode the test information into a first semantic vector. The encoding model is based on the BERT model.
[0086] The first semantic vector is used to measure the semantic similarity between the test requirement and other test examples.
[0087] S502. Based on the first semantic vector, retrieve at least one reference test sample from the preset historical test sample library.
[0088] Optionally, at least one reference test sample can be retrieved from a preset historical test sample library based on the first semantic vector by calculating the cosine similarity of the vectors.
[0089] For example, the cosine similarity between the first semantic vector and the original requirement description of each historical test record in the historical test sample library can be calculated respectively. The results are then sorted according to the similarity results to obtain a sorting result. The first M historical test records are then selected from the sorting result according to a preset first quantity threshold M, and used as reference test samples. Here, M is a positive integer, and M is greater than or equal to 1.
[0090] By using the first semantic vector, at least one reference test case can be retrieved from a pre-defined historical test case library, which can improve the accuracy of the obtained reference test cases and thus significantly reduce the risk of erroneous transfer caused by semantic ambiguity.
[0091] In one possible implementation, Figure 6 This is a flowchart illustrating the process of retrieving at least one test dataset corresponding to a test requirement in the agent-based testing method provided in this application embodiment, referring to... Figure 6 As shown, in S503 above, at least one test dataset corresponding to the test requirements is retrieved from a preset test database based on the test information, including: S601. Based on the test information, generate the second semantic vector corresponding to the test requirements.
[0092] Optionally, the test actions, test objects, and key parameters in the test information can be concatenated and input into a pre-trained encoding model to encode the test information into a second semantic vector. The encoding model is based on the BERT model.
[0093] The first semantic vector is used to measure the semantic similarity between the test requirements and other test datasets.
[0094] S602. Based on the second semantic vector, retrieve at least one candidate dataset from the preset test database.
[0095] Optionally, at least one candidate dataset corresponding to the test requirement can be retrieved from a preset test database based on the second semantic vector by calculating the cosine similarity of the vectors.
[0096] For example, the cosine similarity between the second semantic vector and each test dataset in the test database can be calculated, and the results can be sorted according to the similarity scores. Then, the top N historical test records are selected from the sorted results according to a preset second quantity threshold N, and used as reference test examples. Here, N is a positive integer, and N is greater than or equal to 1.
[0097] S603. Based on each candidate dataset, determine at least one test dataset corresponding to the test requirements.
[0098] Optionally, after obtaining each candidate dataset, the candidate dataset can be validated or filtered to obtain at least one test dataset corresponding to the test requirements.
[0099] By generating a second semantic vector corresponding to the test requirements, and retrieving at least one candidate dataset from a pre-defined test database based on this second semantic vector, at least one test dataset corresponding to the test requirements can be determined based on each candidate dataset. This ensures that high-quality datasets that are semantically highly related but have different labels are not overlooked. Simultaneously, it enables intelligent preliminary screening, significantly improving efficiency.
[0100] In one possible implementation, Figure 7 This is a flowchart illustrating the process of determining at least one test dataset corresponding to test requirements in the agent-based testing method provided in this application embodiment, with reference to... Figure 7 As shown, in S603 above, at least one test dataset corresponding to the test requirement is determined based on each candidate dataset, including: S701. Determine the target parameter inheritance tree corresponding to the test requirement from the multiple parameter inheritance trees obtained in the pre-constructed form.
[0101] It is understandable that the testing system pre-stores multiple pre-constructed parameter inheritance trees. Each parameter inheritance tree is used to structurally manage the dynamic data structure of parameter dependencies and hierarchical relationships between test requirements and datasets. Each node in the parameter inheritance tree is defined as a triple T(D, P, A). Here, D is the natural language description, i.e., the test requirement; P is the parent node reference; and A is the parameter set, i.e., the test dataset. The parameter inheritance trees can be loaded into memory or a vector database during testing system initialization.
[0102] For example, a test requirement corresponds to at least one parameter inheritance tree.
[0103] Optionally, the target parameter inheritance tree corresponding to the test requirement can be determined from multiple pre-constructed parameter inheritance trees based on the test requirements and the pre-stored mapping relationship between test requirements and parameter inheritance trees. The number of target parameter inheritance trees can be one or more.
[0104] S702. Based on the target parameter inheritance tree, filter each candidate dataset to obtain the target candidate dataset.
[0105] Optionally, after obtaining the target parameter inheritance tree, each candidate dataset can be filtered through each node in the target parameter inheritance tree to obtain the target candidate dataset.
[0106] For example, it can be determined whether each candidate dataset has a corresponding node in the target parameter inheritance tree. If it does, the candidate dataset is used as a target candidate dataset; if it does not, the candidate dataset is not used as a target candidate dataset.
[0107] S703. Based on the target candidate dataset, determine at least one test dataset.
[0108] Optionally, after obtaining the target candidate dataset, the target candidate dataset can be used as the test dataset.
[0109] Optionally, after obtaining the target candidate dataset, at least one test dataset can be obtained based on the relevant nodes of the target candidate dataset in the target parameter inheritance tree.
[0110] By using the candidate datasets and the pre-constructed parameter inheritance tree, at least one test dataset corresponding to the test requirements is determined. This fundamentally avoids test failures caused by "missing data" or "unsupported environment," ensuring that the final selected test dataset is truly executable and non-blocking. Furthermore, it supports refined parameter inheritance and reuse.
[0111] In one possible implementation, S703 above determines at least one test dataset based on the target candidate dataset, including: Based on the target candidate dataset, find the current node from the target parameter inheritance tree; based on the current node, at least one parent node of the current node, and / or at least one child node of the current node, determine at least one test dataset.
[0112] Optionally, the current node corresponding to the target candidate dataset can be found from the target parameter inheritance tree.
[0113] In one example, the target candidate dataset and the dataset corresponding to at least one parent node of the current node can be used as a test dataset.
[0114] In another example, the target candidate dataset and the dataset corresponding to at least one child node of the current node can be used as a test dataset.
[0115] In another example, the target candidate dataset, the dataset corresponding to at least one parent node of the current node, and the dataset corresponding to at least one child node of the current node can all be used as a test dataset.
[0116] By using the target candidate dataset, the current node is found from the target parameter inheritance tree. Based on the current node, at least one parent node of the current node, and / or at least one child node of the current node, at least one test dataset is determined. This enables precise context binding and dynamic reasoning. Furthermore, it supports multi-dimensional, combinatorial intelligent data synthesis, significantly improving the complexity and realism of test scenarios, and covering a large number of edge-combination test scenarios that are difficult to prepare in advance. Moreover, it also enables cross-level knowledge reuse and inheritance.
[0117] In one possible implementation, Figure 8 A flowchart illustrating the invocation of a result extraction tool in the agent-based testing method provided in this application embodiment, referring to... Figure 8 As shown, in step S203 above, the result extraction tool is invoked based on the current test results, including: S801. Determine the test type corresponding to the current test result.
[0118] Optionally, after obtaining the current test result, the test type corresponding to the current test result can be determined. The test type includes at least one of the following: content comparison, path verification, visual feature verification, and status code verification.
[0119] In one example, the test type corresponding to the current test result can be determined based on the test information corresponding to the current test result.
[0120] In another example, the test type can be determined based on the data format returned by the current test results. For example, if the test returns DOM structure, text content, or element attributes, it can be identified as a content comparison test; if it returns an image region or confidence score, it can be identified as a visual feature test; if it returns a URL path, it can be identified as a path verification test; and if it returns an HTTP status code or response headers, it can be identified as a status code test.
[0121] S802. Based on the test type, send a call request to the target result extraction tool in at least one result extraction tool.
[0122] Optionally, a target extraction tool can be determined based on the test type and the pre-stored mapping relationship between test types and result extraction tools, and a call request can be sent to the target result extraction tool among at least one result extraction tool.
[0123] For example, a call request can be sent to the target result extraction tool in at least one result extraction tool via the Model Context Protocol (MCP).
[0124] The call request includes the test type and the current test result.
[0125] For example, when the test type is content comparison, the target result extraction tool can be a text matching tool; when the test type is visual feature type, the target result extraction tool can be an image recognition model; when the test type is status code type, the target result extraction tool can be a network packet capture parser; when the test type is path verification type, the target result extraction tool can be a URL listening module.
[0126] S803, the target result extraction tool extracts the results to be verified from the current test results based on the call request.
[0127] Optionally, the target result extraction is based on the call request, and the result to be verified is extracted from the current test result according to preset rules.
[0128] By determining the test type corresponding to the current test result and sending a call request to the target result extraction tool in at least one result extraction tool based on the test type, the target result extraction tool can extract the result to be verified from the current test result based on the call request. This enables professional, accurate, and efficient information extraction for different types of test results, which not only improves the verification accuracy but also enhances the scalability and stability of the testing system.
[0129] Based on the same inventive concept, this application also provides an agent-based testing device corresponding to the agent-based testing method. Since the principle of the device in this application is similar to that of the agent-based testing method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.
[0130] Reference Figure 9 As shown, Figure 9 This application provides a schematic diagram of a test device based on an intelligent agent, applied to a test system. The test system includes at least one execution tool and at least one result extraction tool. The device includes: a first generation module 901, a second generation module 902, an extraction module 903, and a determination module 904; wherein, The first generation module 901 is used to generate test context information corresponding to the test requirements based on the test requirements. The second generation module 902 is used to input the test context information into a pre-trained agent, and the agent generates the current action according to the test context information, calls at least one target execution tool in the execution tool, and the target execution tool performs the test of the current action to obtain and output the current test result. The extraction module 903 is used to call the result extraction tool based on the current test results, and the result extraction tool extracts the results to be verified from the current test results; The determination module 904 is used to input the result to be verified into the agent, and the agent determines whether to end the test based on the result to be verified. If so, a test report corresponding to the test requirement is generated. Otherwise, the agent generates the next action of the current action based on the test context information and the result to be verified, and calls the execution tool and the result extraction tool to iterate until the agent determines to end the test.
[0131] Optionally, the first generation module 901 is specifically used for: Based on the testing requirements, structured test information corresponding to the testing requirements is generated. The testing requirements are represented by natural language, and the test information includes: test actions, test objects, and key parameters. Based on the test information, generate test context information corresponding to the test requirements.
[0132] Optionally, the first generation module 901 is specifically used for: Based on the test information, at least one reference test case corresponding to the test requirement is retrieved from the preset historical test case library; Based on the test information, at least one test dataset corresponding to the test requirements is retrieved from the pre-set test database; Based on the test information, reference test cases, and test datasets, generate test context information corresponding to the test requirements.
[0133] Optionally, the first generation module 901 is specifically used for: Based on the test information, generate a first semantic vector corresponding to the test requirement; Based on the first semantic vector, at least one reference test sample is retrieved from the preset historical test sample library.
[0134] Optionally, the first generation module 901 is specifically used for: Based on the test information, a second semantic vector corresponding to the test requirement is generated; Based on the second semantic vector, at least one candidate dataset is retrieved from the preset test database; Based on each of the candidate datasets, at least one test dataset corresponding to the test requirement is determined.
[0135] Optionally, the first generation module 901 is specifically used for: From the multiple pre-constructed parameter inheritance trees, determine the target parameter inheritance tree corresponding to the test requirement; Based on the target parameter inheritance tree, each of the candidate datasets is filtered to obtain the target candidate dataset; Based on the target candidate dataset, at least one of the test datasets is determined.
[0136] Optionally, the first generation module 901 is specifically used for: Based on the target candidate dataset, the current node is found from the target parameter inheritance tree; At least one test dataset is determined based on the current node, at least one parent node of the current node, and / or at least one child node of the current node.
[0137] Optionally, the extraction module 903 is specifically used for: Determine the test type corresponding to the current test result. The test type includes at least one of the following: content comparison, path verification, visual feature, and status code. Based on the test type, a call request is sent to the target result extraction tool in at least one result extraction tool. The call request includes the test type and the current test result. The target result extraction tool extracts the results to be verified from the current test results based on the call request.
[0138] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.
[0139] This application also provides an electronic device, such as... Figure 10 As shown, Figure 10 The schematic diagram of the electronic device structure provided in the embodiments of this application includes: a processor 1001, a memory 1002, and optionally, a bus 1003. The memory 1002 stores machine-readable instructions executable by the processor 1001 (e.g., ...). Figure 9The device includes the first generation module 901, the second generation module 902, the extraction module 903, and the determination module 904 (and the corresponding execution instructions, etc.). When the electronic device is running, the processor 1001 and the memory 1002 communicate through the bus 1003. When the machine-readable instructions are executed by the processor 1001, the steps of the above-mentioned agent-based test method are performed.
[0140] This application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of the above-described agent-based testing method.
[0141] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.
[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.
[0143] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.
Claims
1. A testing method based on intelligent agents, characterized in that, Applied to a testing system, the testing system including at least one execution tool and at least one result extraction tool, the method includes: Generate test context information corresponding to the test requirements based on the test requirements; The test context information is input into the pre-trained agent, which generates the current action based on the test context information, calls at least one of the target execution tools, and the target execution tool performs the test of the current action to obtain and output the current test result. The result extraction tool is invoked based on the current test results, and the result extraction tool extracts the results to be verified from the current test results; The result to be verified is input into the agent, which determines whether to end the test based on the result. If so, a test report corresponding to the test requirement is generated. Otherwise, the agent generates the next action of the current action based on the test context information and the result to be verified, and calls the execution tool and the result extraction tool to iterate until the agent determines to end the test.
2. The agent-based testing method according to claim 1, characterized in that, The step of generating test context information corresponding to the test requirements based on the test requirements includes: Based on the test requirements, structured test information corresponding to the test requirements is generated. The test requirements are represented by natural language, and the test information includes: test actions, test objects, and key parameters. Based on the test information, generate test context information corresponding to the test requirements.
3. The agent-based testing method according to claim 2, characterized in that, The step of generating test context information corresponding to the test requirement based on the test information includes: Based on the test information, at least one reference test example corresponding to the test requirement is retrieved from a preset historical test example library; Based on the test information, at least one test dataset corresponding to the test requirement is retrieved from a preset test database; Based on the test information, the reference test examples, and the test datasets, test context information corresponding to the test requirements is generated.
4. The agent-based testing method according to claim 3, characterized in that, The step of retrieving at least one reference test sample corresponding to the test requirement from a preset historical test sample library based on the test information includes: Based on the test information, generate a first semantic vector corresponding to the test requirement; Based on the first semantic vector, at least one reference test sample is retrieved from the preset historical test sample library.
5. The agent-based testing method according to claim 3, characterized in that, The step of retrieving at least one test dataset corresponding to the test requirement from a preset test database based on the test information includes: Based on the test information, a second semantic vector corresponding to the test requirement is generated; Based on the second semantic vector, at least one candidate dataset is retrieved from the preset test database; Based on each of the candidate datasets, at least one test dataset corresponding to the test requirement is determined.
6. The agent-based testing method according to claim 5, characterized in that, The step of determining at least one test dataset corresponding to the test requirement based on each of the candidate datasets includes: From the multiple pre-constructed parameter inheritance trees, determine the target parameter inheritance tree corresponding to the test requirement; Based on the target parameter inheritance tree, each of the candidate datasets is filtered to obtain the target candidate dataset; Based on the target candidate dataset, at least one of the test datasets is determined.
7. The agent-based testing method according to claim 6, characterized in that, The step of determining at least one of the test datasets based on the target candidate datasets includes: Based on the target candidate dataset, the current node is found from the target parameter inheritance tree; At least one test dataset is determined based on the current node, at least one parent node of the current node, and / or at least one child node of the current node.
8. The agent-based testing method according to claim 1, characterized in that, The step of calling the result extraction tool based on the current test result includes: Determine the test type corresponding to the current test result. The test type includes at least one of the following: content comparison, path verification, visual feature, and status code. Based on the test type, a call request is sent to at least one of the target result extraction tools in the result extraction tools, the call request including the test type and the current test result; The target result extraction tool extracts the result to be verified from the current test results based on the call request.
9. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when the electronic device is running, are executed by the processor to perform the steps of the agent-based testing method as described in any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the agent-based testing method as described in any one of claims 1 to 8.