A Java testing closed-loop method and system based on AI Agent-driven approach

Abstract

This invention discloses a Java testing closed-loop method and system based on AI Agent-driven approaches. The closed-loop method includes: receiving test requirements input by the user and converting natural language into task requests that the system can recognize; scheduling and controlling the entire testing process; generating new Java test code based on coverage analysis results; uniformly managing the interaction between two AI Agents and the local execution environment, with the two AI Agents completing capability calls through standardized interfaces to achieve module decoupling and unified process control; executing corresponding operations according to forwarded instructions and providing feedback on the execution results. Through multiple iterations, the method achieves automatic approximation and continuous improvement of test coverage, significantly enhancing test depth and accuracy.

Description

Technical Field

[0001] This invention relates to the field of Java automated testing, and in particular to a closed-loop method and system for Java testing driven by an AI Agent. Background Technology

[0002] In the existing software development lifecycle, automated testing is a crucial step in ensuring software quality and stability, but the following technical problems exist in practical applications: Low efficiency and high labor costs in test case generation: Traditional automated testing mainly relies on manual writing of test cases, which not only consumes a lot of manpower, but also leads to problems such as long testing cycles and slow response to changes.

[0003] Insufficient dynamic coverage enhancement capability: Existing automated test case generation technologies mostly focus on generating test cases in one go, lacking a dynamic supplementation mechanism driven by coverage feedback. As a result, there is still room for improvement in coverage depth and coverage accuracy in complex branch structures, abnormal paths, and multi-condition combination scenarios.

[0004] Limited automation and intelligence in test execution: Existing automated testing systems mostly focus on the test execution phase, with limited support for supporting processes before and after testing. Steps such as code retrieval, branch switching, dependency management, test environment parameter configuration, and code change pre-review still mainly rely on manual operation or fixed script processing. The overall process is characterized by rigid rules and insufficient responsiveness, lacking intelligent recognition and adaptive management capabilities based on context information, thus affecting the automation level and execution efficiency of the test process.

[0005] In existing Java automated testing systems, test execution is typically based on mature unit testing frameworks such as JUnit and TestNG, and is uniformly executed through build tools like Maven or other build systems. After test execution, statistical reports such as line coverage and branch coverage are generated using coverage analytics tools like JaCoCo. In engineering practice, this test execution process is usually integrated with version control systems like Git and continuous integration platforms like Jenkins to form an automatically triggered test pipeline, automating code retrieval, build compilation, and test execution. This type of technical solution effectively improves test execution efficiency and automates test result output, making it the mainstream test execution method currently available.

[0006] In terms of test execution, existing technologies are mostly based on unit testing frameworks and continuous integration pipelines to achieve automatic triggering and report output, with their core capabilities focused on "execution according to rules." The test process typically follows preset scripts to complete steps such as code retrieval, building, execution, and report generation. The execution strategy is fixed and lacks the ability to dynamically adjust based on test results. Coverage reports are only used to display or archive statistical results and are not used to drive subsequent test optimization. The overall execution logic has a linear and unidirectional structure and lacks intelligent analysis and adaptive scheduling capabilities.

[0007] In terms of test case generation, existing technologies, whether algorithm-based automated generation tools or code generation methods based on large models, typically rely on manual triggering. The generation process largely depends on manual input of code or instructions and ends after a single generation. The generated results require manual intervention or a continuous integration system to trigger test execution again. When coverage is insufficient, whether to continue generating tests, how to determine the generation scope, and when to stop optimization still mainly depend on manual judgment, resulting in limited automation of the overall process.

[0008] Furthermore, the current test execution process and test generation process operate independently, resulting in process fragmentation and insufficient intelligence. Test execution results are not automatically fed back to the generation module, and the generation module does not continuously iterate and optimize based on coverage gaps. There is a lack of unified scheduling and feedback mechanisms between the two, making it impossible to form a closed-loop structure of "execution-analysis-supplementation-re-execution," and making it difficult to achieve self-iteration in test case generation and continuous improvement in coverage. Summary of the Invention

[0009] In view of the above problems, the present invention is proposed to provide an AI Agent-driven Java test closed-loop method and system to overcome or at least partially solve the above problems.

[0010] According to one aspect of the present invention, a Java test closed-loop method based on AI Agent is provided, the test closed-loop method comprising: Receive test requests from users and convert natural language into task requests that the system can recognize; The entire testing process is scheduled and controlled. Generate new Java test code based on the coverage analysis results; The interaction between the two AI Agents and the local execution environment is managed in a unified manner. The two AI Agents complete the capability call through a standardized interface, realizing module decoupling and unified process control. Perform the corresponding operations according to the forwarded instructions and provide feedback on the execution results.

[0011] Optionally, receiving the test requirements input by the user and converting natural language into a task request that the system can recognize specifically includes: Receive test instructions and pass the input in natural language to the test execution agent; The test execution agent performs semantic parsing and intent recognition on the input content and extracts test-related control parameters. When the user does not explicitly provide some parameters, the system automatically completes the default values ​​based on the preset knowledge base configuration and the prompt word rules of the large language model, forming a set of test task parameters. Before the test is executed, the test execution agent calls the local tool execution module to complete code download, target branch switching, and build environment verification operations according to the control parameters in the task model. When the task model includes code review identifiers, the test execution agent obtains code differences by executing modules using local tools. The test executes the large language model inside the intelligent agent to perform semantic analysis and structured parsing of the discrepancies. During the analysis, the system simultaneously calls upon enterprise-level code writing standards, interface design standards, and exception handling standards stored in the knowledge base, compares and matches the changed code with the standard entries, performs consistency verification and compliance judgment, and identifies code locations that do not conform to the standards or have potential defects.

[0012] Optionally, the test control parameters include: target branch information; baseline branch information; local code path; test directory path; coverage target threshold; whether to enable test generation; and whether to perform code retrieval and code review.

[0013] Optionally, the scheduling and decision-making control of the entire testing process specifically includes: After the test execution is completed, the test execution agent generates a JaCoCo coverage report file through the local tool execution module, and then calls the test case generation agent to analyze the current coverage results and extract the initial coverage and the set of uncovered code.

[0014] Optionally, generating new Java test code based on the coverage analysis results specifically includes: The test case generation agent performs semantic analysis and path reasoning based on the uncovered code set, generates new Java test code files, and returns the generation results to the test execution agent. After receiving new test code, the test execution agent executes the test commands again through the local tool execution module and generates new test reports and coverage reports; the new coverage data is then transmitted to the test case generation agent for analysis.

[0015] Optionally, the unified management of the interaction between the two AI Agents and the local execution environment, with the two AI Agents calling capabilities through standardized interfaces to achieve module decoupling and unified process control, specifically includes: Self-repair mechanism for failure generation: When an exception occurs during the compilation or execution of newly generated test code, the test execution agent reads the compilation error information or runs the exception stack, and passes the error log along with the corresponding test code to the test case generation agent; Coverage self-reinforcing closed-loop structure: After all test cases are successfully executed, the agent generated from the test cases is judged based on the following logic to determine the threshold: Has the current coverage rate reached the preset target? Has the current number of iterations exceeded the maximum limit? When the termination condition is met, the final test result is output and the process ends; If the termination condition is not met, repeat the above process to perform the next round of test case generation task.

[0016] Optionally, the step of performing the corresponding operation according to the forwarded instruction and feeding back the execution result specifically includes: After each test process is completed, the system writes the test results to the memory storage module; In subsequent testing tasks, the agent calls upon historical data provided by the memory module to allow users to more flexibly retrieve test records; The system also incorporates a knowledge base module to store enterprise-level code standards, interface constraints, test environment parameters, exception handling rules, and test case templates. When executing and generating test cases, the knowledge base content is incorporated: During code pre-review, verify that the submitted test code is consistent with the company's specifications; Intelligent maintenance of default parameters in the test codebase and environment; Test case technology stack, writing standards, and code examples; Maintenance system requires specification documents and general design documents.

[0017] Optionally, the test record includes: test branch information; coverage value; set of uncovered code; number of test loops; and report path information.

[0018] This invention also provides a Java test closed-loop system based on AI Agent, applying the aforementioned AI Agent-driven Java test closed-loop method. The test closed-loop system includes: The user interaction module is used to receive test requirements input by users and convert natural language into task requests that the system can recognize. The AI-automated testing agent module is used for scheduling and decision control of the entire testing process; The AI-automated test case generation Agent module is used to generate new Java test code based on coverage analysis results. The public service integration module is used to uniformly manage the interaction between the two AI Agents and the local execution environment. The two AI Agents complete the capability call through standardized interfaces, realizing module decoupling and unified process control. The local tool execution module is used to perform corresponding operations according to the forwarded instructions and to provide feedback on the execution results.

[0019] This invention provides a Java testing closed-loop method and system based on AI Agent-driven approaches. The closed-loop method includes: receiving test requirements input by the user and converting natural language into task requests that the system can recognize; scheduling and controlling the entire testing process; generating new Java test code based on coverage analysis results; uniformly managing the interaction between two AI Agents and the local execution environment, with the two AI Agents completing capability calls through standardized interfaces to achieve module decoupling and unified process control; executing corresponding operations according to forwarded instructions and providing feedback on the execution results. Through multiple iterations, the test coverage is automatically approximated and continuously improved, thereby significantly increasing test depth and accuracy.

[0020] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a Java test closed-loop method based on AI Agent-driven implementation, as provided in this embodiment of the invention; Figure 2 A block diagram of a Java test closed-loop system driven by AI Agent provided in an embodiment of the present invention; Figure 3 A detailed flowchart of a Java test closed-loop method based on AI Agent-driven method provided in this embodiment of the invention. Detailed Implementation

[0023] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0024] The terms "comprising" and "having," and any variations thereof, in the specification, embodiments, claims, and drawings of this invention are intended to cover non-exclusive inclusion, such as including a series of steps or units.

[0025] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0026] like Figure 1 As shown, a Java testing closed-loop method based on AI Agent is presented. The method includes: receiving test requirements input by the user and converting natural language into system-recognizable task requests; scheduling and decision-making control of the entire testing process; generating new Java test code based on coverage analysis results; uniformly managing the interaction between two AI Agents and the local execution environment, with the two AI Agents completing capability calls through standardized interfaces to achieve module decoupling and unified process control; executing corresponding operations according to forwarded instructions and providing feedback on the execution results.

[0027] like Figure 2 As shown, this invention constructs an automated testing system based on dual AI Agent collaboration. This architecture uses a unified public service layer as the core scheduling unit, and through the collaborative work of two AI Agents, it automates the entire process of test preparation, test execution, coverage analysis, test supplement generation, and iterative enhancement. The system generally includes the following modules: User Interaction Module: This module receives test requests from users and converts natural language into task requests that the system can recognize. It sends test task instructions to the Public Service Integration Module but does not directly participate in test execution or code generation. Its role is to build a unified human-computer interaction entry point and serve as the trigger point for system processes. The AI-powered automated testing agent module is responsible for scheduling and decision-making control throughout the entire testing process. This module is built on the Coze platform and interacts with the public service integration module via the Coze SDK and Coze API.

[0028] The AI-automated test case generation agent module generates new Java test code based on coverage analysis results. Developed using the Spring AI framework, this module performs joint analysis of source code and coverage data. It is invoked through the public service integration module, and its generated results are written to a specified test file. The test agent then triggers test execution again, thus participating in a closed-loop process. Public Service Integration Module: The public service integration module is the central layer of the system, which is used to uniformly manage the interaction between the two AI Agents and the local execution environment. Through this module, the two AI Agents are not directly coupled with the underlying tools, but complete the capability call through standardized interfaces, thereby achieving module decoupling and unified process control.

[0029] Local Tool Execution Module: The local tool execution module is the system's underlying execution unit, used to actually run engineering commands. This module does not have autonomous decision-making capabilities; it only executes corresponding operations according to the instructions forwarded by the public service integration module and returns the execution results to the upper-level module.

[0030] like Figure 3 As shown, the system of the present invention forms the following collaborative relationship through the above five modules: Users initiate test requests through the user interaction module; The request enters the public service integration module, which then forwards it to the AI ​​automated testing agent module; The test agent, based on user needs, calls local tools to execute modules to complete test preparation work such as code retrieval, branch switching, and code pre-review; The test agent invokes local tools to execute modules, completes test execution, and generates a coverage report. When a user enables the test case generation function and the coverage is detected to be below the preset threshold, the test agent calls the test case generation agent through the public service integration module. The test case generation agent locates uncovered code snippets based on the coverage report, performs code context analysis using a large language model, generates new Java test cases according to a specified format and technology stack, and outputs the corresponding test code files. The test agent triggers the local tool execution module to perform the test again; Repeat the above process to form a coverage-driven test loop.

[0031] II. Semantic-Driven Testing Process Mechanism for User Interaction Module This invention constructs a semantically driven test task control mechanism. The system receives test instructions through a user interaction module and transmits the input in natural language form to the test execution agent. The test execution agent performs semantic parsing and intent recognition on the input content, extracting test-related control parameters. When the user does not explicitly provide some parameters, the system automatically completes the default values ​​based on a preset knowledge base configuration and large language model prompt word rules, thereby forming a complete set of test task parameters. The test control parameters include: Target branch information; Baseline branch information; Local code path; Test directory path; Coverage target threshold; Enable test generation; Whether to perform code retrieval and code review.

[0032] Before test execution, the test execution agent, based on the control parameters in the task model, invokes the local tool execution module to complete operations such as code download, target branch switching, and build environment verification. When the task model contains code review identifiers, the test execution agent first obtains code differences through the local tool execution module, including newly added or modified classes, methods, and changed code snippets. Subsequently, the large language model within the test execution agent performs semantic analysis and structured parsing of the differences. During the analysis, the system simultaneously invokes enterprise-level code writing standards, interface design standards, and exception handling standards stored in the knowledge base, comparing and matching the changed code with the standard entries. It performs consistency checks and compliance judgments on aspects such as code structure, naming conventions, exception handling logic, and parameter validation completeness, thereby identifying code locations that do not conform to standards or have potential defects. After receiving optimization suggestions, the user can choose whether to terminate the test process.

[0033] Through the above call chain structure, a hierarchical control mode of "user interaction module - public service integration module - test execution intelligent agent - tool execution module" is realized, so that the test preparation phase and the test execution phase are completed continuously under a unified scheduling logic, thereby realizing the overall connection and centralized management of the test process.

[0034] III. Coverage Feedback-Driven Self-Reinforcing Closed-Loop Mechanism After test execution is complete, the test execution agent generates a JaCoCo coverage report file through the local tool execution module. Then, the public service module calls the test case generation agent to analyze the current coverage results, extracting the initial coverage and the set of uncovered code. The test case generation agent performs semantic analysis and path reasoning based on the uncovered code set, generating new Java test code files and returning the results to the test execution agent. Upon receiving the new test code, the test execution agent executes the test commands again through the local tool execution module, generating new test reports and coverage reports. The new coverage data is then again transmitted to the test case generation agent for analysis.

[0035] Self-repair mechanism for test code generation failures: When a newly generated test code encounters an exception during compilation or execution, the test execution agent reads the compilation error information or runs the exception stack trace, and passes the error log along with the corresponding test code to the test case generation agent. The test case generation agent performs semantic localization and repair based on the error log, generates optimized test code, and returns it to the test execution agent for re-execution. This self-repair loop is performed a limited number of times until the test passes or the failure limit is reached.

[0036] Coverage self-reinforcing closed-loop structure: After all test cases are successfully executed, the agent generated from the test cases is judged based on the following logic to determine the threshold: Has the current coverage rate reached the preset target? Has the current number of iterations exceeded the maximum limit? Has the coverage rate of the current round been effectively improved compared to the previous round?

[0037] When the termination condition is met, the final test result is output and the process ends. If the termination condition is not met, the above process will be repeated to perform the next round of test case generation task.

[0038] The above process is executed cyclically under unified scheduling and control, forming the following closed-loop control structure: test execution → coverage resolution → threshold judgment → test generation → re-execution. This cyclical control mechanism achieves a self-reinforcing operation mode where coverage improves round by round.

[0039] IV. Test Data Memory and Knowledge Base Driven Mechanism After each test cycle, the system writes the test results to the memory storage module. The recorded data includes: Test branch information; Coverage percentage; Uncovered code collection; Test the number of loops; Report path information.

[0040] In subsequent testing tasks, the agent can access historical data provided by the memory module, allowing users to more flexibly retrieve test records.

[0041] The system also incorporates a knowledge base module to store enterprise-level code standards, interface constraints, test environment parameters, exception handling rules, and test case templates. When executing and generating test cases, the knowledge base content can be incorporated. During code pre-review, verify that the submitted test code is consistent with the company's specifications; Intelligent maintenance of default parameters in the test codebase and environment; Test case technology stack, writing standards, and code examples; The maintenance system requires specification documents and general design documents to facilitate code understanding.

[0042] By combining the memory module with the knowledge base, the agent can generate new test cases by referencing historical coverage patterns and iterative experience, while also adhering to enterprise standards and environmental constraints. This enables high-precision test case generation and a more automated and intelligent testing process.

[0043] Beneficial effects: Coverage-Driven Self-Reinforcing Closed Loop: This invention constructs a coverage feedback-driven test closed-loop control mechanism. The system takes the uncovered code regions in the coverage report as input, automatically generates runnable Java test code, and forms a loop structure of "test execution → coverage analysis → test supplement generation → re-execution". Through multiple iterations, it achieves automatic approximation and continuous improvement of test coverage, thereby significantly improving test depth and accuracy.

[0044] AI Agent-Enabled End-to-End Automation and Unified Scheduling: This invention empowers the entire testing process through an AI Agent, including test preparation, process management, code change pre-review, test case generation, and execution decisions. A unified scheduling mechanism ensures end-to-end automated operation, reducing manual intervention, improving testing efficiency and process reliability, while guaranteeing a high degree of consistency and controllability between generated test cases and execution results.

[0045] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A Java test closed-loop method based on AI Agent-driven approach, characterized in that, The test closed-loop method includes: Receive test requests from users and convert natural language into task requests that the system can recognize; The entire testing process is scheduled and controlled. Generate new Java test code based on the coverage analysis results; The interaction between the two AI Agents and the local execution environment is managed in a unified manner. The two AI Agents complete the capability call through a standardized interface, realizing module decoupling and unified process control. Perform the corresponding operations according to the forwarded instructions and provide feedback on the execution results.

2. The Java test closed-loop method based on AI Agent-driven according to claim 1, characterized in that, The process of receiving user input for testing requirements and converting natural language into a system-recognizable task request specifically includes: Receive test instructions and pass the input in natural language to the test execution agent; The test execution agent performs semantic parsing and intent recognition on the input content and extracts test-related control parameters. When the user does not explicitly provide some parameters, the system automatically completes the default values ​​based on the preset knowledge base configuration and the prompt word rules of the large language model, forming a set of test task parameters. Before the test is executed, the test execution agent calls the local tool execution module to complete code download, target branch switching, and build environment verification operations according to the control parameters in the task model. When the task model includes code review identifiers, the test execution agent obtains code differences by executing modules using local tools. The test executes the large language model inside the intelligent agent to perform semantic analysis and structured parsing of the discrepancies. During the analysis, the system simultaneously calls upon enterprise-level code writing standards, interface design standards, and exception handling standards stored in the knowledge base, compares and matches the changed code with the standard entries, performs consistency verification and compliance judgment, and identifies code locations that do not conform to the standards or have potential defects.

3. The Java test closed-loop method based on AI Agent-driven according to claim 2, characterized in that, The test control parameters include: target branch information; baseline branch information; local code path; test directory path; coverage target threshold; whether to enable test generation; and whether to perform code retrieval and code review.

4. The Java test closed-loop method based on AI Agent-driven according to claim 1, characterized in that, The scheduling and decision-making control of the entire testing process specifically includes: After the test execution is completed, the test execution agent generates a JaCoCo coverage report file through the local tool execution module, and then calls the test case generation agent to analyze the current coverage results and extract the initial coverage and the set of uncovered code.

5. The Java test closed-loop method based on AI Agent-driven according to claim 1, characterized in that, The process of generating new Java test code based on coverage analysis results specifically includes: The test case generation agent performs semantic analysis and path reasoning based on the uncovered code set, generates new Java test code files, and returns the generation results to the test execution agent. After receiving new test code, the test execution agent executes the test commands again through the local tool execution module and generates new test reports and coverage reports; the new coverage data is then transmitted to the test case generation agent for analysis.

6. The Java test closed-loop method based on AI Agent-driven according to claim 1, characterized in that, The unified management of the interaction between the two AI Agents and the local execution environment, with the two AI Agents calling capabilities through standardized interfaces, achieves module decoupling and unified process control, specifically including: Self-repair mechanism for failure generation: When an exception occurs during the compilation or execution of newly generated test code, the test execution agent reads the compilation error information or runs the exception stack, and passes the error log along with the corresponding test code to the test case generation agent; Coverage self-reinforcing closed-loop structure: After all test cases are successfully executed, the agent generated from the test cases is judged based on the following logic to determine the threshold: Has the current coverage rate reached the preset target? Has the current number of iterations exceeded the maximum limit? When the termination condition is met, the final test result is output and the process ends; If the termination condition is not met, repeat the above process to perform the next round of test case generation task.

7. The Java test closed-loop method based on AI Agent-driven according to claim 1, characterized in that, The specific steps of executing the corresponding operation according to the forwarded instructions and feeding back the execution result include: After each test process is completed, the system writes the test results to the memory storage module; In subsequent testing tasks, the agent calls upon historical data provided by the memory module to allow users to more flexibly retrieve test records; The system also incorporates a knowledge base module to store enterprise-level code standards, interface constraints, test environment parameters, exception handling rules, and test case templates. When executing and generating test cases, the knowledge base content is incorporated: During code pre-review, verify that the submitted test code is consistent with the company's specifications; Intelligent maintenance of default parameters in the test codebase and environment; Test case technology stack, writing standards, and code examples; Maintenance system requires specification documents and general design documents.

8. The Java test closed-loop method based on AI Agent-driven according to claim 7, characterized in that, The test log includes: test branch information; coverage value; set of uncovered code; number of test loops; and report path information.

9. A Java test closed-loop system based on AI Agent, employing the Java test closed-loop method based on AI Agent as described in any one of claims 1-8, characterized in that, The test closed-loop system includes: The user interaction module is used to receive test requirements input by users and convert natural language into task requests that the system can recognize. The AI-automated testing agent module is used for scheduling and decision control of the entire testing process; The AI-automated test case generation Agent module is used to generate new Java test code based on coverage analysis results. The public service integration module is used to uniformly manage the interaction between the two AI Agents and the local execution environment. The two AI Agents complete the capability call through standardized interfaces, realizing module decoupling and unified process control. The local tool execution module is used to perform corresponding operations according to the forwarded instructions and to provide feedback on the execution results.

Images