Code generation system and method based on test-driven reasoning

Abstract

The application discloses a code generation system and method based on test-driven reasoning, and belongs to the technical field of computer systems.The code generation system comprises a test preposition module, which is used for extracting warehouse context before code implementation and driving a large language model to generate an initial unit test suite, and taking the test as an executable behavior boundary; a code generation module, which is used for generating initial implementation code based on a test specification; a reflection improvement module, which is used for performing and reflecting iteration cycles on the code and the test, and modifying the source code and the unit test suite according to execution state feedback, so as to realize double-track joint optimization; and an intelligent agent toolbox, which provides context retrieval and code interaction capabilities.The application breaks through the limitation of traditional methods that take the test as an immutable constraint, allows the model to correct the code and the test simultaneously according to feedback, alleviates self-test bias, and makes the unit test suite converge to the real specification; and significantly improves the stability and accuracy of the generated code in the warehouse-level code generation task.

Description

Technical Field

[0001] This invention relates to the field of computer system technology, and in particular to a code generation system and method based on test-driven reasoning. Background Technology

[0002] Large Language Models (LLMs) have demonstrated outstanding performance in a range of software engineering tasks, particularly in code generation. Their potential has garnered significant attention from both academia and industry.

[0003] To improve model performance, researchers have integrated software engineering methodologies into the code generation process based on large language models. Pair Coder constructs a multi-agent framework that separates high-level planning from concrete implementation, thereby improving code quality. In addition to the planning phase, the integration of software testing methods has proven particularly effective. Existing research shows that providing models with external unit test suites significantly improves their performance on the MBPP and HumanEval datasets compared to relying solely on problem descriptions. Furthermore, AgentCoder, through a multi-agent system, decouples the roles of programmers and testers, demonstrating that explicit test generation effectively improves code correctness. Retrieval augmented generation (RAG) is also widely used to address complex repository-level code completion tasks. Existing retrieval augmentation methods typically employ sparse bag-of-words models (such as Jaccard similarity matching) or dense vector retrieval to identify and provide semantically similar code snippets. However, the effectiveness of these methods is often limited by the quality of the retrieved context, and because they treat unit test suites only as static, post-hoc verification tools, they fail to fully leverage the potential of testing in guiding model inference.

[0004] However, relying on models to generate their own unit test suites carries inherent risks. Recent research has analyzed two debugging paradigms: post-execution debugging and in-execution debugging. The study found that post-execution debugging is often affected by test biases introduced by self-generated tests, leading to misleading feedback; while in-execution debugging allows large language models to mitigate this bias by utilizing intermediate states. Furthermore, most current testing-centric research is limited to function-level tasks. In these scenarios, the process of generating independent functions from natural language descriptions is relatively simple, and the corresponding unit test suite synthesis is relatively straightforward (e.g., writing basic assertion statements). However, in project repository-level tasks, the complexity of dependencies and the need to simulate the runtime environment significantly increase the difficulty of generating unit test suites. Whether self-testing strategies remain effective in this scenario remains inconclusive.

[0005] A key limitation of existing research is the hard segmentation of agent roles: large language models are typically only tasked with correcting implementation code, without being given the task of verifying or correcting the validity of the unit test suite itself. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a code generation system and method based on test-driven reasoning, which solves the technical problems in existing code generation technologies, such as the lack of executable specifications for guidance, the use of generated tests only as static post-verification tools, the difficulty in co-optimizing code implementation and test feedback, and the insufficient correctness and robustness of generated code.

[0007] This invention discloses a code generation system based on test-driven reasoning, including a test pre-module, a code generation module, a reflection and improvement module, and an intelligent agent toolbox; The pre-test module is used to schedule the intelligent agent toolbox to extract the code context related to the target code to be generated from the code repository before the target code is generated, and drive the large language model to automatically generate an initial unit test suite based on the code context. A code generation module is used to generate initial implementation code for the target code based on the specifications of the initial unit test suite; The reflection and improvement module performs an iterative loop of reflection based on the initial implementation code and the initial unit test suite. It uses the current execution status feedback generated during the reflection loop to jointly reason about the expected correctness of the current unit test suite and the logical rigor of the current implementation code. If the current unit test suite is deemed to be incorrect, it is modified. If the current implementation code is deemed to be logically flawed, it is modified. The module synchronously outputs the updated implementation code and unit test suite, generating updated execution status feedback based on these until the end condition of the reflection loop is met. Ultimately, this completes the dual-track joint optimization of the current implementation code and the current unit test suite, yielding results including the target code and the final unit test suite. The intelligent agent toolkit provides context retrieval and code execution interaction to support the large language model in retrieving, editing, and executing operations on the code repository, as well as submitting results.

[0008] Optionally, the pre-test module uses structured system prompts to enable the large language model to output the initial unit test suite before writing the initial implementation code, and uses the initial unit test suite as an executable behavior boundary specification. This behavior boundary specification is then used in the reflection and improvement module to guide the writing of subsequent current implementation code.

[0009] Optionally, the logic is not rigorous because the current implementation code has omitted non-standard execution paths.

[0010] Optionally, the non-standard execution path includes boundary value inputs, abnormal inputs, or special branches, or the processing result does not meet expectations.

[0011] Optionally, the execution status feedback includes at least one of the following types: syntax and compiler-level feedback, complete stack trace information in runtime exception feedback, difference between expected and actual output in logic assertion feedback, runtime timeout or memory overflow errors in boundary and performance feedback, and test specification feedback.

[0012] Optionally, it also includes a state-aware context guidance module, used to dynamically inject guiding instructions into the context of the large language model according to the current iteration stage: Once the large language model submits its initial unit test suite for the first time, an instruction is injected to guide it in generating the initial implementation code. Once the large language model submits its initial implementation code, an instruction is injected to guide it through sandbox execution and verification. Once the large language model receives the execution status feedback, it injects corresponding analysis prompts based on the test pass results.

[0013] Another aspect of the present invention discloses a code generation method based on test-driven reasoning, using the aforementioned code generation system based on test-driven reasoning, comprising the following steps: Step S1: Before the implementation code of the target code, extract the code context related to the target code to be generated from the code repository, and drive the large language model to generate an initial unit test suite based on the code context; Step S2: Generate the initial implementation code of the target code based on the specification of the initial unit test suite; Step S3 involves performing an iterative cycle of execution and reflection on the initial implementation code and the initial unit test suite. Based on the current execution status feedback generated during the reflection iteration cycle, a joint inference is made regarding the expected correctness of the current unit test suite and the logical rigor of the current implementation code. If the current unit test suite is determined to be incorrect, it is modified. If the current implementation code is determined to be logically flawed, it is modified. Simultaneously, the updated implementation code and unit test suite are output. Updated execution status feedback is generated based on the updated implementation code and unit test suite until the termination condition of the reflection iteration cycle is met. Finally, the dual-track joint optimization of the current implementation code and the current unit test suite is completed, yielding results including the target code and the final unit test suite. Step S4: Submit the results using the intelligent agent toolbox.

[0014] Optionally, in step S3, the specific steps of simultaneously modifying the current implementation code and unit test suite generated during the iteration process based on the execution status feedback include: When the execution status feedback contains assertion errors and the large language model determines that the current implementation code is consistent with the behavior boundary specification, it is determined that the unit test suite environment is distorted, which drives the large language model to relax the test boundary or correct the simulation logic to update the unit test suite. When the execution status feedback includes runtime crashes, infinite loop timeouts, or a significant deviation between the actual output and the expected output, the unit test suite standard is deemed valid, driving the large language model to locate the line number of the crashing code to correct the implementation code.

[0015] Optionally, in step S3, a prefix caching mechanism is enabled during multiple iterations to cache the context tensors processed in previous iterations.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: 1. The test-driven reasoning-based code generation system of this invention employs a "test-first" strategy as a high-level chain of thought in code reasoning. Unlike natural language processing systems, which often suffer from ambiguity or vagueness, unit tests, as executable specifications, rigorously define the solution space and boundary conditions. This constraint forces the model to transform natural language requirements into rigorous, executable unit tests before coding. This constraint significantly reduces logical errors in the generated code.

[0017] 2. The test-driven reasoning-based code generation system of this invention introduces a dual-track refinement mechanism to address test bias. This overcomes the limitation of traditional methods that treat tests as immutable constraints, allowing the model to simultaneously correct both the code and the tests themselves based on test feedback. This effectively alleviates "self-testing bias" and enables the generated unit test suite to gradually converge towards the real specification with iterations.

[0018] 3. The code generation system based on test-driven reasoning of the present invention, in terms of system architecture, adopts a single-agent framework to retain the complete dialogue history and context, and gives the single agent the autonomy to dynamically modify code and tests, simulating a coherent "Test-Driven Development (TDD)" thinking process, enabling the model to iteratively align code and tests, and avoiding the complexity overhead caused by role coordination in multi-agent systems.

[0019] 4. The code generation system and method based on test-driven reasoning of the present invention, in terms of model generalization ability, enables smaller open-source models (such as Qwen3-Coder with 30 parameters) to surpass the code generation capabilities of closed-source super-large models (GPT-5-mini) with advanced RAG technology. It exhibits extremely high stability and accuracy in repository-level code generation tasks, verifying the effectiveness of the framework as a generalized reasoning framework. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the embodiments will be briefly introduced below. The features and advantages of the present invention can be more clearly understood by referring to the accompanying drawings. The accompanying drawings are schematic and should not be construed as limiting the present invention in any way. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the code generation system based on generative test-driven reasoning of the present invention. Detailed Implementation

[0022] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0023] A specific embodiment of the present invention, such as Figure 1 As shown, a code generation system based on generative test-driven reasoning is disclosed, including a test pre-module, a code generation module, a reflection and improvement module, and an intelligent agent toolbox.

[0024] Specifically, test generation refers to designing and generating test code for the target code before generating the target code. This test code aims to comprehensively cover the expected behavior of the target code, including regular functional tests, boundary condition tests, and exception handling tests. In practice, the specific form of "test" will vary depending on the granularity of the code generation task.

[0025] Furthermore, the pre-test module is used to schedule the intelligent agent toolbox to extract the code context related to the target code to be generated from the code repository before the target code is generated.

[0026] Specifically, the extracted code context includes, but is not limited to: the docstring related to the target code, upstream and downstream call relationships (i.e., the caller code that calls the target code), other processing methods belonging to the same code class as the target code (a code class usually contains multiple processing methods, which can be understood as the context within the same file), and reference code in the repository that has similar functions to the extracted code context.

[0027] Subsequently, the pre-test module automatically generates an initial unit test suite by driving the large language model to call the standard file editing tools in the intelligent agent toolkit. This generation process relies on the existing intelligent agent tool invocation mechanism and can automatically adapt to different programming language environments. For example, the suite generated for Python can be a test suite based on pytest or unittest, while for Java it can be a test class based on JUnit.

[0028] This invention comprehensively understands the requirements of the target code according to the above process. By generating an initial unit test suite, it eliminates ambiguities in the requirements and defines precise behavioral boundaries for the target code. These behavioral boundaries are used to guide subsequent implementation code writing in the reflection and improvement module, improving code quality. Specifically, the initial unit test suite constructs typical daily use scenarios, boundary conditions, and abnormal input conditions around the requirement description, transforming the implicit functional expectations in the requirements into executable unit test suites. These unit test suites explicitly specify the output results, state changes, and exception handling methods that the target code should produce under different input conditions, thus forming specific constraints on the behavior of the target code.

[0029] Optionally, the behavioral boundaries include the expected behavior of the target code under normal input (normal functional performance), the exception handling logic for illegal or unexpected input (such as throwing a specific error), and the expected behavior under extreme conditions such as null values ​​and extreme values.

[0030] Furthermore, the code generation module generates initial implementation code for the target code based on the specifications of the initial unit test suite, and then enters the reflection and improvement module for an iterative loop of execution and reflection. Based on execution status feedback, the code generation module and the reflection and improvement module simultaneously modify the currently generated target code and the current unit test suite, achieving dual-track joint optimization of the target code and the unit test suite, and obtaining results including the finally confirmed target code and unit test suite. Specifically, the execution status feedback includes normal status signals when the test passes completely (such as a system exit code of 0), and multi-dimensional exception contexts generated during sandbox execution verification of the current unit test suite and the current target code. This execution status feedback includes, but is not limited to: (1) Syntax and compiler-level feedback: such as syntax errors or static type checking prompts during script parsing; (2) Runtime exception feedback: that is, the complete stack trace information thrown when the code crashes during execution, such as array out of bounds, null pointer reference (NullPointerException) or division by zero error; (3) Logical assertion feedback: When the test fails, the test framework returns detailed comparison information, including the specific location of the assertion failure and the difference between the "expected output" and the "actual output". (4) Boundary and performance feedback: For example, timeout warnings caused by code getting stuck in an infinite loop, or out of memory (OOM) errors; (5) Test standardization feedback: When the system execution finds that the code logic is reasonable, but repeated errors occur due to the assertions of the unit test suite itself being too strict, improper environment simulation (Mock) or outdated parameter types, this test standardization feedback will directly drive the reflection and improvement module to downgrade or correct the unit test suite.

[0031] Furthermore, the intelligent agent toolkit provides context retrieval and code execution interaction, supporting directory browsing, structural analysis, global search, symbol resolution, file editing, and test execution and result submission, providing support for the pre-test module, code generation model, and reflective improvement module. Specifically, result submission refers to the final output of the confirmed target code and its accompanying test suite as the final deliverable to the user after the dual-track joint optimization of the code and unit test suite reaches the preset pass standard (i.e., all unit tests pass and the unit test suite is sufficiently robust and comprehensive), or after reaching the maximum number of iterations set by the system. This marks the official end of the code generation task.

[0032] Furthermore, the workflow of the code generation module of the present invention is strictly controlled by the protocol, and is started through system instructions and user prompts, clearly defining the role of the Large Language Model (LLM) and injecting a specific repository context (such as target code metadata).

[0033] Specifically, the generation of the target code and unit test suite in this invention is driven by a large language model (such as Qwen, DeepSeek, or the GPT series). Hints are provided within the given system instructions and repository context. Under this premise, the large language model generates text sequences through an autoregressive approach. Among them, in the context of a given system command and repository, hints are provided. Under the condition of text sequence The expression for the conditional probability distribution is:

[0034] in, This represents the joint probability of a text sequence Y generated by a large language model, given system instructions and repository contextual hints R. This indicates that the large language model is based on the existing preceding sequence for the i-th lexical unit. System commands and repository context hints Generate the i-th word element The conditional probability, i=1,2,…n, where n represents the text sequence. The total number of tokens contained (i.e., the total length of the sequence). It is a text sequence The preceding sequence consists of all the lexical units preceding the i-th lexical unit.

[0035] Furthermore, , Represents a text sequence The i-th token generated in the process.

[0036] For each word, the underlying layer of the large language model captures the long-term dependencies of the complex repository context through a multi-head attention mechanism. Specifically, this complex repository context and long-term dependencies are quantitatively represented by the context window length supported by the large language model, typically covering a span of 8,192 to 128,000 tokens, sufficient to map dozens of interconnected underlying code files and function call chains. During attention computation, the large language model utilizes the current token and all historical inputs (i.e., system instructions and repository context hints). and the generated preceding sequences The hidden layer vectors are multiplied by the model's pre-trained linear mapping weight matrix (query weight matrix). With key weight matrix Specifically, the query weight matrix maps hidden layer vectors to query vectors, and the key weight matrix maps hidden layer vectors to key vectors, thus calculating their respective query and key matrices. The large language model dynamically calculates the dot product of the query and key matrices to obtain attention weights. After normalization using a feedforward neural network and the Softmax function, the probability distribution of each word in the vocabulary is output. This probability distribution is represented as a floating-point vector with a dimension equal to the total size of the underlying vocabulary of the large model. Each value in this floating-point vector represents the word at the corresponding index within the vocabulary as the next word. The confidence probability of the sampled data (with values ​​in the range [0,1] and the sum of the vector elements is 1) is used to generate word-by-word meta-generators for the text sequence.

[0037] Specifically, after receiving the target code generation task, the aforementioned pre-test module uses a large language model. First, act as a test designer, generating an initial test suite for the target code. Let... It is a collection of external tools that can be called upon by a large language model. This is the target code to be generated. After the large language model extracts key context through the tool module, the system instructions, through prompt word constraints, require it to output an initial unit test suite before writing the initial implementation code. Specifically, this is achieved through Structured System Prompts. During the task initialization phase, system instructions with strict output format specifications are injected into the large language model, requiring it to first output an initial unit test suite, expressed as:

[0038] in, This represents the generation mapping function when a large language model acts as a test designer.

[0039] Understandably, this function represents: within the global context of a given system instruction and the current code repository. and target code Given the input information, the Big Prophecy model utilizes a set of callable external tools. Based on the conditional basis (i.e., the model can autonomously decide how to use these tools), after autoregressive inference calculations, an initial unit test suite is finally generated and output. The complete dynamic process.

[0040] This invention's initial test suite generation step treats it as a high-level, formalized thought process. It drives the large language model by injecting prompts containing target code names, file paths, line numbers, import paths, and contextual prefixes. The system explicitly instructs the large language model to focus on usage examples and write executable initial unit test suites that demonstrate expected function calls. To ensure the stability of the generated initial unit test suites, the pre-test module guides the large language model to prioritize building typical call examples of the main "happy path," rather than getting bogged down in excessive generation of edge cases or stress tests. When the large language model submits the initial unit test suite via the result submitter... This signifies that the large language model has established clear and executable behavioral boundaries.

[0041] Understandably, the "happy path" of the main branch refers to the default execution path of the target code or module under ideal conditions where there are no exceptions, errors, or boundary conditions. It represents the core business logic and expected functionality of the code; that is, the test scenario where the target code can run smoothly and return the expected correct output when receiving fully compliant and valid input. In terms of specific implementation, this system uses instruction injection technology based on Constrained Prompt Engineering to ensure the execution of this priority.

[0042] Specifically, when the pre-test module constructs the context input to the large language model, the system explicitly injects priority control instructions into the prompt template. For example, by injecting mandatory descriptions into the system instructions, such as "test functions that validate regular valid inputs must be output first, followed by tests for null values ​​or out-of-bounds exceptions," this context injection technique based on structured prompts can directly guide the output probability distribution of the large language model during the autoregressive inference stage without changing the underlying weights of the model, thereby forcing the agent to focus on generating tests for the core logic.

[0043] Furthermore, the code generation module and the reflection and improvement module constitute a dynamic "execution-reflection-correction" feedback loop to eliminate the inherent biases in the generation of large language models. This is the core of solving the self-testing bias of large language models. The module consists of three stages: initial implementation generation, sandbox execution verification, and dynamic reflection and dual-track optimization.

[0044] Specifically, the initial implementation generation refers to the large language model. Based on the initial unit test suite generated by the pre-test module As a draft specification, it serves as the target code to be generated. Generate initial implementation code System instructions will guide the large language model in writing the initial implementation code. First, draft a logic plan and analyze how to meet the initial unit test suite. The test was conducted, and the required data structures and algorithm steps were outlined.

[0045] It is worth noting that the system instruction grants the large language model the authority to self-correction logic: if the large language model is writing its initial implementation code... When you realize that the initial unit test suite you generated has logical flaws, you can trust your own judgment and write the correct logic instead of blindly conforming to the wrong tests.

[0046] Specifically, the aforementioned sandbox execution verification refers to the system automatically performing initial implementation code verification. Weave it into the repository's physical environment and execute the initial unit test suite via the Test Runner. Receive initial execution status feedback .

[0047] Furthermore, the expression for the execution status feedback is:

[0048] in, This represents the execution status feedback for the t-th iteration; This represents the implementation code for the t-th iteration; Represents the unit test suite for the t-th iteration; Furthermore, the initial execution status feedback This includes the pass rate of the unit test suite, failure assertions, deviations between expected and actual values, and detailed error stack trace information. The system will report this initial execution status. Provided to large language models In order to enable large language models The initial unit test suite generated for it Compared with the initial implementation code Conduct iterative reflection.

[0049] Furthermore, the dynamic reflection and dual-track optimization of this invention are key differences from traditional frameworks. Traditional large-model debugging suffers from "self-testing bias," where the model treats its own generated tests as absolute truth, thus tampering with correct code logic to pass incorrect unit test suites. This invention innovatively introduces a dual-track optimization mechanism for large language models. Received initial execution status feedback Then, simultaneously implement the code for the t-th iteration. and unit test suite If the logical correctness is questioned, a joint reflection is conducted, and updates to the implementation code and unit tests are output simultaneously to obtain the implementation code of the target code in the (t+1)th iteration. and unit test suite The expression is:

[0050] in, This represents the implementation code for the (t+1)th iteration; This represents the unit test suite for the (t+1)th iteration; This represents the execution status feedback for the t-th iteration; This represents the joint reasoning and state update function of the large language model during improved reflection.

[0051] It is understandable that the model is implemented with the code for the t-th iteration. Unit test suite Execution status feedback And system prompts and context As a combined input, it autonomously makes decisions and schedules text editing tools to ultimately map the output to the implementation code and unit test suite after the (t+1)th update.

[0052] Large language models can determine the current unit test suite by using the latest execution status feedback and past memories. The expectation was incorrect (tests were modified), which may also indicate a problem with the current implementation of the target code. The logic is not rigorous (code modification). Specifically, rigorous logic means that the implementation code covers daily scenarios, boundary conditions, and exceptional cases, with complete branch judgments and expected results. Incomplete logic means that the implementation code omits non-standard execution paths (such as boundary values, abnormal inputs, and special branches) or the handling does not meet expectations, requiring modification. Through this two-way reflection, the quality of the unit test suite (TOA, Test-Prediction Consistency) and the implementation code (TSC, Test Self-Consistency) are mutually calibrated in multiple iterations, ultimately achieving co-improvement. Specifically, when the sandbox execution status feedback... The system displays an assertion error, but the model backtracking system commands and repository context hints... Then determine the current implementation code When the core business logic is consistent with the specifications, it is usually determined that the unit test suite is overly stringent or the environment is distorted. For example, the unit test suite may have hard-coded matching for string case, or made absolute value assertions for unpredictable timestamps or random numbers; or the structure of the mock objects may not match the actual dependencies. The large language model generates diagnostic decisions to relax test boundaries or correct mock logic, and then calls a text editing tool to partially rewrite the current unit test suite. For example, the absolute equality assertion (==) may be changed to regular expression matching, type validation (isinstance), or exception handling assertions (pytest.raises). When the sandbox execution status feedback... When a unit test suite contains explicit runtime crashes (such as NullPointerException, array out-of-bounds errors), timeouts caused by infinite loops, or when the actual output deviates significantly from the expected output of a valid unit test suite (logical calculation errors), the large language model determines the validity of the unit test suite. The standard is valid, so the focus shifts to the current implementation code. Initiate correction. The large language model locates the sandbox execution status feedback. Find the specific line number of the code that caused the crash, and use a text editor to modify it, such as "add null check guards", "correct loop boundary conditions", or "adjust algorithm branch logic".

[0053] Furthermore, the aforementioned intelligent agent toolbox model refers to a collection of external tools that can be invoked by a large language model. The goal is to enable large language models to perform contextual exploration within code repositories. Specific implementations include: Directory Viewer: Used to visualize the file system hierarchy of a code repository, helping the agent understand the project's organizational structure and module division.

[0054] Structure Analyzer: Through the parsing capability of Abstract Syntax Tree (AST), it extracts the class, method, and function signatures of the target file, enabling the intelligent agent to quickly grasp the macro structure of the file while shielding a large amount of code implementation details.

[0055] File Reader: Allows large language models to accurately retrieve the specific content of a particular file within a limited range of lines, preventing the text of extremely large files from exceeding the context window of the large language model.

[0056] File Editor: Allows large language models to modify code files, specifically including line insertion and string replacement operations, enabling precise modification of code files.

[0057] Global Search: Based on the powerful text search tool (ripgrep), it performs efficient lexical string matching across the entire code repository to locate global variables, utility classes, or configuration items.

[0058] Symbol Resolver: A Deep Integration Language Server Protocol (LSP) service, unlike lexical search, this tool precisely resolves symbol definitions and cross-file references, solving the problem of tracking complex dependencies.

[0059] Test Runner: Calls the pytest testing framework in Python to execute unit tests generated by the agent in an isolated environment, providing objective status feedback as the basic data for feedback loop closure.

[0060] The Result Submitter acts as the core trigger for state machine transitions. When submitting artifacts (implementation code or test suites) for each iteration, the agent must include a natural language description to reduce model illusions and improve the quality of generated artifacts.

[0061] Furthermore, to prevent context drift during multiple rounds of reflection, the system employs a state-aware context guidance module. This module dynamically injects guiding prompts into the context of the large language model based on the current stage. This prompts are first introduced when the large language model submits its initial unit test suite. At that time, the system will inject prompts to guide it to the next step of generating the specific initial implementation code of the target code. When the large language model submits its initial implementation code Afterwards, the system will inject prompts to guide it through sandbox execution verification to obtain real execution status feedback, enabling dynamic reflection and dual-track optimization; when the large language model receives status feedback... Then, the system will provide different guidance based on the status feedback: if all current unit test suites pass, it will guide the large language model. The system strengthens the currently generated initial unit test suite to ensure sufficient testing strength. If a test fails (some or all test cases fail), the system dynamically injects failure analysis prompts into the large language model, guiding it to analyze whether the failure is caused by code logic or test writing errors, effectively mitigating context drift and attention loss caused by multiple rounds of interaction.

[0062] The entire iteration loop will terminate when one of the following two conditions is met: 1. The agent is highly confident that the current unit test suite... The current implementation code is correct, error-free, and adequately tested. If the implementation is complete and neither requires further modification, then a commit with type: "final" is issued through the result committer. This type indicates that the current commit is the final version and requires the agent to explain why it believes that the current version has reached the optimal state and no further improvement is needed.

[0063] 2. When the loop reaches the preset maximum computing resource budget limit (maximum number of iterations set to 5 rounds), the system will forcibly terminate the loop and output the current optimal result to ensure that resource consumption is controllable.

[0064] Due to the iterative nature of the TDD-Agent code generation system, it inherently consumes more tokens as the dialogue history (including tool calls, test reports, reflective reasoning, and code revisions) accumulates. More tokens represent more runtime and higher overhead. Therefore, the system of this invention enables a prefix caching mechanism during its underlying deployment. This mechanism significantly amortizes the computational cost of reprocessing historical data by caching the context tensors (KV cache) processed in previous rounds, ensuring that the marginal computational cost of subsequent iterations remains within a controllable range. Through optional early stopping, a maximum iteration limit, and the prefix caching mechanism, an optimal balance between computational resource consumption and code generation quality is achieved.

[0065] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A code generation system based on test-driven reasoning, characterized in that, It includes a pre-testing module, a code generation module, a reflection and improvement module, and an intelligent agent toolbox; The pre-test module is used to schedule the intelligent agent toolbox to extract the code context related to the target code to be generated from the code repository before the target code is generated, and drive the large language model to automatically generate an initial unit test suite based on the code context. A code generation module is used to generate initial implementation code for the target code based on the specifications of the initial unit test suite; The reflection and improvement module is used for an iterative loop of reflection based on the initial implementation code and the initial unit test suite. It performs joint reasoning on the expected correctness of the current unit test suite and the logical rigor of the current implementation code based on the current execution status feedback generated in the reflection iteration loop. When it is determined that the current unit test suite is incorrect, the current unit test suite is modified. If the current implementation code is deemed to have flawed logic, modify the current implementation code. It simultaneously outputs the updated implementation code and unit test suite, updates the execution status feedback based on the updated implementation code and unit test suite, until the end condition of the reflection iteration loop is reached, and finally completes the dual-track joint optimization of the current implementation code and the current unit test suite, and obtains the result including the final target code and unit test suite; The intelligent agent toolkit provides context retrieval and code execution interaction to support the large language model in retrieving, editing, and executing operations on the code repository, as well as submitting results.

2. The code generation system based on test-driven reasoning according to claim 1, characterized in that, The pre-test module uses structured system prompts to enable the large language model to output the initial unit test suite before writing the initial implementation code. The initial unit test suite is then used as an executable behavior boundary specification, which is used in the reflection and improvement module to guide the writing of subsequent current implementation code.

3. The code generation system based on test-driven reasoning according to claim 1, characterized in that, The logic is not rigorous because the current implementation code has omitted a non-standard execution path.

4. The code generation system based on test-driven reasoning according to claim 3, characterized in that, The non-standard execution path includes boundary value input, abnormal input, special branches, or processing results that do not meet expectations.

5. The code generation system based on test-driven reasoning according to claim 1, characterized in that, The execution status feedback includes at least one of the following types: syntax and compiler-level feedback, complete stack trace information in runtime exception feedback, difference between expected and actual output in logic assertion feedback, runtime timeout or memory overflow errors in boundary and performance feedback, and test specification feedback.

6. The code generation system based on test-driven reasoning according to claim 1, characterized in that, It also includes a state-aware context guidance module, which dynamically injects guiding instructions into the context of the large language model based on the current iteration stage: Once the large language model submits its initial unit test suite for the first time, an instruction is injected to guide it in generating the initial implementation code. Once the large language model submits its initial implementation code, an instruction is injected to guide it through sandbox execution and verification. Once the large language model receives the execution status feedback, it injects the corresponding analysis prompts based on the test pass results.

7. A code generation method based on test-driven reasoning, using the code generation system based on test-driven reasoning according to any one of claims 1-6, characterized in that, Includes the following steps: Step S1: Before generating the target code, extract the code context related to the target code to be generated from the code repository, and drive the large language model to generate an initial unit test suite based on the code context; Step S2: Generate the initial implementation code of the target code based on the specification of the initial unit test suite; Step S3: Perform a reflective iterative loop on the initial implementation code and the initial unit test suite, and perform joint reasoning on the expected correctness of the current unit test suite and the logical rigor of the current implementation code based on the current execution status feedback generated in the reflective iterative loop; when it is determined that the current unit test suite is incorrect, modify the current unit test suite. If the current implementation code is deemed to have flawed logic, modify the current implementation code. It simultaneously outputs the updated implementation code and unit test suite, updates the execution status feedback based on the updated implementation code and unit test suite, until the end condition of the reflection iteration loop is reached, and finally completes the dual-track joint optimization of the current implementation code and the current unit test suite, and obtains the result including the final target code and unit test suite; Step S4: Submit the results using the intelligent agent toolbox.

8. The code generation method based on test-driven reasoning according to claim 7, characterized in that, In step S3, when it is determined that the current unit test suite is incorrect, the specific steps for modifying the current unit test suite include: When the current execution status feedback contains assertion errors and the large language model determines that the current implementation code is consistent with the behavior boundary specification, it is determined that the current unit test suite environment is distorted, driving the large language model to relax the test boundaries or correct the simulation logic to modify the current unit test suite.

9. The code generation method based on test-driven reasoning according to claim 7, characterized in that, In step S3, when it is determined that the current implementation code logic is not rigorous, the specific steps for modifying the current implementation code include: When the current execution status feedback includes runtime crashes, infinite loop timeouts, or a significant deviation between the actual output and the expected output, it is determined that the current unit test suite standard is valid, driving the large language model to locate the line number of the crashed code in order to correct the current implementation code.

10. The code generation method based on test-driven reasoning according to claim 7, characterized in that, In step S3, a prefix caching mechanism is enabled during multiple iterations to cache the context tensors processed in previous iterations.

Images