Code generation method based on multi-agent self-reflection framework

Abstract

The invention relates to a code generation method based on a multi-agent self-reflection framework, which combines a staged optimization process of prompt generation, test case generation, code generation, result check and code repair, and effectively solves the technical problem of unstable code generation quality in the prior art. By introducing a pruning method to perform quality screening on output of each stage and utilizing an intelligent feedback mechanism to realize efficient error repair and iteration, the code generation accuracy, reliability and efficiency are remarkably improved, and an innovative and efficient technical solution is provided for intelligent code generation of complex programming tasks.

Description

Technical Field

[0001] The present invention belongs to the intersection of artificial intelligence, natural language processing and software engineering, and specifically relates to a code generation method based on a multi-agent self-reflection framework. Background Art

[0002] In recent years, code generation technology has been widely studied and applied in the fields of artificial intelligence, natural language processing, and software engineering. Its goal is to automatically generate code that conforms to natural language descriptions. This technology can significantly improve the efficiency and quality of software development, so it has become one of the current research hotspots. Current technologies are mainly represented by large language models (LLMs), such as ChatGPT, which can generate code based on the user's natural language description through powerful natural language understanding and generation capabilities. These models have shown high effectiveness in simple tasks and have shown good application prospects.

[0003] In order to improve the performance of large language models, researchers have introduced the "Chain of Thought" (CoT) method to help the model better understand programming tasks by guiding it to reason step by step. For example, the "Structured CoT" (SCoT) method uses a predefined semantic step framework to combine prompts with programming paradigms to improve the logic and accuracy of code generation. In addition, in recent years, studies have also proposed a multi-agent framework. This method uses distributed task decomposition and collaboration to enable multiple agents to complete task understanding, code generation, test case generation, and error repair steps. Representative frameworks include CodeCoT and MapCoder. CodeCoT uses three agents to complete task analysis, code generation, and test case generation in turn, and regenerates code that fails the test. MapCoder generates code using a step-by-step solution description and verifies the correctness of the code with test data.

[0004] Although these technologies have improved the efficiency and quality of code generation to a certain extent, there are still many problems and shortcomings. First, large language models cannot fully guarantee the correctness of syntax and semantics during code generation. Especially in complex tasks, the generated code may contain logical errors or syntactic errors, causing the code to fail to run properly. Second, the prompt method based on the "thinking chain" is highly sensitive to the quality of the prompts. Slight differences in the prompts may lead to unstable generation results, especially in complex tasks. Low-quality code may be output due to incomplete reasoning chains or loose logic.

[0005] In addition, although the multi-agent framework optimizes the generation process through collaboration, there is still much room for improvement. The sequential execution mode of multi-agents means that once the upstream agent has a deviation in its understanding of the task, this error will be passed on to the subsequent agent, causing the entire generation process to fail. In addition, the existing framework is relatively simple in code quality control and fails to fully utilize the diversity of different agent outputs to optimize the final result. More importantly, the current framework usually relies on simple tests to verify code quality and lacks an intelligent feedback mechanism to analyze and fix the generated erroneous code, resulting in low generation efficiency and waste of computing resources.

[0006] In summary, although the existing code generation technology has made significant progress, there are still significant shortcomings in the reliability of code generation quality, the stability of prompts, the efficiency of multi-agent collaboration, and the intelligence of error repair. These problems limit the application potential of existing technologies in complex programming tasks. Summary of the invention

[0007] In view of the above problems existing in the prior art, the technical problem to be solved by the present invention is: the technical problem of unstable quality of code generation in the prior art.

[0008] To solve the above technical problems, the present invention adopts the following technical solution: a code generation method based on a multi-agent self-reflection framework, comprising the following steps:

[0009] S1: Prompt generation. Provide a natural language description of the programming task to the prompt agent. The prompt agent generates a series of "thinking chain" prompts based on the task description and stores the "thinking chain" prompts in the CoT prompt pool. The clarity, relevance, conciseness and context adaptability of all "thinking chain" prompts are evaluated by the pruning method. When the clarity, relevance, conciseness and context adaptability of the "thinking chain" prompt meet the corresponding preset values, the "thinking chain" prompt is a high-quality CoT prompt, otherwise it is a low-quality CoT prompt and is deleted.

[0010] S2: Test case generation. High-quality CoT is prompted to the test agent. The test agent generates multiple test cases based on the high-quality CoT prompt and stores them in the test case pool. The test cases are filtered through the pruning method to eliminate empty input, incomplete or invalid test cases. The remaining test cases constitute a high-quality test case set.

[0011] S3: Code generation. Pass the high-quality CoT prompt to the coding agent. The coding agent generates multiple code snippets based on the high-quality CoT prompt and stores the multiple code snippets in the code snippet pool. By executing the generated code snippets in the local environment, evaluate the syntax, logic, and executability of all code snippets. The code snippets without syntax errors are further evaluated for logic, and the code snippets without logic errors are evaluated for executability. Finally, the executable code snippets are obtained.

[0012] Adopt a pruning method to delete the code snippets with mismatched parentheses or unclosed strings in the executable code snippets, and the remaining code snippets are defined as high-quality code snippets.

[0013] S4: Result checking. Use a high-quality test case set to verify the high-quality code snippets obtained in S3.

[0014] For the i-th type of high-quality code snippet, n high-quality test cases in the test case set are used to verify the i-th type of high-quality code snippet, and n test results are obtained:

[0015] a) If the i-th type of high-quality code snippet passes n high-quality test cases, store the i-th type of high-quality code snippet in the ranked code set;

[0016] b) If the i-th type of high-quality code snippet passes m high-quality test cases, where m < n, repair the i-th type of high-quality code snippet;

[0017] For the repaired i-th type of high-quality code snippet, use the n - m high-quality test cases that did not pass in the previous round for verification:

[0018] ① If the number of high-quality test cases that did not pass the test is less than n - m, determine that the i-th type of high-quality code snippet is a repairable snippet, and repair the i-th type of high-quality code snippet again. Use the high-quality test cases that did not pass in the previous round to test the i-th type of high-quality code snippet after the second repair until the number of high-quality test cases that did not pass is equal to the number of high-quality test cases that did not pass in the previous time. Record the repair times of the i-th type of high-quality code snippet and store it in the ranked code set.

[0019] ② If the number of high-quality test cases that did not pass the test is equal to n - m, determine that the i-th type of high-quality code snippet is not repairable and discard the i-th type of high-quality code snippet.

[0020] c) If the i-th type of high-quality code snippet does not pass n high-quality test cases, the i-th type of high-quality code snippet is determined to be not repairable and discarded.

[0021] i traverses its values, and the verification of all high-quality code snippets is completed.

[0022] S5: Code ranking and output. The code snippets in the ranked code set are ranked from high to low according to the number of high-quality test cases passed. If the number of high-quality test cases passed is the same, the one with fewer repairs is ranked higher. Finally, the top k code snippets in the ranked code set are presented to the user.

[0023] Furthermore, the step of repairing the i-th high-quality code snippet in S4 includes:

[0024] The i-th high-quality code snippet and the high-quality test cases that failed the test are provided to the repair agent. The repair agent generates targeted repair suggestions and evaluates the clarity, relevance, simplicity and context adaptability of the repair suggestions through the pruning method. When the clarity, relevance, simplicity and context adaptability of the repair suggestion meet the corresponding preset values, the repair suggestion is defined as a high-quality repair suggestion, otherwise it is defined as a low-quality repair suggestion. The high-quality repair suggestion and the i-th high-quality code snippet are passed back to the coding agent for code repair to generate the repaired i-th high-quality code snippet.

[0025] Compared with the prior art, the present invention has at least the following advantages: 1. The present invention effectively solves the problem that the code generated by a large language model may have grammatical errors, logical defects or semantic incompleteness, resulting in the code being unable to meet user needs or pass test case verification through a self-reflective pruning mechanism. After the code is generated, the system evaluates the generated code through compiler feedback and test case results to identify potential grammatical errors, logical vulnerabilities and functional incompleteness. Low-quality code is eliminated through the pruning mechanism, and only high-quality code fragments that pass the evaluation are retained, thus ensuring that the final generated code can meet user needs and pass verification smoothly.

[0026] 2. The present invention effectively solves the problem that in the traditional multi-agent framework, errors in the task understanding stage may be transmitted to the code generation, testing and repair stages, resulting in a decline in the quality of the generated code by introducing a self-reflection mechanism. In the prompt generation stage, the system not only relies on the language model to generate preliminary prompts, but also prunes low-quality prompts that do not meet the standards by evaluating the clarity, relevance, conciseness and contextual adaptability of each prompt, thereby reducing the transmission of errors. In this way, the system ensures that the impact of erroneous input is minimized at each stage, ensuring that the subsequent code generation, testing and repair stages are more accurate and efficient.

[0027] 3. The present invention solves the problem that the test cases generated by the existing methods may have limited coverage and cannot fully verify the functional correctness and boundary conditions of the generated code by introducing a multi-round iterative pruning and optimization mechanism. After the code is generated, the system not only generates preliminary test cases, but also comprehensively verifies the generated code through a variety of test cases to ensure that different inputs, boundary conditions and possible abnormal situations are covered. At the same time, the system removes low-quality test cases through a pruning mechanism, retains high-quality test cases that can efficiently verify the functional correctness of the code, and ensures the comprehensiveness and depth of the test.

[0028] 4. The present invention improves the repair efficiency by introducing a combination of an automated repair agent (Repir Agent) and compiler feedback, effectively solving the problem that the repair process of the error code lacks systematicity and cannot effectively combine test feedback to generate high-quality repair suggestions, resulting in the problem that the quality and efficiency of the generated code cannot be further improved. When the error code is detected, the system first inputs the compiler feedback and the failed test cases into the large language model, provides specific repair suggestions through the large language model, and then automatically modifies the code according to these suggestions. The repaired code will be verified by testing again to ensure that the repair suggestions are efficient and effective. This process not only improves the repair efficiency, but also avoids inefficient and repeated repairs, ensuring that the generated code can reach high quality standards in the shortest time. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the overall framework disclosed in the present invention.

[0030] Figure 2 It is the hierarchical structure of the pruning process in the present invention.

[0031] Figure 3 To compare the Pass@1 scores of different models on different datasets, the present invention is CodeCoR.

[0032] Figure 4 Comparison of the average edit distance of different models. DETAILED DESCRIPTION

[0033] The present invention is described in further detail below.

[0034] The method of the present invention combines natural language processing technology, the high-level language understanding and generation capabilities of large language models, and a multi-agent collaboration mechanism to automatically generate high-quality code that meets the requirements of natural language description. The invention involves the analysis of programming tasks, code generation, test case generation, and error repair, and is widely used in intelligent auxiliary programming tools and automated code generation systems in software development.

[0035] See also Figure 1 and Figure 2 , 1. A code generation method based on a multi-agent self-reflection framework, characterized in that it includes the following steps:

[0036] The present invention proposes a multi-agent framework based on self-reflection, which effectively improves the quality and efficiency of code generation by introducing a pruning optimization mechanism and a phased intelligent workflow.

[0037] S1: Prompt generation. Provide a natural language description of the programming task to the prompt agent. The prompt agent generates a series of "Chain of Thought" (CoT) prompts based on the task description and stores the "Chain of Thought" prompts in the CoT prompt pool. The clarity, relevance, conciseness and context adaptability of all "Chain of Thought" prompts are evaluated by the pruning method. When the clarity, relevance, conciseness and context adaptability of the "Chain of Thought" prompt meet the corresponding preset values, the "Chain of Thought" prompt is a high-quality CoT prompt, otherwise it is a low-quality CoT prompt and is deleted.

[0038] Clarity is a measure of whether the prompt can concisely convey the core intent of the task. A clear prompt should avoid complex grammar or vague expressions, and allow the recipient to accurately understand its meaning. Clarity is usually calculated by analyzing the language structure, sentence length, and readability in the prompt. If the prompt is too complex or difficult to understand, it may be judged as low quality.

[0039] Relevance focuses on how relevant the prompt is to the current task. A highly relevant prompt should be closely related to the task goal and directly respond to the core requirements of the task. If the prompt contains content that is not relevant to the task or is too general, it will reduce relevance and thus affect the effectiveness of the prompt. Relevance can be calculated by evaluating the degree of semantic match between the task description and the prompt content. Tools such as semantic similarity are usually used to quantify its relevance.

[0040] Conciseness refers to whether the prompt avoids redundancy and unnecessary complexity when conveying information. A concise prompt should include necessary information without excessive or repeated information. Conciseness is usually assessed by analyzing the length and information density of the prompt. If the prompt contains lengthy expressions or each step is too complicated, it will be considered not concise.

[0041] Contextual adaptability is to determine whether the prompt can be adjusted according to the context, constraints and environment of the task. In a specific task context, the prompt needs to fully consider the specific requirements and preconditions of the task to ensure that it adapts to different contexts. Contextual adaptability is usually evaluated by comparing the fit between the steps in the prompt and the task context. If the prompt fails to take into account the specific environment or context of the task, it will be considered poorly adapted.

[0042] Under these criteria, thought chain prompts are divided into low-quality and high-quality categories. Low-quality prompts are usually prompts that fail to meet the basic requirements in one of the above aspects. For example, if a prompt is vague, irrelevant to the task, lengthy, or lacks contextual adaptability, it will be rated as a low-quality prompt. On the other hand, high-quality prompts usually perform well in all evaluation dimensions. They are clear, concise, closely related, and adaptable to the context and constraints of the task. S2: Test case generation. High-quality CoT prompts are given to the test agent. The test agent generates multiple test cases based on the high-quality CoT prompts and stores them in the test case pool. Test cases are screened by pruning methods to eliminate empty inputs, incomplete formats, or invalid test cases (such as lack of expected data or boundary conditions). The remaining test cases constitute a high-quality test case set.

[0043] S3: Code generation. The high-quality CoT hint is passed to the coding agent. The coding agent generates multiple code snippets based on the high-quality CoT hint and stores the multiple code snippets in the code snippet pool. By executing the generated code snippets in the local environment, all code snippets are evaluated for syntax, logic, and executableness. Code snippets without syntax errors are evaluated for logic, and code snippets without logic errors are evaluated for executableness. Finally, executable code snippets are obtained.

[0044] The pruning method is used to remove the code snippets with unmatched brackets or unclosed strings in the executable code snippets, and the remaining code snippets are defined as high-quality code snippets with correctness and execution efficiency potential.

[0045] When evaluating the syntax, logic, and executability of multiple code snippets, first use the compiler to verify whether all code snippets comply with the grammatical rules of the programming language. For code snippets that comply with the grammatical rules of the programming language, use test cases to evaluate whether the code snippets that comply with the grammatical rules of the programming language are correct, whether they meet the task requirements and can handle boundary conditions, and retain the code snippets that meet the requirements. Finally, by running the code snippets screened in the previous step and observing their execution results, retain the code snippets that can be executed correctly and produce the expected results.

[0046] S4: Result checking. Use a high-quality test case set to verify the high-quality code snippets obtained in S3.

[0047] For the i-th type of high-quality code snippet, n high-quality test cases in the test case set are used to verify the i-th type of high-quality code snippet, obtaining n test results.

[0048] a) If the i-th type of high-quality code snippet passes n high-quality test cases, store the i-th type of high-quality code snippet in the ranked code set.

[0049] b) If the i-th type of high-quality code snippet passes m high-quality test cases, where m < n, repair the i-th type of high-quality code snippet.

[0050] For the repaired i-th type of high-quality code snippet, use the n - m high-quality test cases that did not pass in the previous round for verification:

[0051] ① If the number of high-quality test cases that did not pass the test is less than n - m, determine that the i-th type of high-quality code snippet is a repairable snippet, and repair the i-th type of high-quality code snippet again. Then use the high-quality test cases that did not pass in the previous round to test the i-th type of high-quality code snippet after the second repair until the number of high-quality test cases that did not pass is equal to the number of high-quality test cases that did not pass in the previous time. Record the repair times of the i-th type of high-quality code snippet and store it in the ranked code set.

[0052] ② If the number of high-quality test cases that did not pass the test is equal to n - m, determine that the i-th type of high-quality code snippet is not repairable, and discard the i-th type of high-quality code snippet.

[0053] c) If the i-th type of high-quality code snippet does not pass n high-quality test cases, the i-th type of high-quality code snippet is determined to be not repairable, and discard the i-th type of high-quality code snippet.

[0054] Traverse its values of i, then the verification of all high-quality code snippets is completed.

[0055] S5: Code ranking and output. Rank the code snippets in the ranked code set in descending order according to the number of high-quality test cases passed. If the number of high-quality test cases passed is the same, rank them higher if the number of repair times is less. Finally, present the top k code snippets in the ranked code set to the user to ensure their optimal functional correctness and execution efficiency.

[0056] Specifically, the steps for repairing the i-th type of high-quality code snippet in S4 include:

[0057] Provide the i-th high-quality code snippet and the high-quality test cases that failed the test to the repair agent (Repair Agent, in the large language model). The repair agent generates targeted repair suggestions and evaluates the clarity, relevance, conciseness and context adaptability of the repair suggestions through pruning methods. When the clarity, relevance, conciseness and context adaptability of the repair suggestions meet their respective preset values, the repair suggestion is defined as a high-quality repair suggestion, otherwise it is defined as a low-quality repair suggestion. The high-quality repair suggestion and the i-th high-quality code snippet are passed back to the coding agent for code repair to generate the repaired i-th high-quality code snippet. The repaired code is tested and pruned again, and a new round of iteration is started until the generated code passes most test cases.

[0058] Specifically, the compiler feedback, the i-th high-quality code snippet, and the high-quality test cases that failed the test are input into the large language model. The large language model first analyzes the feedback information and gives repair suggestions based on the identified problems. Then, the large language model modifies the code based on these repair suggestions and generates a repaired code snippet.

[0059] Figure 2 The hierarchical structure of the pruning process in the present invention shows the specific process of pruning low-quality outputs in the prompt generation, code generation, test case generation and code repair stages.

[0060] In order to verify the effectiveness and advantages of the present invention in code generation tasks, we used two widely used programming datasets for evaluation: HumanEval and MBPP. The HumanEval dataset contains 164 Python programming questions, which are designed to evaluate the model's algorithmic reasoning ability, grammatical understanding ability, and programming flexibility; the MBPP dataset provides a variety of Python programming tasks, which are mainly used to evaluate the model's actual coding ability. To increase the challenge, we also used enhanced versions of these datasets, namely HumanEval-ET and MBPP-ET. These two datasets introduced more boundary conditions and complex test cases, with the aim of examining the performance of the model when facing more complex tasks.

[0061] We compare with multiple baseline models, especially with the currently widely adopted multi-agent frameworks such as MapCoder and CodeCoT, as well as single-agent methods such as SCoT, Self-Planning, and Reflection. All experiments are conducted in Python, using GPT-3.5-turbo as the base language model to perform code generation tasks, and the experiments are run on a dedicated server to ensure that other services or processes do not affect the measurement results.

[0062] We use several common evaluation indicators to measure the performance of this invention, which comprehensively consider the correctness, quality and efficiency of code generation. The main evaluation indicators include:

[0063] Pass@1: indicates whether the generated code can be executed correctly during the first test, which can measure the functional correctness of the generated code.

[0064] Average edit distance: Calculates the minimum transformation distance (insertion, deletion, substitution) between the generated code and the reference code, which is used to evaluate the structural fidelity of the generated code.

[0065] BLEU score: A metric borrowed from the field of machine translation that is used to evaluate the language accuracy and fluency of generated code versus reference code.

[0066] These indicators comprehensively reflect the functionality, structural integrity, and language quality of the generated code.

[0067] In our experiments, we first input programming problems from the HumanEval and MBPP datasets into the models we compared, including the present invention, SCoT, Self-Planning, MapCoder, and CodeCoT. After each model generated the corresponding code snippets, we performed a Pass@1 test on the codes to check whether they can correctly perform the expected tasks, and calculated the edit distance and BLEU score between the generated codes and the reference codes. In the experiments, we paid special attention to the performance differences of different models in generating codes, especially in terms of code correctness, structural fidelity, and language fluency.

[0068] In addition, to evaluate the running efficiency of the model, we also recorded the execution time, CPU and memory usage, disk I / O, and network I / O of each model. These metrics help us comprehensively evaluate the performance of the invention in resource consumption and understand its cost-effectiveness in code generation tasks.

[0069] Experimental results show that the present invention performs well in multiple indicators and is significantly better than existing multi-agent frameworks and single-agent methods. In terms of Pass@1 score, the present invention achieved 86.6% on the HumanEval dataset, far exceeding Self-Planning (65.2%), SCoT (60.6%), and Reflection (68.1%). On the MBPP dataset, the Pass@1 score of the present invention is 79.2%, which is also significantly higher than other methods (such as Self-Planning 58.6%, SCoT 67.0%, and Reflection 55.7%).

[0070] The present invention also shows excellent performance in terms of edit distance and BLEU score. On the HumanEval dataset, the average edit distance of the present invention is 378.79 and the average BLEU score is 0.276, both of which are better than most baseline models. In particular, on the MBPP dataset, the average edit distance of the present invention is 166.61 and the BLEU score is 0.351, which is significantly higher than Self-Planning and SCoT, highlighting the high semantic and structural consistency between the code generated by the present invention and the reference code.

[0071] In terms of execution efficiency, the present invention also shows superior performance. Compared with other frameworks, the running time of the present invention is 123.69 seconds, which is much lower than SCoT (251.79 seconds) and Self-Planning (242.92 seconds). In addition, the CPU utilization rate of the present invention is only 0.8%, which is much lower than SCoT's 5.2%, which shows that the present invention can efficiently complete tasks while saving computing resources. In terms of memory consumption, the present invention only uses 0.01GB of memory, compared to SCoT's 0.21GB, which shows the advantage of the present invention in memory management.

[0072] The present invention also performs well in terms of disk I / O and network resource consumption. Its disk write volume is 11.49 MB, which is significantly lower than SCoT's 162.90 MB, and the network traffic is more balanced than SCoT, indicating that the present invention is very efficient in terms of resource consumption.

[0073] Figure 3 To compare the Pass@1 scores of different models on different datasets, the present invention is CodeCoR, Figure 3 The Pass@1 scores of various code generation models (such as Few-Shot, ReAct, CodeCoR, etc.) on four datasets (HumanEval, HumanEval-ET, MBPP, MBPP-ET) are shown. The Pass@1 score indicates whether the code generated by the model in the first attempt can correctly perform the task. The higher the value, the higher the functional correctness of the code generated by the model.

[0074] Figure 4The average edit distance of our framework (CodeCoR) is smaller than that of existing methods, indicating that CodeCoR has a higher similarity with the target code when generating code and can more effectively generate solutions close to the correct code. A lower average edit distance means that the edit difference between the generated code and the reference code is smaller, indicating that CodeCoR has higher accuracy and smaller error rate in code generation tasks. In addition, CodeCoR has the largest average BLEU score, indicating that it is not only similar to the target code when generating code, but also has higher quality code when generating code, and can produce more solutions that are semantically consistent with the reference code. The BLEU score is an indicator that measures the similarity between the generated text and the reference text. A higher score means that CodeCoR performs better than other methods in code generation tasks and can generate high-quality code that is more in line with expectations. .

[0075] In summary, the experimental results fully demonstrate the significant advantages of the present invention in terms of correctness, quality, and resource consumption of generated code. It not only outperforms existing methods in traditional evaluation indicators such as Pass@1 score, edit distance, and BLEU score, but also has obvious cost-effectiveness in terms of execution efficiency and resource management. These experimental results show that the present invention is very competitive in code generation tasks, and is particularly suitable for practical application scenarios that have strict requirements for high-quality code generation and efficient resource management.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A code generation method based on a multi-agent self-reflection framework, characterized by: Including the following steps: S1: Prompt generation: Provide the natural language description of the programming task to the prompt agent; the prompt agent generates a series of "chain of thought" prompts according to the task description and stores the "chain of thought" prompts in the CoT prompt pool; Evaluate the clarity, relevance, conciseness, and context adaptability of all "chain of thought" prompts through pruning methods. When the clarity, relevance, conciseness, and context adaptability of the "chain of thought" prompt all meet their respective preset values, the "chain of thought" prompt is a high-quality CoT prompt; otherwise, it is a low-quality CoT prompt and is deleted; S2: Test case generation: Provide the high-quality CoT prompt to the test agent; The test agent generates multiple test cases according to the high-quality CoT prompt and stores them in the test case pool; Screen the test cases through pruning methods,剔除 empty inputs, incomplete or invalid test cases. The remaining test cases constitute a high-quality test case set; S3: Code generation: Pass the high-quality CoT prompt to the coding agent; the coding agent generates multiple code snippets according to the high-quality CoT prompt and stores the multiple code snippets in the code snippet pool; evaluate the syntax, logic, and executability of all code snippets by executing the generated code snippets in the local environment. Code snippets without syntax errors are further evaluated for logic, and code snippets without logic errors are evaluated for executability. Finally, code snippets with executability are obtained; Use pruning methods to delete code snippets with mismatched parentheses or unclosed strings in the executable code snippets. The remaining code snippets are defined as high-quality code snippets; S4: Result checking: Use the high-quality test case set to verify the high-quality code snippets obtained in S3; For the i-th high-quality code snippet, n high-quality test cases in the test case set are used to verify the i-th high-quality code snippet, obtaining n test results: a) If the i-th high-quality code snippet passes n high-quality test cases, store the i-th high-quality code snippet in the ranked code set; b) If the i-th high-quality code snippet passes m high-quality test cases, where m < n, repair the i-th high-quality code snippet; For the repaired i-th high-quality code snippet, use the n - m high-quality test cases that did not pass in the previous round for verification: ① If the number of high-quality test cases that did not pass the test is less than n - m, determine that the i-th high-quality code snippet is a repairable snippet, and repair the i-th high-quality code snippet again. Use the high-quality test cases that did not pass in the previous round to test the i-th high-quality code snippet after the second repair until the number of high-quality test cases that did not pass is equal to the number of high-quality test cases that did not pass in the previous time. Record the repair times of the i-th high-quality code snippet and store it in the ranked code set; ② If the number of high-quality test cases that did not pass the test is equal to n - m, determine that the i-th high-quality code snippet is irreparable and discard the i-th high-quality code snippet; c) If the i-th high-quality code snippet fails n high-quality test cases, the i-th high-quality code snippet is determined to be unrepairable and is discarded; i traverses its values, and the verification of all high-quality code snippets is completed; S5: Code ranking and output: The code snippets in the ranked code set are ranked from high to low according to the number of high-quality test cases that they pass. If the number of high-quality test cases that they pass is the same, the code snippets with fewer repairs are ranked higher. Finally, the top k code snippets in the ranked code set are presented to the user.

2. A code generation method based on a multi-agent self-reflection framework as claimed in claim 1, characterized in that: The step of repairing the i-th high-quality code fragment in S4 includes: The i-th high-quality code snippet and the high-quality test cases that failed the test are provided to the repair agent, which generates targeted repair suggestions and evaluates the clarity, relevance, conciseness and context adaptability of the repair suggestions through the pruning method. When the clarity, relevance, conciseness and context adaptability of the repair suggestion meet their respective preset values, the repair suggestion is defined as a high-quality repair suggestion, otherwise it is defined as a low-quality repair suggestion; the high-quality repair suggestion and the i-th high-quality code snippet are passed back to the coding agent for code repair to generate the repaired i-th high-quality code snippet.

Images