Intelligent optimization method for unit test coverage rate guided by using control flow

Abstract

The invention discloses a unit test coverage rate intelligent optimization method guided by using a control flow. The method comprises the following steps: S1, obtaining and preprocessing a to-be-tested code; s2, a cycle generation stage: calling a predefined tool by using a large generative model to dynamically obtain a code to be tested, and generating a unit test code; s3, repairing the unit test codes which fail to compile or run; s4, acquiring the coverage rate and coverage information of the unit test code; performing control flow analysis on the to-be-detected code according to the coverage information and the coverage rate, and outputting a control flow information set; and uncovered control flow information is fed back to the generation cycle stage as output. According to the method, the automation degree of unit test generation based on the large model is high, the effectiveness of the codes generated by the large model is effectively improved, the coverage rate of the unit test codes generated by the large model is obviously improved, and the problems that the effectiveness of the codes generated by other unit test automatic generation tools based on the large model is low and the coverage rate in a complex scene is low are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of software testing. Specifically, it relates to a unit test coverage intelligent optimization method and system using control flow guidance. BACKGROUND

[0002] Unit testing is a testing method that verifies the smallest testable unit in software, which can quickly find and locate defects in software, and is an important means to ensure software quality in software engineering. The coverage of unit testing is used to measure the coverage of unit testing on the code to be tested, which can be measured from the dimensions of lines, statements, branches, etc. High-coverage unit testing can more completely verify the code to be tested. However, manually writing high-coverage unit testing requires high time and labor costs. In order to overcome this challenge, unit test automatic generation tools have been proposed and widely used.

[0003] Randoop is an automatic unit test generation tool based on feedback-oriented random testing technology. It generates method call sequences randomly and uses execution feedback to avoid invalid or redundant tests, thereby efficiently generating regression test cases.

[0004] Evosuite is a unit test generation tool based on genetic algorithms. It optimizes test cases through evolutionary computation to maximize branch coverage and automatically generates high-coverage JUnit test cases.

[0005] Recently, researchers have combined large models with unit test automatic generation. By using the code semantic understanding ability obtained after learning the patterns and logic in the massive code base, the large model generates unit test code with high readability and better maintainability, which overcomes the problem of low readability and difficulty in maintaining the unit test code generated by traditional unit test automatic generation methods. However, existing unit test automatic generation methods based on large models still have the problems of low code effectiveness, low coverage of generated unit tests in complex scenarios, and strong dependence on the length of the input context of the large model. SUMMARY

[0006] In view of the above defects of the prior art, the present application provides a unit test coverage intelligent optimization method using control flow guidance, which makes the unit test generation based on large models highly automated, effectively improves the effectiveness of the code generated by the large model, significantly improves the coverage of the unit test code generated by the large model, and solves the problems of low code effectiveness and low coverage in complex scenarios of other unit test automatic generation tools based on large models.

[0007] To achieve the above purpose, the technical solution adopted by the present application is:

[0008] Firstly, a unit test coverage intelligent optimization method using control flow guidance includes the following steps:

[0009] S1. Obtain and preprocess the code to be tested;

[0010] S2, Generation Loop Phase: Use the generative large model to call predefined tools to dynamically obtain the code to be tested and generate unit test code;

[0011] S3. Fix the unit test code that failed to compile or run;

[0012] S4. Obtain the coverage rate and coverage information of the unit test code; perform control flow analysis on the code under test based on the coverage information and the coverage rate, and output a set of control flow information; feed back the uncovered control flow information as output to the generation loop stage.

[0013] Preferably, S1 includes:

[0014] S11. Obtain the project to be tested based on the project root path provided by the user;

[0015] S12. Traverse the files in the project to be tested, and filter out the files containing the code to be tested based on the file extension and content;

[0016] S13. Preprocess the code to be tested in the file to generate a clean and structured code representation; the preprocessing includes: removing comments, standardizing code format, extracting function or class definitions, identifying testable units and resolving code dependencies through static analysis tools, and constructing a dependency graph.

[0017] Preferably, S2 includes:

[0018] S21. Construct predefined prompt words; combine the code to be tested and the predefined prompt words into an input string; input the input string into the generative large model to generate the unit test code;

[0019] S22. Compile and run the unit test code in a predefined environment; if the code fails to compile or run, repair the unit test, generate new unit test code and obtain the corresponding coverage; if the coverage reaches the coverage threshold, exit the loop; otherwise, update the unit test code and its corresponding coverage and enter the next round of loop compilation and execution; output the unit test code and its corresponding coverage at the end of the loop.

[0020] Preferably, the predefined prompts include unit test generation instructions, the test framework used, test case templates, and coverage targets.

[0021] Preferably, in S2, the input string also includes historical unit test code and its corresponding coverage.

[0022] Preferably, S4 includes:

[0023] S41. Use a coverage information analysis tool to analyze the execution results of the unit test code and obtain the coverage information of the code under test;

[0024] S42. Determine whether the coverage rate has reached the user-defined threshold; if not, combine the coverage information with the code under test for control flow analysis, and according to the coverage information feedback mechanism, return the uncovered control flow information as output feedback to the generation loop stage for the next round of generation; if the user-defined threshold has been reached, output the unit test code.

[0025] Preferably, in S42, the coverage information feedback mechanism includes: integrating the coverage information into the predefined prompt words.

[0026] Preferably, the coverage information is stored in structured data, including row coverage information and branch coverage information.

[0027] Preferably, the control flow analysis includes:

[0028] Information about covered lines is obtained through line coverage information; information about covered branches is obtained through branch coverage information; and the control flow graph is obtained through the code under test. The control flow graph contains basic blocks and the jump conditions between basic blocks.

[0029] Traverse the basic blocks in the control flow graph. If a basic block is not in the covered line information, mark the basic block. Traverse the jump conditions in the control flow graph. If the jump condition encountered is not in the covered branch information, mark the jump condition.

[0030] Create a new control flow information set, traverse all possible control flows in the control flow graph, where a control flow is a subgraph of the control flow graph. If the current control flow contains a marked basic block or jump condition, add the current control flow information to the control flow information set; output the control flow information set.

[0031] Secondly, a unit test coverage intelligent optimization system using control flow guidance includes:

[0032] The code-to-test preprocessing module is used to acquire and preprocess the code to be tested;

[0033] The generation loop module is used to combine preprocessed code to be tested, predefined prompts and coverage information into the generative large model, output unit test code and compile and run it in a predefined environment to obtain the running results; it provides predefined tools to enable the large model to dynamically obtain the current project and code to be tested information during the generation process;

[0034] The code repair module is used to repair the unit test code that fails to compile or run.

[0035] The coverage information feedback module is used to obtain the coverage rate and coverage information of the unit test code; perform control flow analysis on the code under test based on the coverage information and the coverage rate, and output a set of control flow information.

[0036] The aforementioned intelligent unit test coverage optimization system using control flow guidance is used to implement the intelligent unit test coverage optimization method and its steps using control flow guidance as described in the first aspect.

[0037] Compared with the prior art, the beneficial effects of the present invention are reflected in:

[0038] 1) This invention incorporates control flow analysis into the coverage information feedback mechanism to obtain currently uncovered control flow information as feedback information, enabling large models to generate unit test code that is specifically designed to improve coverage.

[0039] 2) This invention provides predefined tools for large models during the generation cycle, enabling large models to dynamically acquire information about the current project and the code to be tested during the generation process. This allows large models to better understand the current project structure and the relevant context of the code to be tested, reducing the error rate of generated code.

[0040] 3) This invention uses a large language model as the core generator, which improves the quality of automated unit test generation. It also dynamically optimizes test cases through error correction iteration and coverage information feedback, ensuring that the generated unit tests have a low error rate and high coverage. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the method flow of Embodiment 1 of the present invention. Detailed Implementation

[0042] To make the technical means, inventive features, objectives, and effects of the invention readily understandable, the invention is further described below with reference to specific illustrations. However, the invention is not limited to the embodiments described below.

[0043] It should be understood that the structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only used to complement the content disclosed in the specification for those skilled in the art to understand and read, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0044] Example 1:

[0045] like Figure 1 The unit test coverage intelligent optimization method shown includes the following steps:

[0046] S1. Code Preprocessing Stage: Select code to be tested from the project and preprocess it, including:

[0047] S11. Obtain the project to be tested provided by the user. This project can be a code repository or a local code directory. The user needs to provide the root path of the project, and the system will traverse all files starting from this path.

[0048] S12. Iterate through the files in the project under test, filtering for files containing testable code based on file extensions and content. Exclude test files, configuration files, and other non-code files. Filtering rules can be customized according to user needs.

[0049] S13. Preprocess the code to be tested to generate a clean and structured code representation; preprocessing includes:

[0050] Remove comments by using code parsing tools to automatically remove single-line comments, multi-line comments, and docstrings from the code, reducing noise interference.

[0051] Standardize code formatting and use code formatting tools to unify code indentation, spaces, and line breaks to ensure code consistency;

[0052] Extract function or class definitions, identify testable units such as functions, methods, or classes using static analysis tools, and extract information such as their names, parameters, and return types;

[0053] Analyze code dependencies, examine import statements or dependencies in the code, construct a dependency graph, and ensure that external dependencies are handled correctly during subsequent test generation.

[0054] S2, Generation Loop Phase: Combine the preprocessed code to be tested, predefined prompts and coverage, and use the generative large model to generate the corresponding unit test code, then compile and run it. At the same time, provide callable tools for the large model so that the large model can dynamically obtain the current project and code to be tested information during the generation process.

[0055] S21. Combine the preprocessed code to be tested and the predefined prompts into an input string. The predefined prompts contain unit test generation instructions, the test framework used, the test case template (including setup, execution, and assertion steps), and the coverage target (such as achieving high line coverage or branch coverage).

[0056] If the generated unit test code and corresponding coverage from the previous steps exist, add them to the input string. Then input the string into the generative large model to generate the unit test code.

[0057] During the generation process, large models can call predefined tools to dynamically obtain the structure and code information of the current project.

[0058] Predefined tools include obtaining the current project directory structure and reading files related to the code to be tested;

[0059] S22. Compile and run the generated unit test code in a predefined environment. This environment includes the necessary dependency libraries, testing framework, and compilation tools. Record the execution results, including compilation errors, runtime errors, and coverage. Coverage is collected in real time using tools.

[0060] The unit test generation process of this invention includes:

[0061] First, initialize the algorithm's loop count limit (LoopCountLimit), coverage threshold (CoverageThreshold), and loop count (LoopCount). The loop execution condition is that the number of loops does not exceed the loop count limit. During unit test code generation, first combine predefined prompts, the code to be tested, the unit test code, and coverage into an input string. Then, input this string along with predefined tools into the large model to generate unit test code. Next, check if the generated unit test code can be compiled and run. If errors exist, repair the generated unit test code; the repair process is described in S3. Then, calculate the coverage of the generated unit tests on the code to be tested and determine if the generated coverage reaches or exceeds the coverage threshold. If it does, exit the loop; otherwise, replace the old unit test code with the generated unit test code, including the corresponding coverage. Finally, after the loop ends, output the last generated unit test code and its corresponding coverage as the result.

[0062] S3, Code Repair Phase: The generated unit test code is combined with the error information from the compilation and execution process to repair the code;

[0063] S31. First, determine if the generated unit test code encountered any errors during compilation or runtime. Error types include syntax errors, undefined functions or types, and failed assertions. If no errors are found, the code and execution results are used as output for the code repair phase.

[0064] S32. If an error is found, combine the code to be tested, the generated unit test code, the error message, and the repair request into a single prompt. For example, a repair request could be: "Please fix the errors in the following unit test code to ensure the code compiles and runs correctly." The prompt structure is: Repair Request + Code to be Tested + Unit Test Code + Error Message, with each part separated by a delimiter.

[0065] S33. Set the relevant parameters for repairing the large model, including kernel sampling p (value is 1.0), maximum length of generated text max_length (value is 8192), and temperature parameter temperature (value is 1.0). Input the prompt word into the repaired large model to generate the repaired unit test code.

[0066] S34. Compile and run the repaired unit test code in the predefined environment. Determine if the maximum number of iterations for the code repair phase has been reached (e.g., 5 times). If not, and the code still has errors, repeat steps S31-S34.

[0067] The specific implementation of unit test code repair in this embodiment of the invention is as follows: First, the loop count limit is initialized to 5 and the counter Count is 0. If the counter does not exceed the loop count limit, the unit test code is compiled and run to obtain error information during the process. Then, it is determined whether the error information is empty; if it is empty, the loop ends; otherwise, execution continues. Subsequently, the predefined repair prompt words, the code to be tested, the unit test code, and the error information are combined into an input string, the counter Count is incremented by 1, and the string is input into the large repair model to generate the repaired unit test code. If the generated program is still incorrect, the process will continue until the counter exceeds the loop count limit.

[0068] S4, Coverage Feedback Phase: Analyze the execution results of the generated unit test code, obtain coverage information, and provide feedback.

[0069] S41. Use coverage information analysis tools to analyze the execution results of the unit test code output during the code repair phase to obtain coverage information of the code under test, including line coverage, branch coverage, and statement coverage. Coverage information is stored in structured data, including a list of covered and uncovered code lines and branches.

[0070] S42. Determine if the coverage rate has reached the user-defined threshold (e.g., 75% line coverage). If not, combine the coverage information (uncovered lines of code, branches) with the code under test for control flow analysis, and feed the uncovered control flow information back to the generation loop stage as output for the next generation round; if the threshold has been reached, use the generated unit test code as the output of the coverage feedback stage.

[0071] Coverage information feedback mechanism: During the generation cycle, coverage information is integrated into the prompts. For example, prompts such as "Current coverage is X%; please generate test cases to cover the following uncovered lines: [list of line numbers] and branches: [list of branches]" can be added to guide the large model to generate targeted test cases, thereby gradually improving coverage.

[0072] The control flow analysis method of this invention, which combines coverage information with the code under test, is specifically implemented as follows:

[0073] First, the system obtains covered line information through line coverage information, covered branch information through branch coverage information, and a control flow graph (CFB) from the code under test. The CFB contains basic blocks and transition conditions between these blocks. Then, it iterates through the basic blocks in the CFB; if a basic block is not found in a covered line, it is marked. It also iterates through the transition conditions; if a transition condition is not found in a covered branch, it is marked. Next, an empty control flow information set is created, and all possible control flows (where a control flow is a subgraph of the CFB) are traversed. If the current control flow contains a marked basic block or transition condition, it is added to the control flow information set. Finally, the control flow information set is output.

[0074] This patent uses control flow analysis to construct the control flow graph of the code under test, and combines coverage information to obtain path information that can reach the parts of the code under test that are not covered by the current test code as auxiliary information, so that the large model can generate test code with higher coverage.

[0075] Example 2:

[0076] This invention also provides a unit test coverage intelligent optimization system using control flow guidance, the system being used to run the aforementioned unit test coverage intelligent optimization method using control flow guidance, comprising:

[0077] The code-to-test preprocessing module is used to: obtain the project to be tested provided by the user, traverse the project to obtain the code in the project that can be used as test objects as the code to be tested, and preprocess the code to be tested;

[0078] The generation loop module is used to: combine the preprocessed code to be tested, predefined prompts and coverage information into the generative large model; put the unit test code output by the large model into a predefined environment for compilation and execution to obtain the running results; and provide predefined tools to enable the large model to dynamically obtain information about the current project and code to be tested during the generation process.

[0079] The code repair module is used to: determine whether the unit test code generated by the loop generation module has errors. If so, the code generated by the loop generation module is output. If not, the code, error message and repair request are combined into a prompt word. The prompt word is input into the repair model to obtain the repaired code and running results as output, so as to improve the effectiveness of the unit test code.

[0080] The coverage information feedback module is used to: analyze the output results of the code repair module, obtain the coverage of the code under test and determine whether the threshold has been reached. If not, the coverage information (uncovered code lines and branches) is combined with the code under test to perform control flow analysis, and the currently uncovered control flow information is fed back to the generation loop module as output. If yes, the generated unit test code is used as output.

[0081] Example 3: Comparative Experiment

[0082] Table 1: Comparison of LLMCovImpv with other methods

[0083]

[0084] LLMCovImpv is a prototype implementation of the method described in this patent on the Java language and related infrastructure; Chattester is a state-of-the-art automated unit test generation tool based on a conversational large model; Randoop is a traditional automated unit test generation tool based on feedback-oriented random test generation technology.

[0085] Table 1 shows the instruction coverage, branch coverage, and line coverage results of the unit test code generated by LLMCOvImpv, Chattester, and Randoop on seven different projects. To reduce the impact of randomness in the generation of large model code, five experiments were conducted for each target project, and the evaluation values ​​of the results are presented. It can be seen that, compared with the latest automated unit test generation tools based on large models and traditional automated unit test generation tools, the instruction coverage, branch coverage, and line coverage of the unit test code generated by LLMCOvImpv are all improved compared to Chattester on these seven projects, indicating that this method can indeed generate code with higher coverage.

[0086] Example 4: Ablation experiments of different components.

[0087] Table 2: Ablation Experiment Results

[0088]

[0089] Here, we conduct ablation experiments after removing some components from LLMcovImpv to illustrate the impact of each component on the results. Two methods were used in the ablation experiments: one removed the control flow analysis and coverage feedback components (the column in the table indicating the method without control flow analysis, hereinafter referred to as Ablation Experiment 1), and the other removed the iterative process (the column in the table indicating the method without iterative processes, hereinafter referred to as Ablation Experiment 2). It can be observed that, compared to Ablation Experiment 1 and Ablation Experiment 2, the unit tests generated by this method show a certain degree of decrease in instruction coverage, branch coverage, and line coverage across the selected seven items. This indicates that control flow analysis, coverage feedback, and iterative processes can support the generation of unit tests with higher coverage.

[0090] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A unit test coverage intelligent optimization method using control flow guidance, characterized in that, Includes the following steps: S1. Obtain and preprocess the code to be tested; S2, Generation Loop Phase: Use the generative large model to call predefined tools to dynamically obtain the code to be tested and generate unit test code; S3. Fix the unit test code that failed to compile or run; S4. Obtain the coverage rate and coverage information of the unit test code; perform control flow analysis on the code under test based on the coverage information and the coverage rate, and output a set of control flow information; The uncovered control flow information is fed back as output to the generation loop stage.

2. The intelligent optimization method for unit test coverage using control flow guidance according to claim 1, characterized in that, S1 includes: S11. Obtain the project to be tested based on the project root path provided by the user; S12. Traverse the files in the project to be tested, and filter out the files containing the code to be tested based on the file extension and content; S13. Preprocess the code to be tested in the file to generate a clean and structured code representation; Preprocessing includes: removing comments, standardizing code format, extracting function or class definitions, identifying testable units and resolving code dependencies using static analysis tools, and constructing a dependency graph.

3. The intelligent optimization method for unit test coverage using control flow guidance according to claim 1, characterized in that, S2 include: S21. Construct predefined prompt words; combine the code to be tested and the predefined prompt words into an input string; input the input string into the generative large model to generate the unit test code; S22. Compile and run the unit test code in a predefined environment; if the code fails to compile or run, repair the unit test, generate new unit test code and obtain the corresponding coverage; if the coverage reaches the coverage threshold, exit the loop; otherwise, update the unit test code and its corresponding coverage and enter the next round of loop compilation and execution; output the unit test code and its corresponding coverage at the end of the loop.

4. The intelligent optimization method for unit test coverage using control flow guidance according to claim 3, characterized in that, The predefined prompts include unit test generation instructions, the test framework used, test case templates, and coverage targets.

5. The intelligent optimization method for unit test coverage using control flow guidance according to claim 3, characterized in that, In S2, the input string also includes historical unit test code and its corresponding coverage.

6. The intelligent optimization method for unit test coverage using control flow guidance according to claim 1, characterized in that, S4 include: S41. Use a coverage information analysis tool to analyze the execution results of the unit test code and obtain the coverage information of the code under test; S42. Determine whether the coverage rate has reached the user-defined threshold; if not, combine the coverage information with the code under test for control flow analysis, and according to the coverage information feedback mechanism, return the uncovered control flow information as output feedback to the generation loop stage for the next round of generation; if the user-defined threshold has been reached, output the unit test code.

7. The intelligent optimization method for unit test coverage using control flow guidance according to claim 6, characterized in that, In S42, the coverage information feedback mechanism includes: integrating the coverage information into the predefined prompt words.

8. The intelligent optimization method for unit test coverage using control flow guidance according to claim 7, characterized in that, The coverage information is stored in structured data, including row coverage information and branch coverage information.

9. The intelligent optimization method for unit test coverage using control flow guidance according to claim 1, characterized in that, The control flow analysis includes: Information about covered lines is obtained through line coverage information; information about covered branches is obtained through branch coverage information; and the control flow graph is obtained through the code under test. The control flow graph contains basic blocks and the jump conditions between basic blocks. Traverse the basic blocks in the control flow graph. If a basic block is not in the covered line information, mark the basic block. Traverse the jump conditions in the control flow graph. If the jump condition encountered is not in the covered branch information, mark the jump condition. Create a new control flow information set, traverse the control flows in the control flow graph, where each control flow is a subgraph of the control flow graph. If the current control flow contains a marked basic block or jump condition, add the current control flow information to the control flow information set; output the control flow information set.

10. A unit test coverage intelligent optimization system using control flow guidance, characterized in that, include: The code-to-test preprocessing module is used to acquire and preprocess the code to be tested; The generation loop module is used to combine preprocessed code to be tested, predefined prompts and coverage information into the generative large model, output unit test code and compile and run it in a predefined environment to obtain the running results; it provides predefined tools to enable the large model to dynamically obtain the current project and code to be tested information during the generation process; The code repair module is used to repair the unit test code that fails to compile or run. The coverage information feedback module is used to obtain the coverage rate and coverage information of the unit test code; perform control flow analysis on the code under test based on the coverage information and the coverage rate, and output a set of control flow information. The aforementioned intelligent unit test coverage optimization system using control flow guidance is used to implement the intelligent unit test coverage optimization method and its steps using control flow guidance as described in claim 1.

Images