An intelligent incremental generation method of unit test cases for uncovered branches

Abstract

The application provides a unit test case intelligent incremental generation method for uncovered branches, and relates to the field of software testing. Specifically, combining code analysis and coverage analysis technology, each time the large model is called to supplement the test case, the test case is generated only for the uncovered branch of a branch node in the code, the existing test case which has coverage contribution to the current node is used as a seed input, and the coverage state of the test case at the current code branch node is also given, so as to reduce the difficulty of the large model in generating the test case, and improve the efficiency and accuracy of the incremental generation test case.

Description

Technical Field

[0001] This invention relates to the field of software testing, specifically to a method for intelligent incremental generation of unit test cases for uncovered branches. Background Technology

[0002] Software testing is a crucial step in ensuring software quality and improving its reliability. Unit testing, as the most fundamental testing phase, can identify code defects early, significantly reducing the cost of subsequent testing and fixing. Writing comprehensive and effective unit test cases is core to ensuring test quality. Manually designing test cases in unit testing is an extremely tedious and time-consuming task, heavily reliant on the experience of testers, and has become one of the main efficiency bottlenecks in the development process. To address this challenge, automated test case generation technologies have emerged. Symbolic execution and test case generation technologies based on artificial intelligence (such as genetic algorithms, constraint solving, and deep learning) have made significant progress. These technologies can automatically generate test inputs by analyzing the paths, branches, and logic of program code, effectively achieving approximately 80% code coverage in practice (such as statement coverage and branch coverage). This undoubtedly greatly liberates productivity, automatically covering a large number of basic and routine execution paths.

[0003] However, existing technologies reveal a significant "Pareto effect" or "long tail problem" in practice: automated tools can achieve the first 80% of coverage relatively efficiently, but improving the remaining 20% ​​of coverage (usually corresponding to the most complex and hidden paths in the program) becomes exceptionally difficult. This last 20% often involves complex logical constraints, requiring the satisfaction of multiple interdependent and complex combinations of conditions, exceeding the effective capabilities of traditional symbolic execution solvers; or it involves boundary and exceptional cases, those deep boundary conditions and exception handling processes that are extremely difficult to trigger through random or conventional strategies.

[0004] The addition of test cases for this final 20% coverage still heavily relies on test engineers manually constructing, analyzing, and debugging them based on their deep understanding of the business logic and code architecture. This process has proven to be highly time-consuming and intellectually intensive, becoming the ultimate bottleneck in improving test coverage and efficiency. It prevents the overall unit testing process from achieving true full automation, and the marginal effect of testing costs increases dramatically at this stage.

[0005] Therefore, there is an urgent need in this field for a solution that can effectively address the aforementioned last 20% of challenges, aiming to automatically or strongly assist in generating those high-value, high-difficulty test cases, in order to break through the ceiling of existing automated testing technologies and ultimately achieve a double leap in testing efficiency and coverage. Summary of the Invention

[0006] The technical problem solved by this invention is to propose an intelligent incremental generation method for test cases for uncovered branches, which aims to address the challenge that the efficiency of supplementing test cases drops sharply as coverage increases in unit testing, thereby effectively improving the efficiency and coverage of test case supplementation.

[0007] The solution of this invention is: an intelligent incremental generation method for unit test cases for uncovered branches, comprising: S1. Using the source code as input, analyze and generate a simplified function control flowchart. The function control flowchart consists of multiple branches composed of nodes and edges. A node with multiple outgoing edges is called a branch node. S2. Determine if the coverage rate has reached the threshold. If it has, the test ends. If the coverage rate requirement is not met, proceed to S3. The coverage rate is initially obtained by executing existing test cases. S3. Perform the coverage-oriented depth-first traversal function control flowchart. If the depth-first traversal termination condition is met, the test ends (the depth-first traversal here is also called depth-first search DFS); otherwise, go to S4. S4. Determine if the current node is a branch node. If not, return to S3; otherwise, execute S5. S5. Determine if the current branch node has reached the polling limit. If it has, mark the subsequent branches of the current branch node that are not covered as unreachable branches. Further determine if the current branch node has a covered branch. If not, the test ends. If it does, traverse the successor nodes of the covered branches and use the current successor node as the current node to execute S4. If the polling limit has not been reached, go to S6. S6. Determine whether the current branch node is fully covered. If yes, return to S3; otherwise, parse the current branch node to determine the test cases that contribute to the coverage of the current branch node and the branches they cover, and go to S7. S7. Select one use case from those that contribute to the coverage of the current node as a seed use case, and add the information of the branches it has covered to the prompt words. By adjusting the input data, generate new use cases that can cover other branches of the current node; and increment the polling count of the current node by 1; go to S8. S8. Execute the generated new test cases and overlay the coverage onto the existing test cases, then return to S2.

[0008] Preferably, the abstract syntax tree is traversed and analyzed based on Clang's RecursiveASTVisitor, and a simplified function control flow graph is generated by merging basic blocks.

[0009] Preferably, the parsing in S6 is to perform coverage reverse attribution analysis on the current branch node, that is, to calculate which test cases cover the node and to clarify the specific branch covered by each test case.

[0010] Preferably, in S7, the information of the branches that have been covered is added to the prompt words and provided to the large model. Based on the seed use case, the input data is adjusted to generate new use cases that can cover other branches of the current node.

[0011] Preferably, the control flowchart for the coverage-oriented depth-first traversal function means that when the current branch node has been processed and its successor node needs to be traversed, the successor node of the covered branch is traversed first.

[0012] A unit test case intelligent incremental generation system for uncovered branches includes a simplified control flow graph generation module, a coverage judgment module, a traversal module, a judgment module, a test case generation module, and a test case execution module. The simplified control flow graph generation module takes source code as input and analyzes and generates a simplified function control flow graph. The function control flow graph consists of multiple branches composed of nodes and edges. A node with multiple outgoing edges is called a branch node. The coverage determination module determines whether the coverage has reached a threshold. If it has, the test ends; if the coverage requirement is not met, the traversal module is triggered. The coverage is initially obtained by executing existing test cases. The traversal module performs a coverage-oriented depth-first traversal function to control the flow chart, determine the current node, and notify the judgment module. The judgment module determines whether the current node is a branch node. If not, it triggers the traversal module. Otherwise, it checks whether the current branch node has reached the polling limit. If it has, it marks any subsequent uncovered branches of the current branch node as unreachable branches. It further checks whether the current branch node has any covered branches. If not, the test ends. If it does, it traverses the successor nodes of the covered branches and uses the current successor node as the current node for a new judgment. If the polling limit has not been reached, it checks whether the current branch node is fully covered. If so, it triggers the traversal module again. If it is not fully covered, it parses the current branch node to determine the test cases that contribute to the coverage of the current branch node and the branches they cover, and sends the parsing results to the test case generation module. The test case generation module selects one test case from those that contribute to the coverage of the current node as a seed test case, adds information about the branches it has covered to the prompt words, and generates new test cases that can cover other branches of the current node by adjusting the input data; and increments the polling count of the current node by 1. The test case execution module executes the newly generated test cases, overlays the coverage onto the existing test cases, and sends the new coverage to the coverage judgment module.

[0013] The advantages of this invention compared to the prior art are: This invention proposes an incremental unit test case generation technique for uncovered branches based on the mutation of existing test cases by combining function control flowcharts, coverage analysis, and intelligent test case generation technology. This technique improves the efficiency and accuracy of incremental test case supplementation and reduces test case redundancy.

[0014] This invention discloses an intelligent incremental generation technology for unit test cases for uncovered branches. When the code coverage of existing test cases does not reach a preset threshold, this technology guides the large model to generate test cases that cover specific uncovered branches by comprehensively utilizing coverage analysis results, program control flow graph information, and existing test case sets, gradually increasing the coverage until the set threshold is met. This effectively solves the long-tail problem of difficulty in supplementing test cases in the high coverage stage.

[0015] This method, based on existing test cases, leverages a function control flow graph to focus on a branch node on the control flow graph that is not fully covered. It analyzes test cases that contribute to the coverage of this node, as well as the specific branches covered by the test cases. Using this test case as a seed test case, it guides the large model to generate new test cases for the branches not covered by the current node, until all branch nodes of the control flow graph are traversed. This method greatly reduces the difficulty of test case generation, enabling the large model to generate highly targeted test cases efficiently and accurately. It not only significantly improves generation efficiency and coverage but also ensures the quality of test case generation and reduces test case redundancy. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the process of the present invention. Figure 2 Here is the code for the example test.cpp; Figure 3 Here is the control flow diagram for the testStruct function; Figure 4 This refers to the coverage of test case_1; Figure 5 This refers to the coverage of test cases 1 and 2. Figure 6 This refers to the coverage of test cases 1, 2, and 3. Detailed Implementation

[0017] The present invention will be further described below with reference to the embodiments.

[0018] A method for intelligent incremental generation of unit test cases for uncovered branches, such as Figure 1 The following are included: S1. Taking the source code as input, the abstract syntax tree is traversed and analyzed based on Clang's RecursiveASTVisitor, and a simplified function control flow graph is generated by merging basic blocks; the function control flow graph consists of multiple branches composed of nodes and edges, and a node with multiple outgoing edges is called a branch node. S2. Determine if the coverage rate has reached the threshold. If it has, the test ends. If the coverage rate requirement is not met, proceed to S3. The coverage rate is initially obtained by executing existing test cases. S3. Perform a coverage-oriented depth-first traversal function control flowchart. That is, when the current branch node has been processed and it is necessary to traverse its successor node, the successor node (also known as child node) of the covered branch is traversed first. S4. Determine if the current node is a branch node. If not, return to S3; otherwise, execute S5. S5. Determine if the current branch node has reached the polling limit. If it has, mark the subsequent branches of the current branch node that are not covered as unreachable branches. Further determine if the current branch node has a covered branch. If not, the test ends. If it does, traverse the successor nodes of the covered branches and use the current successor node as the current node to execute S4. If the polling limit has not been reached, go to S6. S6. Determine whether the current branch node is fully covered. If so, return to S3. Otherwise, parse the current branch node to determine the test cases that contribute to the coverage of the current branch node and the branches they cover. The parsing here is to perform coverage reverse attribution analysis on the current branch node, that is, to calculate which test cases cover the node and to clarify the specific branches covered by each test case.

[0019] S7. Select one use case from those that contribute to the coverage of the current node as a seed use case, add the branch information it has covered to the prompt words, and generate new use cases that can cover other branches of the current node by adjusting the input data; and increment the polling count of the current node by 1. This step involves adding the information of the branches already covered to the prompt words and providing them to the large model. The requirement is to generate new test cases that cover other branches of the current node by adjusting the input data based on the seed test cases. For details on how to generate test cases for the large model, please refer to CN119988237A, or other methods can be used.

[0020] S8. Execute the generated new test cases and overlay the coverage onto the existing test cases, then return to S2.

[0021] The present invention further provides an intelligent incremental generation system for unit test cases for uncovered branches, including a simplified control flow graph generation module, a coverage judgment module, a traversal module, a judgment module, a test case generation module, and a test case execution module; The simplified control flow graph generation module takes source code as input and analyzes and generates a simplified function control flow graph. The function control flow graph consists of multiple branches composed of nodes and edges. A node with multiple outgoing edges is called a branch node. The coverage determination module determines whether the coverage has reached a threshold. If it has, the test ends; if the coverage requirement is not met, the traversal module is triggered. The coverage is initially obtained by executing existing test cases. The traversal module performs a coverage-oriented depth-first traversal function to control the flow chart, determine the current node, and notify the judgment module. The judgment module determines whether the current node is a branch node. If not, it triggers the traversal module. Otherwise, it checks whether the current branch node has reached the polling limit. If it has, it marks any subsequent uncovered branches of the current branch node as unreachable branches. It further checks whether the current branch node has any covered branches. If not, the test ends. If it does, it traverses the successor nodes of the covered branches and uses the current successor node as the current node for a new judgment. If the polling limit has not been reached, it checks whether the current branch node is fully covered. If so, it triggers the traversal module again. If it is not fully covered, it parses the current branch node to determine the test cases that contribute to the coverage of the current branch node and the branches they cover, and sends the parsing results to the test case generation module. The test case generation module selects one test case from those that contribute to the coverage of the current node as a seed test case, adds information about the branches it has covered to the prompt words, and generates new test cases that can cover other branches of the current node by adjusting the input data; and increments the polling count of the current node by 1. The test case execution module executes the newly generated test cases, overlays the coverage onto the existing test cases, and sends the new coverage to the coverage judgment module.

[0022] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the intelligent incremental generation method for unit test cases for uncovered branches.

[0023] The present invention also provides a computer software product, comprising: a processor and a storage device; Storage device for storing one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the intelligent incremental generation method for unit test cases for uncovered branches.

[0024] Example The following examples further illustrate the implementation of the present invention, but the implementation of the present invention is not limited thereto.

[0025] This example uses the project "test" as an example. The project "test" contains one source file, test.cpp, as follows: Figure 2 As shown (status displayed in Notepad++, with the numbers on the left representing line numbers). We will use the function `testStruct` in the source file `test.cpp` as an example to describe in detail the intelligent incremental generation process of test cases.

[0026] Assuming we have an initial test case_1 with input parameters s->a=0; s->b=0, this invention supplements the incremental test cases for the testStruct function in test.cpp with the following steps: The first step is to analyze the code in test.cpp to obtain a simplified control flow diagram of the testStruct function, such as... Figure 3 As shown: The second step is to execute the initial test cases and calculate the function coverage. The function has four branches. When the input to test case_1 is s->a=0; s->b=0, it covers two of the branches, resulting in a branch coverage rate of 50%. Figure 4 As shown, G represents coverage, R represents no coverage, and Y represents partial coverage.

[0027] The third step is to traverse the function flowchart. When the first branch node is reached, if (s->a>0), get the coverage of this node: case_1 has a coverage contribution to this node, contributing the False branch of this node. Fourth step: Using case_1 as the seed case, the prompt states that this case overrides the False branch at the branch node if (s->a>0). Please modify the input values ​​of this case to override the True branch. The large model generates a new case_2 with inputs s->a=1; s->b=0. Step 5: Execute test case_2 and merge its coverage with that of test case_1. At this point, it covers the True and False branches of the if (s->a>0) node, as well as the False branch of if (s->b>0), achieving a branch coverage of 75%. Figure 5 As shown: Step 6: Traverse the function's control flow graph again. When traversing to the if (s->a>0) node, if both of its two successor branches are covered, continue traversing. When traversing to the if (s->b>0) node, obtain its coverage status: Case 1 has a coverage contribution to this node, contributing the False branch of this node. Step 7: Using case_1 as the seed case, the prompt states that this case overrides the False branch at the branch node if (s->b>0). Please modify the input values ​​of this case to override the True branch. The large model generates a new case_3 with inputs s->a=0; s->b=1. Step 8: Execute test case_3 and merge its coverage with that of test cases_1 and_2. At this point, all four branches are covered, and the branch coverage is 100%. Figure 6 As shown, the traversal ends, and test case generation is complete.

[0028] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the intelligent incremental generation method for unit test cases for uncovered branches.

[0029] The present invention further provides a computer software product, including: a processor and a storage device; Storage device for storing one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the intelligent incremental generation method for unit test cases for uncovered branches.

[0030] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

[0031] The parts of this invention not described in detail are common knowledge to those skilled in the art.

Claims

1. A method for intelligent incremental generation of unit test cases for uncovered branches, characterized in that... include: S1. Using the source code as input, analyze and generate a simplified function control flowchart. The function control flowchart consists of multiple branches composed of nodes and edges. A node with multiple outgoing edges is called a branch node. S2. Determine if the coverage rate has reached the threshold. If it has, the test ends. If the coverage rate requirement is not met, proceed to S3. The coverage rate is initially obtained by executing existing test cases. S3. Perform the coverage-oriented depth-first traversal function control flowchart. If the depth-first traversal termination condition is met, the test ends; otherwise, go to S4. S4. Determine if the current node is a branch node. If not, return to S3. Otherwise, execute S5; S5. Determine if the current branch node has reached the polling limit. If it has, mark the subsequent branches that are not covered by the current branch node as unreachable branches. It further determines whether the current branch node has a covered branch. If not, the test ends. If it does, it traverses the successor nodes of the covered branches and uses the current successor node as the current node to execute S4. If the polling limit has not been reached, it goes to S6. S6. Determine whether the current branch node is fully covered. If yes, return to S3; otherwise, parse the current branch node to determine the test cases that contribute to the coverage of the current branch node and the branches they cover, and go to S7. S7. Select one of the use cases that contribute to the coverage of the current node as a seed use case, and add the information of the branches it has covered to the prompt words. By adjusting the input data, generate new use cases that can cover other branches of the current node. Then increment the polling count of the current node by 1 and go to S8; S8. Execute the generated new test cases and overlay the coverage onto the existing test cases, then return to S2.

2. The intelligent incremental generation method for unit test cases for uncovered branches according to claim 1, characterized in that: Clang's RecursiveASTVisitor traverses and analyzes the abstract syntax tree, generating simplified function control flow graphs by merging basic blocks.

3. The intelligent incremental generation method for unit test cases for uncovered branches according to claim 1, characterized in that: The parsing in S6 involves performing coverage reverse attribution analysis on the current branch node, which means calculating which test cases cover the node and identifying the specific branch covered by each test case.

4. The intelligent incremental generation method for unit test cases for uncovered branches according to claim 1, characterized in that: In S7, information about the branches that have already been covered is added to prompts and provided to the large model. The requirement is to generate new use cases that can cover other branches of the current node by adjusting the input data based on the seed use cases.

5. The intelligent incremental generation method for unit test cases for uncovered branches according to claim 1, characterized in that: The control flowchart for the coverage-oriented depth-first traversal function means that when the current branch node has been processed and it is necessary to traverse its successor node, the successor node of the covered branch is traversed first.

6. A smart incremental generation system for unit test cases for uncovered branches, characterized in that... This includes a simplified control flow graph generation module, a coverage judgment module, a traversal module, a judgment module, a test case generation module, and a test case execution module; The simplified control flow graph generation module takes source code as input and analyzes and generates a simplified function control flow graph. The function control flow graph consists of multiple branches composed of nodes and edges. A node with multiple outgoing edges is called a branch node. The coverage determination module determines whether the coverage has reached a threshold. If it has, the test ends; if the coverage requirement is not met, the traversal module is triggered. The coverage is initially obtained by executing existing test cases. The traversal module performs a coverage-oriented depth-first traversal function to control the flow chart, determine the current node, and notify the judgment module. The judgment module determines whether the current node is a branch node; if not, it triggers the traversal module. Otherwise, determine whether the current branch node has reached the polling limit. If it has, mark the subsequent uncovered branches of the current branch node as unreachable branches. It further determines whether the current branch node has a covered branch. If not, the test ends. If it does, it traverses the successor nodes of the covered branches and uses the current successor node as the current node to re-determine the branch. If the polling limit has not been reached, it determines whether the current branch node is completely covered. If so, it re-triggers the traversal module. If not fully covered, the current branch node is parsed to determine the test cases that contribute to the coverage of the current branch node and the branches they cover, and the parsing results are sent to the test case generation module. The test case generation module selects one test case from those that contribute to the coverage of the current node as a seed test case, adds information about the branches it has covered to the prompt words, and generates new test cases that can cover other branches of the current node by adjusting the input data. And increment the polling count of the current node by 1; The test case execution module executes the newly generated test cases, overlays the coverage onto the existing test cases, and sends the new coverage to the coverage judgment module.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the intelligent incremental generation method for unit test cases for uncovered branches as described in any one of claims 1 to 5.

8. A computer software product, characterized in that... include: Processors and storage devices; Storage device for storing one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the intelligent incremental generation method for unit test cases for uncovered branches as described in any one of claims 1 to 5.

Images