Large language model test case generation method based on path semantic perception

Abstract

The invention discloses a path semantic perception-based large language model test case generation method, which comprises the following steps of: analyzing a to-be-tested item by utilizing a static analysis technology, and constructing an item-level dependency call graph to extract code elements and dependency relationships thereof; obtaining a dependency entity of the function to be tested according to the project dependency graph, and analyzing an execution path of the dependency entity; generating semantic intention description of each path by utilizing a large language model based on a thinking chain path analysis method, and generating test cases consistent with path behaviors according to the semantic intention description; importing the generated test case into a test framework for execution, obtaining a path coverage report and identifying an uncovered path; and according to a coverage feedback result, generating a supplementary test case for the uncovered path by using a large language model, and realizing iterative optimization of the test case. According to the method, path-level refined test case generation can be realized, the test coverage rate and the generation accuracy are improved, manual intervention is reduced, and the efficiency and the intelligent level of automatic testing are improved.

Description

Technical Field

[0001] This invention relates to a test case generation method, belonging to the field of software engineering and automated testing technology. Background Technology

[0002] Unit testing is a crucial step in the software development process used to verify the correctness of the smallest functional units of a program (such as functions or methods). High-quality unit testing can identify and locate potential defects early in the software development process, preventing errors from spreading in later stages and effectively ensuring the reliability and maintainability of the software system.

[0003] As software systems become increasingly larger and more complex, the cost of manually writing and maintaining high-quality unit tests has risen significantly, becoming one of the major burdens in the software development process. To reduce the cost of manual testing, researchers have proposed various automated unit test generation techniques. Existing automated methods mainly fall into three categories: search-based test generation methods, constraint-solving-based test generation methods, and randomized strategy-based test generation methods. These methods typically aim to maximize program coverage by generating a set of test cases that cover different code paths through heuristic search or constraint solving. Tools like EvoSuite utilize evolutionary algorithms to automate test suite generation, improving the level of test automation to some extent. However, these methods still differ from manual testing in terms of the semantic rationality, readability, and matching degree with actual requirements of the generated tests, limiting their application in engineering practice. Furthermore, when faced with programs containing complex control flow, recursive structures, or non-trivial constraints, traditional methods have significant limitations in terms of generation efficiency and coverage.

[0004] To further improve the structure and accuracy of generated tests, researchers have proposed a unit test generation method based on symbolic execution. This type of method systematically explores the program's execution path by symbolizing input variables and solving for path constraints, thereby generating more comprehensive test data. Although symbolic execution methods perform well in small or medium-sized programs, they are susceptible to path explosion when dealing with large industrial-grade code and are difficult to effectively scale to complex nested structures or deeply dependent programs.

[0005] In recent years, with the rapid development of large language models in code understanding and generation, researchers have begun to explore their application in automated unit test generation. These methods typically input the implementation of the focus method and the context of similar methods into the large language model, leveraging the model's ability to automatically generate test cases using code semantic modeling capabilities. Research results show that large model-based generation methods can generate a certain number of executable test cases in various programming architecture scenarios and improve the automation level of test generation to some extent. However, existing large language model-based test generation techniques still have the following shortcomings: (1) Existing models mainly rely on large-scale data-driven learning mechanisms, lack symbolic reasoning capabilities, and are difficult to accurately analyze the logical branches and constraints in the program, resulting in the generated test cases failing to cover the critical path; (2) The lack of a path semantic sensitive analysis mechanism makes it impossible to effectively explore multi-path execution scenarios in multi-branch control flow and deep nested structures; (3) The context information in the construction is not fully utilized. When there is too little input context, the model has difficulty understanding the semantics of the program. However, too much context will distract the model and affect its understanding of the focus method and the generation effect.

[0006] In summary, while existing automated unit test generation technologies have evolved from traditional search and symbolic execution methods to large language model-driven approaches, they still struggle to simultaneously guarantee semantic correctness and high test coverage while also considering contextual understanding and program logic reasoning capabilities. Therefore, there is an urgent need for an automated test generation method that combines path semantic sensitivity analysis and semantic reasoning mechanisms to fully leverage the test generation potential of large language models in complex software systems and improve test quality and generation efficiency. Summary of the Invention

[0007] The purpose of this invention is to provide a method for generating test cases for a large language model based on path semantic awareness. By combining program dependency analysis and semantic reasoning capabilities, it can achieve automated and high-coverage software unit test generation and optimization, thereby effectively improving the intelligence level and semantic consistency of testing, reducing manual testing costs, and enhancing the reliability and maintainability of software.

[0008] To achieve the above objectives, the technical solution of this invention discloses a method for generating test cases for a large language model based on path semantic awareness, characterized by comprising the following steps: Step 1: Project dependency call graph construction, including: Static analysis techniques are used to analyze the project under test and extract code elements including functions, structures, and variables. Based on the calls, references, and dependencies between code elements, a project dependency call graph is constructed to describe the dependency structure between various code elements in the project, providing contextual information for subsequent intent recognition; Step 2, the method for generating path semantic awareness test cases, includes: Based on the project dependency call graph, the function to be tested and its related dependency entities are obtained, providing contextual semantic support for subsequent path analysis; Static analysis tools are used to extract the control flow graph of the function under test and identify all feasible execution paths on the control flow graph. In the path enumeration process, a goal-driven heuristic path selection mechanism is adopted to automatically prune redundant paths in order to address the potential path explosion problem, thereby controlling the analysis complexity while ensuring test coverage. The thinking chain path analysis method is used to perform semantic reasoning and behavioral intent analysis on each feasible execution path, and generate corresponding path semantic intent descriptions; Based on the generated path semantic intent, a large language model is invoked to generate test cases that are consistent with the path semantics, so as to achieve accurate test generation and high coverage verification at the path level. Step 3: Test case optimization based on coverage feedback, including: Import the generated test cases into the existing test framework for execution and collect path coverage reports; Identify uncovered paths based on the path coverage report results; For the uncovered paths, supplementary test cases are generated again using the large language model; Repeat the optimization process until the coverage reaches the preset coverage.

[0009] Preferably, in step one, the constructed project dependency graph uses code entities as nodes and call, reference, and containment relationships as edges, depicting the hierarchical structure and calling relationships between code elements. This fully reflects the logical connections between various functional units in the project and intuitively reflects the interaction paths between functions and their upstream and downstream dependent entities. Preferably, in step two, the nodes of the control flow graph represent statements or basic blocks, and the edges represent possible control flow transition relationships.

[0010] Preferably, in step two, the heuristic path selection mechanism includes: when the number of execution paths exceeds a preset threshold, the heuristic path selection mechanism is automatically activated. The heuristic path selection mechanism selects the most representative subset of paths from the full set of paths with branch coverage as the optimization objective.

[0011] Preferably, the heuristic path selection mechanism first constructs a "list of branches to be covered" and records the true / false branch status of all branch statements; then, it iteratively selects paths that can cover new branches and continuously updates the "set of covered branches" until the coverage goal is achieved or the path budget is exhausted.

[0012] Preferably, in step two, in order to generate the path execution intent description, the system first constructs a contextual prompt for the current path, which includes dependent entities and code path information, function input parameters and return types. Then, the contextual information is input into the large language model, and the model is guided to generate the semantic intent description of the path through instruction prompts.

[0013] Preferably, in step two, generating test cases based on path semantic intent includes: using the path semantic intent as input prompts, calling a large language model to generate test cases consistent with the path behavior; and combining the dependent entities and context information of the function under test to ensure that the generated test cases have semantic consistency and executability.

[0014] Preferably, in step three, the feedback-based test case optimization includes: importing the generated test cases into the existing test framework for execution and collecting path coverage reports; identifying unreached execution paths based on the coverage results; and generating new supplementary test cases using a large language model for the uncovered paths, and iteratively executing and optimizing them until the path coverage reaches a preset threshold.

[0015] This invention proposes a path semantic awareness-based large language model test case generation method. By introducing static dependency analysis and path-level semantic generation mechanisms, it achieves a fully automated test generation system from dependency modeling and path intent reasoning to coverage feedback optimization. First, the method uses static analysis techniques to construct a project dependency graph, systematically identifying code entities and their call relationships, providing structured support for test path identification and context understanding. Then, based on the dependency graph and control flow analysis, and leveraging the thought chain reasoning capabilities of the large language model, semantic modeling and intent generation are performed on different execution paths, thereby generating a test case set consistent with path behavior. Finally, through a coverage feedback-driven test optimization mechanism, the method continuously analyzes uncovered paths in the test report, iteratively generating supplementary test cases to achieve dynamic improvement of the test set and continuous enhancement of coverage.

[0016] The method disclosed in this invention uses path semantic analysis as its core, combining static structural modeling with large-scale model reasoning to overcome the limitations of traditional automated test generation in terms of path coverage and semantic consistency. Through the synergistic effect of project dependency graphs, control flow analysis, and coverage feedback, this invention provides a closed-loop test generation process from dependency identification and path understanding to coverage optimization. This process can improve the coverage breadth, logical rationality, and semantic accuracy of automated tests, providing strong technical support for efficient testing and quality assurance of complex software systems. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the method for generating test cases for a large language model based on path semantic awareness, as described in this invention. Detailed Implementation

[0018] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0019] like Figure 1 As shown in the figure, the method for generating test cases for a large language model based on path semantic awareness disclosed in this invention includes the following three steps: 1) Project dependency graph construction; 2) Test case generation based on path semantic awareness; 3) Test optimization based on coverage feedback.

[0020] The following section details each step of the method for generating test cases for large language models based on path semantic awareness: (a) Steps for constructing a project dependency graph: Project dependency graph construction aims to systematically identify various code entities and their static dependencies in the project under test through static analysis techniques, providing structured dependency support for subsequent path semantic sensitivity analysis and test case generation.

[0021] The project dependency graph construction steps specifically include: using static analysis tools (such as Doxygen) to perform comprehensive static analysis of the given project's source code, identifying code entities within the project, including functions, structs, enumeration type variables, and modules, and analyzing their definition locations to establish an entity index system for the project code. Based on this, the call relationships between code entities are extracted, such as call relationships between methods and reference relationships between methods and structs or classes, thereby comprehensively depicting the functional interactions and dependency hierarchy within the project. Finally, by analyzing relationships such as function calls, member references, and module dependencies, the identified code entities and their dependencies are modeled in graph structure form, constructing a project dependency graph with code entities as nodes and dependency relationships as edges. This project dependency graph, with code entities as nodes and call, reference, and inclusion relationships as edges, depicts the hierarchical structure and call relationships between code elements, fully reflecting the logical connections between various functional units in the project, and intuitively showing the interaction paths between functions and their upstream and downstream dependent entities. This graph model can accurately locate the upstream and downstream dependent entities of the function under test, providing a traceable dependency foundation for path enumeration, semantic analysis, and test case generation, thus laying the foundation for subsequent control flow analysis and path semantic awareness test generation.

[0022] (ii) Test case generation based on path semantic awareness The path semantic awareness-based test case generation aims to combine static analysis with the reasoning capabilities of large language models. By leveraging the semantic reasoning and generation capabilities of large language models, it can automatically generate high-quality test cases for a given function under test, achieving precise test generation for different execution paths.

[0023] The generation of test cases based on path semantic awareness consists of two stages: the context dependency and path extraction stage, and the test generation stage based on path semantic intent.

[0024] The project dependency graph describes the call and reference relationships between functions, modules, structures, and global variables in the project. Therefore, the obtained relevant dependency entities include information such as the external methods called by the function under test, the referenced structures, and the associated global variables, to construct the function's contextual dependency environment, thus providing sufficient contextual information for subsequent control flow analysis. Subsequently, the system uses static analysis tools (such as Joern) to obtain the control flow graph of the function under test. The nodes of the control flow graph represent statements or basic blocks, and the edges represent possible control flow transition relationships. All possible feasible execution paths are enumerated on the control flow graph. By performing a depth-first traversal of the control flow graph, the system can enumerate all feasible execution paths of the function, denoted as P={P1, P2, …, P n Each path consists of lines of code in the function under test, denoted as P. n = <S1, S2,…,Sk ; φ n >, where φ n This represents the logical constraints of the path.

[0025] To address the path explosion problem that may occur during path enumeration, this invention designs a goal-driven heuristic path selection mechanism to prune redundant paths, thereby mitigating the path explosion problem and ensuring the controllability and efficiency of the analysis process. This ensures comprehensive analysis while controlling the number of paths, guaranteeing efficiency and executability. Specifically, when the number of execution paths exceeds a preset threshold (e.g., 10), the system automatically initiates the path pruning mechanism. This mechanism uses branch coverage as the optimization objective, selecting the most representative subset of paths from the full path set. First, the system constructs a "list of branches to be covered," recording the true / false branch status of all branch statements. Then, it iteratively selects paths that can cover new branches, continuously updating the "set of covered branches" until the coverage objective is achieved or the path budget is exhausted. For paths with loop structures, the system further adopts a "0-1-N" loop boundary strategy, limiting the number of loop unrolls to three typical scenarios (0 times, 1 time, 2 times) to cover key logic such as loop entry, single iteration, and multiple iterations. Finally, the logical condition φ for each candidate path is defined. n Feasibility assessments are performed, and unreachable or duplicate paths are eliminated, resulting in a set P of paths with manageable size and semantic complementarity. * ={P1, P2, …, P m}(in This provides a streamlined analytical basis for subsequent test generation.

[0026] In the test generation phase based on path semantic intent, this invention utilizes a large language model to perform semantic understanding and test design on the filtered path set. For each feasible path P obtained from the filtering... nThis invention leverages the path semantic understanding and reasoning capabilities of a large language model to semantically model conditional judgments, state changes, and branching logic within a path, generating corresponding path execution intent descriptions. Specifically, the system first constructs contextual hints, including dependent entities, code path information, function input parameters, and return types. This contextual information is input into the large language model, which is then guided by instruction prompts to generate a semantic intent description of the path. The semantic intent expresses the path's behavioral goals and testing concerns. Based on the generated semantic intent, this invention further invokes the large language model, which then infers and generates test inputs consistent with the path behavior, obtaining specific test cases. This results in a high-quality test case set that accurately covers path logic, reflects semantic differences, and possesses path-level discriminative power and semantic consistency. This stage achieves a fully automated process from path-level semantic understanding to executable test generation, providing precise and efficient technical support for model-driven software testing. By combining dependency context and path semantics, it achieves dual-constraint generation of test cases from structure to semantics, improving the relevance and accuracy of automatically generated tests.

[0027] (III) Test optimization mechanism based on coverage feedback Coverage feedback-based test optimization aims to improve the completeness and effectiveness of the test set through a closed loop of execution verification and coverage analysis. After generating the initial test case set, this invention further introduces a coverage feedback-based test optimization mechanism to improve the coverage of the test set. This mechanism achieves dynamic improvement and adaptive enhancement of the test case set through a "execution-feedback-optimization" cycle. First, the system executes the test case set generated in the previous step using the existing testing framework, collects runtime logs and code coverage information during the test execution of the function under test, including indicators such as statement coverage, branch coverage, and path coverage, and generates a structured test coverage report. The coverage results are statistically analyzed using a combination of static and dynamic analysis. Dynamic execution monitoring is used to capture runtime branch hits, while static analysis results are used to identify potential branches or paths that have not yet been reached. Subsequently, in the feedback analysis phase, the coverage report is analyzed, and the system compares the coverage results with the previously extracted path set {P1, P2, …, P}. m} Compare and identify the set of paths ΔP = {P} that have not yet been triggered by test cases. u ∈ P | Uncovered}, locate the execution paths that have not been reached or fully triggered, along with their associated conditional constraints and contextual dependencies. For these uncovered paths, the system again invokes the large language model, combining the logical constraints φ of the path. uCombined with contextual dependency information, new supplementary semantic intent descriptions and candidate test inputs are generated to specifically generate test cases to supplement test blind spots and attempt to trigger these uncovered paths. The generated supplementary test cases are incorporated into the test set and the execution and analysis process is repeated. After multiple rounds of iterative optimization, the system is iteratively adjusted based on the coverage improvement until a preset coverage threshold is reached or other termination conditions are met. For example, when the overall coverage increment of the test set is lower than a preset threshold (e.g., 2%) or reaches the target coverage level (e.g., 90% branch coverage), the system automatically terminates the optimization process and outputs the final high-quality test set. This achieves dynamic improvement and coverage completion of the initially generated test cases, enhancing the path coverage capability and overall test quality of the test cases.

[0028] This invention constructs an intelligent test generation framework with path semantic sensitivity analysis at its core and a large language model as its driving force. This framework not only enables accurate test generation based on path semantics but also continuously optimizes test quality with the support of a coverage feedback mechanism, improving the automation and coverage completeness of test generation. This invention can be widely applied in the fields of software testing and quality assurance, reducing the workload of manually writing tests, improving testing efficiency and reliability, and providing an efficient and scalable technical solution for intelligent software development and verification.

Claims

1. A method for generating test cases for a large language model based on path semantic awareness, characterized in that, Includes the following steps: Step 1: Project Dependency Call Graph Construction, including: using static analysis techniques to parse the project under test and extract code elements including functions, structures, and variables; constructing a project dependency call graph based on the call, reference, and dependency relationships between code elements to describe the dependency structure between various code elements in the project, providing contextual information for subsequent intent recognition; Step 2: Path Semantic Aware Test Case Generation Method, including: based on the project dependency call graph, obtaining the function under test and its related dependent entities to provide contextual semantic support for subsequent path analysis; using static analysis tools to extract the control flow graph of the function under test, and identifying all feasible execution paths on the control flow graph. During path enumeration, a goal-driven heuristic path selection method is adopted to address the potential path explosion problem. The system employs a selection mechanism to automatically prune redundant paths, thereby controlling analysis complexity while ensuring test coverage. A thought chain path analysis method is used to perform semantic reasoning and behavioral intent parsing on each feasible execution path, generating a corresponding path semantic intent description. Based on the generated path semantic intent, a large language model is invoked to generate test cases consistent with the path semantics, achieving precise test generation and high coverage verification at the path level. Step three involves test case optimization based on coverage feedback, including: importing the generated test cases into the existing test framework for execution and collecting path coverage reports; identifying uncovered paths based on the path coverage report results; generating supplementary test cases for uncovered paths using the large language model; and repeating the execution and optimization process until the coverage reaches the preset coverage level.

2. The method for generating test cases for a large language model based on path semantic awareness as described in claim 1, characterized in that, In step one, the constructed project dependency graph uses code entities as nodes and call, reference, and containment relationships as edges to depict the hierarchical structure and call relationships between code elements, fully reflecting the logical connections between various functional units in the project, and intuitively reflecting the interaction paths between functions and their upstream and downstream dependent entities.

3. The method for generating test cases for a large language model based on path semantic awareness as described in claim 1, characterized in that, In step two, the nodes of the control flow graph represent statements or basic blocks, and the edges represent possible control flow transition relationships.

4. The method for generating test cases for a large language model based on path semantic awareness as described in claim 1, characterized in that, In step two, the heuristic path selection mechanism includes: when the number of execution paths exceeds a preset threshold, the heuristic path selection mechanism is automatically activated. The heuristic path selection mechanism uses branch coverage as the optimization objective and selects the most representative subset of paths from the full set of paths.

5. The method for generating test cases for a large language model based on path semantic awareness as described in claim 4, characterized in that, The heuristic path selection mechanism first constructs a "list of branches to be covered", recording the true / false branch status of all branch statements; then, iteratively selects paths that can cover new branches, continuously updating the "set of covered branches" until the coverage goal is achieved or the path budget is exhausted.

6. The method for generating test cases for a large language model based on path semantic awareness as described in claim 1, characterized in that, In step two, to generate the path execution intent description, the system first constructs a contextual hint for the current path, which includes dependent entities and code path information, function input parameters and return types. Then, the contextual information is input into the large language model, and the model is guided to generate the semantic intent description of the path through instruction prompts.

7. The method for generating test cases for a large language model based on path semantic awareness as described in claim 1, characterized in that, Step two, generating test cases based on path semantic intent, includes: using the path semantic intent as input prompts, calling a large language model to generate test cases consistent with the path behavior; and combining the dependent entities and context information of the function under test to ensure that the generated test cases have semantic consistency and executability.

8. The method for generating test cases for a large language model based on path semantic awareness as described in claim 1, characterized in that, In step three, the feedback-based test case optimization includes: importing the generated test cases into the existing test framework for execution and collecting path coverage reports; identifying unreached execution paths based on the coverage results; and generating new supplementary test cases using a large language model for the uncovered paths, and iteratively executing and optimizing them until the path coverage reaches a preset threshold.

Images