A test case generation method and system based on a large language model and a search test algorithm

Abstract

The present application relates to a kind of test case generation method and system based on large language model and search formula test algorithm, belong to software testing technical field.The method includes: the static analysis of the program to be tested is carried out, and coverage probe and branch distance probe are inserted, and code path division is carried out;Initial test population with path diversity is generated using large language model combined with path division information;With genetic algorithm driving iteration evolution of test population and recording each generation coverage and branch distance information;When detecting coverage stagnation, identify low-coverage objective function and approximate test case closest to distance coverage by branch distance analysis, construct structured prompt words containing branch distance value and approximate test case parameters, call large language model to generate targeted test case seed;After compiling verification, effective seed is injected into population to realize search space redirection.The present application introduces branch distance measurement into the construction process of large language model prompt words, so that the model obtains the quantitative guidance of "how much difference can be covered", and the initial population generation guided by path division reduces the search cold start cost, realizes the deep integration of search test and large language model, and effectively improves the automatic test case coverage rate of complex software such as rail transit simulation system.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, specifically to a test case generation method and system based on a large language model and a search-based testing algorithm, which is particularly suitable for automated testing of software systems with complex control logic and multi-subsystem interaction constraints, such as rail transit simulation systems. Background Technology

[0002] Track simulation systems are core software systems in the rail transit field, capable of simulating the interaction of multiple signal sources under real rail transit conditions. They encompass the operational states of various subsystems, including train operation control, track signal state switching, interlocking logic verification, track circuit occupancy detection, and Automatic Train Protection (ATP). Because direct system testing on operating lines carries extremely high safety risks and costs, simulation environments are necessary to simulate and verify system functionality. These systems involve complex train operation dynamics models, signal control logic, and interaction protocols between multiple subsystems. Their code typically contains deeply nested conditional branches, strict timing constraints, and multi-parameter coupled input spaces. Ensuring the correctness of such systems is crucial for rail transit operational safety, thus requiring high-quality, high-coverage test cases. However, as systems evolve and expand, the number of subsystems and control items requiring integration gradually increases, leading to greater complexity. Manually writing test cases is inefficient and fails to cover all extreme cases and boundary conditions. Therefore, automated test case generation technology has significant engineering application value.

[0003] Currently, there are two main technical approaches in the field of automated test case generation:

[0004] The first category is test case generation methods based on search-based algorithms. These methods use genetic algorithms as their core, encoding test cases as chromosomes. Through genetic operations such as crossover, mutation, and selection, the test population is iteratively evolved, with code coverage serving as the fitness function to guide the search direction. This method can systematically explore the program's input space and generate test suites with high coverage in simple programs and general conditional branching scenarios. However, this type of method heavily relies on the gradient information of the fitness function. When the code under test contains complex object constructions, deep API call chains, or specific numerical constraints, the search process easily gets stuck in a coverage stagnation state, and relying solely on random mutation is insufficient to generate effective test inputs that satisfy semantic constraints. For track simulation systems, their function inputs often need to meet specific operational rule constraints (such as train interval conditions, signal interlocking safety constraints, block section occupancy logic, etc.), and purely random search is extremely difficult to overcome such semantic barriers.

[0005] The second category is test case generation methods based on large language models. These methods leverage the code understanding and generation capabilities of large language models to directly generate complete test methods by constructing prompts containing contextual information about the method under test. The generated test cases are highly readable and can handle complex object construction scenarios. However, these methods lack awareness of the program's runtime state and covered paths. The generation process is essentially a one-time static inference, unable to dynamically adjust the generation strategy based on existing test coverage. This easily leads to a large number of redundant tests covering the same paths, resulting in low overall coverage improvement efficiency. Furthermore, large language models suffer from the "illusion" problem, potentially generating test code that is uncompileable or exhibits runtime errors, requiring additional verification and repair mechanisms.

[0006] Existing automated test case generation technologies all have their limitations when dealing with software systems such as track simulation systems that have complex control logic and multi-subsystem interaction constraints. There is an urgent need for a method that can integrate the systematic coverage exploration capabilities of search-based testing algorithms with the semantic understanding and code generation capabilities of large language models to effectively improve the test coverage of such complex software systems. Summary of the Invention

[0007] To overcome the problems in existing technologies, such as the tendency of search-based testing algorithms to stagnate under complex constraints and the lack of coverage awareness in large language models leading to redundant testing, this invention provides a test case generation method and system based on large language models and search-based testing algorithms to solve the aforementioned problems in existing technologies.

[0008] A test case generation method based on a large language model and a search-based testing algorithm, the method comprising:

[0009] According to one aspect of the present invention, a test case generation method based on a large language model and a search-based testing algorithm is provided, specifically including the following steps:

[0010] S1. Perform static analysis and instrumentation on the program under test, extract function-level information, construct a control flow graph, insert coverage probes and branch distance probes, and perform code path partitioning based on the control flow graph;

[0011] S2. Design a chromosome encoding scheme for test cases, use a large language model combined with code path partitioning information to generate an initial test population with path diversity, and define a fitness function based on branch coverage.

[0012] S3. Drive the iterative evolution of the test population with a genetic algorithm, perform selection, crossover and mutation operations, execute test cases after each generation and record the cumulative coverage and branch distance information of each uncovered branch;

[0013] S4. Monitor the coverage growth trend. When coverage stagnation is detected, analyze the accessibility of each uncovered branch by combining branch distance data, identify the low coverage objective function, and select the approximate test cases that are closest to coverage.

[0014] S5. Extract the context information and branch distance data of the low coverage objective function, construct a structured prompt word containing function source code, parameter constraints, uncovered branch conditions, branch distance values ​​and approximate test cases, call the large language model to generate targeted test case seeds, and compile, verify and automatically repair the generated results;

[0015] S6. Convert the validated test case seeds into the chromosome encoding format of the genetic algorithm, inject them into the current test population to replace individuals with low fitness, resume the main loop of the search-based test, and redirect the search space; when the preset termination condition is met, output the final test suite.

[0016] In addition to the aspects and any possible implementations described above, a further implementation is provided in which the static analysis and coverage instrumentation of the program under test in S1 includes:

[0017] S11. Source Code Parsing and Function Information Extraction: Perform syntax parsing on the source code of the program under test, construct an Abstract Syntax Tree (AST), and extract the signature information of all testable functions, including function name, input parameter types and value ranges, return value types, exception declarations, and calling dependencies between functions.

[0018] S12. Control Flow Graph and Branch Structure Construction: Construct a control flow graph (CFG) for each testable function, identify all conditional branch nodes and loop structures, and count the total number of branches for each function to provide basic data for subsequent coverage measurement and stagnation analysis.

[0019] S13. Coverage and Branch Distance Probe Instrumentation: Insert coverage probes and branch distance probes at each branch node of the program under test. The coverage probe records the number of hits for each branch; the branch distance probe calculates and records the numerical distance between the current execution state and the branch condition when the branch condition is not met, which is used to identify the test case closest to the coverage target.

[0020] S14. Code Path Partitioning: Based on the control flow graph constructed in S12, the code path of the program under test is divided. The execution paths formed by different combinations of branch conditions are classified into several path clusters. Each path cluster represents a type of semantically similar program behavior, providing guidance information for the subsequent generation of an initial population with path diversity using a large language model.

[0021] In addition to the aspects and any possible implementations described above, a further implementation is provided in which the chromosome encoding scheme for designing test cases in S2, the initial test population generated using a large language model, and the fitness function based on branch coverage are defined as follows:

[0022] S21. Test Case Chromosome Encoding Design: Each test case is encoded as a chromosome, which consists of an ordered sequence of method call statements. Each statement contains a method identifier, input parameter values, and object references. For numeric parameters, real number encoding is used; for object parameters, constructor call chain encoding is used.

[0023] S22. Initial Population Generation Based on the Large Language Model: Utilizing the code path cluster information divided in S14, construct prompt words containing corresponding path conditions and parameter constraints for each path cluster. Call the large language model to generate several test cases for each path cluster. After compiling and verifying all generated test cases, merge them to form the initial test population. If the number of valid test cases generated by the large language model is insufficient for the preset population size P, supplement with randomly generated test cases to ensure the population reaches the preset size.

[0024] S23. Fitness Function Definition: Define a fitness function with branch coverage as its core. For a single test case... Its fitness value This is equal to the ratio of the number of newly covered branches after the test case is executed to the total number of branches in the function under test. For the entire test suite... Its overall fitness value equal The ratio of the total number of branches covered by all test cases to the total number of branches in the program under test.

[0025] In addition to the aspects and any possible implementations described above, a further implementation is provided in which the iterative evolution of the test population driven by a genetic algorithm in S3 includes:

[0026] S31. Population Fitness and Branch Distance Assessment: At the beginning of each generation, all test cases in the population are executed one by one. Branch coverage information of each individual is collected using a coverage probe, and branch distance data of each individual for uncovered branches is collected using a branch distance probe. The fitness value of each individual and the cumulative branch coverage of the entire population are calculated. ( (For the current algebra).

[0027] S32. Genetic Operation Execution: The three genetic operations, selection, crossover, and mutation, are executed sequentially. The selection operation uses a tournament selection strategy, selecting individuals with higher fitness from the current population to enter the next generation; the crossover operation uses single-point crossover, randomly selecting a crossover point in the method call sequences of the two parent chromosomes to exchange segments; the mutation operation includes parameter value perturbation, addition or deletion of method calls, where parameter value perturbation is randomly shifted around the current value according to a Gaussian distribution.

[0028] S33. Coverage and Branch Distance Recording: After each generation of evolution, record the cumulative branch coverage of the current population. And store it in the coverage history sequence. Simultaneously update the minimum branch distance record table for each uncovered branch. ,in For the uncovered branch identifier, This represents the minimum branch distance value for this branch in the current population. This serves as the identifier for the test case that generates the minimum distance. The record is used for subsequent stagnation detection and branch distance guided analysis.

[0029] In addition to the aspects described above and any possible implementations, a further implementation is provided in which the objective function for monitoring coverage growth trends and identifying low coverage in S4 includes:

[0030] S41. Coverage Growth Rate Calculation: Calculate the coverage growth rate after each generation of evolution. ,in For the first The cumulative branch coverage of the generation.

[0031] S42. Stagnation Status Determination: Set a coverage growth threshold. and continuous stagnation algebra threshold When continuous The coverage growth rate of each generation meets the requirements. When this happens, the current search process is determined to have entered a coverage stagnation state.

[0032] S43. Low Coverage Objective Function Identification and Branch Distance Analysis: After a stagnation state is triggered, traverse the branch coverage data of all functions in the tested program, sort them from low to high according to function-level branch coverage, and select the branch with the lowest coverage. These functions form a set of low-coverage objective functions. Subsequently, regarding For each uncovered branch of each function, the branch distance table is recorded from S33. The system queries the minimum branch distance value of the target branch and its corresponding approximate test case. The approximate test case refers to the test case in the current population that is closest to the target branch. The branch distance information and actual execution parameters it carries will serve as key inputs for subsequent targeted generation of the large language model.

[0033] As described above and in any possible implementation, a further implementation is provided in which the generation of targeted test case seeds based on a large language model in S5 includes:

[0034] S51. Objective function context information and branch distance data extraction: For a set of low-coverage objective functions Each function in Extract the complete source code, function signature, parameter types and constraint information, source code of the called dependent methods, list of uncovered branches of the function, and minimum branch distance value and corresponding approximate test case information of each uncovered branch identified in S43.

[0035] S52. Construction of Structured Prompts for Branch Distance Enhancement: The context information extracted in S51 is assembled into structured prompts, which include the following parts: (a) a task instruction, indicating that test cases covering specific branches of a specified function need to be generated; (b) the function source code and its dependency context; (c) parameter constraint description; (d) a description of the uncovered target branch; (e) branch distance analysis information and approximate test case code for each uncovered branch, including the actual execution parameters and gap analysis of the test case; and (f) output format requirements, specifying the code template that the generated test cases must follow.

[0036] S53. Large Language Model Test Case Generation: The structured prompts are input into the large language model, which generates a set of test case seeds for functions with low coverage based on its understanding of the code semantics. The seeds are compilable test method code, containing calls to the target function and preset input parameter values.

[0037] S54. Compilation, Verification, and Repair of Generated Test Cases: Each test case seed generated in S53 is compiled and verified. For test cases that fail to compile, the compilation error information is extracted and combined with the original test code to construct a repair prompt, which is then used to automatically repair the test case by calling the large language model again. The repair process executes a maximum of a preset number of commands. If the test case still fails to compile after multiple iterations, it is discarded.

[0038] As described above and in any possible implementation, a further implementation is provided in which population enhancement and search space redirection in S6 include:

[0039] S61. Test Case Chromosome Encoding Conversion: Convert the test case seeds that have passed compilation verification in S54 according to the chromosome encoding scheme defined in S21, and map their method call sequences and parameter values ​​to the standard chromosome representation.

[0040] S62. Population Injection and Update Strategy: Inject the transformed test case chromosomes into the current test population. An elite replacement strategy is used during injection, that is, the individuals in the current population are sorted from low to high fitness values, and the newly injected chromosomes replace an equal number of individuals with the lowest fitness to ensure the population size remains unchanged.

[0041] S63. Search Loop Resumption and Termination Determination: After completing the population injection, resume the main search-based test loop in S3 and continue the iterative evolution of the genetic algorithm. The entire test generation process will terminate and the final test suite will be output when any of the following termination conditions are met: (a) the cumulative branch coverage reaches the preset target coverage threshold; (b) the total number of generations reaches the preset maximum generation limit; (c) the number of interventions by the large language model reaches the preset maximum number of interventions limit.

[0042] This invention also provides a test case generation system based on a large language model and a search-based testing algorithm. The system is used to implement the method described above, and includes:

[0043] The program analysis and instrumentation module is used to perform static analysis on the source code of the program under test, extract information such as function signatures, parameter types and dependencies, construct a control flow graph, insert coverage probes and branch distance probes at branch nodes, and perform code path partitioning based on the control flow graph;

[0044] The search-based test engine module is used to maintain the test population, perform selection, crossover and mutation operations of the genetic algorithm, drive the iterative evolution of test cases, and execute test cases after each generation to evaluate fitness and collect branch distance data.

[0045] The coverage monitoring and stagnation detection module is used to collect and record branch coverage and branch distance data of each generation of test populations, calculate the coverage growth rate, determine whether it has entered a stagnation state, and identify the low coverage objective function and its approximate test cases when stagnation is triggered.

[0046] The large language model test generation module is used to generate a diverse initial population based on code path partitioning during the initial population construction phase, and is activated when coverage stagnates. It extracts context information and branch distance data of the low coverage objective function, constructs structured prompt words to enhance branch distance, calls the large language model to generate targeted test case seeds, performs compilation verification and automatic repair, and finally injects the verified test cases into the population of the search-based test engine.

[0047] Beneficial effects of the present invention

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] (1) Achieving deep integration and complementary advantages between search-based testing and large language models: This invention not only generates test cases using large language models when search encounters bottlenecks, but also utilizes large language models combined with code path partitioning information from static analysis to generate an initial population with path diversity during the initial population construction stage, ensuring that large language models are integrated throughout the entire test generation process. Search-based testing provides systematic coverage optimization capabilities, while large language models provide code semantic understanding and domain knowledge application capabilities. The two complement each other in the initialization and stagnation intervention stages, thereby effectively improving the overall coverage of automated testing.

[0050] (2) Intelligent intervention mechanism based on coverage stagnation detection: This invention determines whether the search process has stalled by monitoring the coverage growth rate, and only triggers the intervention of the large language model when the search algorithm itself cannot break through the coverage bottleneck. This on-demand triggering mechanism avoids the waste of computing resources caused by frequent calls to the large language model, while ensuring that each intervention is aimed at the real coverage bottleneck, thus improving the efficiency of the test generation process.

[0051] (3) Branch distance-based directional analysis and cue word construction significantly improve the accuracy of large language model generation: This invention innovatively introduces the branch distance metric from search-based testing into the cue word construction process of large language models. During stagnation intervention, not only are low-coverage objective functions and uncovered branches identified, but branch distance analysis also precisely locates the test cases closest to the coverage target and their actual execution parameters, incorporating this quantitative information into the cue words. The large language model thus obtains a quantitative guide to "how much difference is needed for coverage," enabling precise parameter adjustments based on existing tests, rather than guessing from scratch, significantly improving the effectiveness and coverage contribution of generated test cases.

[0052] (4) Particularly applicable to software systems with complex control logic constraints, such as track simulation systems: Track simulation systems involve numerous functions related to signal control logic, interlocking safety rules, and train operation constraints. Traditional search-based testing often stalls when faced with such constraints. This invention utilizes the understanding of knowledge in train operation control, signaling systems, and interlocking logic by a large language model, enabling the generation of test inputs that satisfy operational rule constraints. This helps the search algorithm overcome such semantic barriers and significantly improves test coverage of key functional modules. Attached Figure Description

[0053] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the description of the invention and its embodiments to explain the invention, but do not constitute a limitation thereof. In the drawings:

[0054] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0055] Figure 2 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0056] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples, so that the process of how the present invention uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly. It should be noted that, as long as there is no conflict, the various embodiments and features in the various embodiments of the present invention can be combined with each other, and the resulting technical solutions are all within the protection scope of the present invention.

[0057] A test case generation method based on a large language model and a search-based testing algorithm, the process of which is as follows: Figure 1 As shown, it includes the following steps:

[0058] S1. Perform static analysis and coverage instrumentation on the program under test:

[0059] S11. Source Code Analysis and Function Information Extraction:

[0060] The source code of the program under test (e.g., a rail transit simulation system) is parsed to construct an Abstract Syntax Tree (AST). Meta-information of all common testable functions is extracted from the AST, including: function name, input parameter list (including parameter names and types), return type, possible exception types, and a list of other methods called within the function body. Simultaneously, the call dependencies between functions are analyzed to construct a function call graph.

[0061] For example, in a rail transit simulation system, the following key functions are analyzed:

[0062] simulateTrainMovement(TrainState state, double dt, TrackSectionsection): Train operation simulation function. Inputs include the current state of the train (position, speed, acceleration), simulation time step, and track section information.

[0063] computeSignalState(int blockId, boolean[] trackOccupancy, InterlockTable table): Signal state calculation function, input block section number, track circuit occupancy status array and interlock table;

[0064] checkRouteConflict(Route route1, Route route2, InterlockTable table): Route conflict detection function, takes two route objects and an interlock constraint table as input;

[0065] calculateBrakingDistance(double speed, double gradient, doublefrictionCoeff): This function calculates the braking distance. It takes the current train speed, track gradient, and friction coefficient as input.

[0066] S12. Control Flow Graph and Branch Structure Construction:

[0067] For each extracted function, construct a Control Flow Graph (CFG). Traverse all conditional statements (e.g., if-else, switch-case) and loop structures (e.g., for, while) within the function body, marking each conditional statement as a branch node and recording the true and false branches of that branch. Analyze the statistics for each function. Total number of branches The total number of branches in the tested program is then calculated. ,in This represents the total number of testable functions.

[0068] For example, the control flow graph of the simulateTrainMovement function contains the following branches:

[0069] Branch b1: if (dt <= 0) — Check if the simulation time step is valid;

[0070] Branch b2: if (state.speed < 0 || state.speed > section.maxSpeed) — Determine if the train speed exceeds the section speed limit;

[0071] Branch b3: if (isOccupied(section.nextBlock)) — Check if the preceding occupancy section is occupied;

[0072] Branch b4: if (state.position > section.length) — Determines whether the train has left the current track section.

[0073] The control flow graph for the `calculateBrakingDistance` function contains the following branches:

[0074] Branch b5: if (speed <= 0) — Boundary condition handling path for when the train has stopped;

[0075] Branch b6: if (gradient < -maxGradient && frictionCoeff < minFriction) — Emergency braking path under extreme downhill and low friction conditions;

[0076] Branch b7: if (brakingDistance > remainingDistance) — Overspeed protection trigger path when the braking distance exceeds the remaining distance.

[0077] S13. Coverage and branch distance probe setup:

[0078] At each branch node in the source code of the program under test, two types of probe code are automatically inserted: coverage probe and branch distance probe.

[0079] A coverage probe is a counter increment statement; when a test case execution passes through a certain branch, the corresponding counter increments by one. The counter values ​​of all probes are stored in a global coverage record table, CoverageMap.

[0080] When a branch condition is not met, the branch distance probe calculates the numerical distance between the current execution state and the state where the branch condition is met, and stores the result in a global branch distance record table. In the numerical comparison condition `if (a < b)`, when the condition is not met (a >= b), the branch distance is calculated as |a - b| + 1; for compound conditions (such as a < b && c < d), the sum of the branch distances of each sub-condition is taken. The smaller the branch distance, the closer the test case is to covering that branch.

[0081] For example, for the branch b6 in the calculateBrakingDistance function: if (gradient < -maxGradient && frictionCoeff < minFriction), when a test case is executed with gradient = -20 and frictionCoeff = 0.12 (where maxGradient = 30 and minFriction = 0.08), since gradient = -20 > -30 does not satisfy the first sub-condition, the branch distance is |-20 - (-30)| + 1 = 11; at the same time, frictionCoeff = 0.12 > 0.08 does not satisfy the second sub-condition, and the branch distance is |0.12 - 0.08| + 1 ≈ 1.04. The combined branch distance is 11 + 1.04 = 12.04.

[0082] S14. Code path partitioning:

[0083] Based on the control flow graph constructed in S12, partition the code paths of the program under test. By analyzing the combination of branch conditions in the control flow graph of each function, classify the possible execution paths into several path clusters, and each path cluster represents a class of program behaviors with similar semantics.

[0084] For example, for the calculateBrakingDistance function, the following path clusters can be partitioned according to its control flow graph: Path cluster A (normal braking path: speed > 0 and gradient and frictionCoeff are within the normal range); Path cluster B (parking boundary path: speed <= 0); Path cluster C (extreme condition emergency braking path: gradient < -maxGradient or frictionCoeff < minFriction); Path cluster D (overspeed protection path: brakingDistance > remainingDistance). Each path cluster will be used as the guiding target for the subsequent large language model to generate initial test cases.

[0085] S2. Design the chromosome encoding scheme for test cases, use the large language model to generate the initial test population, and define the fitness function:

[0086] S21. Design of chromosome encoding for test cases:

[0087] Each test case is encoded as a chromosome, structured as an ordered sequence of method call statements. Each statement consists of a quadruple (method identifier, target object reference, list of input parameter values, return value reference). For numeric parameters (such as int and double), their numerical values ​​are used as the gene positions; for string parameters, the character sequence is used as the gene positions; for complex object parameters, their construction process is expanded into a nested subsequence of method calls. The length of the chromosome (i.e., the number of method call statements) is variable, but a maximum length limit is set. .

[0088] For example, a test case chromosome for the simulateTrainMovement function can be encoded as:

[0089] Statement 1: state = createTrainState(500.0, 60.0, 0.0) — Creates a train state object (position 500 meters, speed 60 km / h, acceleration 0).

[0090] Statement 2: section = getTrackSection("SectionA", 80.0, 1200.0) — Get track section information (section name, speed limit 80km / h, length 1200 meters)

[0091] Statement 3: result = simulateTrainMovement(state, 1.0, section) — Calls the function under test, passing in a simulation step size of 1 second.

[0092] Statement 4: assertNotNull(result) — asserts that the result is not null.

[0093] S22. Initial population generation based on a large language model:

[0094] Using the code path cluster information divided in S14, construct prompt words containing corresponding path conditions and parameter constraints for each path cluster, and call the large language model to generate several test cases for each path cluster.

[0095] For example, for path cluster C (emergency braking paths under extreme conditions), the prompt is: "Please generate test cases for the function calculateBrakingDistance(double speed, double gradient, double frictionCoeff), aiming to cover braking calculation paths under extreme downhill and low-friction conditions, where gradient should be less than -30, frictionCoeff should be less than 0.08, and speed should be a positive number. Please generate test cases with three different parameter combinations." The test cases generated by the large language model based on this prompt can include exploration of extreme paths from the early stages of the search.

[0096] All generated test cases are compiled, verified, and then merged to form the initial test population. If the number of valid test cases generated by the large language model is insufficient to meet the preset population size... (For example If the initial population reaches the preset size, randomly generated test cases are used to supplement it. Compared to purely random generation, this method gives the initial population better code path diversity, enabling it to cover more branches in the early stages of the search and reducing the cold start cost of the genetic algorithm.

[0097] S23. Definition of fitness function:

[0098] Define a fitness function based on branch coverage. For a single test case. The formula for calculating its fitness value is:

[0099]

[0100] in, The set of branches for the current test objective. Indicates the number of elements in the set.

[0101] For the entire test suite Its overall fitness (i.e., cumulative branch coverage) is:

[0102]

[0103] in This represents the set of all branches of the program under test. The fitness function uses maximizing coverage as the search objective, guiding the genetic algorithm to explore towards uncovered branches.

[0104] S3. Iterative evolution of the test population driven by genetic algorithms:

[0105] S31. Population fitness and branching distance assessment:

[0106] In the At the beginning of generational evolution ( Starting from 1), compile and execute the current population one by one. This involves all individual test cases. Using the coverage probe inserted in S13, branch coverage data is collected after each individual's execution; using the branch distance probe, branch distance data for each uncovered branch is collected. Based on the fitness function defined in S23, the fitness value of each individual is calculated. Simultaneously, the coverage data of all individuals are aggregated to calculate the cumulative branch coverage rate of the current population. .

[0107] S32, Genetic Operation Execution:

[0108] (1) Selection operation: A tournament selection strategy is adopted. Each time, τ individuals (tournament size) are randomly selected from the population. ,For example Select the individuals with the highest fitness from these individuals and place them in the mating pool. Repeat this process until the mating pool reaches the population size. .

[0109] (2) Crossover operation: Randomly pair parent individuals from the mating pool, with crossover probability (For example The system determines whether to perform crossover. If crossover is performed, a crossover point is randomly selected from the method call sequences of the two parent chromosomes, and the method call fragments after the crossover point are swapped to generate two offspring individuals.

[0110] (3) Mutation operation: For each offspring individual, with the mutation probability (For example Each gene locus is mutated independently. Numerical parameters are mutated by adding a Gaussian distribution to the current value. The random disturbance quantity, of which For variable asynchronous long parameters; the variation of the method call sequence includes randomly inserting a new method call, randomly deleting an existing method call, or randomly replacing the target method of a method call.

[0111] S33, Coverage and Branch Distance Records:

[0112] After each generation of evolution, the cumulative branch coverage of the current population will be calculated. Recorded in the coverage history sequence Meanwhile, update the minimum branch distance record table for each uncovered branch. ,in For the uncovered branch identifier, It is the minimum branch distance value for this branch in the current population (i.e., the value closest to this branch among all individuals). The test case identifier that generates this minimum distance.

[0113] For example, after a certain generation of evolution, the branch distance table The following information may be logged: The minimum distance of branch "calculateBrakingDistance_b6" is 12.04, as determined by the test case.

[0114] Generate (executed with gradient=-20, frictionCoeff=0.12). This record table is used to accurately locate approximate test cases during subsequent stall detection.

[0115] S4. Coverage Stagnation Detection and Low Coverage Objective Function Identification:

[0116] S41. Coverage growth rate calculation:

[0117] In each generation After evolution is complete, calculate the coverage growth rate of the most recent generation:

[0118]

[0119] in and The first generation and first The cumulative branch coverage of the generation.

[0120] S42. Stagnation State Determination:

[0121] Two preset threshold parameters: coverage growth threshold (For example (i.e., 0.1%) and the threshold for continuous stagnation algebra. (For example The search process is considered to have entered a coverage stagnation state when the following conditions are met:

[0122] For continuous generation, that is, the first Generation to the first All generations meet That is, continuous. The coverage growth rate of each generation is lower than the threshold. This indicates that the search capability of the genetic algorithm has been largely exhausted in the current search space region, and external assistance is needed to break through the coverage bottleneck.

[0123] S43. Identification of low-coverage objective functions and branch distance analysis:

[0124] Once S42 determines that it has entered a stagnant state, the following two-stage analysis is performed:

[0125] In the first stage, the control flow graph information of all functions constructed in S12 is traversed, and the statistics of each function are analyzed. Current branch coverage:

[0126]

[0127] All functions according to Sort by low to high, and select the lowest-ranked. Functions (e.g.) ), forming a set of low-coverage objective functions .

[0128] The second stage, for For each uncovered branch of each function, the branch distance table is recorded from S33. The system queries the minimum branch distance value of the target branch and its corresponding approximate test case. The approximate test case refers to the test case in the current population that is closest to the target branch. The actual execution parameters and branch distance value it carries will become the key input for subsequent prompt words in the construction of the large language model.

[0129] For example, in a rail transit simulation system, after multiple generations of genetic algorithm search, the following low coverage functions and their branch distance analysis results may be identified:

[0130] The `calculateBrakingDistance` function has a 30% coverage rate and excludes the emergency braking branch.

[0131] The minimum branch distance is 12.04, which is approximately the test case distance. Execute with gradient=-20 and frictionCoeff=0.12 (gradient<-30 and frictionCoeff<0.08 are required);

[0132] The `checkRouteConflict` function: 35% coverage, minimum branch distance of 8.0 for uncovered multi-path conflict branches, approximate test cases.

[0133] Two partially overlapping routes were set up, but the specific combination of interlocking constraints was not satisfied;

[0134] The `computeSignalState` function has a coverage rate of 40%. The minimum branch distance for the downgraded branch of uncovered signals is 5.0. This is an approximate test case. The trigger condition of two adjacent partitions being occupied but not three adjacent partitions being occupied simultaneously has been set.

[0135] These branch distance information precisely reveal "how much difference is needed to achieve coverage," providing a clear direction for parameter adjustment in large language models.

[0136] S5. Targeted test case seed generation based on large language model:

[0137] S51. Extraction of objective function context information and branch distance data:

[0138] for Each low coverage objective function Extract the following contextual information:

[0139] (1) The complete source code of the function;

[0140] (2) The list of input parameters of the function and the type, value range and physical meaning of each parameter (if available);

[0141] (3) The source code or interface signature of the key dependent methods called inside the function;

[0142] (4) The list of branches that the function is not currently covered, including the conditional expression for each uncovered branch;

[0143] (5) The minimum branch distance value of each uncovered branch identified in S43 and the corresponding approximate test case code, including the actual execution parameter value of the test case and the specific conditions that were not met;

[0144] (6) Representative test case code that has covered some branches of the function in the existing test population, as a reference example.

[0145] S52. Structured prompt word construction:

[0146] The information extracted from S51 is assembled into structured prompt words, and its template format is as follows:

[0147] "You are a professional software testing engineer. Please generate unit test cases for the following functions that can cover the specified branches."

[0148] [Source code of the function under test]

[0149] {Complete source code for the function}

[0150]

Function Parameter Description

[0151] {Parameter name, type, value range, physical meaning}

[0152] [Dependency Method Signature]

[0153] {Signature or brief description of key dependency methods}

[0154] [Currently uncovered target branch]

[0155] {List of conditional expressions for uncovered branches}

[0156] Branch distance analysis and approximation test cases

[0157] {Minimum branch distance for each uncovered branch, corresponding approximate test case code, and actual parameter values ​​during execution. For example: target branch gradient < -30 && frictionCoeff < 0.08, the current most recent test case is executed with gradient=-20 and frictionCoeff=0.12, branch distance=15.04, suggested adjustment direction: reduce gradient to below -35 and reduce frictionCoeff to below 0.05}

[0158] [Reference Test Cases]

[0159] {Existing representative test case code}

[0160] Please adjust the parameters based on the approximate test cases from branch distance analysis to generate {n} test cases. Each test case must cover at least one of the uncovered branches. Ensure that the input parameter values ​​meet business constraints, and the output format is a standard {test framework} test method.

[0161] For example, for the `calculateBrakingDistance` function, the generated prompts explicitly state that it needs to cover "emergency braking paths under extreme downhill conditions (gradient < -maxGradient) and low friction conditions (frictionCoeff < minFriction)," while providing branch distance analysis information: "The closest test case currently executed with gradient=-20 and frictionCoeff=0.12, with a branch distance of 15.04 (gradient < -30 and frictionCoeff < 0.08 are required). It is recommended to reduce gradient to below -35 and frictionCoeff to below 0.05." This quantified branch distance information allows large language models to make precise parameter adjustments based on the execution of existing tests, rather than guessing parameter values ​​from scratch, significantly improving the relevance of generated test cases.

[0162] S53, Large Language Model Test Case Generation:

[0163] The structured prompts built by S52 are input into a large language model (such as DeepSeek-V3.2 or other large language models with code generation capabilities). Based on its understanding of function code semantics and domain knowledge, the model generates a set (usually n, e.g., n=5) of test case seeds for uncovered branches. Each seed is a piece of compilable test method code, containing a call to the target function and semantically inferred input parameter values.

[0164] For example, for the extreme downhill emergency braking branch of the `calculateBrakingDistance` function, the model might generate the following test cases:

[0165] @Test

[0166] public void testCalculateBrakingDistance_SteepDownhillLowFriction() {

[0167] double speed = 80.0; / / Train speed 80km / h

[0168] double gradient = -35.0; / / Extreme downhill gradient -35‰

[0169] double frictionCoeff = 0.05; / / Low friction coefficient in rainy weather

[0170] double result = calculateBrakingDistance(speed, gradient,frictionCoeff);

[0171] assertTrue(result > 0);

[0172] assertTrue(result > 1000); / / Under extreme conditions, the braking distance should exceed 1000 meters.

[0173] }

[0174] This test case utilizes the understanding of train braking dynamics by a large language model and carefully constructs input parameters for extreme downhill slopes and slippery road surfaces, enabling it to trigger the exception handling branch in the emergency braking calculation path.

[0175] S54. Compilation, verification, and repair of generated test cases:

[0176] The following verification and repair process is performed on each test case seed generated in S53:

[0177] (1) Try compiling the test case code. If the compilation is successful, mark it as a valid seed.

[0178] (2) If compilation fails, extract the error information returned by the compiler (such as type mismatch, method signature error, missing dependency import, etc.).

[0179] (3) Assemble the original test code and compilation error information together into a repair prompt, and call the large language model again to attempt repair. The prompt template is: "The following test case has a compilation error, please repair it. [Original code] {code} [Compilation error] {error message} Please output the complete test method code after repair.";

[0180] (4) The above repair process will iterate for a maximum of r rounds (e.g., r=3); if it still fails to compile after r rounds, the test case seed will be discarded.

[0181] This verification and repair mechanism filters out unexecutable test code generated by the "illusion" of the large language model, ensuring that all test case seeds injected into the population are compileable and executable valid code.

[0182] S6. Population Enhancement and Search Space Redirection:

[0183] S61, Test case chromosome encoding conversion:

[0184] The test case seeds that have passed compilation verification in S54 are structurally transformed according to the chromosome encoding scheme defined in S21. Specifically, the test method body of each seed is parsed, the sequence of method call statements is identified, the method identifier, input parameter values, and object reference relationships of each statement are extracted, and mapped to a standard chromosome quadruple sequence representation.

[0185] S62. Population Injection and Update Strategy:

[0186] An elite replacement strategy is employed to inject the transformed test case chromosomes into the current population. The specific steps are as follows: First, the current population... All individuals are sorted from low to high fitness values; then, the newly generated individuals are used... chromosomes ( Replace the seed with the one that has passed the verification with the one that has the lowest fitness after sorting. Individual. If Exceeding population size If the preset injection ratio limit (e.g., 20%) is set, then the top of the highest fitness values ​​will be selected. A seed is injected to prevent LLM-generated seeds from over-dominantly dominating the population and to maintain the population's genetic diversity.

[0187] S63. Determining whether to resume or terminate the search loop:

[0188] After the population injection is complete, the stall generation counter is reset to zero, and the main search-based test loop in S3 is resumed to continue the iterative evolution of the genetic algorithm from the new population state. Since the injected LLM seed carries specific information about uncovered branches, the genetic algorithm can perform further crossover and mutation operations from these new starting points to explore previously inaccessible search space regions, thereby redirecting the search direction.

[0189] The entire test generation process will stop when any of the following termination conditions are met:

[0190] (a) Cumulative branch coverage Reaching or exceeding the preset target coverage threshold (For example );

[0191] (b) Total number of generations Reaching the preset maximum algebra limit (For example );

[0192] (c) Number of times the large language model intervenes Reaching the preset maximum number of interventions (For example ).

[0193] After termination, all test cases that can be successfully executed in the current population are merged and deduplicated, and the final test suite is output.

[0194] As an embodiment of the invention, the invention also provides a test case generation system based on a large language model and a search-based testing algorithm, such as... Figure 2 As shown, the system is used to implement the method, and the system includes:

[0195] The program analysis and instrumentation module performs static analysis on the source code of the program under test (such as a rail transit simulation system), constructs an abstract syntax tree and control flow graph, extracts meta-information such as function signatures, parameter types, and dependencies, automatically inserts coverage probes and branch distance probes at branch nodes, and performs code path partitioning based on the control flow graph. The module's output includes a function meta-information library, a control flow graph library, code path cluster partitioning results, and the instrumented executable program under test. For example, for a rail transit simulation system containing 50 testable functions, this module can parse the signatures, parameter constraints, and a total of over 300 branch nodes of all functions, complete probe instrumentation, and partition the code path into approximately 80 path clusters.

[0196] The search-based test engine module is used to implement test case evolution search based on genetic algorithms. This module maintains the test population, including chromosome encoding of test cases, initial population generation, fitness evaluation (by calling instrumented programs to execute test cases and collect coverage data), and the execution of genetic operations such as selection, crossover, and mutation. This module is the core driving engine of the entire system, responsible for the main loop. Under typical configuration, this module runs with a population size of 100 individuals, with each generation taking approximately several seconds to tens of seconds (depending on the execution time of the program under test), continuously iterating until a stall signal is received or a termination condition is met.

[0197] The coverage monitoring and stagnation detection module continuously monitors the trends in coverage and branch distance during test evolution. This module receives coverage and branch distance data for each generation from the search-based test engine module, maintains a historical coverage sequence and branch distance record table, calculates the coverage growth rate, and applies preset threshold conditions (coverage growth threshold). and stagnation algebra The module determines whether a coverage stagnation has occurred. When stagnation is detected, the module further analyzes the function-level coverage data, identifies the set of target functions with low coverage, and filters out approximate test cases for each uncovered branch through branch distance analysis. The complete analysis results are then passed to the large language model test generation module.

[0198] The large language model test generation module undertakes two core responsibilities: first, in the initial population construction phase, it generates diverse initial test cases for each path cluster based on code path cluster partitioning information; second, it generates targeted test case seeds guided by branch distance when coverage stagnation is triggered. This module comprises five sub-modules: the context extraction sub-module obtains the source code and branch information of low-coverage functions from the program analysis and instrumentation module; the branch distance analysis sub-module obtains branch distance data and approximate test cases from the coverage monitoring module; the prompt word construction sub-module assembles the context information and branch distance data into structured prompt words with enhanced branch distance; the model invocation sub-module interacts with external large language model services (such as DeepSeek-V3.2) through API interfaces, sending prompt words and obtaining the generated test case code; and the compilation and verification sub-module performs compilation testing and multiple rounds of automatic repair verification on the generated code. The verified test case seeds are finally converted into chromosome encoding format and injected into the population of the search-based test engine module, triggering the redirection of the search space.

[0199] The foregoing description illustrates and describes several preferred embodiments of the present invention. However, as previously stated, it should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept described herein through the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A test case generation method based on a large language model and a search-based testing algorithm, characterized in that, Includes the following steps: S1. Program Analysis and Instrumentation: Perform static analysis on the program under test, extract function signatures, parameter types and dependencies, construct a control flow graph, insert coverage probes and branch distance probes, and perform code path partitioning based on the control flow graph; S2. Test Population Initialization: Design the chromosome encoding scheme for test cases, use the large language model combined with the code path partitioning information to generate an initial test population with path diversity, and define a fitness function based on branch coverage. S3. Search-based test main loop execution: The test population is driven by a genetic algorithm to perform selection, crossover and mutation operations, and test cases are executed after each generation, and the cumulative branch coverage and branch distance information of each uncovered branch are recorded. S4. Coverage Stagnation Detection and Branch Distance-Guided Target Analysis: Monitor the coverage growth trend. When the coverage growth rate is lower than the preset threshold for multiple consecutive generations, it is determined to be in a stagnant state. Identify the target function set with the lowest coverage and combine it with branch distance data to filter out the approximate test cases that are closest to each uncovered branch. S5. Branch distance enhancement of large language model targeted test generation: Extract the context information and branch distance data of the low coverage objective function, construct a structured prompt word containing function source code, parameter constraints, uncovered branch conditions, branch distance values ​​and approximate test cases, call the large language model to generate targeted test case seeds, and compile, verify and automatically repair the generated results; S6. Population Enhancement and Search Space Redirection: Convert the validated test case seeds into the chromosome encoding format of the genetic algorithm, inject them into the current test population to replace individuals with low fitness, and resume the main loop of the search-based test; when the preset termination condition is met, output the final test suite.

2. The test case generation method based on a large language model and a search-based testing algorithm according to claim 1, characterized in that, The branch distance probe in step S1 specifically involves inserting branch distance calculation code at each conditional branch node of the program under test. When the branch condition is not met, the numerical distance between the current execution state and the condition is met is calculated and stored in the global branch distance record table. The code path division specifically involves classifying the execution path into several path clusters based on the combination relationship of branch conditions in the control flow graph. Each path cluster represents a type of semantically similar program behavior.

3. The test case generation method based on a large language model and a search-based testing algorithm according to claim 1, characterized in that, The initial population generation in step S2 is specifically as follows: using the path cluster information generated by the code path division, constructing prompt words containing corresponding path conditions and parameter constraints for each path cluster, calling the large language model to generate test cases for each path cluster, and merging them into an initial population after compilation and verification; if the number of effective test cases is insufficient for the preset population size, randomly generated test cases are used to supplement it.

4. The test case generation method based on a large language model and a search-based testing algorithm according to claim 1, characterized in that, The stagnation state determination in step S4 is specifically as follows: set a coverage growth threshold ε and a continuous stagnation generation threshold m. When the coverage growth rate of m consecutive generations is lower than ε, the search process is determined to have entered a coverage stagnation state. The target analysis guided by branch distance is specifically as follows: for each uncovered branch of the low coverage objective function, query the minimum branch distance value and the corresponding approximate test case from the branch distance record table. The approximate test case is the test case in the current population that is closest to covering the target branch.

5. The test case generation method based on a large language model and a search-based testing algorithm according to claim 1, characterized in that, The structured prompts in step S5 include the following parts: task instructions, complete source code of the function under test and its dependency context, parameter constraint descriptions, conditional expressions for uncovered target branches, minimum branch distance values ​​for each uncovered branch and actual execution parameters and gap analysis of corresponding approximate test cases, existing representative test cases, and output format requirements.

6. The test case generation method based on a large language model and a search-based testing algorithm according to claim 1, characterized in that, The compilation verification and automatic repair in step S5 are as follows: the test cases generated by the large language model are compiled one by one. If the compilation fails, the error information returned by the compiler is extracted, and the error information and the original test code are used together to construct a repair prompt word. The large language model is called again for repair. The repair process executes a maximum of r preset iterations. If the compilation still fails, the test case is discarded.

7. The test case generation method based on a large language model and a search-based testing algorithm according to claim 1, characterized in that, In step S6, the population injection adopts an elite replacement strategy, which means that all individuals in the current population are sorted from low to high fitness values, and the same number of individuals with the lowest fitness are replaced with newly generated test case chromosomes. An upper limit for the injection ratio is set. When the number of seeds to be injected exceeds the upper limit, only the seeds with the highest fitness are selected for injection in order to maintain the genetic diversity of the population.

8. A test case generation system based on a large language model and a search-based testing algorithm, implementing the method as described in any one of claims 1-7, characterized in that, The system includes: Program Analysis and Instrumentation Module: Used to perform static analysis on the source code of the program under test, extract function signatures, parameter types and dependencies, construct control flow graphs, insert coverage probes and branch distance probes, and perform code path partitioning based on the control flow graph; The search-based test engine module is used to maintain the test population, perform selection, crossover, and mutation operations of the genetic algorithm, drive the iterative evolution of test cases, and collect coverage data and branch distance data by executing test cases to evaluate fitness. Coverage monitoring and stagnation detection module: used to record branch coverage and branch distance data of each generation of test population, calculate coverage growth rate, determine whether it has entered a stagnation state, and identify low coverage objective function and its approximate test cases when stagnation is triggered; The large language model test generation module is used to generate a diverse initial population based on code path partitioning during the initial population construction phase, extract context information and branch distance data of low coverage functions when coverage stagnates, construct structured prompt words with enhanced branch distance, call the large language model to generate targeted test case seeds, perform compilation verification and automatic repair, and convert the verified seeds into chromosome codes before injecting them into the test population.

Images