A diversified test case generation method and system based on a generator

Abstract

The application discloses a diversified test case generation method and system based on a generator construction, and belongs to the technical field of software automatic testing and program analysis. In order to solve the technical problems of huge SMT solving overhead, random variation destroying variable dependence leading to invalid input, and deterministic assignment leading to lack of diversity of test cases in the prior art, the normalized constraint is obtained by simplifying and variable dependence eliminating on the program path constraint; the model converter chain and the target constraint set are generated by reduction processing; the initial model is obtained by splitting and dependence analysis on the target constraint, constructing a random sampler and solving; the test input model is generated by mapping and converting the result and the initial model after merging through the model converter chain by using the random sampler for sampling. The application can significantly share the solving overhead and improve the generation throughput, ensure the input effectiveness, and improve the diversity of test cases and the code coverage.

Description

Technical Field

[0001] This invention belongs to the field of software automated testing and program analysis technology, specifically relating to a method and system for generating diverse test cases based on generator construction. Background Technology

[0002] Symbolic execution is a program analysis technique that systematically explores the program's state space, widely used in automated test case generation and software vulnerability discovery (such as buffer overflow and null pointer reference detection). Its standard workflow includes the following key steps: First, symbolic environment modeling. Symbolic execution engines (such as open-source tools KLEE, Angr, TritonDSE, etc.) do not run the program using concrete input data, but instead define input data (such as command-line arguments, file content, network packets) as symbolic variables. Next is path exploration and state branching. The engine simulates the program's instruction execution. When the execution flow encounters a conditional branch instruction (such as if (x > 10)) and this condition depends on a symbolic variable, the execution engine performs a branching operation, generating two new program states: one state executes along the true branch, and the other along the false branch. In each program state, the engine maintains a set of path constraints (usually denoted as ). This set is the logical conjunction of all branch conditions traversed from the program entry point to the current instruction. For example, if the program successively traverses the branches x>0 and y<5, then the path constraint is x>0∧y<5. To verify the reachability of a path or generate test cases that trigger that path, the engine converts the collected path constraints into SMT formulas and calls the SMT solver to solve them.

[0003] Modern SMT solvers (such as Z3 and CVC4) are not simply black-box functions; they contain complex transformation pipelines, which are crucial for understanding the improvements of this invention. To handle complex constraint formulas, SMT solvers employ a series of reasoning steps called tactics. At the start of the solution process, the original path constraints are treated as a goal. The solver sequentially applies tactics (such as simplify, propagate-values, elim-uncnstr, etc.) to transform the current goal into one or more logically equivalent or implied, structurally simpler sub-goals. For example, the elim-uncnstr tactic identifies variables that appear only once in the formula and are not otherwise restricted, removing them from the goal and thus reducing the problem's dimensionality. The solve-eqs tactic attempts to eliminate variables in equality constraints through Gaussian elimination or other algebraic methods. The solver's ultimate goal is to construct a model that satisfies the original constraints (i.e., the specific assignment of variables). This process is the reverse of the simplification process. Once the simplest sub-objective at the bottom layer is solved to obtain a partial model, the solver calls the model transformers corresponding to each strategy to reverse-map the sub-objective models layer by layer, ultimately reconstructing the model of the original objective. Existing model transformers are typically deterministic. For variables eliminated during simplification (i.e., free variables), the model transformer usually assigns a fixed default value. For example, in the Z3 solver implementation, for integer variables eliminated by the elim-uncnstr strategy, the model transformer typically assigns them a value of 0 (or other fixed constant) during reconstruction, rather than randomly selecting one from its feasible region.

[0004] The closest prior art to this invention is a symbolic execution method based on a particular solution mode, and a hybrid fuzzing technique that combines random mutation with this method.

[0005] In current industry and academia (such as SAGE, Mayhem, and QSYM systems), the use of SMT solvers is limited to finding specific solutions. For each expensive path constraint solution request, the SMT solver returns only one specific input sample that satisfies the constraint. For example, for the constraint x>10, the solver may always return x=11 (depending on the specific implementation and context), rather than actively returning x=100 or x=255. If testers want to generate multiple different test inputs for the same program path to trigger different memory states, the current practice is to obtain the first solution, construct a new constraint, and call the solver again. This iterative solving leads to extremely high time overhead.

[0006] To alleviate the problem of slow symbolic execution solving speed, existing technologies typically employ a hybrid fuzzing strategy. This involves using symbolic execution to generate a high-quality seed input, which is then randomly mutated by a fuzzer (such as AFL or LibFuzzer). The technical implementation involves the symbolic execution engine solving the constraints to obtain a seed input that satisfies complex conditions (e.g., by checking with a Magic Number). The fuzzer then performs random operations on the seed, such as bit flipping, byte addition / subtraction, and block replacement, attempting to generate a large number of variant inputs. This approach of solving a solution first and then randomly mutating it performs poorly when facing strong or structured constraints, mainly in the following two aspects: (1) Difficulty in maintaining constraint satisfiability: Random mutation operations are "blind," failing to understand the dependencies between variables. (2) Limitations in solution space exploration: Because existing SMT solvers (as mentioned above) tend to assign default values ​​(e.g., 0) to free variables, the seeds generated by symbolic execution often cluster in certain "corners" of the solution space. For example, for the constraint buffer_size < 100, the solver may always return 0. Relying on this single, specific seed for mutation makes it difficult to explore other regions of the solution space (such as the boundary value 99 or the intermediate value 50), which may result in missing some program errors that are only triggered under specific data distributions (such as integer overflow or memory corruption in specific patterns).

[0007] In-depth analysis of existing symbolic execution and hybrid fuzzy testing techniques revealed the following significant technical defects and shortcomings in their constraint solving mechanisms and input generation strategies: First, the particular solution mechanism results in excessively high unit cost and low efficiency for input generation. In existing technologies, SMT solvers (such as Z3 and CVC4) integrated into the backend of symbolic execution engines (such as KLEE and Angr) generally employ a particular solution mode, which suffers from severe efficiency bottlenecks when handling automated testing tasks. Solving path constraints is typically a computationally intensive problem involving complex bit vector operations and logical reasoning. Under existing mechanisms, each time the system performs a time-consuming and expensive solution operation, it can only obtain a unique test input that satisfies the constraints. This mechanism, where a single high-cost query corresponds to a single output, leads to extremely high average computational cost for generating a single test case. If the testing requirements necessitate generating N different test inputs for the same program path (to trigger different memory states), existing technologies must repeatedly call the solver N times. Furthermore, to avoid generating duplicate solutions, each call must add a new constraint negating the current solution to the original constraint. As the number of iterations increases, the constraint formula becomes increasingly complex, causing the solution time to grow non-linearly, severely limiting the throughput of symbolic execution per unit time. In the process of finding the first solution, the solver actually constructs a large amount of intermediate reasoning information, but this intermediate reasoning information is discarded after returning a single result. To find other solutions along the same path, the reasoning must be restarted, and previous computational results cannot be reused.

[0008] Secondly, hybrid strategies based on random mutation cannot maintain the satisfaction of complex constraints. To avoid the high overhead of frequent solutions, existing hybrid fuzzing techniques employ a strategy of first solving for a seed and then performing random mutation. This strategy has inherent mechanism flaws when facing tight constraints prevalent in programs. Variables in path constraints often have strict logical or algebraic dependencies (e.g., ...). Or CRC32(Data) == Checksum). Existing random mutation operators (such as bit flipping and byte replacement) are based on heuristic design, lacking an understanding of constraint semantics and thus exhibiting blindness. When the mutation algorithm randomly modifies the value of the independent variable (such as x or Data), it cannot synchronously update the value of the dependent variable (such as y or Checksum) according to the functional relationship. Because the above dependency relationship is broken, the new input data generated by mutation no longer satisfies the original path constraints. In actual testing, these inputs are rejected in the very early stages of program execution because they cannot pass the branch check, and cannot reach the target code region at all. Statistics show that when facing checksums or complex logical branches, the efficiency of inputs generated by random mutation is extremely low (often close to 0%). This causes the test system to waste a lot of CPU time on executing invalid inputs, failing to effectively expand code coverage, and making the hybrid method actually degenerate into inefficient random guessing.

[0009] Finally, deterministic solution strategies lead to a singular solution space exploration and a lack of diversity. Existing SMT solvers, in order to optimize the solution speed for a single objective, employ deterministic simplification and assignment strategies in their internal implementation. This results in a severe loss of generality in automated testing scenarios. During constraint solving, for free variables that are not strictly constrained, existing model converters tend to eliminate them and assign fixed default values ​​(usually 0, False, or the lowest lower bound of the type). Assuming the path constraint only requires input_x > 10, the solver will most likely always return input_x = 11 or other fixed values, without actively exploring other feasible values ​​such as 100 or MAX_INT. This deterministic mechanism causes the generated test cases to always cluster in certain specific corners of the solution space, failing to cover the broader area of ​​the solution space. However, many deep software vulnerabilities (such as buffer overflows and integer overflows) are often only triggered when variables take specific non-default values ​​(such as large integers, negative numbers, or specific positioning patterns). This lack of randomness and diversity in existing solution methods easily leads to the failure to detect such hidden vulnerabilities.

[0010] The single-solve, single-generation model in symbolic execution leads to significant time overhead and low input generation throughput due to frequent calls to the SMT solver. In traditional symbolic execution architectures, the path constraint solving process constitutes the main performance bottleneck. Existing technologies generally rely on specific solution-solving patterns, which have a strict linear correspondence: each call to the SMT solver can only produce one specific test input that satisfies the current path constraint. Since SMT solving is a computationally intensive problem, a single solution often takes a significant amount of time. If the testing task needs to generate a large number of test cases (N examples) for the same program path, existing technologies must adopt an iterative solution approach, i.e., repeatedly calling the solver N times, and each time adding a new negation clause to the original constraint to avoid generating duplicate solutions. This approach causes the total time cost of generating test inputs to increase linearly or even superlinearly with the number of generated inputs, greatly limiting the execution efficiency and throughput of symbolic execution tools per unit time.

[0011] When handling strongly constrained program paths, random mutations can disrupt dependencies between variables, leading to a large number of invalid test inputs and wasted computational resources. Existing techniques such as hybrid fuzzing often employ random mutation strategies to circumvent costly solution processes, involving bit flipping or random replacement of a seed input generated by symbolic execution. However, modern software is rife with complex strong constraints, such as checksums, hash verification, cryptographic algorithms, or complex algebraic equations. These constraints involve strict logical or mathematical dependencies between variables. Existing random mutation operators lack an understanding of the semantics of these underlying constraints, resulting in indiscriminate modification. Randomly modifying an independent variable (e.g., the message body) within a dependency without simultaneously updating the corresponding dependent variable (e.g., the checksum) directly violates the constraint's satisfiability. This causes a massive number of newly generated inputs to be rejected by branch conditions at the program's entry point or early stages, preventing them from reaching deeper code logic. This not only fails to improve test coverage but also results in a significant waste of CPU and time resources.

[0012] The deterministic assignment strategy of existing SMT solvers leads to a lack of test case diversity due to solution space exploration bias. Existing SMT solvers (such as Z3) are primarily designed to determine the satisfiability of logical formulas, rather than generating diverse test cases. Therefore, during the model building phase, the solver's internal strategies (such as the model converter) tend to employ deterministic simplification methods. For free variables that are not strictly constrained, the solver typically eliminates them and assigns fixed default values ​​(e.g., 0, False, or the minimum boundary value of the type). This deterministic assignment strategy causes test cases generated by symbolic execution to tend to cluster in certain corners of the solution space, failing to distribute them evenly or widely throughout the feasible solution space. However, many hidden software vulnerabilities (such as integer overflows triggered by specific values ​​or logical errors triggered by specific patterns) often lie deep within the solution space, only being triggered when variables take non-default random values. This lack of randomness and diversity in the generation method of existing technologies easily leads to the failure to detect such deep-seated vulnerabilities, reducing the completeness of software testing.

[0013] In summary, existing solutions using specific methods are inefficient, while stochastic mutation methods have inherent limitations in maintaining the satisfiability of complex constraints. Current technologies lack a mechanism that can efficiently and continuously sample multiple solutions satisfying dependencies directly from the solution space of constraints. Summary of the Invention

[0014] The purpose of this invention is to address the problems of high time overhead and low input generation throughput caused by frequent calls to the SMT solver due to the single-time solution-corresponding-single-generation pattern; the problem of a large number of test input failures and wasted computing resources caused by random mutations disrupting the dependencies between variables when handling strongly constrained paths; and the problems of solution space exploration bias and lack of test case diversity caused by the deterministic assignment strategy of existing solvers. This invention proposes a method and system for generating diverse test cases based on generator construction. By constructing a reusable generator and combining it with reversible random model transformation and hierarchical range sampling, batch generation of test inputs under a single solution is achieved. This significantly amortizes the solution overhead and improves the generation throughput, while ensuring that the generated samples naturally satisfy the complex dependency logic between variables and achieving uniformly distributed random sampling within the solution space. Ultimately, this improves the efficiency of symbolic execution testing, input validity, and code coverage.

[0015] To achieve the above objectives, the present invention adopts the following technical solution.

[0016] A method for generating diverse test cases based on a generator includes the following steps: Obtain program path constraints, simplify and eliminate variable dependencies on the program path constraints to obtain normalized constraints; The normalized constraints are reduced to generate a model converter chain and a set of target constraints. The target constraint set is decomposed and its dependencies are resolved, a random sampler is constructed and solved to obtain the initial model; The random sampler is used to sample variables. The sampling results are then merged with the initial model and mapped and transformed through the model converter chain to generate the test input model.

[0017] Further, program path constraints are obtained, and these constraints are simplified and variable dependencies are eliminated to obtain normalized constraints, including: Identify the bit vector extraction operation in the program path constraints; Intermediate variables are introduced for different bit segments corresponding to the bit vector extraction operation; Construct bit vector concatenation constraints to establish the mapping relationship between the intermediate variables and the original variables, thus obtaining normalized constraints.

[0018] Further, the normalized constraints are reduced to generate a model converter chain and a target constraint set, including: The normalization constraints are gradually simplified by applying a reversible strategy to obtain an intermediate target state; Generate a corresponding model converter layer based on the intermediate target state, and record the mapping rules between the intermediate target state and the original state; By repeatedly executing the simplification and recording process, the model converter chain is constructed, and a set of target constraints that cannot be further simplified is output.

[0019] Furthermore, a reversible strategy is applied to progressively simplify the normalization constraints to obtain intermediate target states, including: Identify the unconstrained variables in the normalized constraints that appear only on one side of the equation and are not restricted by other constraints; An auxiliary Boolean variable is introduced, and a model converter containing conditional branching logic is constructed for the unconstrained variable. Establish a reversible mapping rule from the simplified model state back to the original model state to obtain the intermediate target state.

[0020] Furthermore, applying a reversible strategy to progressively simplify the normalization constraints to obtain an intermediate target state also includes: Identify the equality constraints in the normalization constraints, and eliminate variables using algebraic elimination to obtain the assignment dependencies between variables; The assignment dependency relationship is recorded in the model converter layer to obtain the intermediate target state.

[0021] Further, the target constraint set is subjected to constraint decomposition and dependency resolution, a random sampler is constructed and solved to obtain the initial model, including: By performing optimistic solution processing on the target constraint set, redundant constraints are identified and filtered to obtain the final constraint set; Based on the dependencies between the variables resolved by the final constraint set, a random sampler is constructed to determine the sampling order of the variables; The complex logic in the final constraint set is solved using a satisfaction mode theory solver to obtain an initial model that serves as the basis for generation.

[0022] Furthermore, by performing optimistic solution processing on the target constraint set, redundant constraints are identified and filtered to obtain the final constraint set, including: The target constraint set is split into a range constraint set and other constraint sets; The other constraint sets are subjected to satisfaction tests using random samples generated by a range-based sampler, and difficult clauses that cannot be satisfied are filtered out. The range constraint set is merged with the difficult clause to construct the final constraint set.

[0023] Furthermore, based on the dependencies between the variables resolved in the final constraint set, a random sampler is constructed to determine the sampling order of the variables, including: Analyze the boundary relationships of the variables in the final constraint set, and construct a directed dependency graph with the variables as nodes; The directed dependency graph is topologically sorted to determine the sampling order of the dependent variable and the associated variable; The dynamic boundary expressions based on the preceding variables for each variable node are analyzed to obtain the sequential sampling sequence.

[0024] Further, the random sampler is used to sample variables, and the sampling results are merged with the initial model. Then, the model is mapped and transformed through the model converter chain to generate a test input model, including: The dynamic value range of each variable is calculated sequentially according to the sampling sequence, and the variable value is determined within the dynamic value range by a preset sampling strategy to obtain the variable sampling result; The variable sampling results are combined with the initial model to obtain a combined model that includes the initial assignment scheme for all variables; The combined model is input into the model converter chain, and the variable space mapping transformation is performed in reverse order of the reduction phase to obtain the test input candidate model to be verified. The test input candidate model is substituted into the program path constraints for validity verification. If the verification is successful, the final test input model is generated. Alternatively, if the verification fails, a rollback process of resampling or exact solution is performed.

[0025] A generator-based system for generating diverse test cases includes: The preprocessing module is used to obtain program path constraints, simplify the program path constraints and eliminate variable dependencies to obtain normalized constraints. The reduction module is used to perform reduction processing on the normalization constraints and generate a model converter chain and a target constraint set. The post-processing module is used to perform constraint decomposition and dependency resolution on the target constraint set, construct a random sampler, and solve for the initial model. The model generation module is used to sample variables using the random sampler, merge the sampling results with the initial model, and then perform mapping transformation through the model converter chain to generate a test input model.

[0026] The present invention has achieved the following beneficial effects.

[0027] 1. This invention replaces the traditional single-solution-specific-address approach with a repeatedly invoked generator function, achieving a paradigm shift from constraint-based problem-solving to a compilation-based process. By utilizing the computational overhead of a single SMT query to build the generator, subsequent batch generation of test inputs requires only low-complexity algebraic operations, significantly distributing the solution cost. Under the same time budget, this invention can generate tens of times more test cases, greatly improving the throughput and input generation efficiency of symbolic execution tools.

[0028] 2. This invention employs reversible stochastic model converter technology to ensure that the generated test inputs have extremely high logical validity. By improving the reduction strategy, the randomness of the independent variables is accurately propagated to the dependent variable through mathematical transformation, ensuring that the generated samples strictly adhere to the algebraic and logical dependencies between variables during the construction phase. Compared to random mutation methods that blindly destroy data dependencies, the inputs generated by this invention can successfully pass complex checksums and branch checks, ensuring that the testing process focuses on exploring the deep logic of the program.

[0029] 3. This invention utilizes a hierarchical range sampling algorithm to achieve comprehensive and diverse exploration of the solution space, significantly improving code coverage. By constructing a variable dependency graph and performing topological sorting, uniform random sampling is performed within the dynamically calculated interval, breaking the limitations of traditional solvers that favor default values ​​(such as 0) or specific boundary values. This diverse input distribution can effectively trigger program behaviors that only appear under specific intermediate values ​​or random combinations, increasing the probability of discovering potential vulnerabilities.

[0030] 4. This invention introduces an optimistic simplification and correctness checking mechanism, effectively solving the problems of timeout or deadlock under complex constraints. By identifying and actively removing redundant constraints that are highly likely to be satisfied but difficult to solve, the generator logic is simplified, and a fast verification step is used to filter samples. This mechanism achieves a balance between efficiency and reliability, utilizing the probabilistic advantage of multiple sampling to ensure that the system maintains an efficient approximate solution path and generality even when facing extremely difficult constraints.

[0031] 5. This invention improves the processing accuracy of complex bit manipulation constraints through variable slicing preprocessing technology. By introducing intermediate variables to explicitly represent bit extraction operations, implicit bit-level references are transformed into explicit variable-level dependencies, eliminating spurious dependencies. This technique makes the dependency graph construction more accurate, providing crucial preprocessing support for subsequent generator construction, constraint simplification, and efficient sampling, further enhancing the adaptability to complex program instructions. Attached Figure Description

[0032] Figure 1 This is a simplified flowchart of a generator-based method for generating diverse test cases, as shown in the embodiment. Figure 2 This is a flowchart illustrating a generator-based method for generating diverse test cases in one of the embodiments. Figure 3 This is a schematic diagram of the strategy for eliminating unconstrained variables in the embodiment; Figure 4 This is a flowchart illustrating the application of the optimistic solution scheme in the embodiments. Figure 5 This is a schematic diagram illustrating the layered range sampling principle in the embodiment. Figure 6 This is a schematic diagram of the method generation process in the embodiments; Figure 7 This is a module diagram of a generator-based system for generating diverse test cases, as shown in the embodiment. Detailed Implementation

[0033] To make the various technical features, advantages, or effects of the present invention more apparent and understandable, detailed descriptions are provided below through embodiments.

[0034] This invention provides a method for generating diverse test cases based on a generator. For example... Figure 1 and Figure 2 As shown, this method runs on a general-purpose computer system and solves the output of a symbolic execution engine. Specifically, the symbolic execution engine includes, but is not limited to, KLEE, Angr, TritonDSE, and SymCC. This method includes the following steps.

[0035] Step S1: Obtain program path constraints, simplify and eliminate variable dependencies on the program path constraints to obtain normalized constraints.

[0036] Specifically, the preprocessing stage simplifies the constraint structure and eliminates unnecessary variable dependencies as much as possible without performing logical reduction. The input consists of path constraints collected by the symbolic execution engine while exploring the program path; these path constraints are quantifier-based positional vector logical formulas.

[0037] In an optional embodiment of the present invention, the simplification of the program path constraints specifically includes: using logical identities to perform basic simplification of the input satisfaction modulo theory formula; applying predefined rewriting templates to normalize common complex constraint patterns in the program; and identifying and removing constraints that are highly likely to be satisfied but difficult to solve based on heuristic rules, so as to achieve optimistic simplification.

[0038] In an optional embodiment of the present invention, step S1 may include: Step S11: Identify the bit vector extraction operation in the program path constraint.

[0039] Step S12: Introduce intermediate variables for the different bit segments corresponding to the bit vector extraction operation.

[0040] Step S13: Construct bit vector concatenation constraints, establish the mapping relationship between the intermediate variables and the original variables, and obtain normalized constraints.

[0041] Specifically, by introducing intermediate variables and splicing constraints to eliminate spurious dependencies introduced by the extract operation, it is helpful to construct a more accurate dependency graph.

[0042] In an optional embodiment of the present invention, if terms (extract7 6x) and (extract5 4x) exist, this method introduces new variables x7_6 and x5_4, and adds constraints: x = concat(..., x7_6, ..., x5_4, ...) Here, concat represents the bit vector concatenation operation; x7_6 and x5_4 represent intermediate variables introduced after slicing; and x represents the original variable.

[0043] Specifically, regarding variable slicing, in addition to the methods mentioned above that introduce intermediate variables and splicing constraints, bit-level dependencies can also be directly established during the dependency graph construction phase. In this approach, the nodes of the dependency graph are specific bits of the variables, and the dependency analysis problem of bit operations is solved through fine-grained graph algorithms, without the need to explicitly rewrite the formulas.

[0044] Step S2: Reduce the normalized constraints to generate a model converter chain and a target constraint set.

[0045] Specifically, this stage utilizes a series of invertible tactics to progressively transform the target constraints into simpler forms and constructs a corresponding model converter chain. Unlike traditional solvers that directly solve a single model, this invention solves the input path constraints as a generator object that can be called multiple times. The reduction stage is a cyclical process that repeatedly executes the relevant simplification steps until the constraints cannot be further simplified, and each reduction generates a model converter layer.

[0046] In an optional embodiment of the present invention, step S2 may include: Step S21: Apply a reversible strategy to gradually simplify the normalization constraints to obtain an intermediate target state.

[0047] Step S22: Generate a corresponding model converter layer based on the intermediate target state, and record the mapping rules between the intermediate target state and the original state.

[0048] Step S23: By repeatedly executing the simplification and recording process, the model converter chain is constructed, and the set of target constraints that cannot be further simplified is output.

[0049] Specifically, the target constraint set includes the effective range of variable values ​​derived through the bit vector boundary propagation (Propagate-Bv-Bounds) strategy, in preparation for subsequent range sampling.

[0050] In an optional embodiment of the present invention, step S21 may include: Step S211: Identify the unconstrained variables in the normalized constraints that appear only on one side of the equation and are not restricted by other constraints.

[0051] Specifically, such as Figure 3 As shown, taking constraint equations as an example: Where y represents the variable on the left side of the equation; x represents the unconstrained variable.

[0052] Step S212: Introduce auxiliary Boolean variables and construct a model converter containing conditional branching logic for the unconstrained variables.

[0053] Specifically, for the identified unconstrained variable x, the system does not directly assign a fixed value, but instead constructs a model converter containing conditional branching logic. During this conversion process, auxiliary Boolean variables are introduced to control the value logic of different branches during the model generation phase.

[0054] In an optional embodiment of the present invention, the calculation method of variable y is determined based on the state of the auxiliary variable using the if-then-else statement, thereby ensuring that the generated model can cover different value spaces through algebraic operations. Figure 3 As shown, the auxiliary Boolean variable can be k!0.

[0055] Step S213: By performing a transformation operation, a reversible mapping rule is established to trace back from the simplified model state to the original model state, and an intermediate target state is obtained.

[0056] Specifically, the mapping rule records the path back from the simplified target state to the original state. This mechanism allows for the assignment of random values ​​to free variables in subsequent generation stages, and then, through a model converter chain, the simplified model obtained in the post-processing stage is inversely converted into a concrete model that satisfies the original constraints. This enables the generation of diverse y values ​​through the calculation formula of variable y, thereby generating diverse test input models.

[0057] In an optional embodiment of the present invention, the backtracking process of the intermediate target state is manifested as backtracking from the target state goal_1 to the original state goal_0, correspondingly converting the simplified model model_1 into the specific model model_0.

[0058] In an optional embodiment of the present invention, step S21 may further include: Step S214: Identify the equality constraints in the normalization constraints.

[0059] Step S215: Eliminate the variables in the equality constraints using algebraic elimination to obtain the assignment dependencies between variables.

[0060] Specifically, the equations are solved using the Solve-Eqs method, which uses Gaussian elimination or other algebraic methods to eliminate variables in the equality constraints.

[0061] Step S216: Record the assignment dependency relationship in the model converter layer to obtain the intermediate target state.

[0062] Step S3: Perform constraint decomposition and dependency resolution on the target constraint set, construct a random sampler, and solve to obtain the initial model.

[0063] Specifically, after reduction processing in the post-processing stage, the remaining constraints are divided into range constraints and complex constraints. This stage further simplifies the solution objective through an optimistic solution strategy. This invention can be developed based on the Z3 solver, utilizing its strategy and model converter interface; alternatively, it can be replaced with Satisfaction Modulus Theory (SMT) solvers that support bit vector theory and provide access permissions to the reduction process, such as CVC5, Boolector, Yices2, or STP. Furthermore, the solver can also be implemented by a separate constraint compiler, i.e., directly parsing and compiling standard SMT-LIB format formulas into generator executable code with random sampling capabilities through built-in modules.

[0064] In an optional embodiment of the present invention, step S3 may include: Step S31: By performing optimistic solution processing on the target constraint set, redundant constraints are identified and filtered to obtain the final constraint set.

[0065] Specifically, such as Figure 4 As shown, this step aims to identify and filter constraints that are easy to satisfy through random sampling.

[0066] In an optional embodiment of the present invention, step S31 may include: Step S311: Split the target constraint set into a range constraint set and other constraint sets.

[0067] Specifically, the Eager Split operation is performed to explicitly split the original set of input constraints into Range Constraints and Other Constraints.

[0068] Step S312: Use random samples generated by a range-based sampler to perform a satisfaction test on the other constraint sets, and filter out difficult clauses that cannot be satisfied.

[0069] Specifically, the other constraint sets are further divided into several subgroups, and the satisfiability of each constraint group is tested using random samples generated by a range-based sampler. If a set of constraints is easily satisfied in the random sampling test, it is considered redundant and removed from the solution objective; only constraints that cannot be satisfied by random samples are retained as hard-to-satisfy clauses.

[0070] Step S313: Merge the range constraint set (Final Constraints) with the difficult clause to construct the final constraint set.

[0071] Specifically, such as Figure 4 As shown, the separated range constraint set is merged with the selected difficult clauses to construct the final constraint set. This set serves as the input target for constructing the sampler and calling the conventional satisfaction modulo theory solver in subsequent steps.

[0072] Step S32: Based on the dependencies between the variables resolved in the final constraint set, construct a random sampler to determine the sampling order of the variables.

[0073] Specifically, such as Figure 5 As shown, this stage aims to transform interdependent range constraints into a dynamic set of boundaries that can be sampled sequentially.

[0074] In an optional embodiment of the present invention, step S32 may include: Step S321: parse the variable boundary relationships in the final constraint set and construct a directed dependency graph with the variables as nodes.

[0075] Specifically, the range constraints are resolved by treating variables as nodes and the boundary constraint relationships between variables as directed edges to construct a dependency graph. For example... Figure 5 (a) and Figure 5 As shown in (b), if the boundary of variable x in the constraint is determined by variable y, then a directed edge is established from the independent variable node y to the dependent variable node x.

[0076] In an optional embodiment of the present invention, if a cyclic dependency exists in the dependency graph, the present invention may introduce a cycle-breaking mechanism. When a cycle is detected, the cyclic dependency is transformed into a directed acyclic graph by fixing the value of a variable in the cycle or sampling it as an independent variable; or constraint propagation technology is used to replace topological sorting sampling for the cyclic dependency part.

[0077] Step S322: Perform topological sorting on the directed dependency graph to determine the sampling order of the dependent variable and the associated variable.

[0078] Specifically, the ordering principle is as follows: independent variables that are depended upon are sampled first, and related variables that depend on other variables are sampled later. This order ensures that when sampling any variable, all preceding variables involved in its boundary expression have obtained definite values.

[0079] Step S323: parse the dynamic boundary expressions based on the preceding variables corresponding to each variable node to obtain the sequential sampling sequence.

[0080] Specifically, such as Figure 5 As shown in (c), dynamic lower bound and upper bound expressions based on their preceding variables are parsed for each variable. The parsing results include: for independent variables at the initial level, their boundaries are usually preset minimum values ​​{MIN} or maximum values ​​{MAX}; for related variables with dependencies, their boundaries are expressed as functions of preceding variables. For example, the lower bound of variable z is parsed as a set of functions of preceding variables y and x {(bvmul y #x3), x}, which means that the actual lower bound of z will be the maximum value of the calculated results of these expressions.

[0081] Step S33: Use the satisfaction module theory solver to solve the complex logic in the final constraint set to obtain the initial model as the basis for generation.

[0082] Specifically, for the remaining small number of complex logics that cannot be processed by range boundary analysis, including nonlinear operations or complex logic spanning multiple variables, a specific model is obtained as an initial model by calling a conventional satisfaction modulo theory solver. This initial model will serve as the basis for subsequent generation stages and will be merged with the range sampling results.

[0083] In an optional embodiment of the present invention, the output of the post-processing stage is a generator function. When the subsequent generation stage begins execution, by calling this generator function and inputting a random seed, a specific test input model that satisfies the path constraints can be output.

[0084] Step S4: Use the random sampler to sample variables, merge the sampling results with the initial model, and then perform mapping transformation through the model converter chain to generate the test input model.

[0085] Specifically, when the symbolic execution engine needs new test input, it invokes the already built generator. The generation process is as follows: Figure 6 As shown in the figure. Through the above implementation method, the present invention achieves one-time solution and infinite generation, improving the generation efficiency and coverage of test inputs while ensuring constraint satisfaction.

[0086] In an optional embodiment of the present invention, step S4 may include: Step S41: Calculate the dynamic value range of each variable sequentially according to the ordered sampling sequence, and determine the variable value within the dynamic value range using a preset sampling strategy to obtain the variable sampling result.

[0087] Specifically, based on a determined topological order, a range-based sampler is used to sequentially calculate the dynamic closed interval range of each variable. Uniform random sampling is then performed within the calculated closed intervals to obtain the specific value model of the variables.

[0088] In an optional embodiment of the present invention, for the variable x8, the sampler samples according to its range to obtain a specific assignment model. For example, when the value range is [#x30, #x40], the generated assignment model can be represented as (define-fun x8 () (_ BitVec 8) #x30).

[0089] Specifically, regarding sampling strategies, to optimize software testing results, this invention can employ boundary preference sampling, weighted random sampling, or non-uniform distribution sampling, where non-uniform distribution sampling includes normal distribution or geometric distribution. After calculating the variable value interval [min, max], the probability of selecting boundary values, special values, or specific distribution pattern values ​​is increased to simulate specific data input patterns and enhance the detection capability for specific vulnerabilities.

[0090] Step S42: The variable sampling results are merged with the initial model to obtain a combined model containing the preliminary assignment scheme of all variables.

[0091] Specifically, such as Figure 6 As shown, the range sampling results are merged with the particular model obtained by a conventional satisfaction model theory solver. The combined model includes preliminary assignment schemes for the range variables and complex constraint variables. For example, the sampled values ​​of variable x8 are merged with the particular model values ​​of variable x2 into a single combined model.

[0092] Step S43: Input the combined model into the model converter chain, and perform variable space mapping transformation in the reverse order constructed in the reduction stage to obtain the test input candidate model to be verified.

[0093] Specifically, the converter executes in reverse order, following a Last-In-First-Out (LIFO) sequence, mapping the simplified model variables back to the original variable space. For example... Figure 6As shown, the model converter performs a model-add operation to introduce new associated variables and processes auxiliary variables. For free variables in the converter chain, the generator assigns new random values ​​and calculates the values ​​of dependent variables through algebraic operations or conditional branching logic. After the entire chain of model conversions, a specific test input candidate model containing all the original symbolic variables is generated. In an optional embodiment of the invention, the auxiliary variables can be k!0, and the conditional branching logic is implemented using an ite(If-Then-Else) statement.

[0094] Step S44: Substitute the candidate test input model into the program path constraints for validity verification, and generate the final test input model after successful verification, or perform a rollback process of resampling or exact solution when verification fails.

[0095] Specifically, due to the application of optimistic simplification strategies in preprocessing and postprocessing, the generated model may not satisfy the original path constraints. The system substitutes the generated test input candidate model m into the original path constraints. Perform a quick verification.

[0096] In an optional embodiment of the present invention, the verification and rollback process includes: If the verification relationship is satisfied That is, the generated candidate model m can satisfy the original path constraints. If the model is found to be a valid test input, then the model will be determined and output.

[0097] If the verification relationship is not satisfied, a resampling mechanism is triggered. If the verification still fails after multiple consecutive sampling attempts, it indicates that optimistic simplification has led to a deviation in the solution space, and the system reverts to calling a conventional satisfaction modulus theory solver for an accurate solution.

[0098] This invention also provides a system for generating diverse test cases based on a generator, such as... Figure 7 As shown, it includes: The preprocessing module is used to obtain program path constraints, simplify the program path constraints and eliminate variable dependencies to obtain normalized constraints. The reduction module is used to perform reduction processing on the normalization constraints and generate a model converter chain and a target constraint set. The post-processing module is used to perform constraint decomposition and dependency resolution on the target constraint set, construct a random sampler, and solve for the initial model. The model generation module is used to sample variables using the random sampler, merge the sampling results with the initial model, and then perform mapping transformation through the model converter chain to generate a test input model.

[0099] Through the above embodiments, the present invention achieves one-time solution and infinite generation, improving the generation efficiency and coverage of test inputs while ensuring constraint satisfaction.

[0100] Method performance testing: This invention verifies the effectiveness of its method through comparative experiments on real-world assemblies. The experiments used three widely used symbolic execution engines—TritonDSE, Angr, and KLEE—as benchmarks, replacing their constraint solving parts with the generator construction scheme of this invention (marked as +GS). Branch coverage tests were conducted on 17 real-world open-source projects, including freetype2, halfbuzz, libjpeg, and libxml2. The test results are shown in Table 1.

[0101] Table 1. Experimental results comparing branch coverage before and after the generator construction scheme (+GS). In the table, the left side of each item represents the coverage of the original engine, and the bolded value on the right side represents the coverage after integrating the method of this invention (+GS).

[0102] Experimental results show that the present invention exhibits significant performance improvements under different engine configurations: (1) Under the TritonDSE engine: Regardless of whether the configuration is no seed or with seeds, the branch coverage of multiple projects increased significantly after the introduction of the method of this invention. For example, in the freetype2 project, the branch coverage under the no seed configuration increased from 1326 to 3743; in the libjpeg project, the coverage jumped from 238 to 905.

[0103] (2) Under the Angr engine: This invention can reliably improve coverage for both breadth-first search (BFS) and depth-first search (DFS) strategies. Taking libxml2 as an example, under the BFS strategy, the number of branches covered increases significantly from 1285 to 3034; under the DFS strategy, it also increases from 1254 to 2246.

[0104] (3) Under the KLEE engine: The method of this invention outperforms the original engine under the three search strategies of BFS, DFS and random-path. In particular, in the harfbuzz project, the branch coverage under the BFS strategy increased from 3152 to 4416; in the libxml2 project, the coverage under the random-path strategy increased significantly from 2647 to 4436.

[0105] In summary, the experimental data clearly demonstrate that by constructing a reusable generator, this invention can effectively solve the bottleneck of traditional symbolic execution when dealing with complex path constraints. It can significantly enhance path exploration capabilities under various mainstream engines and different search strategies, thereby improving code branch coverage.

[0106] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention should be covered within the protection scope of the present invention, which is defined by the claims.

Claims

1. A method for generating diverse test cases based on a generator, characterized in that, Includes the following steps: Obtain program path constraints, simplify and eliminate variable dependencies on the program path constraints to obtain normalized constraints; The normalized constraints are reduced to generate a model converter chain and a target constraint set; The target constraint set is decomposed and its dependencies are resolved, a random sampler is constructed and solved to obtain the initial model; The random sampler is used to sample variables. The sampling results are then combined with the initial model and mapped through the model converter chain to generate the test input model.

2. The method as described in claim 1, characterized in that, Obtain program path constraints, simplify and eliminate variable dependencies on these constraints to obtain normalized constraints, including: Identify the bit vector extraction operation in the program path constraints; Intermediate variables are introduced for different bit segments corresponding to the bit vector extraction operation; Construct bit vector concatenation constraints to establish the mapping relationship between the intermediate variables and the original variables, thus obtaining normalized constraints.

3. The method as described in claim 1, characterized in that, The normalized constraints are reduced to generate a model converter chain and a target constraint set, including: The normalization constraints are gradually simplified by applying a reversible strategy to obtain an intermediate target state; Generate a corresponding model converter layer based on the intermediate target state, and record the mapping rules between the intermediate target state and the original state; By repeatedly executing the simplification and recording process, the model converter chain is constructed, and a set of target constraints that cannot be further simplified is output.

4. The method as described in claim 3, characterized in that, The normalization constraints are gradually simplified using a reversible strategy to obtain intermediate target states, including: Identify the unconstrained variables in the normalized constraints that appear only on one side of the equation and are not restricted by other constraints; An auxiliary Boolean variable is introduced, and a model converter containing conditional branching logic is constructed for the unconstrained variable. Establish a reversible mapping rule from the simplified model state back to the original model state to obtain the intermediate target state.

5. The method as described in claim 3, characterized in that, The normalization constraints are gradually simplified using a reversible strategy to obtain an intermediate target state, which also includes: Identify the equality constraints in the normalization constraints, and eliminate variables using algebraic elimination to obtain the assignment dependencies between variables; The assignment dependency relationship is recorded in the model converter layer to obtain the intermediate target state.

6. The method as described in claim 1, characterized in that, The target constraint set is subjected to constraint decomposition and dependency resolution, a random sampler is constructed and solved to obtain the initial model, including: By performing optimistic solution processing on the target constraint set, redundant constraints are identified and filtered to obtain the final constraint set; Based on the dependencies between the variables resolved by the final constraint set, a random sampler is constructed to determine the sampling order of the variables; The complex logic in the final constraint set is solved using a satisfaction mode theory solver to obtain an initial model that serves as the basis for generation.

7. The method as described in claim 6, characterized in that, By performing optimistic solution processing on the target constraint set, redundant constraints are identified and filtered to obtain the final constraint set, including: The target constraint set is split into a range constraint set and other constraint sets; The other constraint sets are subjected to satisfaction tests using random samples generated by a range-based sampler, and difficult clauses that cannot be satisfied are filtered out. The range constraint set is merged with the difficult clause to construct the final constraint set.

8. The method as described in claim 6, characterized in that, Based on the dependencies between variables resolved in the final constraint set, a random sampler is constructed to determine the sampling order of variables, including: Analyze the boundary relationships of the variables in the final constraint set, and construct a directed dependency graph with the variables as nodes; The directed dependency graph is topologically sorted to determine the sampling order of the dependent variable and the associated variable; The dynamic boundary expressions based on the preceding variables for each variable node are analyzed to obtain the sequential sampling sequence.

9. The method as described in claim 1, characterized in that, The random sampler is used to sample variables. The sampling results are then merged with the initial model and mapped through the model converter chain to generate a test input model, including: The dynamic value range of each variable is calculated sequentially according to the sampling sequence, and the variable value is determined within the dynamic value range by a preset sampling strategy to obtain the variable sampling result; The variable sampling results are combined with the initial model to obtain a combined model that includes the initial assignment scheme for all variables; The combined model is input into the model converter chain, and the variable space mapping transformation is performed in reverse order of the reduction phase to obtain the test input candidate model to be verified. The test input candidate model is substituted into the program path constraints for validity verification. If the verification is successful, the final test input model is generated. Alternatively, if the verification fails, a rollback process of resampling or exact solution is performed.

10. A diverse test case generation system based on generator construction, characterized in that, include: The preprocessing module is used to obtain program path constraints, simplify the program path constraints and eliminate variable dependencies to obtain normalized constraints. The reduction module is used to perform reduction processing on the normalization constraints and generate a model converter chain and a target constraint set. The post-processing module is used to perform constraint decomposition and dependency resolution on the target constraint set, construct a random sampler, and solve for the initial model. The model generation module is used to sample variables using the random sampler, merge the sampling results with the initial model, and then perform mapping transformation through the model converter chain to generate a test input model.

Images