A software automatic testing method based on function-level selection and symbolic hybrid execution

By combining fuzzing and symbolic execution in software testing, using symbolic method to call expressions and path constraint segmentation technology, the problems of complex constraint solution and environment modeling in the existing technology are solved, and more efficient and accurate software is achieved. Vulnerability discovery.

CN114138669BActive Publication Date: 2024-12-06NAT UNIV OF DEFENSE TECH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202111510194.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-10
Publication Date
2024-12-06
Estimated Expiration
2041-12-10

AI Technical Summary

Technical Problem

Existing software testing methods are inefficient, costly and error-prone, especially when dealing with complex constraint solutions and environmental modeling, which leads to low efficiency in software vulnerability discovery.

Method used

Using a mixed execution method based on function-level selection, the symbolic execution of complex functions is skipped by generating symbolic methods and calling expressions, the path constraints are divided and the solution is solved using the fuzzing method, which reduces the difficulty and time cost of constraint solving, improves the testing efficiency and Accuracy.

Benefits of technology

It achieves the testing effect of detecting software vulnerabilities more accurately and efficiently, avoids symbol execution failures, and improves the efficiency and accuracy of software testing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114138669B_ABST
    Figure CN114138669B_ABST
Patent Text Reader

Abstract

The invention discloses a software automatic testing method based on function-level selection symbolic hybrid execution, comprising the following steps: step S1. determining complex functions contained in a software program and generating a complex function set; step S2. using a symbolic execution method to collect path constraints of the software to be tested, and when encountering a complex function F belonging to the complex function set, skipping the symbolic execution of the complex function F, creating a symbolic method call expression F' for representing the symbolic execution of the complex function F, and generating a path constraint containing the symbolic method call expression F'; step S3. dividing all the path constraints collected in step S2 into a relevant part constraint P1 containing the symbolic method call expression F' and an irrelevant part constraint P2, using a solver to solve the actual value of the symbolic variable contained in the irrelevant part constraint P2, and then solving P1 through a fuzzy test method to generate a new test case; the invention can discover software vulnerabilities more accurately and efficiently.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a software automatic testing method based on function-level selection symbolic hybrid execution. Background Art

[0002] With the increasing development of information technology, software has penetrated into all areas of social development. However, the software development process is complex. Once improperly developed, software vulnerabilities will occur, causing unnecessary losses during the use of the software. Software testing is a major method for detecting software vulnerabilities. The current mainstream method in the industry is to test the quality of software products by manually designing test cases. However, manually generating test cases is usually inefficient, costly, and prone to errors. Software vulnerabilities can be regarded as erroneous statements hidden under certain conditions. In order to increase the probability of software testing discovering software vulnerabilities, software testing is committed to generating test cases with high code coverage for the program to be tested. Fuzzing and symbolic execution based on coverage guidance are currently two widely studied and used testing technologies.

[0003] The fuzzing method is to dynamically execute the target software after setting the initial test cases, and select existing test cases based on coverage for random mutation to generate new test cases. This process will be repeated until no more branches or statements can be covered. When the test cases are dynamically executed, if an abnormal crash of the software is detected, it is considered that a software vulnerability has been discovered. Since fuzz testing uses random mutation, it is difficult to generate test cases that can directly cover complex conditions, and the test efficiency is often relatively low.

[0004] Symbolic execution is an accurate software testing method for exploring program paths. Symbolic execution extracts symbolic paths from the program and generates test cases based on the path conditions of each symbolic path. Symbolic execution uses constraint solving to effectively explore paths, especially for paths with low probabilities. However, symbolic execution often encounters the following problems: 1. Challenges of complex constraint solving: Constraint solving is the basis of symbolic execution. During the symbolic execution process, all path constraints will call the constraint solver to solve and determine the path reachability. When encountering complex constraint solving, such as complex nonlinear conditions in the program, some path constraints are too large or difficult to solve, resulting in frequent calls to the solver, which makes constraint solving occupy most of the symbolic execution resource consumption and even causes symbolic execution to fail; 2. Challenges of environmental modeling problems: When the analyzed program calls a function in a third-party library and the source code of the library is not available, symbolic execution may not be able to explore the entire path space of the program. Manual modeling handles function calls without source code, but it is usually not automatic and requires a lot of manual work.

[0005] Existing complex constraint solving and environment modeling methods can be divided into two categories: upper approximation methods and lower approximation methods. Upper approximation methods simplify complex constraints into simple constraints, such as over-approximating nonlinear constraints with linear constraints, and introducing new symbolic variables for function calls without source code; lower approximation methods simplify complex constraints by using specific values ​​to replace symbolic values ​​and using specific execution to execute functions without source code. However, both upper approximation methods and lower approximation methods have obvious defects. Upper approximation methods make symbolic execution inaccurate and produce erroneous results; lower approximation methods use specific values ​​or actual execution methods, which will lead to path loss, that is, incomplete exploration of program path space. Summary of the invention

[0006] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide a software automatic testing method based on function-level selection symbolic hybrid execution that can more accurately and efficiently discover software vulnerabilities.

[0007] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0008] A software automatic testing method based on function-level selection symbolic hybrid execution includes the following steps:

[0009] Step S1. Determine the complex functions included in the software program and generate a complex function set;

[0010] Step S2. Use the symbolic execution method to collect the path constraints of the software to be tested. When a complex function F belonging to the complex function set is encountered, the symbolic execution of the complex function F is skipped, and a symbolic method call expression F' is created to represent the symbolic execution of the complex function F, and a path constraint containing the symbolic method call expression F' is generated;

[0011] Step S3. Divide all the path constraints collected in step S2 into relevant part constraints P1 containing the symbolic method call expression F' and irrelevant part constraints P2 not containing the symbolic method call expression F', use the solver to solve the actual values ​​of the symbolic variables contained in the irrelevant part constraints P2, and then solve P1 through the fuzzy testing method to generate a new test case.

[0012] Furthermore, the complex function in step S1 is a function that satisfies a preset complex function rule and / or a function that is preset as a complex function.

[0013] Furthermore, the preset complex function rule is: the function structure contains multiple loops or has nonlinear operations.

[0014] Furthermore, the creation of the symbolic method call expression F' in step S2 is used to represent the execution of the complex function F, and the generation of the path constraint containing the symbolic method call expression F' is specifically as follows: the complex function is instrumented, and when the conditional statement contains the complex function collected by the instrumentation, the symbolic method call expression F' is actually introduced into the path constraint generated by the conditional statement.

[0015] Furthermore, the generated path constraints also include constraints related to complex functions introduced into the context information.

[0016] Furthermore, the fuzzy testing method steps in step S3 include: constructing a dependency tree between each symbolic variable, and performing fuzzy testing on the symbolic expression in sequence according to the dependency order between each symbolic variable in the dependency tree.

[0017] Furthermore, the solver in step S3 is an SMT solver.

[0018] The present invention also proposes a software automatic testing device based on function-level selection symbolic hybrid execution, comprising:

[0019] A complex function determination module, used to determine the complex functions included in the software program and generate a complex function set;

[0020] A path constraint collection module is used to collect path constraints of the software under test by using a symbolic execution method. When a complex function F belonging to the complex function set is encountered, the symbolic execution of the complex function F is skipped, and a symbolic method call expression F' is created to represent the symbolic execution of the complex function F, and a path constraint containing the symbolic method call expression F' is generated;

[0021] The path solving module is used to divide all the path constraints collected by the path constraint collection module into relevant part constraints P1 containing the symbolic method call expression F' and irrelevant part constraints P2 not containing the symbolic method call expression F', use the solver to solve the actual values ​​of the symbolic variables contained in the irrelevant part constraints P2, and then solve P1 through the fuzzy testing method to generate a new test case.

[0022] The present invention further provides a computer device, comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the above method.

[0023] The present invention also provides a computer-readable storage medium storing a computer program, and the computer program implements the above-mentioned method when executed.

[0024] Compared with the prior art, the advantages of the present invention are:

[0025] (1) In software testing, a combination of fuzzing and hybrid symbolic execution is adopted. In the symbolic execution process, the execution of complex functions is effectively avoided by introducing symbolic method call expressions. On the one hand, the difficulty of path constraint modeling is reduced. On the other hand, by describing complex functions in path constraints, the efficiency of constraint solving is improved, symbolic execution failure is avoided, the accuracy of symbolic execution is guaranteed, and the efficiency of symbolic execution is effectively improved, thereby achieving a more accurate and efficient software testing effect of discovering software vulnerabilities.

[0026] (2) During the path constraint solving process, the part of the path constraint related to the symbolic method call expression is segmented, and low-cost fuzzing is used instead of high-cost solver solving, which reduces the time cost of constraint solving and effectively improves the efficiency of software testing.

[0027] (3) By presetting functions with complex function rules and / or presetting functions as complex functions before symbolic execution, the recognition effect of complex functions can be effectively improved, the efficiency of symbolic execution can be improved, and thus the efficiency of software testing can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0029] Figure 1 It is a software example program in the embodiment of the present invention.

[0030] Figure 2 The present invention is a flowchart of a method for automatic software testing based on function-level selection of symbolic mixed execution in an embodiment of the present invention.

[0031] Figure 3 A schematic diagram of the process of solving constraints of a software example program by a software automatic testing method based on function-level selection and symbolic hybrid execution in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present invention is not limited thereby.

[0033] The steps of a software automatic testing method based on function-level selection symbolic hybrid execution in this embodiment include:

[0034] Step S1. Determine the complex functions included in the software program and generate a complex function set;

[0035] Step S2. Use the symbolic execution method to collect the path constraints of the software to be tested. When a complex function F belonging to the complex function set is encountered, the symbolic execution of the complex function F is skipped, and a symbolic method call expression F' is created to represent the symbolic execution of the complex function F, and a path constraint containing the symbolic method call expression F' is generated;

[0036] Step S3. Divide all the path constraints collected in step S2 into relevant part constraints P1 containing the symbolic method call expression F' and irrelevant part constraints P2 not containing the symbolic method call expression F', use the solver to solve the actual values ​​of the symbolic variables contained in the irrelevant part constraints P2, and then solve P1 through the fuzzy testing method to generate a new test case.

[0037] The software test of this embodiment adopts a combination of fuzz testing and hybrid symbolic execution, and in the symbolic execution process, the execution of complex functions is effectively avoided by introducing symbolic method call expressions. On the one hand, the difficulty of path constraint modeling is reduced, and on the other hand, the efficiency of constraint solving is improved by describing complex functions in path constraints, thereby avoiding symbolic execution failures, ensuring the accuracy of symbolic execution, and effectively improving the efficiency of symbolic execution, thereby achieving a more accurate and efficient software testing effect for discovering software vulnerabilities.

[0038] In the process of solving the path constraints, this embodiment divides the part of the path constraints related to the symbolic method call expression, and replaces the high-cost solver solution with low-cost Fuzzing, thereby reducing the time cost of constraint solving and effectively improving the efficiency of software testing.

[0039] This embodiment can effectively improve the recognition effect of complex functions and the efficiency of symbolic execution by presetting functions with complex function rules and / or presetting functions as complex functions before symbolic execution, thereby improving software testing efficiency.

[0040] In this embodiment, the complex function in step S1 is a function that satisfies a preset complex function rule and / or a function that is preset as a complex function.

[0041] The preset complex function rule in this embodiment is: the function structure contains multiple loops or has nonlinear operations.

[0042] In this embodiment, the creation of the symbolic method call expression F' in step S2 is used to represent the execution of the complex function F, and the generation of the path constraint containing the symbolic method call expression F' is specifically to instrument the complex function to collect context information about the complex function. When the conditional statement contains the complex function collected by instrumentation, the symbolic method call expression F' is actually introduced into the path constraint generated by the conditional statement. The context information includes variable values, function call stack, etc. The path constraint is composed of a combination of multiple conditional statements, and a combination of multiple expressions constitutes a path constraint. When the conditional statement contains a complex function collected by instrumentation, by introducing the symbolic method call expression F' into the path constraint generated by the conditional statement, the conditional statement may contain the symbolic method call expression F'.

[0043] In this embodiment, the generated path constraints also include constraints related to complex functions introduced into the context information.

[0044] In order to improve the efficiency of the fuzzing method in step S3 of this embodiment, it is necessary to construct a dependency tree between each symbolic variable and perform fuzzing on the symbolic expressions in the order of dependency.

[0045] In this embodiment, the solver in step S3 is an SMT solver.

[0046] Generally, the software testing framework based on hybrid symbolic execution is roughly divided into four steps: 1. Use test cases to actually execute the program; 2. Use symbolic execution technology to collect path constraints for conditional statements during the actual execution of the program; 3. After the actual execution is completed, the symbolic executor will convert all collected path constraints into SMT formulas, select a constraint for inversion according to the search strategy, and then call the SMT solver to solve it to obtain a new set of test cases; 4. Continue the next execution until the program crashes, then write down the test case that caused the crash.

[0047] In this embodiment, the symbolic execution framework designed based on the software automatic testing method of function-level selection symbolic hybrid execution optimizes the process of collecting path constraints in step 2 when introducing symbolic method call expressions, that is, inserting stubs for complex functions, mainly collecting context information about the complex functions, such as variable values, function call stacks, etc. When the conditional statement contains a complex function collected by stubs, the symbolic method call expression F' is actually introduced into the path constraint generated by the conditional statement. In addition, constraints related to complex functions in the context information will be introduced as needed.

[0048] At the same time, the fuzzy test Fuzzing method is combined for solving, and the process of solving the path constraints in step 3 is optimized. Since the symbolic method call expression is introduced in step 2, in order to deal with the problem of symbolic method call expressions that cannot be converted into SMT formulas, this embodiment takes into account the constraint partitioning method, and hands over the expressions that the SMT solver cannot handle to the fuzzy test Fuzzing. The difficulty of fuzzy test Fuzzing lies in the randomly generated input, whose value range is too large. Without additional information guidance, it is difficult to quickly hit the required solution. Therefore, in this embodiment, the actual value solved by SMT is used to limit the value space of fuzzy test Fuzzing, and construct the dependency tree of each symbolic variable, and perform fuzzy test Fuzzing on the symbolic expression in the order of dependency. Because additional information is provided to guide the fuzzy test Fuzzing to quickly hit the solution, the time overhead is greatly reduced.

[0049] This embodiment also includes a software automatic testing device based on function-level selection symbolic hybrid execution, including:

[0050] A complex function determination module, used to determine the complex functions included in the software program and generate a complex function set;

[0051] A path constraint collection module is used to collect path constraints of the software under test by using a symbolic execution method. When a complex function F belonging to the complex function set is encountered, the symbolic execution of the complex function F is skipped, and a symbolic method call expression F' is created to represent the symbolic execution of the complex function F, and a path constraint containing the symbolic method call expression F' is generated;

[0052] The path solving module is used to divide all the path constraints collected by the path constraint collection module into relevant part constraints P1 containing the symbolic method call expression F' and irrelevant part constraints P2 not containing the symbolic method call expression F', use the solver to solve the actual values ​​of the symbolic variables contained in the irrelevant part constraints P2, and then solve P1 through the fuzzy testing method to generate a new test case.

[0053] This embodiment also includes a computer device, including a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the above-mentioned method.

[0054] This embodiment also includes a computer-readable storage medium storing a computer program, and the computer program implements the above-mentioned method when executed.

[0055] Figure 1 The example software program of the software automatic test in this embodiment is shown, and the testing process of the example software program is as follows:

[0056] Step S1: The sin and cos functions in the program use complex nonlinear operations, which are generally difficult to solve; at the same time, the function usually belongs to an external library function and meets the preset complex function rules, so in step 1 of this embodiment, the sin and cos functions are determined as complex functions and added to the complex function set.

[0057] Step S2: Use the symbolic execution method to collect the path constraints of the software sample program. The three input parameters a, b, and c of the sample program are used as symbolic variables. When the software sample program executes to the error() function, the collected path constraints are Figure 1 Note below that since sin and cos are in the set of complex functions, sin(a) and cos(b) are introduced as symbolic method call expressions. The constraints related to these two complex functions are sin(a)>cos(b) and sin(a)>0.

[0058] Step S3: Divide all the collected path constraints into relevant partial constraints P1 including the symbolic method call expressions sin(a) and cos(b): (sin(a)>cos(b) ∧ sin(a)>0) and the irrelevant part constraint P2 that does not contain a symbolic method call expression: (a>0 ∧ b>0);

[0059] Assuming that the actual values ​​of symbolic variables a and c given by the SMT solver are 5.0 and 1 respectively, the next step is to let fuzz testing to solve them;

[0060] When fuzzing, after substituting a=5.0 into the symbolic method call expression sin(a), it is found that some constraints of P1 are violated. The error reason is that there is a related constraint sin(a)>0, and sin(5.0)<0 violates the constraint, so the symbolic variable a cannot be equal to 5.0. The additional constraint a!=5.0 is added to the P2 constraint (this operation does not exceed a fixed number of iterations. If the condition is still not met after exceeding the number of iterations, "unknown" is returned as the fuzzing solution), and the SMT solver solves P2 again to return a new solution;

[0061] Assume that the actual values ​​of symbolic variables a and c given by the SMT solver are 1.5 and 1 respectively; then the fuzzing method is used to solve P1;

[0062] It is found that the constraint about sin(a)>0 is satisfied, and the value of sin(a) is known. Then the only unknown value of the symbolic variable b among the remaining constraints is the value of the symbolic variable. Then the value of b is repeatedly fuzzed to test the satisfiability of the remaining constraints. It is found that when b is 1, the constraint of sin(a)>cos(b) can be satisfied. All the symbolic variables related to the constraints are known, so the solution process is terminated and the solution of a=1.5, b=1, c=1 is returned, which is the new test case.

[0063] It should be noted that for this example program test process, in step S3, the relevant partial constraint P1 is divided (assuming that the partial constraint P2 is not considered at present): (sin(a)>cos(b) ∧ sin(a)>0), this constraint contains two complex functions sin and cos. Assume that cos(b) is fuzzed first, when b=-3.14..., cos(b) is equal to -1, and then sin(a) is fuzzed. At this time, the condition that needs to be satisfied is sin(a)>cos(b), then when a=-3, sin(a)>cos(b) is satisfied, but sin(a)>0 is not satisfied. It means that if fuzzing is performed in the above random order, it is very likely that multiple fuzzing will be invalid and the solution time will increase.

[0064] In this example, since part of the constraint sin(a)>0 is the constant 0, this constraint has an implicit requirement on the range of a. That is, when fuzzing satisfies the constraint sin(a)>0, the actual value of sin(a) is generated. Then, the sin(a) part of sin(a)>cos(b) becomes a constant. Fuzzing cos(b) can determine the value of b more quickly and satisfy the overall constraint.

[0065] Therefore, in order to improve the efficiency of the fuzzing method in step S3 of the program example test process, it is necessary to construct a dependency tree between each symbolic variable: 0->sin(a)->cos(b); and perform fuzzing on the symbolic expressions in the dependency order, which further speeds up the efficiency of software automatic testing.

[0066] like Figure 2 The flowchart of the software automatic testing method based on function-level selection and symbolic hybrid execution in this embodiment is shown.

[0067] like Figure 3 The figure shows the process of solving the constraints of the software example program based on the software automatic testing method of function-level selection symbolic hybrid execution in this embodiment.

[0068] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A software automatic testing method based on function-level selection symbolic hybrid execution, characterized in that: The following steps are involved: Step S1. Determine the complex functions included in the software program and generate a complex function set; Step S2. Use the symbolic execution method to collect the path constraints of the software to be tested. When a complex function F belonging to the complex function set is encountered, the symbolic execution of the complex function F is skipped, and a symbolic method call expression F' is created to represent the symbolic execution of the complex function F, and a path constraint containing the symbolic method call expression F' is generated; Step S3. Divide all the path constraints collected in step S2 into relevant part constraints P1 containing the symbolic method call expression F' and irrelevant part constraints P2 not containing the symbolic method call expression F', use a solver to solve the actual values ​​of the symbolic variables contained in the irrelevant part constraints P2, and then solve the relevant part constraints P1 through a fuzzy test method to generate a new test case; The step S2 of creating a symbolic method call expression F' for representing the symbolic execution of the complex function F and generating a path constraint containing the symbolic method call expression F' specifically comprises: inserting a complex function, and when a conditional statement contains a complex function collected by the inserting, introducing the symbolic method call expression F' into the conditional statement to generate a path constraint; the generated path constraint also includes constraints related to the complex function in the context information; The fuzzing method steps in step S3 include: constructing a dependency tree between symbolic variables, and performing fuzzing on symbolic expressions in sequence according to the dependency order between symbolic variables in the dependency tree.

2. The software automatic testing method based on function-level selection symbolic hybrid execution according to claim 1, characterized in that: The complex function in step S1 is a function that satisfies a preset complex function rule and / or a function that is preset as a complex function.

3. The software automatic testing method based on function-level selection symbolic hybrid execution according to claim 2 is characterized in that: The preset complex function rule is: the function structure contains multiple loops or has nonlinear operations.

4. The software automatic testing method based on function-level selection symbolic hybrid execution according to any one of claims 1 to 3, characterized in that: The solver is an SMT solver, and step S3 also includes using the value solved by the SMT solver to limit the value space of the fuzzy test Fuzzing method.

5. A software automatic testing device based on function-level selection symbolic hybrid execution, characterized in that: include: A complex function determination module, used to determine the complex functions included in the software program and generate a complex function set; A path constraint collection module is used to collect path constraints of the software under test by using a symbolic execution method. When a complex function F belonging to the complex function set is encountered, the symbolic execution of the complex function F is skipped, and a symbolic method call expression F' is created to represent the symbolic execution of the complex function F, and a path constraint containing the symbolic method call expression F' is generated; A path solving module is used to divide all the path constraints collected by the path constraint collecting module into relevant part constraints P1 containing the symbolic method call expression F' and irrelevant part constraints P2 not containing the symbolic method call expression F', use a solver to solve the actual values ​​of the symbolic variables contained in the irrelevant part constraints P2, and then solve P1 through a fuzzy test method to generate a new test case; The path constraint collection module creates a symbolic method call expression F' for representing the symbolic execution of a complex function F, and generates a path constraint containing the symbolic method call expression F' by instrumenting the complex function, and when a conditional statement contains the complex function collected by instrumentation, the symbolic method call expression F' is introduced into the conditional statement to generate a path constraint; the generated path constraint also includes constraints related to the complex function in the context information; The steps of the fuzzing test method in the solution path module include: constructing a dependency tree between each symbolic variable, and performing fuzzing test on the symbolic expression in sequence according to the dependency order between each symbolic variable in the dependency tree.

6. A computer device comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program, wherein: The processor is configured to execute the computer program to perform the method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed, the method according to any one of claims 1 to 4 is implemented.

Citation Information

Patent Citations

  • Dynamic symbol execution-based Java program error detection method

    CN108304320A

  • Unexplored branch search in hybrid fuzz testing of software binaries

    US20190220387A1

  • Hybrid genetic concolic co-verification of hardware and software

    US20190266071A1