A hybrid fuzz testing method and system based on multi-element seed selection

Abstract

The application discloses a hybrid fuzzing method and system based on multi-element seed selection. Firstly, the static analysis is carried out on the program to be tested, and suspicious vulnerability point set and program control flow graph information are obtained, and the seed coordinator uses the suspicious vulnerability point set and the control flow graph information to generate a maintainable complete control flow graph; secondly, the missing branch table is maintained in the hybrid fuzzing process, and the execution frequency of the executed branch in the program control flow graph is increased; then, according to the program control flow graph and the branch execution frequency, the branch constraint complexity, the branch depth and the suspicious vulnerability point distance of each missing branch are calculated, and the expected income of the symbolic solution of each missing branch is calculated; finally, the final income of the seed is calculated, and the k seeds with the highest final income in the seed pool are selected and provided to the symbolic executor for solving. The application effectively improves the test efficiency of the hybrid fuzzing, explores more branches of the program to be tested, discovers more program vulnerabilities, and is suitable for all AFL type fuzzers.

Description

Technical Field

[0001] This invention belongs to the field of automated software vulnerability detection, specifically involving a hybrid fuzzing testing method and system based on multiple seed selection. Background Technology

[0002] Fuzzing, as the most popular automated software vulnerability discovery technique, has been widely used in academia and industry due to its advantages such as high automation, low energy consumption, and high utilization of computing resources. However, because it uses a brute-force enumeration method with random operators to generate test cases during the seed mutation phase, fuzzing cannot discover all possible paths in the program under test.

[0003] Hybrid fuzzing combines fuzzing with symbolic execution. It uses fuzzing to explore most branches in the program under test, and symbolic execution to solve program branches with complex constraints that fuzzing cannot overcome. This leverages the advantages of both fuzzing and symbolic execution to improve the overall efficiency of vulnerability discovery.

[0004] Symbolic execution incurs significant time overhead, making it impossible to solve all seeds generated during hybrid fuzzing in real-time. Furthermore, existing hybrid fuzzing tools often rely on the symbolic executor to blindly select targets from a large pool of candidate seeds, resulting in low-quality test cases that fail to trigger new program branches for the fuzzer. Therefore, in hybrid fuzzing, selecting high-quality seeds for the symbolic executor to solve, enabling it to generate high-quality test cases within a limited timeframe and maximizing testing efficiency, is a crucial factor influencing the overall efficiency of hybrid fuzzing.

[0005] Existing open-source tools, patents, and papers related to hybrid fuzz testing:

[0006] 1. The open-source testing tool Driller employs a hybrid testing strategy that starts with requirements. Initially, only fuzzing is initiated. Symbolic execution is only activated when fuzzing fails to make progress. The symbolic executor randomly selects candidate seeds, performs symbolic solving, and returns to continue fuzzing after discovering a new branch. The open-source testing tool QSYM designs an incremental symbolic solving scheme for hybrid fuzzing scenarios. This means that for target constraints, QSYM modifies a portion of the input to satisfy the constraints, rather than solving the constraints from scratch. The paper "Towards Optimal Concolic Testing" designs an optimal switching strategy for hybrid fuzzing scenarios. This involves dynamically evaluating fuzzing and symbolic execution during hybrid fuzzing and then selecting the method with the best expected results for software testing. However, the above methods do not consider the expected benefits of seed selection when performing symbolic execution; instead, they blindly select seeds for symbolic execution.

[0007] 2. CN104375942A provides a binary-oriented hybrid fuzzing method that uses binary code coverage as an intermediate layer and calculates the coverage of the program under test in real time. When the coverage no longer improves, symbolic execution is used to explore new paths. CN109117364A provides a target-oriented test case generation method. This method calculates the distance from each node in the CFG of the program under test to the target, performs guided fuzzing based on this distance, and generates target-oriented test cases if the input covers the target; otherwise, symbolic execution is invoked to solve the problem. CN112181833A discloses an intelligent fuzzing method that converts the input of the program under test into symbolic expressions through symbolic execution and constrains the symbolic expressions of the input variables to execute a defined execution path. However, none of the above methods consider the seed selection problem in hybrid fuzzing. The symbolic executor does not evaluate the benefit of solving the seed, resulting in many invalid calculations by the symbolic executor, leading to low overall detection efficiency in hybrid fuzzing.

[0008] In summary, for hybrid fuzzing systems composed of fuzzing and symbolic execution, the symbolic executor's blind selection of targets for symbolic solving makes it difficult for it to choose high-yield targets from the seed pool, thus affecting the overall efficiency of the hybrid fuzzing system. This invention proposes a hybrid fuzzing method based on multivariate seed selection. By performing multivariate quantitative evaluation on branches not explored by the fuzzer in the program control flow graph, the symbolic executor can solve for targets with higher yields, improving the vulnerability detection efficiency of the hybrid fuzzing system. Summary of the Invention

[0009] To address the above problems, this invention proposes a hybrid fuzzing method and system based on multivariate seed selection. First, the fuzzing system performs static analysis on the program under test, obtaining a set of potential vulnerability points and program control flow graph information. The seed coordinator uses this set of potential vulnerability points and control flow graph information to generate a maintainable and complete control flow graph. Second, the fuzzing system maintains a missing branch table during hybrid fuzzing. Whenever a new seed is added to the seed pool, execution path analysis is performed on the new seed to generate an updated missing branch table, while simultaneously increasing the execution frequency of executed branches in the program control flow graph. Then, the seed coordinator in the system calculates the branch constraint complexity, branch depth, and distance to potential vulnerability points for each missing branch in the missing branch table based on the program control flow graph and branch execution frequency, and uses these three pieces of information to calculate the expected symbolic solution benefit for each missing branch. Finally, for each seed in the seed pool, the sum of the benefits of all missing branches on the seed's execution path is calculated as the seed's final benefit. The k seeds with the highest final benefits in the seed pool are selected and provided to the symbolic executor for solution.

[0010] This invention develops a hybrid fuzzing prototype system based on the AFL fuzz tester and the QSYM symbolic executor. The system incorporates a seed coordinator based on multivariate seed selection for screening high-quality seeds. The fuzz tester performs seed mutation and seed pool maintenance; the seed coordinator calculates the symbolic solution yield of all seeds in the seed pool and marks the k seeds with the highest yields; the symbolic executor performs symbolic solutions on the marked seeds and feeds back the solution results that trigger new paths to the seed pool.

[0011] This invention proposes a hybrid fuzzing method and system based on multivariate seed selection, mainly comprising four stages: fuzzing stage, seed coordination stage, symbolic execution stage, and seed synchronization stage. First, static analysis is performed on the program under test to obtain a set of potential vulnerability points and program control flow graph information that may trigger vulnerabilities. The seed coordinator uses the set of potential vulnerability points and control flow graph information to generate a maintainable complete control flow graph. Second, a missing branch table is maintained during the hybrid fuzzing process. Whenever a new seed is added to the seed pool, execution path analysis is performed on the new seed to generate the latest missing branch table, while simultaneously increasing the execution frequency of already executed branches in the program control flow graph. Then, based on the program control flow graph and branch execution frequency, the seed coordinator calculates the branch constraint complexity, branch depth, and distance to potential vulnerability points for each missing branch in the missing branch table, and uses these three pieces of information to calculate the expected symbolic solution benefit for each missing branch. Finally, for each seed in the seed pool, the sum of the benefits of all missing branches on the seed's execution path is calculated as the seed's final benefit, and the k seeds with the highest final benefits in the seed pool are selected and provided to the symbolic executor for solution.

[0012] 1. Fuzz Testing Phase

[0013] The AFL fuzz tester employs methods such as seed pool maintenance, seed priority calculation, fuzz mutation, and seed synchronization, specifically including the following:

[0014] 1.1 Seed Pool Maintenance

[0015] The fuzz tester constructs an initial seed pool using user-provided test cases, and continuously adds test cases that can trigger new paths in the program to the seed pool during the fuzzing process, serving as candidate test cases for the next round of mutation.

[0016] 1.2 Seed Priority

[0017] The fuzz tester will select seeds from the candidate test cases that execute faster, occupy a larger bitmap coverage, and are discovered later for priority fuzz mutation.

[0018] 1.3 Fuzzy Variation

[0019] The fuzz tester sorts the seeds in the seed pool according to the seed priorities described in 1.1 and 1.2, and then mutates the seeds sequentially. Mutation operations include bit flipping, addition and subtraction, singular value replacement, dictionary value replacement, and concatenation. The test cases generated by the mutation are sent to the program under test for execution.

[0020] 1.4 Seed Synchronization

[0021] The fuzz tester synchronizes seeds that are valuable to it from a specified file directory.

[0022] After completing one round of fuzz mutation, the fuzzer performs seed synchronization on the test cases in the specified file directory. It traverses all test cases in the file directory and attempts to execute them. For each test case execution, it records the triggered branch bitmap coverage information. If a new program branch is triggered, it means that the test case is valuable, and the fuzzer will synchronize this test case to the seed pool.

[0023] 2. Seed Coordination Phase: The seed coordinator calculates and maintains the program control flow graph and missing branch table of the program under test. It marks potential vulnerability points on the program control flow graph and calculates ternary information such as branch constraint complexity, branch depth, and distance to potential vulnerability points for all missing branches. This information is used to calculate the seed's reward and select the k highest-reward seeds, which are then submitted to the symbolic executor for solving. Specifically, this includes the following:

[0024] 2.1 Marking Suspicious Vulnerability Points

[0025] The program under test is compiled using a compiler to generate the original program bytecode BitCodeNormal; the program under test is also compiled using a compiler with memory error detection parameters to generate program bytecode BitCodeCrash with memory error detection flags. The differences between BitCodeNormal and BitCodeCrash are compared, and program code blocks that differ between the two bytecode files are considered suspicious vulnerability points CrashCB (Crash CodeBlock) and recorded in the suspicious vulnerability point file CrashesCBs.json.

[0026] 2.2 Construction of Program Control Flow Graph

[0027] The BitCodeNormal in section 2.1 generates the node and edge information BlockBranchInfo.json of the program control flow graph. The seed coordinator takes BlockBranchInfo.json as input, uses the DiGraphMap tool to construct the dynamic program control flow graph (CFG) of the program under test, and then reads CrashesCBs.json to mark suspicious vulnerability points in the CFG to obtain a dynamic CFG with suspicious vulnerability point information.

[0028] 2.3 Branch Information Update

[0029] Calculation of execution frequency of program branches in CFG: Obtain the seed that triggers a new program branch obtained from mutation generation in 1.3 or seed synchronization in 1.4, and then calculate the execution path for each seed to generate the program code block sequence CodeBlocks accessed during program execution: {CodeBlocks[0], ..., CodeBlocks[i], ..., CodeBlocks[n]}, i = 1, 2...n, where n is the total number of program code blocks accessed (program code blocks are continuous program instruction sequences separated by conditional jump instructions). In the program code block sequence, for any pair of code blocks CodeBlocks[i] and CodeBlocks[i+1], it represents a directed program branch in the CFG graph (where CodeBlocks[i] is the prefix code block and CodeBlocks[i+1] is the suffix code block; after executing the prefix code block, the suffix code block is reached through a conditional jump). The execution frequency of this program branch is incremented by 1, indicating that this program branch was accessed during this execution. The execution frequencies of n-1 program branches in CodeBlocks are calculated sequentially.

[0030] For example in Figure 2 The CFG shown contains the sequence of program execution code blocks cbs = {cb0, cb1, cb2, cb4, cb5, cb7, cb9}.<cbi,cbj> This represents a directed program branch from cbi to cbj, where cbi is the prefix code block of cbj and cbj is the suffix code block of cbi. The dashed branch is shown below.<cb2,cb3> The missing branch is the branch that is adjacent to the execution path but has not been executed.

[0031] Maintenance and update of the missing branch list missBranchList: (1) missBranchList is initially empty; (2) For each code block in the program code block sequence CodeBlocks, such as CodeBlocks[i], obtain all its suffix code blocks OtherCBlocks on CFG by the code block identifier: {OtherCBlocks[0], ..., OtherCBlocks[j], ..., OtherCBlocks[m]}, j = 1, 2...m, m is the total number of suffix code blocks. For a directed program branch composed of CodeBlock[i] and any OtherCBlock[j], if the execution frequency of the program branch is 0, then the program branch...<CodeBlock[i],OtherCBlock[j]> (3) If a missing branch in missBranchList appears in CodeBlocks, then the missing branch has been executed, and it is removed from missBranchList.

[0032] 2.4 Calculation of Lost Branch Revenue

[0033] For all missing branches, calculate information such as branch constraint complexity, branch depth, and distance to suspected vulnerabilities.

[0034] 2.4.1 Calculation of Complexity for Missing Branch Constraints

[0035] For a given missing branch<CodeBlockA,CodeBlockB> Based on the code block identifier of CodeBlockA, retrieve all its suffix code blocks OtherCBlocks from the CFG. For each program branch consisting of code blocks in CodeBlockA and OtherCBlocks, calculate the sum of the execution frequencies of all program branches recorded in 2.3 as ExecCountAll. Since...<CodeBlockA,CodeBlockB> For omitted branches, the default execution frequency is ExecCount(<CodeBlockA,CodeBlockB> If ) = 1, then the probability that the tested program reaches CodeBlockB after executing CodeBlockA is 1 / ExecCountAll, which is regarded as the constraint complexity Complex of the missing branch.

[0036] like Figure 2Missing branches in<cb2,cb3> The prefix code block cb2 corresponds to the suffix code block {b3, b4}. cb2 and {b3, b4} together form the program branch {<cb2,cb3> ,<cb2,cb4>}. Assuming ExecCount(<b2,b4> ) = 9, because<b2,b3> Because of the missing branch, the default value for ExecCount is based on experience.<b2,b3> ) = 1, which can be calculated<b2,b3> The constraint complexity is 1 / (1+9) = 0.1.

[0037] 2.4.2 Calculation of Missing Branch Depth

[0038] For a given missing branch<CodeBlockA,CodeBlockB> The shortest path between CodeBlockA and the main function entry block is calculated using Dijkstra's shortest path algorithm on CFG, which is the depth of the missing branch.

[0039] 2.4.3 Calculation of distance to suspected vulnerabilities

[0040] For a given missing branch<CodeBlockA,CodeBlockB> The set of suspected vulnerability points, CrashCBs, is used (in section 2.2, a CFG graph containing information on suspected vulnerability points is constructed, and the set of suspected vulnerability points, CrashCBs, can be obtained by traversing the CFG graph). The Dijkstra shortest path algorithm is used on the CFG graph to calculate the path length CrashLen between each suspected vulnerability point in CodeBlockB and CrashCBs. Then, the harmonic mean distance is calculated for all the obtained CrashLen, which is used as the suspected vulnerability distance Len of the missing branch.

[0041] 2.5 Seed Selection Strategy

[0042] For each missing branch, calculate its expected solution benefit (BranchBenefit) using the following formula:

[0043]

[0044] Where Complex represents the constraint complexity, Depth represents the depth, Len represents the distance to the suspected vulnerability, and c and α are preset empirical values, usually taken as c=1 and α=2.

[0045] After obtaining all BranchBenefits, for each seed in the seed pool, use the method in section 2.3 to obtain all adjacent missing branches on the execution path of that seed. Calculate the sum of BranchBenefits for all missing branches as the total expected SeedBenefit for that seed. Finally, mark the k (usually empirically taken as 2) seeds with the highest SeedBenefits for symbolic execution.

[0046] 3. Symbolic execution phase

[0047] The constraint solving method using the QSYM symbolic executor specifically includes the following:

[0048] 3.1 Seed Acquisition

[0049] The symbolic executor selects all marked seeds from the seed pool, excludes seeds that have already undergone symbolic execution, and performs symbolic execution on the remaining seeds.

[0050] 3.2 Result Generation

[0051] After obtaining the seed, the symbolic executor performs symbolic execution to solve the constraints of adjacent edges in the seed execution path, generates test cases that satisfy the constraints, and places them in the symbolic execution working directory, waiting for the fuzz tester to synchronize the seed.

[0052] 4. Seed Synchronization Stage

[0053] The coverage verification and seed merging method using the AFL fuzz tester specifically includes the following:

[0054] 4.1 Coverage Test

[0055] After completing the fuzz mutation of the current round, the fuzz tester retrieves all test cases from the working directory of symbolic execution and performs bitmap coverage testing on them.

[0056] 4.2 Seed Merging

[0057] If the current test case can trigger a new bitmap region on the original bitmap of the fuzz tester, then the current test case is valuable and is added to the seed pool.

[0058] Another object of the present invention is to provide a hybrid fuzz testing system based on multivariate seed selection, which mainly includes:

[0059] Compilation and Build Modules: The C-class code is instrumented using the afl-clang / afl-clang++ compilation tools. The resulting executable program is used for testing by the fuzzer and symbolic executor. The program under test is built using clang and clang+AddressSanitizer, generating the original bytecode file and the bytecode file with detection flags. The parts that differ between the two bytecode files are marked as suspicious vulnerability points and saved in CrashesCBs.json. At the same time, the blockBranchInfo.json file, which contains the node and edge information of the program's control flow graph, is generated based on the original bytecode file.

[0060] Fuzzing module: The fuzz tester periodically selects high-quality seeds from the seed pool, uses mutation operators such as bit flipping, addition and subtraction, value replacement, and splicing to mutate the seeds, tests the test cases generated by the mutation, and saves information such as the overlay bitmap during the test to determine whether the test cases can trigger new program branches.

[0061] Seed Coordination Module: The seed coordinator periodically calculates the profit information of the seeds in the fuzzer seed pool, including the calculation of ternary information such as branch constraint complexity, branch depth and distance to suspected vulnerabilities, as well as the calculation of the total seed profit; it marks the k seeds with the highest overall profit and provides them to the symbolic executor for symbolic execution solution;

[0062] Symbolic execution module: The symbolic executor obtains the seeds marked by the seed coordinator and performs symbolic execution to solve them; the solution is placed in the symbolic execution working directory and periodically synchronized to the fuzz tester to help the fuzz tester discover new program paths;

[0063] Seed synchronization module: After completing a stage of seed mutation, the fuzzer synchronizes the seed from the specified file directory, traverses all test cases in the file directory and attempts to execute them. For the execution of each test case, it records the triggered branch bitmap coverage information. If a new program branch is triggered, it means that the test case is valuable. The fuzzer will synchronize this test case to the seed pool for use in subsequent stages of fuzz mutation.

[0064] The main advantages of this invention are:

[0065] 1. This invention employs a hybrid fuzzing method based on multivariate seed selection. It evaluates the expected benefits of seed symbol solving in terms of branch constraint complexity, branch depth, and distance to suspected vulnerabilities. This can effectively promote task collaboration between fuzzing and symbol execution in hybrid testing and improve testing efficiency.

[0066] 2. Compared to traditional hybrid fuzzers like QSYM, the hybrid fuzzing method based on multi-seed selection proposed in this invention effectively solves the problem that the symbolic executor in traditional hybrid fuzzers lacks guidance in seed selection, thus failing to effectively improve the efficiency of the fuzzer. In this invention, by adding a seed coordinator during the hybrid fuzzing process, seeds with higher solution benefits can be filtered out, and the symbolic executor can help the fuzzer discover more paths, thereby improving the overall efficiency of hybrid fuzzing.

[0067] 3. Using this invention can improve the efficiency of vulnerability detection. Within the same testing period, more branches of the tested program can be explored, and more program vulnerabilities can be discovered. Attached Figure Description

[0068] Figure 1 This is a flowchart illustrating the hybrid fuzzy testing method based on multivariate seed selection according to the present invention.

[0069] Figure 2 This is a schematic diagram of the missing branches in the hybrid fuzzy testing method based on multivariate seed selection of the present invention.

[0070] Figure 3 This is a flowchart of the data transfer process in the hybrid fuzzy testing method based on multivariate seed selection in this invention.

[0071] Figure 4 This is a flowchart of the seed synchronization process in the hybrid fuzzy testing method based on multivariate seed selection in this invention. Detailed Implementation

[0072] The technical solutions of the embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0073] like Figure 1 As shown, the hybrid fuzzing method based on multivariate seed selection mainly includes four stages: fuzzing stage, seed coordination stage, symbolic execution stage, and seed synchronization stage.

[0074] 1. Fuzz Testing Phase

[0075] 1.1 Use the afl-clang / afl-clang++ compiler tools to compile and instrument the program under test, and use user-provided test cases to build an initial seed pool.

[0076] 1.2 During the testing process, the fuzz tester prioritizes seeds with faster execution speed and larger bitmap area for fuzz mutation. Mutation operators include bit flipping, addition and subtraction, value replacement, and splicing.

[0077] 1.3 The fuzz tester sends the test cases generated by the mutation as input to the program under test, and records the program branch bitmap coverage information while the program under test is executing. If the test case covers a new branch during the execution process, the current test case is considered valuable and is synchronized to the seed pool.

[0078] 1.4 After performing one round of mutation on all candidate seeds, the fuzzer will perform a seed synchronization from the specified file directory. It will traverse all test cases in the file directory and record their bitmap coverage information. If a new bitmap coverage area is triggered, it indicates that the test case is valuable and the test case will be synchronized to the seed pool.

[0079] 2 Seed Coordination Stage

[0080] 2.1 Before testing, the program under test was compiled and built using clang (a C language compiler) to generate the original program bytecode BitCodeNormal; the program under test was then compiled and built using clang + the memory error detector Address Sanitizer (which adds memory error detection flags to code blocks where memory-related errors such as buffer overflows may occur) to generate program bytecode BitCodeCrash with memory error detection flags. A difference analysis was performed on BitCodeNormal and BitCodeCrash, and code blocks that differed between the two bytecode files were considered suspicious vulnerability points CrashCB (CrashCodeBlock) and recorded in the suspicious vulnerability point file CrashesCBs.json.

[0081] 2.2 Generate the node and edge information BlockBranchInfo.json of the program control flow graph based on BitCodeNormal in 2.1. The seed coordinator takes BlockBranchInfo.json as input, uses the DiGraphMap tool to construct the dynamic program control flow graph (CFG) of the program under test, and then reads CrashesCBs.json to mark suspicious vulnerability points in the CFG, thus obtaining a CFG containing information on suspicious vulnerability points.

[0082] 2.3 During the testing process, as the fuzz tester continuously updates the seed pool, the seed coordinator calculates the execution path for the newly added seeds, generating a sequence of program code blocks accessed during program execution: CodeBlocks: {CodeBlocks[0], ..., CodeBlocks[i], ..., CodeBlocks[n]}, i = 1, 2...n, where n is the total number of accessed program code blocks. In the program code block sequence, for any pair of code blocks CodeBlocks[i] and CodeBlocks[i+1], it represents a directed program branch in the CFG graph. Increasing the execution frequency of this program branch by 1 indicates that this program branch was accessed during this execution. The execution frequencies of the n-1 program branches in CodeBlocks are calculated sequentially.

[0083] While tracing the execution path of the newly discovered seed, the missing branch list is updated: (1) missBranchList is initially empty; (2) For each code block in CodeBlocks, such as CodeBlocks[i], all its suffix code blocks OtherCBlocks are obtained on CFG by the code block identifier: {OtherCBlocks[0], ..., OtherCBlocks[j], ..., OtherCBlocks[m]}, j = 1, 2...m, m is the total number of suffix code blocks. For a directed program branch composed of CodeBlock[i] and any OtherCBlock[j], if the execution frequency of the program branch is 0, then the program branch is a missing branch missBranch =<CodeBlock[i],OtherCBlock[j]> (3) If a missing branch missBranch in missBranchList appears in CodeBlocks, then the missing branch missBranch has been executed, and it is removed from missBranchList.

[0084] The pseudocode for the path tracing algorithm is as follows:

[0085]

[0086] 2.4 Seed Expected Return Calculation: For a new seed, update the missing branch table, calculate information such as high constraint complexity, deep branch depth, and small vulnerability distance for all missing branches, then calculate the return of the missing branches, and finally calculate the seed's expected return, such as... Figure 3 As shown.

[0087] The seed's missing branch table is updated, and the calculation of three pieces of information—high constraint complexity of missing branches, deep branch depth, and small vulnerability distance—is performed. The pseudocode for the processing flow is as follows:

[0088]

[0089]

[0090] 2.4.1 Calculation of the constraint complexity of missing branches: Given a missing branch<CodeBlockA,CodeBlockB> Based on the code block identifier of CodeBlockA, retrieve all its suffix code blocks OtherCBlocks from the CFG. For each program branch consisting of code blocks in CodeBlockA and OtherCBlocks, calculate the sum of the execution frequencies of all program branches recorded in 2.3 as ExecCountAll. Since...<CodeBlockA,CodeBlockB> For missBranch, its default execution frequency is ExecCount(<CodeBlockA,CodeBlockB> If ) = 1, then the probability that the tested program reaches CodeBlockB after executing CodeBlockA is 1 / ExecCountAll, which is regarded as the constraint complexity of the missing branch.

[0091] 2.4.2 Depth calculation of missing branches: Dijkstra's shortest path algorithm is used on CFG to calculate the shortest path between CodeBlockA and the main function entry code block, and the length of the shortest path is recorded as the depth of the missing branch.

[0092] 2.4.3 Calculation of Suspicious Vulnerability Distance for Missing Branches: Given a set of suspected vulnerability points CrashCBs, calculate the path length CrashLen between each suspected vulnerability point in CodeBlockB and CrashCBs using Dijkstra's shortest path algorithm on the CFG. Calculate the harmonic mean distance for all calculated CrashLen, which is taken as the suspected vulnerability distance Len for that missing branch.

[0093] 2.4.4 Calculation of Expected Solution Benefit for Missed Branches: For each missed branch (missBranch) in the MissBranchList table, the expected solution benefit (BranchBenefit) for that missed branch is calculated using the following formula:

[0094]

[0095] Complex, Depth, and Len represent the constraint complexity, depth, and distance to the suspected vulnerability of the missBranch, respectively. c and α are preset constant values, typically taken as c=1 and α=2 based on experience.

[0096] 2.4.5 Calculation of Expected Returns from Symbolic Solving of Seeds: After obtaining the BranchBenefits of all missing branches in 2.4.4, for each seed in the seed pool, the sum of the BranchBenefits of all missing branches is calculated as the total expected return of the seed, SeedBenefit. Finally, the k seeds with the highest SeedBenefits (usually empirically taken as 2) are marked for symbolic execution solving.

[0097] The pseudocode for calculating expected seed returns is as follows:

[0098]

[0099]

[0100] The symbol execution phase 3 includes the following processes:

[0101] The symbolic executor acquires the marked seeds, excludes the seeds that have already been symbolically solved, and performs symbolic execution on the remaining seeds one by one. The test cases that satisfy the constraints are then placed in the symbolic execution working directory.

[0102] The seed synchronization phase includes the following processes:

[0103] like Figure 4 As shown, after completing the fuzzing and mutation work for the current round, the fuzzer periodically synchronizes valuable seeds from the symbolic execution file directory. For each seed in the file directory, the fuzzer attempts to execute it and records its bitmap overlay information. If it can overlay a new bitmap, it means that it has explored a new branch in the program under test, and then synchronizes it to the seed pool.

[0104] In summary, the hybrid fuzzing method based on multi-seed selection of this invention adds a seed coordinator to the workflow of the fuzzer and symbolic executor. This helps the symbolic executor select targets that are not easily explored by the fuzzer and are closer to the location where vulnerabilities may be discovered for symbolic execution, thereby improving the overall efficiency of hybrid fuzzing.

[0105] To verify the effectiveness of this invention, the experimental group used a method based on multivariate seed selection for optimization (us), while the control group used the method from the digfuzz paper (digfuzz) and the savior tool (savior). The test set used six well-known real-world software programs, with each test lasting three hours. To reduce the impact of randomness in fuzz testing, all experiments were repeated three times, and the results were averaged. The final experimental data are shown in the table below:

[0106]

[0107]

[0108] As shown in the table, after using the method described in this invention, both the number of paths and the number of vulnerabilities are improved compared to traditional hybrid fuzzing methods. Therefore, the method described in this invention can effectively improve the efficiency of hybrid fuzzing systems in detecting vulnerabilities.

[0109] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. All equivalent changes and modifications made within the scope of the present invention are within the scope of protection of the present invention.

[0110] The component combination features not described in detail in the specification are those readily conceived in the prior art or easily determined and undisputed when implementing the present invention. The above solutions are merely descriptions of preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily implemented by those skilled in the art within the scope described in this application without altering the basic principles involved in the claims should be included within the scope of protection of this application; that is, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A hybrid fuzzy testing method based on multivariate seed selection, characterized in that... It includes four phases: fuzz testing phase, seed coordination phase, symbolic execution phase, and seed synchronization phase; First, static analysis is performed on the program under test to obtain a set of suspicious vulnerability points that may trigger vulnerabilities and program control flow graph information. The seed coordinator uses the set of suspicious vulnerability points and program control flow graph information to generate a maintainable complete control flow graph. Secondly, during the hybrid fuzzing process, a missing branch table is maintained. Whenever a new seed is added to the seed pool, the execution path of the new seed is analyzed and calculated to generate the latest missing branch table. At the same time, the execution frequency of the executed branches in the program control flow graph is increased. Then, the seed coordinator calculates the branch constraint complexity, branch depth, and distance to suspected vulnerability points for each missing branch in the missing branch table based on the program control flow graph and branch execution frequency, and uses these three pieces of information to calculate the expected benefit of the symbolic solution for each missing branch. The process for calculating the complexity of the omitted branch constraint is as follows: For a given omitted branch...<CodeBlockA,CodeBlockB> Based on the code block identifier of CodeBlockA, obtain all its suffix code blocks OtherCBlocks in CFG; for each program branch composed of code blocks in CodeBlockA and OtherCBlocks, calculate the sum of the execution frequencies of all program branches as ExecCountAll; since<CodeBlockA,CodeBlockB> For omitted branches, the default execution frequency is ExecCount(<CodeBlockA,CodeBlockB> If ) = 1, then the probability that the tested program reaches CodeBlockB after executing CodeBlockA is 1 / ExecCountAll, which is regarded as the constraint complexity Complex of the missed branch; The process of calculating the depth of the missing branch is as follows: For a given missing branch...<CodeBlockA,CodeBlockB> The shortest path between CodeBlockA and the main function entry code block is calculated using Dijkstra's shortest path algorithm on CFG, which is the depth of the missing branch. The process for calculating the distance to a suspected vulnerability is as follows: For a given missed branch<CodeBlockA,CodeBlockB> Given a set of suspected vulnerability points CrashCBs, calculate the path length CrashLen between CodeBlockB and CrashCBs using Dijkstra's shortest path algorithm on the CFG graph. Then calculate the harmonic mean distance for all the obtained CrashLen as the suspected vulnerability distance Len for the missing branch. Finally, for each seed in the seed pool, the sum of the expected symbolic solution returns of all missed branches on the seed's execution path is calculated as the seed's final return. The k seeds with the highest final returns in the seed pool are then provided to the symbolic executor for solving. The test cases generated by the solution are then subjected to coverage verification, and test cases that can trigger new program branches are synchronized to the seed pool.

2. The hybrid fuzzy testing method based on multivariate seed selection according to claim 1, characterized in that... Seed coordination Phase 1: The seed coordinator calculates and maintains the program control flow graph and missing branch table of the program under test, marks potential vulnerability points on the program control flow graph, and calculates the branch constraint complexity, branch depth, and distance to potential vulnerability points for all missing branches. This information is used to calculate the seed's reward and select the k seeds with the highest rewards, which are then submitted to the symbolic executor for solving. Specifically, this includes the following: 2.1 Marking Suspicious Vulnerability Points Use a memory error detector to mark the locations in the program under test where vulnerabilities may occur; 2.2 Construction of Program Control Flow Graph The BitCodeNormal in section 2.1 generates the node and edge information BlockBranchInfo.json of the program control flow graph. The seed coordinator takes BlockBranchInfo.json as input, uses the DiGraphMap tool to construct the dynamic program control flow graph (CFG) of the program under test, and then reads CrashesCBs.json to mark suspicious vulnerability points in the CFG to obtain a dynamic CFG with suspicious vulnerability point information. 2.3 Branch Information Update For each seed that triggers a new program branch, the execution path is calculated to generate a sequence of program code blocks accessed during program execution: CodeBlocks: {CodeBlocks[0], ..., CodeBlocks[i], ..., CodeBlocks[n]}, i = 1, 2 ... n, where n is the total number of program code blocks accessed; in the program code block sequence, for any pair of code blocks CodeBlocks[i] and CodeBlocks[i+1], it represents a directed program branch in the CFG graph. The execution frequency of this program branch is incremented by 1, indicating that this program branch was accessed during this execution. The execution frequencies of n-1 program branches in CodeBlocks are calculated sequentially. 2.4 Calculation of Lost Branch Revenue For all missing branches, calculate the branch constraint complexity, branch depth, and distance to the suspected vulnerability respectively; 2.5 Seed Selection Strategy For each missing branch, calculate its sign-solved expected benefit (BranchBenefit) using the following formula: ； Where Complex represents the constraint complexity, Depth represents the depth, Len represents the distance to the suspected vulnerability, and c and α are preset empirical values; After obtaining all BranchBenefits, for each seed in the seed pool, use the method in 2.3 to obtain all adjacent missing branches on the execution path of that seed; calculate the sum of BranchBenefits of all missing branches as the final SeedBenefit of that seed; finally, mark the k seeds with the highest SeedBenefits for symbolic execution.

3. A hybrid fuzzy testing system based on multivariate seed selection, used to implement the method described in claim 1, characterized in that... It includes the following five modules: Compilation and build module: The C class code is instrumented using the afl-clang / afl-clang++ compilation tools. The generated executable program is used for testing by the fuzzer and symbolic executor. The program under test is built using clang and clang+AddressSanitizer, generating the original bytecode file and the bytecode file with detection flags. The parts that differ between the two bytecode files are marked as suspicious vulnerability points and saved to CrashesCBs.ison. At the same time, the blockBranchInfo.json of the program control flow graph is generated based on the original bytecode file. Fuzzing module: The fuzzer periodically selects seeds from the seed pool that have faster execution speed and occupy a larger bitmap area, uses mutation operators to mutate the seeds, tests the test cases generated by the mutation, and saves the overlay bitmap during the test to determine whether the test cases can trigger new program branches. Seed Coordination Module: The seed coordinator periodically calculates the profit information of the seeds in the fuzz tester's seed pool, including the calculation of branch constraint complexity, branch depth and distance to suspected vulnerabilities, as well as the final profit of the seeds; The k seeds with the highest final returns are marked and provided to the symbolic executor for symbolic execution solution; Symbolic execution module: The symbolic executor obtains the seed marked by the seed coordinator and performs symbolic execution solution; the solution is placed in the symbolic execution working directory and periodically synchronized to the fuzz tester to help the fuzz tester discover new program paths; Seed synchronization module: After completing a stage of seed mutation, the fuzzer synchronizes the seed from the specified file directory, traverses all test cases in the file directory and attempts to execute them. For the execution of each test case, it records the triggered branch bitmap coverage information. If a new program branch is triggered, it means that the test case is valuable. The fuzzer will synchronize this test case to the seed pool for use in subsequent stages of fuzz mutation.

Images