A fuzz testing method based on adaptive multi-objective optimization

Abstract

The application discloses a fuzzy testing method based on adaptive multi-target optimization, which comprises the following steps: S1, selecting seeds from a seed queue, calculating scores of each target combination, and selecting the most suitable target combination according to the scores; S2, determining the energy of the current seed according to the selected target combination; S3, determining the mutation and execution times of the seed according to the allocated energy; and S4, determining to open a non-dominated sorting genetic algorithm according to the current target value. The application has the advantages of simple principle, simple operation, wide application range and the like.

Description

Technical Field

[0001] This invention mainly relates to the field of computer security technology, specifically a fuzzy testing method based on adaptive multi-objective optimization, which is particularly suitable for vulnerability discovery systems in the field of computer security. Background Technology

[0002] Fuzz testing is one of the most successful search-based software testing methods. As a variant of fuzz testing, Coverage-Guided Gray-Box Fuzzing (CGF) has recently received widespread attention from researchers. Essentially, CGF is an optimization problem. The key to optimization problems is searching the input space to find the optimal solution and optimize the objective. Optimizing the objective means searching for the maximum or minimum value of the target. In CGF, the most important objective is code coverage; the goal of CGF is to maximize coverage.

[0003] Single-objective optimization searches for the optimal solution to only one objective. However, in the real world, it is necessary to optimize multiple objectives simultaneously to solve problems, such as detecting different types of errors and improving fuzzing efficiency. Specifically, these objectives should be adaptively selected and prioritized according to the test scenario at different stages of the fuzzing process. For example, when testing code snippets related to memory allocation, seeds related to memory consumption should be given priority; to break embedded branch conditions, seeds with more comparison bytes should be an important objective. Therefore, multi-objective optimization (MOO) has been proposed to effectively study the optimal solution among multiple objectives.

[0004] While CGF considers objectives beyond coverage during the search process, existing tools do not truly support multi-objective optimization. For example, AFL also searches for inputs with two additional objectives: execution time and input size. It selects the input (seed) with the smaller product of these two objectives. Theoretically, considering one solution at a time during the search, such as the product of objectives, can lead to getting trapped in local optima and failing to produce a globally optimal solution. Furthermore, some tools cannot coordinate multiple objectives simultaneously. When a new objective is added, the old objective is discarded. For example, MemLock targets memory consumption errors by selecting a seed with higher memory consumption. It optimizes for both coverage and memory consumption objectives. However, as an AFL-based tool, MemLock completely eliminates AFL's speed objective. According to experiments, this neglect of multiple objectives clearly affects the execution speed of fuzz testing.

[0005] Currently, there are some shortcomings in the existing technology:

[0006] 1. During long-term fuzzing, optimizing one objective may negatively impact another. For example, experiments show that increasing the number of comparison bytes to meet branch conditions slows down the entire fuzzing process. This inherent conflict between objectives necessitates the appropriate coordination of different objectives at different stages to find the globally optimal solution.

[0007] 2. Energy scheduling in CGF is used to control mutations and execution counts (i.e., energy) on the seed, thereby guiding the fuzzing process. Previous work on energy scheduling, such as AFLFast and EcoFuzz, aimed to allocate appropriate energy and conserve energy based on the seed's path discovery ability. However, in multi-objective scenarios, energy scheduling algorithms need to be combined with objective combination selection to control energy allocation.

[0008] 3. Efficiency is a crucial metric in fuzzing. When considering multiple objectives, the selection of objective combinations, as well as the optimization of energy scheduling and mutation strategies, introduce additional overhead. For example, Cerebro uses the Pareto front (i.e., the seed set with the optimal objective value) and non-dominated sorting to search for the optimal solution during the testing process. However, Cerebro's process is executed only once per fuzzing loop, causing the main fuzzing process to wait for the final result and wasting valuable CPU time. Moreover, a single run cannot produce a globally optimal solution; calculating the Pareto front and reaching convergence typically requires over 100 iterations through an evolutionary process. Furthermore, directly employing this evolutionary process to find the optimal solution in fuzzing incurs significant performance overhead. Summary of the Invention

[0009] The technical problem to be solved by this invention is to provide a fuzzy testing method based on adaptive multi-objective optimization that is simple in principle, easy to operate, and has a wide range of applications, in response to the technical problems existing in the prior art.

[0010] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0011] A fuzz testing method based on adaptive multi-objective optimization, comprising:

[0012] Step S1: Select a seed from the seed queue, calculate the score of each target combination, and select the most suitable target combination based on the score;

[0013] Step S2: Determine the energy of the current seed based on the selected target combination;

[0014] Step S3: Based on the allocated energy, determine the mutation of the seed and the number of times it will be executed;

[0015] Step S4: Based on the current target value, decide whether to enable the non-dominated sorting genetic algorithm.

[0016] As a further improvement to the method of the present invention: by seed scheduling, a suitable seed is selected from the seed queue, and an adaptive target combination selection algorithm and an adaptive energy allocation algorithm are called to determine the mutation and execution number of a certain seed; seeds transmitted by other modules are received, and these seeds are saved to maintain the seed queue.

[0017] As a further improvement to the method of the present invention: an adaptive target combination selection strategy based on a multi-player multi-armed slot machine model is adopted, the CGF process under multiple objectives is modeled as a multi-player multi-armed slot machine problem, and the target combination is modeled as different players with their own objectives, so as to handle the target combination selection problem and select the best target combination that has the highest reward for the current fuzzy state.

[0018] As a further improvement to the method of the present invention: the adaptive target combination selection strategy based on the multi-player multi-armed slot machine model includes:

[0019] Payout measurement based on target values: The payout for selecting a target in the multi-armed slot machine model is calculated based on the relationship between the current value and the historical average value of different targets. Specifically:

[0020]

[0021] This metric selects the target with the larger current state value, gradually bringing it closer to the optimal solution;

[0022] The return measurement method for target portfolios: Different target portfolios contain different objectives. When the number of objectives is N, the number of target portfolios is 2. N Based on the target returns within the target portfolio, the return of the target portfolio is calculated as follows:

[0023]

[0024] The return of the target portfolio is the average of the returns of each individual target.

[0025] A scoring algorithm based on the UCB1 algorithm: Calculates scores for different target combinations based on the calculated target combination payoff.

[0026]

[0027] Based on the scoring algorithm described above, the optimal target combination for the current testing phase is adaptively selected.

[0028] As a further improvement to the method of the present invention: the seed is modeled as the arm of a slot machine, and the fuzzing state is divided into exploration and development states; MobFuzz controls the mutation and execution number on the seed through adaptive energy scheduling; through this model, MobFuzz allocates appropriate energy to the seed under the selected combination to achieve the optimal solution.

[0029] As a further improvement to the method of the present invention: the testing process is divided into a development process and an exploration process based on whether there are any newly generated seeds with unknown benefits in the fuzz test; the development process allocates the minimum required energy to the seeds so that they can quickly obtain the size of the seed's benefits; the exploration process allocates more energy to seeds with high potential in order to discover more paths and vulnerabilities.

[0030] As a further improvement to the method of the present invention: the non-dominated sorting genetic algorithm adopts the non-dominated sorting genetic method NIC for fuzz testing, uses iterative steps to search for the optimal solution to the selected target, and interacts with the fuzz testing main process through a seed queue; the non-dominated sorting genetic method NIC is designed based on Pareto front and non-dominated sorting, and is used to search for the optimal solution during the evolution process.

[0031] As a further improvement to the method of the present invention: the non-dominated ranking genetic method (NIC) for fuzzy testing includes:

[0032] Adaptively select the initial population size: Before the genetic algorithm starts, randomly select a certain number of seeds as the initial population.

[0033] Based on the effective mutation count of different mutation operators for different targets, the relationship between each target and the mutation operator with better mutation effect is established. When selecting a mutation operator again, the mutation operator with better effect is selected for the current target.

[0034] By sharing a seed queue, the non-dominated sorting genetic method is organically combined with the fuzz testing process.

[0035] As a further improvement to the method of the present invention: In step S4, after opening the non-dominated sorting genetic algorithm, the optimal solution of the target and the seed of the Pareto front are calculated using the genetic algorithm and returned to the fuzzy testing main process; check whether a new path has been found in the testing process. If so, the seed is kept and placed in the seed queue, and the process is transferred to step S1 to continue testing; otherwise, the program execution is terminated by the user's pause operation or the preset running time.

[0036] As a further improvement to the method of the present invention: the process of steps S2 and S3 includes:

[0037] Start the multi-player, multi-armed slot machine model, calculate the payoff for each target combination, and select the combination with the highest current payoff; determine the mutation of the seed and the number of executions based on the allocated energy; and decide whether to enable the non-dominated sorting genetic algorithm based on the current target value.

[0038] Compared with the prior art, the advantages of the present invention are as follows:

[0039] 1. The fuzzy testing method based on adaptive multi-objective optimization of this invention is simple in principle, easy to operate, and widely applicable. By adaptively selecting target combinations, it can solve the problem of conflict effects between different targets. The adaptive target combination selection strategy is based on a multi-player, multi-armed slot machine model. This strategy models the fuzzy test as a multi-player, multi-armed slot machine model, and selects the most suitable target combination for testing according to the progress of the fuzzy test.

[0040] 2. The fuzzy testing method based on adaptive multi-objective optimization of the present invention, by determining energy scheduling in conjunction with target combination selection, is applicable to energy scheduling in multi-objective situations. For the adaptive energy allocation strategy of seed selection, this strategy identifies the testing process as an exploration state and a discovery state on the already selected target combination, allocating different energy to the seed in different states.

[0041] 3. The fuzzing method based on adaptive multi-objective optimization of this invention targets the non-dominated sorting genetic algorithm for fuzzing. This algorithm uses iterative steps to search for the optimal solution to the selected objective and interacts with the fuzzing main process through a seed queue. This invention can find the optimal solution to the selected objective without introducing additional performance overhead, thereby reducing the performance overhead of multi-objective fuzzing. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the method of the present invention.

[0043] Figure 2 This is a schematic diagram illustrating the implementation principle of the present invention in a specific application example.

[0044] Figure 3 This is a schematic diagram of the seed scheduling principle in a specific application example of the present invention.

[0045] Figure 4 This is a detailed flowchart illustrating the system execution timing in a specific application example of the present invention. Detailed Implementation

[0046] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] like Figures 1-4As shown, the present invention provides a fuzz testing method based on adaptive multi-objective optimization, which includes:

[0048] Step S1: Select a seed from the seed queue, calculate the score of each target combination, and select the most suitable target combination based on the score;

[0049] Step S2: Determine the energy of the current seed based on the selected target combination;

[0050] Step S3: Based on the allocated energy, determine the mutation of the seed and the number of times it will be executed;

[0051] Step S4: Based on the current target value, decide whether to enable the non-dominated sorting genetic algorithm.

[0052] In a specific application example, this invention selects a suitable seed from the seed queue through seed scheduling, invokes an adaptive target combination selection algorithm and an adaptive energy allocation algorithm, and determines the mutation and execution count for a given seed. Furthermore, it receives seeds transmitted from other modules and stores these seeds to maintain the seed queue.

[0053] Furthermore, in specific application examples, this invention employs an adaptive objective combination selection strategy based on a multi-player, multi-armed slot machine model. To solve the adaptive selection problem of objective combinations, this invention models the CGF process under multiple objectives as a multi-player, multi-armed slot machine problem. The goal of the classic multi-armed slot machine model is to maximize the reward in a finite number of trials by selecting the appropriate arm. This invention models the objective combination as different players with their own objectives to handle the objective combination selection problem, choosing the optimal objective combination that offers the highest reward for the current fuzzy state.

[0054] Furthermore, in specific application examples, the adaptive target combination selection strategy based on the multi-player, multi-armed slot machine model includes:

[0055] Payout measurement based on target values: The payout for selecting a target in the multi-armed slot machine model is calculated based on the relationship between the current value and the historical average value of different targets. Specifically:

[0056]

[0057] This metric method can select targets with larger current state values ​​to ensure that they gradually approach the optimal solution.

[0058] The return measurement method for target portfolios: Different target portfolios contain different objectives. When the number of objectives is N, the number of target portfolios is 2. N Based on the target returns within the target portfolio, the return of the target portfolio can be calculated as follows:

[0059]

[0060] The return of the target portfolio is the average of the returns of each individual target.

[0061] A scoring algorithm based on the UCB1 algorithm: Based on the calculated return of the target combination, scores can be calculated for different target combinations.

[0062]

[0063] Based on this scoring algorithm, the optimal target combination for the current testing phase can be adaptively selected.

[0064] In a specific application example, this invention employs an adaptive energy allocation strategy. Specifically, to handle energy scheduling suitable for multi-objective scenarios, the invention models the seed as the arm of a slot machine and divides the fuzzing state into exploration and development states. MobFuzz controls the mutation and execution count on the seed through adaptive energy scheduling. Using this model, MobFuzz allocates appropriate energy to the seed under selected combinations to achieve the optimal solution and avoid energy waste.

[0065] Furthermore, in specific application examples, adaptive energy allocation strategies for seed selection include:

[0066] The state identification method in the fuzzing process: Based on whether there are newly generated seeds with unknown potential gains in the fuzzing process, the testing process can be divided into a development process and an exploration process. The development process allocates the minimum required energy to the seeds, aiming to obtain the seed's potential gain as quickly as possible; the exploration mode allocates more energy to seeds with high potential, which can discover more paths and vulnerabilities.

[0067] Adaptive energy allocation strategy: For different objectives, the minimum energy required to reach a certain value is:

[0068]

[0069] The minimum energy for a combination of objectives containing different objectives is:

[0070]

[0071] In the development state, the minimum energy required to allocate to the seed is:

[0072]

[0073] In the exploration phase, the energy allocated to the seed is adaptively adjusted based on the seed's performance:

[0074]

[0075] This energy is proportional to the minimum energy and the seed's performance.

[0076] In a specific application example, the NIC module of this invention employs a non-dominated sorting genetic algorithm (NIC) for fuzz testing. This algorithm uses iterative steps to search for the optimal solution to the selected objective and interacts with the fuzz testing master process through a seed queue. NIC is designed based on Pareto fronts and non-dominated sorting to search for the optimal solution during the evolutionary process.

[0077] Furthermore, in specific application examples, non-dominated sorting genetic algorithms for fuzz testing include:

[0078] Variable population size: The initial population size is a key factor in non-dominated genetic algorithms, which adaptively select the initial population size based on the number of seeds. Before the genetic algorithm begins, 10% of the total number of seeds is randomly selected as the initial population.

[0079] Mutation operators for multi-objective optimization: Based on the effective number of mutations of different mutation operators for different objectives, the relationship between each objective and the mutation operator with better mutation effect can be established. When selecting a mutation operator again, the mutation operator with better effect can be selected for the current objective.

[0080] Overhead strategy for integration with fuzzing process: By using a shared seed queue, the non-dominated sorting genetic algorithm can be organically integrated with the fuzzing process, avoiding wasted CPU execution attempts. Additionally, the algorithm includes a switch to activate the genetic algorithm when needed during testing.

[0081] like Figure 2 As shown, a test system built according to the above method of the present invention in a specific application example includes:

[0082] Seed scheduling module: This module is responsible for selecting suitable seeds from the seed queue, invoking an adaptive target combination selection algorithm and an adaptive energy allocation algorithm, and determining the mutation and execution count for a given seed. In addition, this module is also responsible for receiving seeds transmitted from other modules, storing these seeds, and maintaining the seed queue.

[0083] Program Mutation and Execution Module: This module is responsible for using mutation operators to mutate the selected seed, generating test cases, i.e., the input to the target program. The execution module uses this input to execute the target program and uses instrumentation to detect whether a new program path has been overwritten. If a new path is found, the input is passed as a seed to the seed scheduling module.

[0084] Non-dominated sorting module: This module is responsible for monitoring the increase or decrease of each target value. When the target value decreases, the non-dominated sorting genetic algorithm is activated to find the Pareto seed that can increase the target value and transmit it to the seed scheduling module.

[0085] As can be seen from the above, combining Figure 1 and Figure 2 In a specific application example, the detailed process of this invention includes:

[0086] Step S1: Select a seed from the seed queue for testing;

[0087] Step S2: Activate the multi-player multi-armed slot machine model, calculate the payout for each target combination, and select the combination with the highest current payout;

[0088] Step S3: Activate the energy scheduling algorithm to allocate energy based on the exploration and discovery status and the selected target combination;

[0089] Step S4: Based on the allocated energy, determine the mutation of the seed and the number of times it will be executed;

[0090] Step S5: Determine whether to enable the NIC module based on the current target value;

[0091] Step S6: The NIC module uses a genetic algorithm to calculate the optimal solution of the objective and the seed of the Pareto front, and returns it to the fuzz test main process;

[0092] Step S7: Check if a new path has been found during the testing process. If so, keep the seed in the seed queue and go to step S1 to continue testing.

[0093] Step S8: MobFuzz terminates the program execution based on the user's pause operation or the preset runtime.

[0094] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0095] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A fuzzy testing method based on adaptive multi-objective optimization, characterized in that, include: Step S1: Select a seed from the seed queue, calculate the score of each target combination, and select the most suitable target combination based on the score; Step S2: Determine the energy of the current seed based on the selected target combination; Step S3: Based on the allocated energy, determine the mutation of the seed and the number of times it will be executed; Step S4: Based on the current target value, decide whether to activate the non-dominated sorting genetic algorithm; An adaptive objective combination selection strategy based on a multi-player multi-armed slot machine model is adopted. The CGF process under multiple objectives is modeled as a multi-player multi-armed slot machine problem, and the objective combination is modeled as different players with their own objectives. This is to handle the objective combination selection problem and select the optimal objective combination that has the highest reward for the current fuzzy state. The seed is modeled as the arm of a slot machine, and the fuzzing state is divided into exploration and development states. MobFuzz controls the mutation and number of executions on the seed through adaptive energy scheduling. Through this model, MobFuzz allocates appropriate energy to the seed under selected combinations to achieve the optimal solution. The non-dominated sorting genetic algorithm adopts the non-dominated sorting genetic method NIC for fuzzing testing. It uses iterative steps to search for the optimal solution to the selected target and interacts with the fuzzing main process through a seed queue. The non-dominated sorting genetic method NIC is designed based on Pareto front and non-dominated sorting and is used to search for the optimal solution during the evolution process.

2. The fuzzy testing method based on adaptive multi-objective optimization according to claim 1, characterized in that, By selecting a suitable seed from the seed queue through seed scheduling, and calling an adaptive target combination selection algorithm and an adaptive energy allocation algorithm, the mutation and number of executions for a certain seed are determined. Receive seeds transmitted from other modules, save these seeds, and maintain a seed queue.

3. The fuzzy testing method based on adaptive multi-objective optimization according to claim 1, characterized in that, The adaptive target combination selection strategy based on the multi-player, multi-armed slot machine model includes: Payout measurement based on target values: The payout for selecting a target in the multi-armed slot machine model is calculated based on the relationship between the current value and the historical average value of different targets. Specifically: This metric selects the target with the larger current state value, gradually bringing it closer to the optimal solution. The payoff metric for target combinations is as follows: different target combinations contain different targets; when the number of targets is N, the number of target combinations is 2N. Based on the payoff of the targets within each target combination, the payoff for that combination is calculated as follows: The return of the target portfolio is the average of the returns of each individual target. A scoring algorithm based on the UCB1 algorithm: Calculates scores for different target combinations based on the calculated target combination payoff. Based on the scoring algorithm described above, the optimal target combination for the current testing phase is adaptively selected.

4. The fuzzy testing method based on adaptive multi-objective optimization according to claim 1, characterized in that, Based on whether there are any newly generated, unknown-benefit seeds in the fuzz test, the testing process is divided into a development process and an exploration process; The development process allocates the minimum energy required to the seed so that it can quickly obtain the seed's benefits; the exploration process allocates more energy to seeds with high potential so as to discover more paths and vulnerabilities.

5. The fuzzy testing method based on adaptive multi-objective optimization according to any one of claims 1-4, characterized in that, The non-dominated sorting genetic algorithm adopts the non-dominated sorting genetic method NIC for fuzzing testing. It uses iterative steps to search for the optimal solution to the selected target and interacts with the fuzzing main process through a seed queue. The non-dominated sorting genetic method NIC is designed based on Pareto front and non-dominated sorting and is used to search for the optimal solution during the evolution process.

6. The fuzzy testing method based on adaptive multi-objective optimization according to claim 5, characterized in that, The non-dominated sorting genetic method (NIC) for fuzz testing includes: Adaptively select the initial population size: Before the genetic algorithm starts, randomly select a certain number of seeds as the initial population. Based on the effective mutation count of different mutation operators for different targets, the relationship between each target and the mutation operator with better mutation effect is established. When selecting a mutation operator again, the mutation operator with better effect is selected for the current target. By sharing a seed queue, the non-dominated sorting genetic method is organically combined with the fuzz testing process.

7. The fuzzy testing method based on adaptive multi-objective optimization according to any one of claims 1-4, characterized in that, In step S4, after opening the non-dominated sorting genetic algorithm, the optimal solution of the objective and the seed of the Pareto front are calculated using the genetic algorithm and returned to the fuzzy testing main process; check whether a new path has been found during the testing process. If so, the seed is kept and placed in the seed queue, and the process proceeds to step S1 to continue testing; otherwise, the program execution is terminated by the user's pause operation or the preset running time.

8. The fuzzy testing method based on adaptive multi-objective optimization according to any one of claims 1-4, characterized in that, The processes of steps S2 and S3 include: Start the multi-player, multi-armed slot machine model, calculate the payoff for each target combination, and select the combination with the highest current payoff; determine the mutation of the seed and the number of executions based on the allocated energy; and decide whether to enable the non-dominated sorting genetic algorithm based on the current target value.

Images