Efficient fuzzy testing method and system for command line option parameters

Abstract

The invention discloses an efficient fuzz testing method and system for command line option parameters, and belongs to the technical field of fuzz testing. The method comprises the following steps: extracting and grouping option parameters in an option group construction module to form an initial option group; performing multi-parameter simultaneous variation on the option group in the joint variation module to generate test input; instrumentation is carried out through an execution and coverage rate analysis module, and the change of the code coverage rate is monitored; the contribution attribution module carries out single-parameter isolation comparison under the condition that the processing coverage rate is improved; and the iterative optimization module updates the seed pool and finally outputs a vulnerability report. The method is suitable for a scene in which a plurality of option parameters need to be subjected to joint variation, and vulnerabilities in the complex scene cannot be omitted.

Description

Technical Field

[0001] This invention belongs to the field of fuzz testing technology, specifically relating to an efficient fuzz testing method and system for command-line option parameters. Background Technology

[0002] Fuzzing is an automated testing technique that automatically or semi-automatically generates random data according to certain rules. This random data is then input into the entry point of a dynamically running program under test, while simultaneously monitoring for any abnormal behavior, such as system crashes or assertion failures, to discover software defects. Fuzzing can uncover defects that cause program malfunctions and is also suitable for discovering unknown vulnerabilities.

[0003] Fuzz testing was first developed by Barton Miller of the University of Wisconsin. Their work not only used random, unstructured test data, but also systematically utilized a range of tools to analyze various software on different platforms and systematically analyzed the errors found in the tests.

[0004] Fuzzing tools are mainly divided into two categories: mutation testing and generation testing. Mutation testing generates test data by changing existing data samples. Generation testing generates new test data by modeling the program input. Fuzzing can be used for white-box, gray-box, or black-box testing. File formats and network protocols are the most common testing targets, but any program input can be used as a test object.

[0005] The main techniques of mutation-based fuzz testing include: seed input (providing initial data as the basis for mutation), mutation engine (using algorithms such as bit flipping or crossover to generate new input), instrumentation (inserting code to hook and trace the execution path), and feedback loop (optimizing the next round of mutation based on monitoring results).

[0006] Specifically, the fuzz testing process is as follows: (1) Seed preparation: Collect or generate initial input seeds, for example, from sample inputs or random data of the program; (2) Input mutation: Modify the seed using mutation operators, such as random bit flipping, inserting / deleting bytes, or syntax-based structured mutation; (3) Program execution: Inject the mutated input into the target program and monitor the execution behavior through instrumentation tools, including code coverage, memory access and exception signals; (4) Feedback analysis: Evaluate the execution results. If a new coverage path or crash is found, retain the mutated input as a new seed; otherwise, discard it or optimize it further. (5) Iterative optimization: Adjust the mutation strategy based on feedback, such as prioritizing the mutation of historically effective seeds to form a closed loop until the time limit or coverage saturation is reached.

[0007] Through the steps described above, fuzzing can automatically explore the input space of a program, achieve efficient vulnerability detection, and provide a solid foundation for software security.

[0008] In existing technologies, fuzzing methods for command-line option parameters are mainly used to discover potential vulnerabilities, security issues, or abnormal behaviors in command-line programs. These methods test the program's processing logic for option parameters by generating mutated inputs. The technical solution most similar to this invention focuses on how to efficiently generate and mutate option parameters to improve test coverage and efficiency. Existing technologies typically rely on feedback-driven or random mutation mechanisms, but often have limitations in handling complex option dependencies, simultaneous mutation of multiple parameters, or semantic constraints. Several typical existing implementation schemes are described below.

[0009] 1. Fuzzing method based on random string mutation. This method treats the entire command line input as a complete string and generates potential option parameter combinations through random mutation. A representative work is the argv-fuzz mode of the AFL (American FuzzyLop) tool, which uses the command line parameters as input seeds for byte-level mutation. This method uses code coverage feedback to guide the mutation process and can automatically discover some hidden options or parameter combinations. In the specific implementation, the user provides an initial command line string as a seed, runs the program through the mutated string, and monitors for crashes or exceptions. This method can explore the unknown parameter space to a certain extent without knowing the option structure of the program. However, this method has obvious defects: (1) The mutation process is inefficient, and a large number of generated inputs are invalid, resulting in long fuzzing time and high resource consumption; (2) It lacks an understanding of option semantics and cannot handle parameter type constraints (e.g., numerical range and string format), which easily generates invalid parameters; (3) It does not support the combination mutation of multiple option parameters, and cannot perceive whether the mutation is of option parameters, a single option, or multiple options, resulting in low fuzzing efficiency.

[0010] 2. Fuzzing method based on option type inference. This method first identifies the parameter type of the option, then generates initial values ​​based on the type characteristics and performs targeted mutation. A representative work is the OSmart tool, which infers the option type through static analysis, assigns an initial value to each option, and performs mutation according to the mutation rules corresponding to the type. This method mutates according to different option types, which can generate more effective parameter inputs and improve the targeting of fuzzing. However, this method has obvious defects: (1) It cannot perform joint mutation on multiple option parameters at the same time, ignores the mutual influence between options, and may miss more complex vulnerability triggering situations; (2) Type inference depends on accurate analysis of the source code. When the source code is more complex, there may be errors in the identification of option types.

[0011] 3. Structured fuzzing method based on syntax and seeds. This method refers to generating structured input seeds by parsing the command line syntax and then mutating them. For example, using the LibFuzzer framework in combination with a custom command line parser, the option syntax rules are extracted from the program's man page or help documentation to generate initial seeds and test parameter combinations using mutation operators. This method supports the simultaneous mutation of multiple option parameters by defining the input structure to simulate option parameters. This method can improve the validity of the input and better cover the parsing logic of the program. However, this method still has obvious defects: (1) It requires manually defining the syntax rules of option parameters, which is suitable for command line programs that are very familiar with the parameters that the options can accept, but it is difficult to effectively fuzz test other unknown or more complex command line programs; (2) It is difficult to effectively set reasonable syntax rules when multiple parameters need to be mutated at the same time.

[0012] In summary, the main drawbacks of existing methods for parsing command-line options include the following.

[0013] 1. Document-based option extraction method.

[0014] Document-based option extraction methods primarily rely on the documentation accompanying the program to obtain command-line options. However, this approach has the following drawbacks: First, it depends on the accuracy and completeness of the documentation. Different versions of the program may have outdated or incomplete documentation, leading to incomplete extracted option information, or even options that do not match the actual available options. Second, the descriptions of options in the documentation are usually in natural language and lack strict grammatical rules, potentially resulting in vague descriptions and inconsistent formatting. This makes it difficult to parse the parameters corresponding to the options using automated methods. Furthermore, the documentation may even lack descriptions of certain option parameters, making it impossible to use the target option correctly.

[0015] 2. Extraction method based on generated options.

[0016] Generative option extraction methods are primarily represented by AFL's argv mode (afl-argv). This method treats the entire command-line input as a test string and uses fuzzing techniques to randomly or systematically mutate this string, triggering different program execution paths during multiple runs to explore potential command-line options. However, this method can only theoretically cover hidden options not documented in the documentation and has the following drawbacks: First, it treats the command line as a single string, lacking an understanding of the semantics and syntax of the options. The generated input is often a random combination, resulting in a large amount of invalid or unparseable input, leading to a significant waste of time and resources. Second, it only works for target programs that receive short options; its parsing ability for long options is weak, making it difficult to extract complete information about long options. Furthermore, generative option extraction methods cannot directly obtain the semantic attributes of options, such as default values, parameter types, or logical relationships between options. This method lacks higher-level structured information and heavily relies on the exploration efficiency of fuzzing. When the number of program options is large or there is complex combinatorial logic, the generative approach struggles to cover all options within a reasonable timeframe, resulting in incomplete extraction results.

[0017] 3. Inference method based on speculation regarding option type.

[0018] OSmart proposes a heuristic method for inferring option types. Its core idea is to infer the option parameter type based on the assignment behavior of `optarg` in the source code. When an option parameter is detected to be processed by functions such as `atoi(optarg)` or `strtol(optarg)`, it is inferred to be a numeric type; when the parameter is operated on by string copy functions such as `strncpy`, it is inferred to be a string type; if no explicit function call is detected, the option parameter is inferred to be a string by default; if the option does not involve parameter assignment, it is inferred to be an empty type. While this approach is very simple to implement, its type inference capabilities have significant shortcomings: the inference range is too limited, supporting only the three basic types of numbers, strings, and empty, and cannot identify more complex parameter types, such as boolean, enumeration, list, or composite parameters, resulting in overly coarse type classification results. Secondly, this method relies entirely on direct API calls in static source code for type inference. When the program uses indirect function calls, custom conversion functions, or encapsulated library functions, this method cannot accurately capture the actual type semantics, leading to inference errors or omissions. Furthermore, this method lacks context sensitivity, relying primarily on a single assignment statement to infer the type, without considering the broader usage of option parameters within the program. When a parameter is used in different forms across different logical branches, this method cannot comprehensively determine its type. It fails to incorporate the actual runtime behavior of the program to infer the type of the parameter received by the option, thus failing to guarantee the completeness and accuracy of type inference.

[0019] Therefore, current fuzzing methods for command-line option parameters, whether based on random string mutation, option type inference, or structured fuzzing based on syntax and seeds, while capable of fuzzing to some extent and even discovering vulnerabilities in programs, suffer from low mutation efficiency and difficulty in supporting joint mutation of multiple parameters. These problems make it difficult for existing technologies to efficiently and comprehensively test the option parameter behavior of complex command-line programs. This invention aims to solve these practical problems and provide an efficient fuzzing method.

[0020] Furthermore, while the existing technology OSmart: Whitebox Program Option Fuzzing proposes a method to infer option types through static analysis, assign initial values ​​to each option, and mutate them according to type rules, this system fixes other option parameters in the input file when fuzzing the option parameters, only mutating the option parameters of the target option, making it difficult to cover complex scenarios. Summary of the Invention

[0021] This invention discloses an efficient fuzzing method and system for command-line option parameters, which is applicable to scenarios that require joint mutation of multiple option parameters and will not miss vulnerabilities in this complex scenario.

[0022] To achieve the above objectives, the technical solution of the present invention includes the following:

[0023] An efficient fuzzing method for command-line option parameters, the method comprising: Extract all option parameters from the source code of the command-line program, and then... After dividing the options into several groups, add them to the seed pool; Select an option group from the seed pool as the current option group and mutate it to generate several mutation inputs; wherein, the current option group is composed of... Options parameters Composed of, the variant input consists of Options parameters composition; Perform fuzz testing on the mutant inputs in a command-line program, and after selecting the mutant input with the highest code coverage as the high-value input, retain the fuzz test results corresponding to the high-value input. Identify the option parameters that contribute the most to high-value inputs. ; The option parameter that contributes the most The feature is added to the seed pool to improve the option parameter that contributes the most to the mutation process. Corresponding option parameters After determining the probability of selection, the process of selecting an option group from the seed pool as the current option group for mutation is repeated until the set conditions are met. Based on the fuzz test results corresponding to the high-value inputs in each round, the fuzz test results of the command-line program are obtained.

[0024] Furthermore, the option parameters Divided into several option groups, including: Identify option parameters using static analysis tools combined with dynamic execution tracing. Parameter types and dependencies; Based on dependency analysis, the option parameters Divide into option groups.

[0025] Furthermore, the algorithms for performing the mutation include: genetic algorithms, subpopulation optimization algorithms, or differential evolution algorithms.

[0026] Furthermore, identify the option parameters that contribute the most to the high-value inputs. The algorithms include: single-parameter isolation attribution techniques, taint tracking techniques, or statistical techniques based on regression models; wherein, the implementation process of the single-parameter isolation attribution technique includes: Retrieve the option parameters that make up the current option group. And the option parameters of this high-value input ; Use the option parameters from the high-value input respectively Replace the corresponding option parameter in the current option group In order to obtain Individual combinations; exist Select the sub-combination with the highest coverage from the sub-combinations, and then select the option parameters from that sub-combination with the highest coverage. The option parameter that contributes the most.

[0027] Furthermore, the setting conditions include: the code coverage of high-value inputs does not improve within a set time period or reaches the initially set fuzzing test upper limit time.

[0028] Furthermore, based on the fuzz test results corresponding to the high-value inputs in each round, the fuzz test results of the command-line program are obtained, including: Based on the fuzz test results corresponding to each round of high-value inputs, obtain the command-line inputs that caused the command-line program to malfunction and generate an error log; For command-line programs that exhibit abnormalities, the vulnerability type of the vulnerability in the command-line program is obtained based on the Sanitizer classification model; By combining the command-line input, error logs, and vulnerability types that caused the command-line program to malfunction, the fuzz test results of the command-line program are obtained.

[0029] An efficient fuzzing system for command-line option parameters, the system comprising: The option group building module is used to extract all option parameters from the source code of a command-line program and store the option parameters. After dividing the options into several groups, add them to the seed pool; The joint mutation module is used to select an option group from the seed pool as the current option group and mutate it to generate several mutation inputs; wherein, the current option group is composed of... Options parameters Composed of, the variant input consists of Options parameters composition; The execution and coverage module is used to perform fuzz testing on the mutant inputs in the command-line program, and after selecting the mutant input with the highest code coverage as the high-value input, retain the fuzz test results corresponding to the high-value input; The contribution attribution module is used to determine the option parameters that contribute the most to high-value inputs. ; The iterative optimization module is used to select the option parameter that contributes the most. The feature is added to the seed pool to improve the option parameter that contributes the most to the mutation process. Corresponding option parameters After determining the probability of selection, the process of selecting an option group from the seed pool as the current option group is repeated until the set conditions are met. Based on the fuzz test results corresponding to the high-value inputs in each round, the fuzz test results of the command-line program are obtained.

[0030] An electronic device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the efficient fuzzing method for command-line option parameters as described above.

[0031] A computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the efficient fuzzing method for command-line option parameters as described above.

[0032] A computer program product, characterized in that, when the computer program product is run on a computer device, it causes the computer device to execute the efficient fuzzing method for command line option parameters described in any of the preceding claims.

[0033] Compared with the prior art, the present invention has at least the following beneficial effects.

[0034] A joint mutation strategy based on option groups was designed and implemented. By simultaneously mutating multiple option parameters to generate test inputs, and combining this with feedback-driven optimization, the problem of multi-parameter combination explosion in existing methods is avoided.

[0035] A single-parameter isolation attribution mechanism was designed and implemented. When joint mutations lead to an increase in coverage, the contributing parameter is accurately determined by generating and comparing sub-combinations that mutate only a single parameter, and the effective mutation features are iteratively retained accordingly.

[0036] A coverage-oriented iterative testing process was designed and implemented. The compilation strategy of option groups is dynamically adjusted based on attribution results to achieve efficient code coverage improvement and vulnerability discovery. Attached Figure Description

[0037] Figure 1 Overall system architecture diagram.

[0038] Figure 2 Logic flowchart. Detailed Implementation

[0039] The present invention will now be described in further detail with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0040] The overall architecture diagram of the system of this invention is as follows: Figure 1 As shown, after inputting the source code of the command-line program, the option parameters are extracted and grouped in the option group construction module to form an initial option group; the option group is simultaneously mutated with multiple parameters in the joint mutation module to generate test input; the code coverage is instrumented and monitored by the execution and coverage analysis module; the contribution attribution module performs single-parameter isolation comparison when the coverage is improved; the iterative optimization module updates the seed pool and finally outputs a vulnerability report.

[0041] The specific implementation process of this solution is as follows: Figure 2 As shown.

[0042] Step 1: Extracting option parameters and constructing option groups.

[0043] Extract all option parameters from the source code of the input command-line program, and use static analysis tools combined with dynamic execution tracing to identify parameter types and dependencies.

[0044] Specifically, the program's source code is traversed to match option prefixes and generate a parameter list. Then, based on dependency analysis, the parameters are divided into option groups. Each option group is represented as a multi-dimensional vector, and a seed pool containing the option groups is output for subsequent mutation.

[0045] Step 2: Generate initial mutation input.

[0046] Based on option groups, a lightweight genetic algorithm is used to generate the initial mutation vector. Specifically, an option group is selected from the seed pool as the parent vector, and mutation operators are applied randomly to generate the input vector. Mutation methods include bit flipping, string replacement, insertion, and deletion. The input vector is then transformed into a command-line string that can be directly received by the target command-line program, serving as the mutation input. This part can be accelerated using multi-threading.

[0047] In other embodiments, the present invention can also use particle swarm optimization or differential evolution algorithms for initial mutation generation. Similar algorithms maintain the diversity of joint mutations and constraint checks, allowing the system to still generate valid test inputs after replacement, thus achieving the effect of covering the combinatorial space.

[0048] Step 3: Execute tests and monitor code coverage.

[0049] The mutated inputs are executed in the target's command-line program, and their behavior is monitored using instrumentation tools. Specifically, gcov and addressSanitizer are integrated during program compilation to run command-line inputs and record code coverage, exception signals, and execution paths. Batch inputs are executed in parallel to calculate the coverage increment for each input. If there is no improvement in coverage, the vector is discarded. Otherwise, the vector is marked as a high-value input and proceeds to step 4.

[0050] In one embodiment, code coverage monitoring can be implemented not only through integration with gcov and AddressSanitizer, but also by using other instrumentation tools such as kcov. These alternatives are functionally equivalent, all capable of monitoring changes in coverage and providing real-time feedback.

[0051] Step 4: Contribution attribution analysis.

[0052] When a mutation leads to increased coverage, a single-parameter isolation mechanism is triggered to determine the contributing parameter. Specifically, for the current option group (parent vector) and the mutated option group, single-mutation sub-combinations are generated. For example, if the current option group is ["a", "0", "123"] and the mutated option group is ["b", "1", "456"], the vector with only one parameter mutated corresponds to a sub-combination {["b", "0", "123"], ["a", "1", "123"] or ["a", "0", "456"]}. These sub-combinations are executed in parallel, and the code coverage corresponding to each sub-combination is calculated. The sub-combination with the highest coverage is selected, and the changed option parameter within it is identified as the option parameter with the greatest contribution. The attribution results are output, preserving the mutation characteristics of high-contribution parameters, i.e., prioritizing the mutation of option parameters corresponding to option combinations with higher code coverage.

[0053] In other embodiments, the present invention may also use taint tracking techniques or statistical attribution based on regression models to replace subcombination comparisons. These substitutions can, to some extent, quantify the contribution of multiple option parameters while preserving the effect of compilation features.

[0054] Step 5: Iteratively optimize the compilation strategy.

[0055] The seed pool and mutation strategy are updated based on the attribution results. Specifically, features with high contribution parameters are added to the seed pool to increase their probability in the next round of mutation. A feedback loop is formed: steps 2-4 are repeated until code coverage no longer improves within a certain time or the initially set fuzzing time limit is reached.

[0056] Step 6: Output the results.

[0057] Summarize all execution results and analyze the command-line input that caused the target command-line program to malfunction (e.g., program crash or memory leak). Use Sanitizer to classify vulnerability types. Simultaneously, if a program exception is encountered during the process, generate an error log and roll back to the previous valid seed.

[0058] In summary, this invention achieves efficient fuzz testing of command-line option parameters through multi-parameter joint mutation based on option groups, single-parameter isolation attribution mechanism, and coverage-oriented iterative optimization process. This solves the problems of low compilation efficiency, inaccurate attribution, and difficulty in improving code coverage in existing technologies. This method significantly improves the vulnerability discovery capabilities for command-line programs.

[0059] Specifically, this invention efficiently achieves simultaneous mutation of multiple option parameters, balancing test coverage and resource consumption during the multi-parameter mutation process, avoiding the inefficiency caused by combinatorial explosion in existing methods. After multi-parameter mutation, this invention accurately determines which specific parameter leads to improved code coverage, not only resolving blind attribution but also minimizing the additional execution overhead caused during attribution, ensuring overall fuzzing efficiency. This invention iteratively optimizes the mutation strategy based on attribution results, improving overall code coverage.

[0060] Although specific embodiments of the invention have been disclosed for illustrative purposes to aid in understanding and implementing the invention, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the invention should not be limited to the content disclosed in the preferred embodiments, and the scope of protection claimed by the invention is defined by the claims.

Claims

1. An efficient fuzzing method for command line option arguments, characterized in that, The method comprises: Extract all the option parameters from the source code of the command line program and add the option parameters After dividing the option parameters into several option groups, add the seed pool; Selecting an option group from a seed pool as a current option group and performing mutation to generate a plurality of mutated inputs; wherein the current option group is composed of option parameters , and the mutated input is composed of option parameters ; After executing the mutation input in the command line program to perform the fuzz test and selecting a mutation input with the highest code coverage as a high-value input, the fuzz test result corresponding to the high-value input is reserved; Determining most contributing option parameters in high value inputs ; adding the feature of the most contributed option parameter to the seed pool to improve the most contributed option parameter in the variation process corresponding to the option parameter selected probability, re-executing the selection of an option group from the seed pool as the current option group for variation until a set condition is reached Based on the fuzz test result corresponding to each round of high-value input, the fuzz test result of the command line program is obtained.

2. The method of claim 1, wherein, Said option parameter Divided into several option groups, including: Identifying option parameters using static analysis tools in conjunction with dynamic execution traces of the parameter types and dependencies; According to dependency analysis, the option parameters are divided into option groups.

3. The method of claim 1, wherein, The algorithm for executing the mutation includes a genetic algorithm, a sub-population optimization algorithm, or a differential evolution algorithm.

4. The method of claim 1, wherein, Determining most contributing option parameters in high value inputs The algorithm includes: single parameter isolation attribution technology, taint tracking technology or regression model based statistical technology; wherein the implementation process of the single parameter isolation attribution technology includes: respectively obtaining the option parameters of the options combined into the current option group and the option parameters of the high-value input ; Respectively using high-value input option parameters Replace the corresponding option parameters in the current option group To get Sub-combination; In the sub-combinations with the highest coverage and select the option parameter in the sub-combination with the highest coverage as the most contributing option parameter. the sub-combinations with the highest coverage and select the option parameter in the sub-combination with the highest coverage as the most contributing option parameter.

5. The method of claim 1, wherein, The setting condition includes that the code coverage of the high-value input does not improve within a set time or reaches an initially set fuzz test upper limit time.

6. The method of claim 1, wherein, Based on the fuzz test result corresponding to each round of high-value input, the fuzz test result of the command line program is obtained, which includes: According to the fuzz test result corresponding to each round of high-value input, the command line input causing the abnormal command line program is obtained, and an error log is generated; For the abnormal command line program, the vulnerability type of the vulnerability in the command line program is obtained based on a Sanitizer classification model; The fuzz test result of the command line program is obtained by comprehensively considering the command line input causing the abnormal command line program, the error log, and the vulnerability type of the vulnerability.

7. An efficient fuzzing system for command line option arguments, characterized in that, The system comprises: Option group building module, used for extracting all option parameters from the source code of the command line program, and adding the option parameters After dividing into several option groups, join the seed pool; A joint variation module is configured to select an option group from the seed pool as a current option group and to generate a plurality of variation inputs by performing variation on the current option group; wherein the current option group is composed of option parameters and the variation input is composed of option parameters . An execution and coverage module for executing the mutation input in the command line program to perform the fuzz test, and selecting a mutation input with the highest code coverage as a high-value input, and then reserving the fuzz test result corresponding to the high-value input; a contribution attribution module for determining which option parameter in the high value input contributed the most ; an iterative optimization module for adding the most contributing option parameter of the seed pool to improve the mutation process of the most contributing option parameter probability of being selected, re-executes the selection of an option group from the seed pool as the current option group for mutation until a set condition is reached; and obtains the fuzz test result of the command line program based on the fuzz test result corresponding to each round of high value input.

8. An electronic device, comprising: The electronic device comprises a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the efficient fuzz test method for command line option parameters according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer program instructions, and the computer program instructions are executed by the processor to implement the efficient fuzz test method for command line option parameters according to any one of claims 1-6.

10. A computer program product, characterised in that, When the computer program product runs on the computer device, the computer device executes the efficient fuzz test method for command line option parameters according to any one of claims 1-6.

Images