An intelligent fuzz testing method and system for network protocol software

Abstract

An intelligent fuzz testing method and system for network protocol software. The steps of the method are as follows: Step S1: State node definition; The test driver obtains the protocol state machine definition by calling the protocol layer interface and feeds it back to the protocol test scheduler; The protocol test scheduler conducts tests on each node in sequence according to the generated protocol state machine definition; Step S2: State node testing; At the start of a test of a state node, the protocol test scheduler depends on the current state node; The protocol test scheduler requests the test driver to provide communication messages, and it forwards the request for generating communication messages to the protocol layer interface; The protocol layer interface generates communication messages and returns them, which are forwarded to the protocol test scheduler; The protocol test scheduler calls the communication agent to send communication messages to the target protocol software to complete the communication interaction. This system is used to implement the above method. The present invention has the advantages of simple principle, easy operation, wide application range, good scalability, etc.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of automated software testing, and particularly refers to a network protocol intelligent fuzz testing method and testing system that support multiple test engines and multiple instrumentation tools. Background Art

[0002] Fuzz testing is an automated software testing technique that discovers software vulnerabilities by providing unexpected inputs to the target software and monitoring abnormal results. The mainstream fuzz testing techniques can be divided into two categories: black-box fuzz testing and grey-box fuzz testing. The black-box fuzz testing technique randomly and quickly generates test cases and inputs them to the target software, and obtains the test results by judging the output information of the target software. Grey-box fuzz testing, also known as intelligent fuzz testing, requires instrumenting the target software during the testing process to obtain feedback information, and using the feedback information to guide the generation of a new round of test cases, making the testing process more targeted and efficient than black-box fuzz testing.

[0003] An intelligent fuzz testing system generally includes three modules, namely a test case mutation module, a test scheduling module, and an instrumentation monitoring module; its basic workflow is as follows:

[0004] The test case mutation module constructs unexpected inputs and sends them to the target software;

[0005] The instrumentation monitoring module collects and feeds back execution information, and determines whether a crash or an exception occurs;

[0006] The test scheduling module filters out more valuable test cases based on the feedback information and decides which strategy to use.

[0007] The above fuzz testing techniques are widely used in the fields of vulnerability mining and software testing, providing important technical support for the timely discovery and repair of target vulnerabilities such as terminal software, industrial control software, and network protocol software.

[0008] Intelligent fuzz testing of network protocol software refers to the adaptation and transformation of traditional fuzz testing for the characteristics of network protocol software to support intelligent feedback testing of network protocol software. Since network protocol software interacts through network packets and needs to perform protocol state transitions during the interaction, and its software forms are diverse, including source code, binary programs, network services in firmware systems, etc., intelligent fuzz testing of network protocol software requires the fuzz testing system to have stronger adaptation capabilities.

[0009] Therefore, the intelligent fuzz testing system needs to support interaction with network protocol software, maintain protocol state transitions, support multiple software forms, support test scheduling, and also adapt to different test engines according to the format requirements of the target software, in order to achieve the scalability of fuzz testing, the applicability of the target, and the efficiency of testing, and support large-scale and automated vulnerability mining for various forms of network protocol software.

[0010] However, there are some deficiencies in the existing technologies for implementing intelligent fuzz testing of network protocols:

[0011] (1) From the perspective of the test engine, intelligent feedback-based fuzz testing engines mainly face the interaction method of command-line input. Moreover, due to the format and stateful characteristics of network protocols, they need to be adapted one by one when applied to different network protocol software targets. Taking the AFL engine as an example, its process of testing the target software is achieved by reading and writing local files and constructing command parameters, without network communication functions and protocol state maintenance, so it cannot interact with network protocol software. And the AFL engine is a classic intelligent feedback engine, and many advanced test engines with more functions are developed based on it, such as AFLSmart, which can mutate based on the characteristics of block data; such as Superion, which can generate test cases based on the syntax of the test object. Since they are all developed based on AFL, and the AFL engine does not have the ability to test network protocol software, these test engines with good characteristics also cannot be applied to the fuzz testing of network protocol software.

[0012] (2) From the perspective of the instrumentation tool, due to the diverse forms of network protocol software, different types of instrumentation tools are required to complete the instrumentation and monitoring of the target software. Network protocol software has both open-source software forms, and source code instrumentation tools are needed to implant probes during compilation; there are also binary executable program forms, and binary user-mode instrumentation tools are needed to collect software runtime information; at the same time, there is also the binary full-system runtime state, such as network services on the firmware system, which follow the system startup and termination, and support-system binary instrumentation tools are needed to monitor and record. If the fuzz testing system cannot support the adaptation of multiple types of instrumentation tools, then its usage scenarios will be limited. For example, even for the same version of OpenSSL, for the source code form, binary user mode, and full-system state, the code of the fuzz testing system needs to be modified for adaptation, greatly reducing its applicable range.

[0013] In the existing related research on intelligent fuzz testing technology for network protocol software, although it can support network communication functions and protocol state maintenance after being improved based on AFL, these newly added protocol test functions are tightly coupled with the test engine and cannot be adapted to other test engines with more functions. Taking AFLNet and StateAFL as examples, by adding functional codes for network communication and state maintenance modules to the AFL engine, they support network protocol fuzz testing. However, they only support the AFL engine and cannot migrate their protocol test functions to other engines of the same type. If you want to use engines such as AFLSmart and Superion for testing, you must modify the source code, which involves a large amount of work, a complex adaptation process, and poor scalability. In addition, AFLNet only supports source code-level instrumentation and user-level instrumentation, and does not support system-level instrumentation; StateAFL only supports source code-level instrumentation and does not support user-level and system-level instrumentation. If you want to support the testing of multi-form network protocol software, you need to adapt the corresponding types of instrumentation tools one by one, which is highly difficult and has poor scalability. Summary of the Invention

[0014] The technical problem to be solved by the present invention lies in: aiming at the technical problems existing in the prior art, the present invention provides an intelligent fuzz testing method and testing system for network protocol software with simple principle, easy operation, wide application range, and good scalability.

[0015] To solve the above technical problems, the present invention adopts the following technical solutions:

[0016] An intelligent fuzz testing method for network protocol software, the steps of which include:

[0017] Step S1: Definition of state nodes;

[0018] The test driver obtains the protocol state machine definition by calling the protocol layer interface and feeds it back to the protocol test scheduler; the protocol test scheduler sequentially tests each node according to the generated protocol state machine definition.

[0019] Step S2: Testing of state nodes;

[0020] At the beginning of a test of a state node, the protocol test scheduler completes the preliminary communication preparation according to the current state node.

[0021] The protocol test scheduler requests the test driver to provide communication messages, and the test driver forwards the request for generating communication messages to the protocol layer interface.

[0022] The protocol layer interface generates communication messages and returns them, which are forwarded to the protocol test scheduler.

[0023] The protocol test scheduler calls the communication agent to send communication messages to the target protocol software to complete subsequent communication interactions.

[0024] As a further improvement of the method of the present invention: the process of generating the communication message by the protocol layer interface includes:

[0025] After receiving the request to generate a communication message, the protocol layer interface initiates a request to provide a test case to the test driver;

[0026] The test driver forwards the test case generation request to the test agent;

[0027] The test agent receives test cases from the test engine and forwards them to the test driver, which provides them to the protocol layer interface;

[0028] The protocol layer interface assembles communication messages according to the test strategy.

[0029] As a further improvement of the method of the present invention, it also includes: the instrumentation tool provides message execution feedback and the test engine performs test case mutation.

[0030] As a further improvement of the method of the present invention, the process of the instrumentation tool providing message execution feedback and the test engine performing test case mutation includes:

[0031] During a state node test, when the protocol test scheduler and the target protocol software interact, the instrumentation tool continuously records the target software's running information;

[0032] After the interaction is completed, the test driver sends a test end signal to the instrumentation tool and a request to collect execution feedback;

[0033] After receiving the signal, the instrumentation tool sends the recorded operation information to the test driver and clears the record;

[0034] The test driver sends the execution feedback to the test agent, which forwards it to the test engine and notifies it that the test has been completed. The test engine is driven to generate new test cases based on the execution feedback information.

[0035] As a further improvement of the method of the present invention, it also includes a test driver adaptation process:

[0036] (a) Protocol layer adaptation: interacting with the protocol-related layers through the interfaces provided by the protocol-independent layers, driving the protocol-related components to implement the protocol definitions and assembling the protocol communication messages from the test cases according to the strategies;

[0037] (b) Test engine adaptation: Initialized by the test driver at the beginning of each message test, specifying the seed path and providing the engine startup parameters; by constructing a test agent as the intermediate test object of the test engine;

[0038] (c) Instrumentation tool adaptation; Initialized to a binary user-mode instrumentation system by the test driver at the start of each message test, providing startup parameters, starting the target software, and performing instrumentation monitoring; The collection of execution feedback starts from the beginning of a test and continuously records the execution path information by the qemu instrumentation tool until the test driver notifies the end of a test, stops the collection, and returns the recorded results. When the next test starts, it clears and starts collecting again.

[0039] The present invention further provides an intelligent fuzz testing system for network protocol software, which includes:

[0040] A protocol test scheduler for providing protocol state scheduling and test scheduling functions;

[0041] A test driver for providing inter-module communication specifications and data transmission scheduling functions;

[0042] A protocol layer interface for undertaking protocol definition and communication message generation functions;

[0043] A test strategy unit for defining message mutation strategies in a rule-based manner, distinguishing variable fields and fixed fields, and providing them to the protocol layer interface; This unit is used to guide how to assemble test cases into communication messages;

[0044] A communication proxy unit for providing network interaction functions with the target software;

[0045] A test engine adaptation interface unit for providing test case generation functions by interacting with the test engine;

[0046] An instrumentation tool adaptation interface unit for providing execution feedback functions by interacting with the qemu instrumentation tool.

[0047] As a further improvement of the system of the present invention: The test scheduling conducts test scheduling on the state nodes of the current test according to the test strategy.

[0048] As a further improvement of the system of the present invention: The protocol layer interface includes a protocol-independent layer for providing a general process and a protocol-dependent layer for providing different protocol processing; The protocol-independent layer provides processing flow specifications and interface functions, provides adaptation functions for different network protocols, and provides protocol state machine definitions and communication message generation functions for the protocol test scheduler; The protocol-dependent layer is used to complete the definition of the state machine and the generation of communication messages, requests the test driver to generate test cases according to the specified test engine and test strategy, and forms communication messages according to the test strategy.

[0049] As a further improvement of the system of the present invention: the test engine adaptation interface unit uses a general test agent to complete the interaction with other test engines; by providing a processing flow specification and interface functions, the test agent adapts the test engine to the framework and provides test cases for other modules.

[0050] As a further improvement of the system of the present invention: the instrumentation tool adaptation interface unit adapts the instrumentation tool to the framework by providing a processing flow specification and interface functions, provides an execution information recording and feedback function for polymorphic target protocol software, and controls the start and termination of the target protocol software.

[0051] Compared with the prior art, the advantages of the present invention are as follows:

[0052] An intelligent fuzz testing method and testing system for network protocol software of the present invention have a simple principle, convenient operation, wide application range, and good scalability. The present invention is an intelligent fuzz testing method for network protocol software that can support multiple test engines and multiple types of instrumentation tools. By introducing a test driver, designing a general test engine adaptation interface and an instrumentation tool adaptation interface, the source code of the test engine does not need to be modified, and a unified interaction specification is provided for the instrumentation tool. The present invention can support multiple test engines, support the adaptation of multiple types of instrumentation tools, complete the generation of test cases and the feedback of execution paths between different modules, and can implement a highly scalable and highly applicable intelligent fuzz testing framework for network protocol software. Description of the Drawings

[0053] Figure 1 is a flow schematic diagram of the method of the present invention.

[0054] Figure 2 is a principle schematic diagram of the present invention in a specific application example. Detailed Embodiments

[0055] The following will further elaborate on the present invention in detail with reference to the specification drawings and specific embodiments.

[0056] Taking the FTP protocol object LightFTP binary user-mode software, AFLSmart test engine, and qemu instrumentation tool of the present invention as examples, the technical solution of the present invention and the process in specific applications are elaborated in detail.

[0057] As Figure 1 shown, an intelligent fuzz testing method for network protocol software of the present invention includes the following steps:

[0058] Step S1: State node definition;

[0059] The test driver obtains the FTP protocol state machine definition by calling the protocol layer interface and feeds it back to the protocol test scheduler. The protocol test scheduler then tests each node in turn based on the generated FTP protocol state machine definition.

[0060] Step S2: status node test;

[0061] When a test of a status node begins, the protocol test scheduler completes the preliminary communication preparation according to the current status node;

[0062] The protocol test scheduler requests the test driver to provide an FTP communication message, and the test driver forwards the request for generating the FTP communication message to the protocol layer interface;

[0063] The protocol layer interface generates and returns FTP communication messages, which are then forwarded to the protocol test scheduler.

[0064] The protocol test scheduler calls the communication agent to send communication messages to the target protocol software LightFTP to complete subsequent communication interactions.

[0065] As a preferred solution, the present invention further includes: the instrumentation tool qemu provides message execution feedback and the test engine AFLSmart performs test case mutation.

[0066] In a specific application example, in the above steps, the process of generating the FTP communication message by the protocol layer interface includes:

[0067] After receiving the request to generate an FTP communication message, the protocol layer interface initiates a request to provide a test case to the test driver;

[0068] The test driver forwards the test case generation request to the test agent;

[0069] The test agent receives test cases from the test engine AFLSmart and forwards them to the test driver, which provides them to the protocol layer interface;

[0070] The protocol layer interface assembles FTP communication messages according to the test strategy.

[0071] In a specific application example, in the above steps, the process in which the instrumentation tool qemu provides message execution feedback and the test engine AFLSmart mutates the test case includes:

[0072] In a state node test, when the protocol test scheduler interacts with the target protocol software LightFTP, the instrumentation tool qemu continuously records the target software LightFTP running information;

[0073] After the interaction is completed, the test driver sends a test end signal and a request to collect execution feedback to the instrumentation tool qemu;

[0074] After the stub tool qemu receives a signal, it sends the recorded running information to the test driver and then clears the record.

[0075] The test driver sends the execution feedback to the test agent, which forwards it to the test engine AFLSmart and notifies that the test is completed, thereby driving the test engine AFLSmart to generate new test cases based on the execution feedback information.

[0076] In a specific application example, the present invention further includes a test driver adaptation process, which includes:

[0077] (a) Protocol layer adaptation;

[0078] Through the interface provided by the protocol-independent layer, interact with the protocol-related layer, drive the FTP protocol-related components to implement the protocol definition and assemble the test cases into FTP protocol communication messages according to the policy.

[0079] (b) Test engine adaptation;

[0080] At the start of each message test, it is initialized by the test driver, specifying the seed path seeds_dir and providing the startup parameters of the AFLSmart engine ". / afl-fuzz -i seeds_dir -o fuzz_output -Q -d -m 1024".

[0081] Then, by constructing a test agent as an intermediate test object for the AFLSmart test engine. The test agent completes receiving test cases from the AFLSmart test engine, forwarding the test cases to the test driver, receiving the execution feedback from the test driver and forwarding it to the AFLSmart test engine, and notifying the AFLSmart test engine that the current test is completed.

[0082] (c) Stub tool adaptation;

[0083] At the start of each message test, it is initialized by the test driver as the binary user-mode stub system qemu, providing the startup parameters "qemu-x86_64 -d nochain LightFTP / fftp_c LightFTP / fftp.conf 2200", starting the target software LightFTP and performing stub monitoring. The collection of execution feedback starts from the beginning of a test. The qemu stub tool continuously records the execution path information until the test driver notifies it that a test ends, stops collecting and returns the recorded results, and clears and starts collecting again when the next test starts.

[0084] Such as Figure 2As shown in the figure, the present invention further provides an intelligent fuzz testing system for network protocol software, which includes:

[0085] A protocol test scheduler for providing protocol state scheduling and test scheduling functions.

[0086] A test driver for providing inter-module communication specifications and data transmission scheduling functions.

[0087] A protocol layer interface for undertaking protocol definition and communication message generation functions.

[0088] A test strategy unit for defining FTP message mutation strategies in a rule-based manner, differentiating variable fields and fixed fields, and providing them to the protocol layer interface. This unit is used to guide how to assemble test cases into FTP communication messages.

[0089] A communication proxy unit for providing network interaction functions with the target software LightFTP.

[0090] A test engine adaptation interface unit for providing test case generation functions by interacting with the test engine AFLSmart.

[0091] An instrumentation tool adaptation interface unit for providing execution feedback functions by interacting with the instrumentation tool qemu.

[0092] In a specific application example, the protocol state scheduling refers to selecting state nodes to be tested according to the generated protocol state machine.

[0093] In a specific application example, the test scheduling refers to carrying out test scheduling on the current tested state nodes according to the test strategy. In this example, the protocol test scheduler uses the existing functions of the fuzz testing tool boofuzz.

[0094] In a specific application example, the functions of the test driver may include but are not limited to:

[0095] By docking with the protocol layer interface, the test engine adaptation interface, and the instrumentation tool adaptation interface, the framework is adapted to common network protocols, multiple test engines, and multiple types of instrumentation tools;

[0096] By interacting with the protocol layer interface, providing communication messages for the protocol test scheduler;

[0097] By interacting with the test engine, providing test cases for the protocol layer interface;

[0098] By interacting with the instrumentation tool, providing execution feedback of the target protocol software for the test engine.

[0099] In a specific application example, the protocol layer interface includes a protocol-independent layer for providing general processes and a protocol-dependent layer for providing different protocol processing. The protocol-independent layer provides processing flow specifications and interface functions, provides adaptation functions for different network protocols, and provides a protocol state machine definition and communication message generation function for the protocol test scheduler. The protocol-dependent layer is used to complete the definition of the FTP state machine and the generation of FTP communication messages, request the test driver to generate test cases according to the specified test engine and test strategy, and form communication messages according to the test strategy.

[0100] In a specific application example, the communication proxy unit is used to receive communication messages transmitted by the protocol test scheduler and perform network interaction with the target protocol software LightFTP. In this example, the communication proxy unit uses the existing functions of the fuzz testing tool boofuzz.

[0101] In a specific application example, the test engine adaptation interface unit uses a general test proxy to complete the interaction with other test engines. By providing processing flow specifications and interface functions, the test proxy is used to adapt the test engine AFLSmart to the framework and provide test cases for other modules. The test engine AFLSmart can interact with other components without modifying the source code.

[0102] In a specific application example, the instrumentation tool adaptation interface unit adapts the instrumentation tool qemu to the framework by providing processing flow specifications and interface functions, provides an execution information recording and feedback function for the multi-form target protocol software, and controls the start and stop of the target protocol software LightFTP.

[0103] The above is only the preferred implementation mode of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be pointed out that for those of ordinary skill in the art, several improvements and refinements made without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. An intelligent fuzz testing method for network protocol software, characterized in that the steps include: Step S1: state node definition; The test driver obtains the protocol state machine definition by calling the protocol layer interface and feeds it back to the protocol test scheduler; The protocol test scheduler tests each node in turn according to the generated protocol state machine definition; Step S2: status node test; When a test of a status node begins, the protocol test scheduler completes the preliminary communication preparation according to the current status node; The protocol test scheduler requests the test driver to provide a communication message, and the test driver forwards the request to generate the communication message to the protocol layer interface; The protocol layer interface generates and returns communication messages, which are then forwarded to the protocol test scheduler. The protocol test scheduler calls the communication agent to send communication messages to the target protocol software to complete subsequent communication interactions; Also included is the test drive adaptation process: (a) Protocol layer adaptation: interacting with protocol-related layers through the interfaces provided by the protocol-independent layer, driving protocol-related components to implement protocol definitions and assembling protocol communication messages from test cases according to strategies; (b) Test engine adaptation: Initialized by the test driver at the beginning of each message test, specifying the seed path and providing the engine startup parameters; by constructing a test agent as the intermediate test object of the test engine; (c) Instrumentation tool adaptation: At the beginning of each message test, the test driver initializes the binary user-mode instrumentation system, provides startup parameters, starts the target software, and performs instrumentation monitoring. Execution feedback is collected from the beginning of a test. The qemu instrumentation tool continuously records execution path information until the test driver notifies it that a test has ended. It then stops collecting and returns the recorded results, and waits until the next test starts to clear and restart collection.

2. The intelligent fuzz testing method for network protocol software according to claim 1, wherein The process of generating the communication message by the protocol layer interface includes: After receiving the request to generate a communication message, the protocol layer interface initiates a request to provide a test case to the test driver; The test driver forwards the test case generation request to the test agent; The test agent receives test cases from the test engine and forwards them to the test driver, which provides them to the protocol layer interface; The protocol layer interface assembles communication messages according to the test strategy.

3. The intelligent fuzz testing method for network protocol software according to claim 1 or 2, characterized in that, Also includes: The instrumentation tool provides message execution feedback and the test engine performs test case mutation.

4. The intelligent fuzz testing method for network protocol software according to claim 3, characterized in that The process of the instrumentation tool providing message execution feedback and the test engine performing test case mutation includes: During a state node test, when the protocol test scheduler and the target protocol software interact, the instrumentation tool continuously records the target software's running information; After the interaction is completed, the test driver sends a test end signal to the instrumentation tool and a request to collect execution feedback; After receiving the signal, the instrumentation tool sends the recorded operation information to the test driver and clears the record; The test driver sends the execution feedback to the test agent, which forwards it to the test engine and notifies it that the test has been completed. The test engine is driven to generate new test cases based on the execution feedback information.

5. An intelligent fuzz testing system for network protocol software, characterized in that, include: Protocol test scheduler, used to provide protocol status scheduling and test scheduling functions; Test driver, used to provide inter-module communication specifications and data transmission scheduling functions; Protocol layer interface, used to define protocols and generate communication messages; A test strategy unit is used to define the message mutation strategy in a regular manner, distinguish variable fields and fixed fields, and provide them to the protocol layer interface; This unit is used to guide how to assemble test cases into communication messages; A communication proxy unit is used to provide network interaction functions with the target software; A test engine adaptation interface unit is used to provide test case generation functions by interacting with the test engine; A stub tool adaptation interface unit is used to provide execution feedback functions by interacting with the stub tool qemu.

6. The intelligent fuzz testing system for network protocol software according to claim 5, wherein The test scheduling is to carry out test scheduling on the status nodes of the current test according to the test strategy.

7. The intelligent fuzz testing system for network protocol software according to claim 5, characterized in that The protocol layer interface includes a protocol-independent layer for providing general processes and a protocol-related layer for providing different protocol processing; the protocol-independent layer provides processing flow specifications and interface functions, provides adaptation functions for different network protocols, and provides protocol state machine definitions and communication message generation functions for the protocol test scheduler; the protocol-related layer is used to complete the definition of the state machine and the generation of communication messages, request the test driver to generate test cases according to the specified test engine and test strategy, and form communication messages according to the test strategy.

8. The intelligent fuzz testing system for network protocol software according to claim 5, characterized in that The test engine adaptation interface unit uses a general test proxy to complete the interaction with other test engines; by providing processing flow specifications and interface functions, the test engine is adapted to the framework using the test proxy to provide test cases for other modules.

9. The intelligent fuzzy testing system for network protocol software according to claim 5, wherein The stub tool adaptation interface unit adapts the stub tool to the framework by providing processing flow specifications and interface functions, provides execution information recording and feedback functions for multi-form target protocol software, and controls the start and termination of the target protocol software.

Images