Network protocol response behavior consistency test method based on differential fuzz testing

Abstract

The application provides a network protocol response behavior consistency test method based on differential fuzz testing, which comprises the following steps: preparing an initial test case for a fuzzer instance, and performing a first round of fuzz testing; the fuzzer instance adds valuable test cases found by the fuzzer instance to a shared seed pool, and obtains test cases from the shared seed pool; mapping responses obtained by stimulating a protocol message to a server of different implementations of the same protocol through a reduction function to obtain a reduction response set, and comparing the mapping results to determine a processing strategy of the responses; and evaluating test effects, so that reasons for causing differences in responses are analyzed, and then vulnerabilities in protocol implementations are more efficiently mined.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and in particular to a method for testing the consistency of network protocol response behavior based on differential fuzzy testing. Background Technology

[0002] Fuzz testing, as a software testing technique, involves inputting a large amount of automatically or semi-automatically generated random data into a program and monitoring for program anomalies to discover potential program errors. Fuzz testing is widely used to detect security vulnerabilities in software or computer systems.

[0003] Differential protocol fuzzing typically uses templates or variations of existing seeds to generate test cases. These test cases are then used as input to stimulate servers with different implementations of the same protocol. The resulting response messages are then analyzed to uncover vulnerabilities in the protocol implementation. However, the testing process inevitably triggers discrepancies caused by different configurations or interpretations of the specification, rather than vulnerabilities in the protocol implementation itself. Summary of the Invention

[0004] This invention provides a network protocol response behavior consistency testing method based on differential fuzzing, the purpose of which is to improve the efficiency of protocol fuzzing defect detection.

[0005] To achieve the above objectives, this invention provides a network protocol response behavior consistency testing method based on differential fuzzy testing, comprising:

[0006] Step 1: Prepare initial test cases for the fuzzer instance and conduct the first round of fuzz testing;

[0007] Step 2: The fuzzer instance adds the valuable test cases it discovers to the shared seed pool and retrieves test cases from the shared seed pool;

[0008] Step 3: Map the responses obtained after stimulating servers with different implementations of the same protocol with the protocol message to a reduced response set using a reduction function, and compare the mapping results to determine the response processing strategy.

[0009] Step 4: Repeat steps 2 and 3 to evaluate the test results.

[0010] The method includes generative fuzz testing mode and variant fuzz testing mode.

[0011] The method specifically includes:

[0012] Step s1: For generative fuzzing, write a template that conforms to the specific format requirements of the fuzzer according to the protocol specification; for variant fuzzing, select some test cases from the existing test cases as initial seeds.

[0013] Step s2: Run the generative fuzzer instance and the mutator fuzzer instance;

[0014] Step s3: For generative fuzzing, the fuzzer instance generates test cases based on the template. For mutative fuzzing, the fuzzer instance adds the initial seed to the seed queue and selects a seed from it to mutate, thus obtaining test cases.

[0015] Step s4: The fuzzer instance inputs test cases to servers with different implementations of the same protocol for testing;

[0016] Step s5: If the current test case covers a new path, a new branch, or triggers a new crash, the fuzzer instance adds it to the shared seed pool and uses the reduction function to map the obtained response to the reduced response.

[0017] Step s6: The generative fuzzer instance obtains test cases from the shared seed pool, generates a template based on the obtained test cases, and modifies the template using a large language model. The mutative fuzzer instance obtains test cases from the shared seed pool as new seeds and adds them to the seed queue. Based on the comparison results of the reduced responses, the response processing strategy is determined.

[0018] Step s7: Repeat steps s3 to s6 until no test cases cover the new path, new branch, or trigger a new crash for a consecutive preset number of rounds, and then evaluate the test results.

[0019] Specifically, step s5 includes:

[0020] Each fuzzer instance discovers test cases that cover new paths, new branches, or trigger new crashes, which are then stored as seeds in different subfolders under a shared seed pool folder. The responses are then mapped to reduced response sets using the reduction function R.

[0021] The weight of a test case in the shared seed pool is evaluated based on the number of paths (p), branches (b), or new crashes (c) covered by the test case. case = p×l+b×m+c×n, where l+m+n=1.

[0022] Specifically, step s6 includes:

[0023] If the number of test cases generated by the fuzzer instance does not reach the preset value, the fuzzer instance will only test the test cases it generates.

[0024] If the number of test cases generated by the fuzzer instance reaches a preset value, the fuzzer instance will start scanning the subfolders maintained by other fuzzer instances under the shared seed pool folder;

[0025] For a mutated fuzzer instance, if the scanned test cases are not in its own seed queue, they are processed according to weight W. case Test cases are added in descending order of size. For generative fuzzer instances, if a scanned test case has not been tested before, its weight W is increased. case Save the largest test case;

[0026] For generative fuzzer instances, the saved test cases are segmented according to fields, and templates that conform to the specific format requirements of the fuzzer are generated using the segmented fields. The protocol specification document is then input into the large language model, which performs legality checks and refinements on the templates.

[0027] The scheme further includes:

[0028] Under the reduction function R, if two reduction responses are inconsistent, it is considered that the responses come from two different protocol implementations, and therefore one of the protocol implementations is considered to have a vulnerability.

[0029] The above-described solution of the present invention has the following beneficial effects:

[0030] The network protocol response behavior consistency testing method based on differential fuzzing described in the above embodiments of the present invention proposes a novel response analysis framework for protocol fuzzing, improving the way test case generation and response analysis are performed from an architectural perspective. Specifically, by combining the advantages of generative fuzzing and mutant fuzzing in test case generation, a shared seed pool is designed to share test cases discovered by generative and mutant fuzzer instances that cover new paths, new branches, or trigger new crashes, thereby generating more legitimate and diverse test cases. The obtained test cases are then used to stimulate servers with different implementations of the same protocol, mapping the responses under a reduction function to analyze the reasons for the differences in responses, and thus more efficiently uncovering vulnerabilities in the protocol implementation.

[0031] Other beneficial effects of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0032] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0033] To make the technical problems, solutions, and advantages of this invention clearer, a detailed description will be provided below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0034] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0035] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a locking connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0036] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0037] like Figure 1 As shown, an embodiment of the present invention provides a network protocol response behavior consistency testing method based on differential fuzzy testing, including:

[0038] Step 1: Prepare initial test cases for the fuzzer instance and conduct the first round of fuzz testing;

[0039] Step 2: The fuzzer instance adds the valuable test cases it discovers to the shared seed pool and retrieves test cases from the shared seed pool;

[0040] Step 3: Map the responses obtained after stimulating servers with different implementations of the same protocol with the protocol message to a reduced response set using a reduction function, and compare the mapping results to determine the response processing strategy.

[0041] Step 4: Repeat steps 2 and 3 to evaluate the test results.

[0042] The key process of this invention mainly includes three stages: test case push, seed acquisition, and response analysis. First, initial test cases are prepared for each fuzzer instance for the first round of fuzz testing. Then, in the test case push stage, each fuzzer instance adds valuable test cases it discovers to a shared seed pool. In the seed acquisition stage, each fuzzer instance retrieves test cases from the shared seed pool. In the response analysis stage, the responses obtained after stimulating servers with different implementations of the same protocol with protocol messages are mapped to reduced response sets using a reduction function, and the mapping results are compared to determine the response processing strategy. These three stages are executed cyclically. Finally, the test results are evaluated.

[0043] The method includes generative fuzz testing mode and variant fuzz testing mode.

[0044] The method specifically includes:

[0045] Step s1: For generative fuzzing, write a template that conforms to the specific format requirements of the fuzzer according to the protocol specification; for variant fuzzing, select some test cases from the existing test cases as initial seeds.

[0046] Step s2: Run the generative fuzzer instance and the mutator fuzzer instance;

[0047] Step s3: For generative fuzzing, the fuzzer instance generates test cases based on the template. For mutative fuzzing, the fuzzer instance adds the initial seed to the seed queue and selects a seed from it to mutate, thus obtaining test cases.

[0048] Step s4: The fuzzer instance inputs test cases to servers with different implementations of the same protocol for testing;

[0049] Step s5: If the current test case covers a new path, a new branch, or triggers a new crash, the fuzzer instance adds it to the shared seed pool and uses the reduction function to map the obtained response to the reduced response.

[0050] Step s6: The generative fuzzer instance obtains test cases from the shared seed pool, generates a template based on the obtained test cases, and modifies the template using a large language model. The mutative fuzzer instance obtains test cases from the shared seed pool as new seeds and adds them to the seed queue. Based on the comparison results of the reduced responses, the response processing strategy is determined.

[0051] Step s7: Repeat steps s3 to s6 until no test cases cover the new path, new branch, or trigger a new crash for a consecutive preset number of rounds, and then evaluate the test results.

[0052] Specifically, step s5 includes:

[0053] Each fuzzer instance discovers test cases that cover new paths, new branches, or trigger new crashes, which are then stored as seeds in different subfolders under a shared seed pool folder. The responses are then mapped to reduced response sets using the reduction function R.

[0054] The weight of a test case in the shared seed pool is evaluated based on the number of paths (p), branches (b), or new crashes (c) covered by the test case. case = p×l+b×m+c×n, where l+m+n=1.

[0055] Specifically, step s6 includes:

[0056] If the number of test cases generated by the fuzzer instance does not reach the preset value, the fuzzer instance will only test the test cases it generates.

[0057] If the number of test cases generated by the fuzzer instance reaches a preset value, the fuzzer instance will start scanning the subfolders maintained by other fuzzer instances under the shared seed pool folder;

[0058] For a mutated fuzzer instance, if the scanned test cases are not in its own seed queue, they are processed according to weight W. case Test cases are added in descending order of size. For generative fuzzer instances, if a scanned test case has not been tested before, its weight W is increased. case Save the largest test case;

[0059] For generative fuzzer instances, the saved test cases are segmented according to fields, and templates that conform to the specific format requirements of the fuzzer are generated using the segmented fields. The protocol specification document is then input into the large language model, which performs legality checks and refinements on the templates.

[0060] The scheme further includes:

[0061] Under the reduction function R, if two reduction responses are inconsistent, it is considered that the responses come from two different protocol implementations, and therefore one of the protocol implementations is considered to have a vulnerability.

[0062] This method, based on generative fuzzing which generates test cases from templates, introduces mutant fuzzing. The combination of the two generates high-quality test cases, which overcomes the shortcomings of the original mutant fuzzing and reduces the possibility of test cases being rejected prematurely by the protocol implementation. It also overcomes the shortcomings of the original generative fuzzing and improves the diversity of test cases, enabling them to more comprehensively cover the target program.

[0063] This method draws on the seed-sharing mechanism of AFL-p and designs a template generation module based on it to realize the transformation from test cases to templates. This enables generative fuzz testing to utilize valuable test cases in the shared seed pool to improve its testing performance. Furthermore, it combines a large language model to further refine the generated templates.

[0064] Regarding differential testing, this method defines a reasonable metric to evaluate whether there are differences or consistency in the responses. Specifically, a reduction function is chosen to abstract this metric. Under the reduction function, if the mapping results from responses from two different protocol implementations are inconsistent, then one of the protocol implementations is considered to have a vulnerability.

[0065] Terminology Explanation

[0066] (1) Message: The data transmission unit in protocol communication, which defines the format and content of communication, and can be a request, response, etc.

[0067] (2) Seed: The initial input data used to generate test cases. By mutating the seed, a variety of test cases can be generated.

[0068] (3) Corpus: Seed set.

[0069] (4) Template: A structured description that defines the format, fields, data types and possible value ranges of protocol messages, and is used to guide testing tools in generating test cases.

[0070] (5) Protocol Specification: A detailed document or description that specifies all aspects of protocol communication, including message format, field definition, data type, state transition, etc.

[0071] (6) Protocol implementation: Software or systems developed based on protocol specifications to actually support protocol communication.

[0072] The network protocol response behavior consistency testing method based on differential fuzzing described in the above embodiments of the present invention proposes a novel response analysis framework for protocol fuzzing, improving the way test case generation and response analysis are performed from an architectural perspective. Specifically, by combining the advantages of generative fuzzing and mutant fuzzing in test case generation, a shared seed pool is designed to share test cases discovered by generative and mutant fuzzer instances that cover new paths, new branches, or trigger new crashes, thereby generating more legitimate and diverse test cases. The obtained test cases are then used to stimulate servers with different implementations of the same protocol, mapping the responses under a reduction function to analyze the reasons for the differences in responses, and thus more efficiently uncovering vulnerabilities in the protocol implementation.

[0073] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for testing the consistency of network protocol response behavior based on differential fuzzy testing, characterized in that, include: Step 1: Prepare initial test cases for the fuzzer instance and conduct the first round of fuzz testing; Step 2: The fuzzer instance adds the valuable test cases it discovers to the shared seed pool and retrieves test cases from the shared seed pool; Step 3: Map the responses obtained after stimulating servers with different implementations of the same protocol with the protocol message to a reduced response set using a reduction function, and compare the mapping results to determine the response processing strategy. Step 4: Repeat steps 2 and 3 to evaluate the test results; The method includes generative fuzz testing mode and variant fuzz testing mode; The method specifically includes: Step s1: For generative fuzzing, write a template that conforms to the specific format requirements of the fuzzer according to the protocol specification; for variant fuzzing, select some test cases from the existing test cases as initial seeds. Step s2: Run the generative fuzzer instance and the mutator fuzzer instance; Step s3: For generative fuzzing, the fuzzer instance generates test cases based on the template. For mutative fuzzing, the fuzzer instance adds the initial seed to the seed queue and selects a seed from it to mutate, thus obtaining test cases. Step s4: The fuzzer instance inputs test cases to servers with different implementations of the same protocol for testing; Step s5: If the current test case covers a new path, a new branch, or triggers a new crash, the fuzzer instance adds it to the shared seed pool and uses the reduction function to map the obtained response to the reduced response. Step s6: The generative fuzzer instance obtains test cases from the shared seed pool, generates a template based on the obtained test cases, and modifies the template using a large language model. The mutative fuzzer instance obtains test cases from the shared seed pool as new seeds and adds them to the seed queue. Based on the comparison results of the reduced responses, the response processing strategy is determined. Step s7: Repeat steps s3 to s6 until no test cases cover the new path, new branch, or trigger a new crash for 10 consecutive rounds, and then evaluate the test results.

2. The network protocol response behavior consistency testing method based on differential fuzzy testing according to claim 1, characterized in that, Step s5 specifically includes: Each fuzzer instance's discovered test cases that cover new paths, new branches, or trigger new crashes are saved as seeds in different subfolders under a shared seed pool folder, and then reduced using a reduction function. Map the response to a reduced set of responses; The weight of a test case in the shared seed pool is evaluated based on the number of paths (p), branches (b), or new crashes (c) covered by the test case. .

3. The network protocol response behavior consistency testing method based on differential fuzzy testing according to claim 2, characterized in that, Step s6 specifically includes: If the number of test cases generated by the fuzzer instance does not reach the preset value, the fuzzer instance will only test the test cases it generates. If the number of test cases generated by the fuzzer instance reaches a preset value, the fuzzer instance will start scanning the subfolders maintained by other fuzzer instances under the shared seed pool folder; For mutated fuzzer instances, if a scanned test case is not in its seed queue, it is sorted according to weight. Test cases are added in descending order of weight. For generative fuzzer instances, if a scanned test case has not been tested before, its weight is increased. Save the largest test case; For generative fuzzer instances, the saved test cases are segmented according to fields, and templates that conform to the specific format requirements of the fuzzer are generated using the segmented fields. The protocol specification document is then input into the large language model, which performs legality checks and refinements on the templates.

4. The network protocol response behavior consistency testing method based on differential fuzzy testing according to claim 3, characterized in that, The method further includes: In the reduction function If the responses from the two reductions are inconsistent, it is considered that the responses come from two different protocol implementations, and therefore one of the protocol implementations is considered to have a vulnerability.

Images