A method for fuzz testing stateful network protocols

Abstract

The application discloses a kind of stateful network protocol-oriented fuzz testing method, constructs the corpus of initial test case, and carries out compiling and inserting plug-in operation;Through state selection algorithm, the target state of each round of fuzz testing is selected;From the test case set, the corresponding test case queue is selected, then a test case is selected from the test case queue to generate new test case by variation operation, then the new test case is sent to network protocol server to carry out fuzz testing;After a round of fuzz testing is completed in network protocol server, the feedback information of network protocol server is collected, including code coverage, state coverage and state transition, and subsequent fuzz testing process is guided according to feedback information.The above-mentioned method can solve the problems existing in state selection, test case scheduling and other aspects of traditional network protocol fuzz testing, improve network protocol fuzz testing efficiency and vulnerability mining capability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network protocol security vulnerability detection, and particularly relates to a fuzzy testing method for stateful network protocols. BACKGROUND

[0002] Network protocols are the foundation of modern Internet, which describe the specification of two communication entities to exchange data. However, due to the complexity and diversity of network protocols, it is challenging to correctly implement network protocols, which may lead to the introduction of vulnerabilities in the implementation process. For example, the Heartbleed vulnerability in OpenSSL, one of the implementations of the famous Transport Layer Security (TLS) protocol, can be used by malicious attackers to leak confidential data in the memory of remote devices. Therefore, vulnerabilities in network protocol implementations are a major threat to the entire cyberspace, and it is crucial to discover these target vulnerabilities. Fuzzy testing is one of the most effective techniques for software security analysis, which has the advantages of high testing efficiency, simplicity and low false positive rate. These advantages make it widely used in various scenarios, including network protocols, operating system kernels, and smart contracts. Due to the complex state space of most network protocols, traditional stateless fuzzy testing methods cannot fully represent the internal state of network protocol servers, resulting in low testing efficiency. Currently, fuzzy testing methods for stateful network protocols can be mainly divided into two categories: stateful black-box fuzzy testing and coverage-based gray-box fuzzy testing. Stateful black-box fuzzy testing methods generate a large number of test cases randomly and use state machine models to guide the fuzzy testing process. However, this method often contains many invalid and redundant test cases. In order to improve the efficiency of fuzzy testing, coverage-based gray-box fuzzy testing has gradually attracted the attention of industry personnel. This method uses runtime feedback information to guide the testing process, which can reduce the blindness of the fuzzy testing process and improve the efficiency of fuzzy testing.

[0003] The existing technology coverage-based gray-box fuzzy testing method mainly uses code coverage to guide the fuzzy testing process. This method first selects the state to be tested according to the state selection strategy, then selects the test case from the test case queue corresponding to the state to be tested, and finally uses the test case to perform fuzzy testing on the network protocol server. However, due to the complex state space of stateful network protocols, traditional fuzzy testing methods guided by code coverage cannot effectively test stateful network protocols, mainly having the following problems: (1) the information for evaluating the state is too single, which cannot accurately evaluate the value of the state; (2) there is a lack of effective state selection algorithm, which may cause the fuzzer to spend a lot of time testing meaningless states, thereby reducing the testing efficiency; (3) the selection and retention process of test cases relies too much on code coverage, ignoring other feedback information. SUMMARY

[0004] The application aims to provide a stateful network protocol-oriented fuzz testing method, which can solve the problems of traditional network protocol fuzz testing in state selection, test case selection, test case reservation and the like, and improve the network protocol fuzz testing efficiency and vulnerability mining capability.

[0005] The application aims to achieve the above technical solutions.

[0006] A stateful network protocol-oriented fuzz testing method, the method comprising:

[0007] Step 1, constructing an initial test case corpus and performing a compilation and patching operation;

[0008] Step 2, initializing a state array and selecting a target state of each round of fuzz testing through a state selection algorithm;

[0009] Step 3, selecting a test case queue corresponding to the target state obtained in step 2 from a test case set, then selecting a test case from the test case queue to generate a new test case through a mutation operation, and then sending the new test case to a network protocol server for fuzz testing;

[0010] Step 4, after the network protocol server completes a round of fuzz testing, collecting feedback information of the network protocol server, including code coverage, state coverage and state transition, and guiding the subsequent fuzz testing process according to the feedback information.

[0011] As can be seen from the above technical solutions of the application, the above method can solve the problems of traditional network protocol fuzz testing in state selection, test case selection, test case reservation and the like, and improve the network protocol fuzz testing efficiency and vulnerability mining capability. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0013] Figure 1 The flowchart of the stateful network protocol-oriented fuzz testing method provided by the embodiment of the application is shown in the figure.

[0014] Figure 2 The flowchart of the state selection process described in the embodiment of the application is shown in the figure.

[0015] Figure 3 The schematic diagram of the implementation scene of the example of the application is shown in the figure. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application, which do not constitute a limitation of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the protection scope of the present application.

[0017] As Figure 1 shown is a flowchart of a method for stateful network protocol fuzz testing provided by an embodiment of the present application, the method comprising:

[0018] Step 1, constructing an initial test case corpus and performing a compilation and instrumentation operation;

[0019] In this step, first, a client and a network protocol server are started and communication is performed; all data packets of the network protocol server corresponding port are acquired and collected; finally, the collected data packets are analyzed using a data packet analysis tool WireShark, and all request data packets are extracted as an initial test case queue set TestCaseSet, i.e. TestCaseSet={T,...,T i ,...,T n}, T i refers to the ith test case queue in TestCaseSet, i = 1,...,n, wherein n is the total number of test case queues;

[0020] The test case queue T i is composed of test cases, i.e. T i ={t i，1 ,...t i，j ,...,t i,m}, t i,j represents the jth test case in the ith test case queue, j = 1,...,m, wherein m is the total number of test cases in T i ;

[0021] Compilation is a process of converting source code into executable programs, and instrumentation is a process of adding additional code to target programs at the time of compilation or runtime. Through instrumentation, relevant feedback information of program runtime can be obtained; specifically, a compilation and instrumentation tool LLVM is used to implement compilation and instrumentation on the target protocol server program, to generate corresponding target code for collecting feedback information of program runtime, including code coverage and state information.

[0022] Step 2, initializing a state array and selecting a target state for each round of fuzz testing through a state selection algorithm;

[0023] In this step, when fuzz testing a stateful network protocol, the state to be tested must be selected first. However, not all states are equally important, so an effective state selection algorithm is needed to prioritize important states. These important states can usually increase code coverage and help discover new states or new state transitions.

[0024] The state array is represented as States = {S1, ..., S k ,...,S u}, S k represents the kth state, k = 1, ..., u, and u is the total number of states. The process of state selection can be seen as a process of fully "utilizing" known important states and actively "exploring" unknown important states. This can be described by the multi-armed bandit problem in reinforcement learning. The customized exponential weight algorithm exp3 is used to guide the state selection process, so that each round of testing can select as many important states as possible, such as Figure 2 FIG. 1 is a flow chart of a state selection process according to an embodiment of the present invention, which specifically includes:

[0025] First, an initial weight w is assigned to each state k=1,...,u k (0), the initial weight is defined as:

[0026]

[0027] Each state k is associated with a reward value reword, which is used to express the trade-off between rewarding states and rarely tested states, defined as:

[0028]

[0029] Among them, states_discovered indicates the new states discovered when the state is selected as the state to be tested; paths_discovered indicates the new code paths discovered when the state is selected as the state to be tested; state_trans indicates the number of state transitions when the state is selected as the state to be tested; fuzzs indicates the number of times the state is triggered during the fuzz test; state_selects indicates the number of times the state is selected;

[0030] Then, in the rth round of fuzzy testing, the cumulative loss value Lk(r) of each state k is calculated. The cumulative loss value Lk(r) is calculated as follows:

[0031]

[0032]

[0033] wherein loss k (r) refers to the loss value of the kth state in the rth round; reword k (r) refers to the reward value of the kth state in the rth round; MaxReword(r) refers to the maximum reward value in all states in the rth round; loss k (r) refers to the loss value of the kth state in the rth round; cumulative loss value L k (r) is used to update the weight of the state, thereby affecting the probability of the state being selected;

[0034] Based on the cumulative loss value L k (r), the weight w k (r) of each state k in the rth round is calculated, and the probability distribution p k (r) of each state k being selected in the rth round is calculated according to the weight w k (r); finally, a state is randomly selected according to the probability distribution p k (r) for fuzzy testing, and the loss value loss k (r) is observed and recorded.

[0035] wherein the weight w k (r) and the probability distribution p k (r) are defined as follows:

[0036]

[0037]

[0038] η is the learning rate, used to control the exploration degree of the algorithm.

[0039] The above process comprehensively evaluates the value of the selected target state from aspects such as state coverage, code coverage, and the number of times the state is tested, and introduces a customized exp3 algorithm to guide the state selection process in the fuzzy testing of the stateful network protocol, which can preferentially select important states for testing, thereby improving the efficiency of the fuzzy testing and the vulnerability mining capability.

[0040] Step 3, selecting the test case queue corresponding to the target state obtained in step 2 from the test case set, and then selecting a test case from the test case queue for mutation operation to generate a new test case, and sending the new test case to the network protocol server for fuzzy testing.

[0041] In this step, how to select the test case with high test efficiency and strong vulnerability mining ability is a difficulty of network protocol fuzz testing, and the traditional method excessively depends on the code coverage index when selecting the test case, and ignores other important state information in the test process, such as the number of state transitions, which makes it difficult to select high-quality test cases, and further affects the effectiveness and efficiency of the test.

[0042] The application evaluates each test case by introducing the fitness concept in the genetic algorithm, each test case has a fitness value Fitness, and the fitness value Fitness is used to quantify the performance of the test case in the test process, and is specifically represented as:

[0043] Fitness=α·F scov +β·F ccov +γ·F Strans

[0044] Wherein F Scov is the fitness based on state coverage; F Ccov is the fitness based on code coverage; F strans is the fitness based on the number of state transitions; alpha, beta, Y are dynamic parameters and alpha+beta+gamma=1;

[0045] For a test case c, the fitness value Fitness c , the probability of being selected is:

[0046]

[0047] Wherein, N is the total number of test cases in the test case queue;

[0048] The application comprehensively evaluates the test case from three dimensions of state coverage, code coverage and number of state transitions, can overcome the limitation of the traditional method of excessively relying on a single code coverage index, can more effectively explore the state space of the stateful network protocol, and further improve the efficiency and vulnerability mining ability of the fuzz testing.

[0049] After selecting the test case, the test case is mutated to generate a new test case, and is sent to the network protocol server for fuzz testing, and the mutation strategy includes bit-level mutation, block-level mutation and byte-level mutation, and the mutation strategy is as shown in Table 1:

[0050] Table 1 Mutation strategy of test case mutation module

[0051]

[0052]

[0053] These strategies are combined together to provide more possibilities for the variation of test cases. The mutation process is divided into two stages. The first stage: according to the size of the test case, the partial bit is flipped, which is divided into bit-level flipping and byte-level flipping, and then the bit-level, block-level and byte-level insertion and deletion operations are performed on the test case; The second stage: this stage is a random combination of various mutation strategies. Specifically, each mutation strategy has a certain probability of being selected. Once selected, the strategy will be applied to the mutation process of the test case to achieve diversified mutation effects.

[0054] After the network protocol server completes a round of fuzz testing, the fitness value Fitness of the corresponding test case is modified according to the feedback information of the network protocol server, and the test cases in the test case queue are adjusted. Specifically, remove_rate(x) is used to represent the removal rate of the xth round of test cases, which is defined as:

[0055] remove_rate(x) = remove_rate(0) x e -λ·x

[0056] Where remove_rate(0) represents the initial removal rate; λ is a decay index; According to experience, the initial value of λ is set to 0.9, and then it is dynamically adjusted according to the test results;

[0057] According to remove_rate(x), the test cases with low fitness in the queue are removed each round.

[0058] The above method has the advantages that it allows rapid removal of low-quality test cases in the early stage, and the removal rate gradually decreases over time, thereby preserving the diversity of test cases in the later stage of testing, avoiding the problem of low efficiency of fuzz testing caused by premature convergence.

[0059] Step 4, after the network protocol server completes a round of fuzz testing, collect the feedback information of the network protocol server, including code coverage, state coverage and state transition, and guide the subsequent fuzz testing process according to the feedback information.

[0060] In this step, the feedback information of the network protocol server is collected. If a new state new _ s t a t e , the new state new_state is added to the state array states;

[0061] According to the collected feedback information of the network protocol server, update the reward value reword of each state in the state array states and the fitness value Fitness of the corresponding test case.

[0062] Finally, a new round of fuzz testing is performed.

[0063] It is worth noting that the contents not described in detail in the embodiments of the present application belong to the prior art known to those skilled in the art.

[0064] For example, as Figure 3 The figure shows the schematic diagram of the example implementation scenario of the present application, which includes a smart network tester and multiple servers carrying different network protocols, such as DTLS, FTP, DNS, etc. The smart network tester is mainly used to send a large number of abnormal, unexpected or randomly generated test cases to the network protocol servers mentioned above and collect the corresponding feedback information of the servers, and generate detailed test reports according to the feedback information, providing data support and decision basis for the optimization and improvement of network protocols.

[0065] Compared with other network protocol fuzz testing methods, the method described in the embodiments of the present application considers the importance of state selection to the fuzz testing process, models the state selection process as an "exploration" and "exploitation" problem, and introduces the exp3 algorithm variant in the multi-armed bandit problem to help the fuzzer select a more potential state as the target state of the fuzz testing, thereby solving the problem of traditional network protocol fuzz testing in state selection.

[0066] At the same time, the present application evaluates the state and test case by comprehensively considering the code coverage, state coverage, state transition number and other multiple dimensions, and no longer relies too much on the code coverage, thereby solving the problems of test case redundancy and insufficient testing in the traditional method.

[0067] In addition, those skilled in the art can understand that all or part of the steps of the above-mentioned embodiment methods can be completed by programs instructing relevant hardware, and the corresponding programs can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a disk or an optical disk, etc.

[0068] The above is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited to this. Any changes or replacements easily thought of by those skilled in the art within the technical range disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims. The information disclosed in the background section of this paper is only intended to deepen the understanding of the overall background of the present application, and should not be regarded as acknowledging or implying in any form that the information constitutes prior art known to those skilled in the art.

Claims

1. A fuzz testing method for stateful network protocols, characterized in that: The method comprises: Step 1: Build the initial test case corpus and perform compilation and instrumentation operations; Step 2: Initialize the state array and select the target state for each round of fuzz testing through the state selection algorithm; The process of step 2 is specifically as follows: The state array is represented as States = {S1, ..., S k ,...,S u }, S k represents the kth state, k = 1, ..., u, where u is the total number of states. A customized exponential weight algorithm exp3 is used to guide the state selection process, so that important states are selected in each round of testing. Specifically: First, an initial weight w is assigned to each state k=1,...,u k (0), the initial weight is defined as: Each state k is associated with a reward value, which represents the trade-off between states that are profitable and states that are rarely tested, and is defined as: Among them, states_discovered indicates the new states discovered when the state is selected as the state to be tested; paths_discovered indicates the new code paths discovered when the state is selected as the state to be tested; state_trans indicates the number of state transitions when the state is selected as the state to be tested; fuzzs indicates the number of times the state is triggered during the fuzz test; state_selects indicates the number of times the state is selected; Then, in the rth round of fuzz testing, the cumulative loss value L for each state k is calculated k (r), cumulative loss value L k (r) is calculated as follows: Among them, loss k (r) refers to the loss value of selecting the kth state in the rth round; reward k (r) refers to the reward value of selecting the kth state in the rth round; MaxReward(r) refers to the maximum reward value of all states in the rth round; loss k (𝜏) refers to the loss value of the k-th state selected in the 𝜏-th round; the cumulative loss value L k (r) is used to update the weight of the state, thereby affecting the probability of the state being selected; Based on the cumulative loss value L k (r) Calculate the weight w of each state k in round r k (r), and according to the weight w k (r) Calculate the probability distribution p of each state k being selected in round r k (r); finally, according to the probability distribution p k (r) Randomly select a state for fuzz testing and observe the recorded loss value loss k (r); The weight w k (r) and the probability distribution p k (r) is defined as follows: η is the learning rate, which is used to control the degree of exploration of the algorithm; Step 3: Select a test case queue corresponding to the target state obtained in step 2 from the test case set, then select a test case from the test case queue to perform a mutation operation to generate a new test case, and then send the new test case to the network protocol server for fuzz testing; Step 4: After the network protocol server completes a round of fuzz testing, feedback information of the network protocol server is collected, including code coverage, state coverage, and state transfer, and subsequent fuzz testing processes are guided by the feedback information.

2. The fuzz testing method for stateful network protocols according to claim 1, characterized in that: In step 1, first start the client and the network protocol server and communicate; obtain all data packets corresponding to the ports of the network protocol server and collect and retain these data packets; finally, use the data packet analysis tool WireShark to parse the collected data packets and extract all request data packets as the initial test case queue set TestCaseSet, that is, TestCaseSet = {T1,..., T i ,..., T n}, T i refers to the i-th test case queue in TestCaseSet, 1 < i < n, where n is the total number of test case queues; Test case queue T i Consisting of test cases, i.e., T i = {t i，1 ,... t i，j ,...,, t i，m}, t i，j represents the j-th test case in the i-th test case queue, 1 < j < m, where m is the total number of test cases in T i ; Then, the compilation and instrumentation tool LLVM is used to perform compilation and instrumentation operations on the target protocol server program to generate corresponding target code for collecting feedback information during program runtime, including code coverage and status information.

3. The fuzz testing method for stateful network protocols according to claim 1, characterized in that: In step 3, each test case is evaluated by introducing the fitness concept in the genetic algorithm. Each test case has a fitness value, which is used to quantify the performance of the test case during the test process. It is specifically expressed as: Fitness=α·F Scov +β·F Ccov +γ·F Strans Among them F Scov is the fitness based on state coverage; F Ccov is the fitness based on code coverage; F Strans is the fitness based on the number of state transitions; α, β, γ are dynamic parameters and α+β+γ=1; For a test case c, its fitness value is Fitness c , then the probability of being selected is: Where N is the total number of test cases in the test case queue; After the test cases are selected, they are mutated to generate new test cases and sent to the network protocol server for fuzz testing. The mutation strategies include bit-level mutation, block-level mutation, and byte-level mutation. The mutation process is divided into two stages. In the first phase, bits are flipped based on the size of the test case, either at the bit level or the byte level. Insertion and deletion operations are then performed on the test case at the bit, block, and byte levels. In the second phase, each mutation strategy has a certain probability of being selected. Once selected, the strategy is applied to the mutation process of the test case to achieve diverse mutation effects. After the network protocol server completes a round of fuzz testing, the fitness value of the corresponding test case is modified according to the feedback information of the network protocol server, and the test cases in the test case queue are adjusted. Specifically, remove_rate(x) is used to represent the ratio of test cases removed in the xth round, which is defined as: remove_rate(x)=remove_rate(0)×e -λ·x Where remove_rate(0) represents the initial removal rate; λ is a decay exponent, the initial value of λ is set to 0.9, and then dynamically adjusted according to the test results; In each round, test cases with low fitness in the queue are removed according to remove_rate(x).

4. The fuzz testing method for stateful network protocols according to claim 1, characterized in that: In step 4, if a new state new_state is generated, the new state new_state is added to the state array states; Update the reward value of each state in the state array states and the fitness value Fitness of the corresponding test case based on the feedback information collected from the network protocol server; Finally, a new round of fuzz testing is performed.

Images