Computer-implemented method for identifying potential denial-of-service attack vulnerabilities in a network protocol program

Abstract

A computer-implemented method for identifying potential denial-of-service attack vulnerabilities in a network protocol program (10) installed on a computing device with limited computing resources, the network protocol program (10) including a plurality of network protocol states (12) that can be traversed by corresponding input (14) executed by the network protocol program (10) and prompting the network protocol program (10) to undergo a network protocol transition (16) from one network protocol state (12) to another, each network protocol transition (16) having a consumption and / or allocation (18) of computing resources.

Description

[0001] DESCRIPTION

[0002] Computer-implemented method for identifying potential denial-of-service attack vulnerabilities in a network protocol program

[0003] TECHNICAL FIELD

[0004] The invention relates to a computer-implemented method for identifying potential denial-of-service attack vulnerabilities in a network protocol program. The invention further relates to a data processing device, a computer program, and a computer-readable data carrier.

[0005] BACKGROUND

[0006] In the present Internet digital world, millions of software programs or components that implement various protocols (e.g., FTP, HTTP, SSH) have been deployed to provide various services, we call these software programs or components protocol programs. These protocol programs play a critical role and compose the functional basics of the Internet. However, due to the importance and accessibility of protocol programs, they become the attackers’ most common targets. The Denial-of- Service (DoS) attack, which has targeted many protocol programs, is one of the most common cyberattacks. Nowadays, DoS attack is becoming more and more frequent and significantly threaten the security of the entire Internet world, causing billions of economic losses.

[0007] The aim of a DoS attack is to impede service availability. According to the type of vulnerabilities exploited by attackers, DoS attacks can be categorized as two general types: 1 ) crash attack: this kind of attack leverages vulnerabilities that can cause the services crashed (e.g., segment fault) and unavailable; 2) resource exhaustion attack: this kind of attack leverages resource exhaustion vulnerabilities to flood the services to exhaust their resources, causing the services unavailable for legitimate users. Resource exhaustion vulnerability is a specific type of fault that causes the consumption and / or allocation of some resource in an undefined or unnecessary way (e.g., memory allocation without limits or throttling). To defend against DoS attacks, several prevention (e.g., filtering) and mitigation (e.g., detection) measures have been proposed. However, as these measures may be created and tested based on known DoS vulnerabilities or flaws, it is unintelligible whether these measures may be still effective facing unknown or new DoS vulnerabilities. Therefore, revealing the DoS vulnerabilities in advance is a promising defense approach which can 1 ) help develop new defense measures or patches in advance for new DoS vulnerabilities, and 2) help test the ability of deployed defense measures for defending new DoS vulnerabilities.

[0008] To reveal DoS vulnerabilities, except manual audit which may be tedious and timeconsuming, automated testing is a more attractive and promising methodology. Fuzzing is a popular automated testing technique much favored by academia and industry in recent five years. For protocol programs, corresponding fuzzing techniques called protocol fuzzing (e.g., AFLNet) are also proposed. Protocol fuzzing integrates the formats and states of protocol into a fuzzing process and tries to fully test each state of protocol. Here are steps to go through in protocol fuzzing:

[0009] 1. Prepare the compiled target protocol program and initial seeds (for protocol, they are message packages), and put the seeds into seed pool.

[0010] 2. Select a state of protocol, then select a seed related to this state from seed pool.

[0011] 3. Mutation on the selected seed, feed the mutated seed to the target program to execute it.

[0012] 4. Observe the status of program execution, if the protocol program goes into new states or behaves valuable status (e.g., crash), put the mutated seed into seed pool. Then go to step 2.

[0013] However, existing protocol fuzzing may be mainly designed for discovering zero- day vulnerabilities that will crash the program. It would be desirable to have a protocol fuzzing technique for discovering resource depletion vulnerabilities by improving the above steps. SUMMARY OF THE INVENTION

[0014] The object of the invention is to provide an improved method for identifying potential denial-of-service attack vulnerabilities in a network protocol program.

[0015] The object of the invention is achieved by the subject-matter of the independent claims. Advantageous embodiments of the invention are subject-matter of the dependent claims.

[0016] In one aspect, the invention provides a computer-implemented method for identifying potential denial-of-service attack vulnerabilities in a network protocol program installed on a computing device with limited computing resources, the network protocol program including a plurality of network protocol states that can be traversed by corresponding input executed by the network protocol program and prompting the network protocol program to undergo a network protocol transition from one network protocol state to another, each network protocol transition having a consumption and / or allocation of computing resources, the method comprising: a) Preparing an input seed from a seed pool, the input seed including a plurality of seed inputs and executing the plurality of seed inputs by the network protocol program; b) Extracting the network protocol states which the network protocol program adopts, and the network protocol transitions which the network protocol program undergoes, during execution of the plurality of seed inputs, and monitoring the highest computing resources consumption and / or allocation for each undergone network protocol transition; c) Representing the network protocol states, the undergone network protocol transitions, and the corresponding highest computing resources consumption and / or allocation in form of a resource-integrated state machine model; and d) Identifying the potential denial-of-service attack vulnerabilities based on the resource-integrated state machine model.

[0017] An advantage of the method may be that DoS attack vulnerabilities that eventually exhaust computing resources can be revealed in advance. Thus, in contrast to known methods that reveal DoS attack vulnerabilities crashing the program, the present method broadens the scope to DoS attacks that may exhaust computing resources in order to impede service availability. Thus, the present method may contribute to improve the cybersecurity of network protocol programs.

[0018] Preferably, the method further comprises: e) Selecting a network protocol state from the resource-integrated state machine model for fuzzing; f) Fuzzing the selected network protocol state by executing a plurality of seed inputs of a fuzzing seed of the seed pool by the network protocol program while carrying out the step b); and g) Updating the resource-integrated state machine model and / or the seed pool based on results of the step f).

[0019] An advantage of the method may be that fuzzing techniques may be employed. Fuzzing comprises modem techniques which, for example, mutate a seed pool in order to generate new seed input for testing. Fuzzing may improve the discovery of vulnerabilities in network protocol programs.

[0020] Preferably, the method further comprises: h) Repeating and / or iterating the steps e) to g) based on the previously updated resource-integrated state machine and / or seed pool.

[0021] An advantage of the method may be that it can be used in an iterative manner by repeating and / or iterating certain steps. Thus, the resource-integrated state machine model can eventually converge or reveal new vulnerabilities with every subsequent repetition and / or iteration. The number of repetitions and / or iterations and / or the calculation time may be set in advance. Thus, users can predefine a certain level of security in advance.

[0022] Preferably, step e) further comprises: e1) Selecting the network protocol state based on an updating time it took previously for extracting new network protocol states and / or new network protocol transitions and / or monitoring new highest computing resources consumptions or allocations.

[0023] The selection of network protocol states for fuzzing can be based on different priorities. For example, if a network protocol state did not contribute in revealing new or unknown network protocol states and / or network protocol transitions and / or computing resources consumptions or allocations previously, the method may focus on other network protocol states instead. This increases the efficiency of the method.

[0024] Preferably, step e) further comprises: e2) Selecting the network protocol state based on the maximum highest computing resources consumption and / or allocation among the extracted network protocol transitions.

[0025] The method may also focus on network protocol states that contribute to a high computing resources consumptions or allocations. These network protocol states may have a high probability to reveal new or unknown network protocol transitions with high computing resources consumptions or allocations. This increases the efficiency of the method.

[0026] Preferably, step f) and / or step g) further comprise: f1) Preparing the fuzzing seed by mutating the seed pool with respect to the selected network protocol state and preferably with respect to computing resources consumption and / or allocation; and / or g1 ) Updating the resource-integrated state machine and / or the seed pool, when a new network protocol state and / or a new network protocol transition is extracted and / or a new highest computing resources consumption and / or allocation is monitored.

[0027] Preferably, the fuzzing seed is prepared by mutation of the seed pool. However, also other preparation methods, for example, generation-based fuzzing methods may be employed. Preferably, the resource-integrated state machine and / or the seed pool are updated, whenever a new or unknown network protocol state and / or a new or unknown network protocol transition is extracted and / or a new or unknown highest computing resources consumption and / or allocation is monitored. This may have the advantage that the resource-integrated state machine model may be improved with every repetition and / or iteration of the method.

[0028] Preferably, the method further comprises: i) Adapting the network protocol program for avoiding the identified potential denial-of-service attack vulnerabilities.

[0029] The method may have the advantage that the network protocol program can be adapted in advance for avoiding identified potential DoS attacks. This increases the cybersecurity of the network protocol program under test.

[0030] In another aspect, the invention provides a data processing device comprising means for carrying out the method of any of the preceding embodiments.

[0031] In another aspect, the invention provides a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of the preceding embodiments.

[0032] In another aspect, the invention provides a computer-readable data carrier having stored thereon the computer program.

[0033] Embodiments of the invention preferably have the following advantages and effects:

[0034] Preferred embodiments of the invention target to discover resource depletion vulnerabilities that are not stressed by existing protocol fuzzing. We preferably integrate resource consumption information into state selection, mutation, feedback, and state machine components of protocol fuzzing to make it feasible and efficient for discovering resource depletion vulnerabilities. Preferred embodiments can automatically test and expose new resource depletion vulnerabilities for reducing the risk of protocol programs being attacked by DoS. Furthermore, embodiments of the invention preferably portray the resource consumption status of protocol programs under each protocol state and can be used as a profiler to help developers find performance problems in programs.

[0035] Existing state-of-the-art protocol fuzzing techniques, like AFLNet, integrate states information of protocol into fuzzing process; use code coverage, protocol states as feedback and program’s crash status as oracle; try to fully test each protocol state and discover new vulnerabilities that can cause protocol programs to crash.

[0036] However, they may not be feasible for discovering resource depletion vulnerabilities. An advantage of preferred embodiments over existing techniques may be that we can automatically test and efficiently discover resource depletion vulnerabilities in protocol programs.

[0037] Preferred embodiments may be applied in several scenarios:

[0038] For protocol program vendors, the developing teams can integrate the present idea to their Continuous Integration (Cl) process as a guarantee for the quality and security of the developed protocol programs. The usage scenario is, whenever a protocol program is updated, it can go through the testing using the present idea before releasing it to the public. In this sense, preferred embodiments of the invention can also help the protocol developing teams to prevent security vulnerabilities involving in the final protocol program version.

[0039] For the software testing teams, they can use embodiments of the invention as a tool for automated testing the protocol programs. This can help the testing teams to detect a large scale of protocol programs without the support of vendors.

[0040] For software security companies, embodiments of the invention preferably can be integrated into their existing dynamic analysis tools as a new feature (by far no existing dynamic analysis tool can be used to efficiently detect resource depletion DoS vulnerabilities in protocol programs), preferably allowing them to find bugs / vulnerabilities. To summarize, companies can use preferred embodiments of the invention to enhance the security and robustness of their protocol programs, while software security companies can use the present idea to broaden their business scope.

[0041] BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Embodiments of the invention are now explained in more detail with reference to the accompanying drawings of which

[0043] Fig. 1 shows a first embodiment of a computer-implemented method for identifying potential denial-of-service attack vulnerabilities in a network protocol program installed on a computing device with limited computing resources;

[0044] Fig. 2 shows a second embodiment of the computer-implemented method; and

[0045] Fig. 3 shows an embodiment of a data processing device adapted to carry out the method according to Fig. 1 and / or 2.

[0046] DETAILED DESCRIPTION OF THE DRAWINGS

[0047] Fig. 1 shows a first embodiment of a computer-implemented method for identifying potential denial-of-service attack vulnerabilities in a network protocol program 10 installed on a computing device with limited computing resources.

[0048] The network protocol program 10 includes a plurality of network protocol states 12 that can be traversed by corresponding input 14 executed on the network protocol program 10. The corresponding input 14 prompts the network protocol program 10 to undergo a network protocol transition 16 from one network protocol state 12 to another. Each network protocol transition 16 has a consumption and / or allocation 18 of computing resources.

[0049] In a step S11 , the method includes: - Preparing an input seed 20 from a seed pool 22, the input seed 20 including a plurality of seed inputs 14 and executing the plurality of seed inputs 14 by the network protocol program 10.

[0050] In a step S12, the method includes:

[0051] - Extracting the network protocol states 12 which the network protocol program 10 adopts, and the network protocol transitions 16 which the network protocol program undergoes, during execution of the plurality of seed inputs 14, and monitoring the highest computing resources consumption and / or allocation 18 for each undergone network protocol transition 16.

[0052] In a step S13, the method includes:

[0053] - Representing the network protocol states 12, the undergone network protocol transitions 16, and the corresponding highest computing resources consumption and / or allocation 18 in form of a resource-integrated state machine model 24.

[0054] In a step S14, the method includes:

[0055] - Identifying the potential denial-of-service attack vulnerabilities based on the resource-integrated state machine model 24.

[0056] Fig. 2 shows a second embodiment of the method. The second embodiment includes the steps S11 to S14, which are not shown again in Fig. 2.

[0057] In a step S21 , the method includes:

[0058] - Selecting a network protocol state 12 from the resource-integrated state machine model 24 for fuzzing.

[0059] In a step S22, the method includes:

[0060] - Fuzzing the selected network protocol state 12 by executing a plurality of seed inputs 14 of a fuzzing seed 26 of the seed pool 22 by the network protocol program 10 while carrying out the step S12.

[0061] In other words, the selected network protocol state 12 is fuzzed, while the network protocol states 12 which the network protocol program 10 adopts, and the network protocol transitions 16 which the network protocol program undergoes, during execution of the plurality of seed inputs 14 of the fuzzing seed 26 are extracted, and the highest computing resources consumption and / or allocation 18 for each undergone network protocol transition 16 is monitored.

[0062] In a step S23, the method includes:

[0063] - Updating the resource-integrated state machine model 24 and / or the seed pool 22 based on results of the step S22.

[0064] In a step S24, the method includes:

[0065] - Repeating and / or iterating the steps S21 to S23 based on the previously updated resource-integrated state machine 24 and / or seed pool 22.

[0066] Fig. 3 shows an embodiment of a data processing device 28 that is adapted to carry out the method as described with reference to Fig. 1 and 2. The data processing device 28 may be the computing device with limited computing resources as precedingly mentioned.

[0067] The data processing device 28 includes an initial module 30, a loop module 32, and an analyzing module 46.

[0068] The initial module 30 includes the network protocol program 10, an initial seed 34 as input seed 20 including the plurality of seed inputs 14, a state extractor function 36 for extracting the network protocol states 12 and the network protocol transitions 16, and a format parser 37 to parse seed inputs 14 and / or message responses from the network protocol program 10.

[0069] The initial module 30 performs the steps S11 to S13 by using the initial seed 34 as input seed 20. Thus, in step S13 performed by the initial module 30, the network protocol states 12 and the network protocol transitions 16 are extracted and the highest computing resources consumption and / or allocation 18 for each network protocol transition 16 is monitored. The loop module 32 receives the extracted network protocol states 12 and network protocol transitions 16 as well as the monitored highest computing resources consumption and / or allocation 18 for each network protocol transition 16 from the initial module 30 and represents them in form of the resource-integrated state machine model 24.

[0070] The loop module 32 may further receive the initial seed 34 from the initial module 30 and include the initial seed 34 in the seed pool 22. Alternatively, or additionally, the seed pool 22 may be generated and / or updated based on the initial seed 34 and results of the step S12 performed by the initial module 30.

[0071] The loop module 32 is configured for repeating and / or iterating the steps S21 to S23 in a plurality of loops. For this, the loop module 32 includes a resource- adaptive component 38 and an executor function 40.

[0072] At the beginning of a loop, the seed pool 22 and the resource-integrated state machine model 24 are send to the resource-adaptive component 38.

[0073] The resource-adaptive component 38 includes a selector function 42 and a mutator function 44.

[0074] The selector function 42 performs the step S21 by selecting a network protocol state 12 from the resource-integrated state machine model 24. The selector function 42 may select a network protocol state 12 based on the maximum highest computing resources consumption and / or allocation 18 among the extracted network protocol transitions 16. Alternatively, or additionally, the selector function 42 may select a network protocol state 12 based on an updating time it took in a previous loop for extracting new or unknown network protocol states 12 and / or new or unknown network protocol transitions 16 and / or monitoring new or unknown highest computing resources consumptions or allocation 18.

[0075] The mutator function 44 is configured for preparing the fuzzing seed 26 by mutating the seed pool 22 with respect to the selected network protocol state 12 and preferably with respect to computing resources consumption and / or allocation 18.

[0076] Next in the loop, the executor function 40 performs the step S22 by fuzzing the selected network protocol state 12 by executing the plurality of seed inputs 14 of the fuzzing seed 26 of the seed pool 22 by the network protocol program 10 while carrying out the step S12. That means, the network protocol states 12 which the network protocol program 10 adopts, and the network protocol transitions 16 which the network protocol program undergoes, during execution of the plurality of seed inputs 14 of the fuzzing seed 26 are extracted, and the highest computing resources consumption and / or allocation 18 for each undergone network protocol transition 16 is monitored.

[0077] Next in the loop, the results of the step S22 are then used for updating, in step S23, the resource-integrated state machine model 24 and / or the seed pool 22.

[0078] The updating of the resource-integrated state machine 24 and / or the seed pool 22 may be performed, whenever a new or unknown network protocol state 12 and / or a new or unknown network protocol transition 16 is extracted and / or a new or unknown highest computing resources consumption and / or allocation 18 is monitored.

[0079] At the end of the loop, the step S24 may be performed and a new loop may be started. The number of repetitions and / or iterations and / or a calculation time may be set in advance. The more repetitions and / or iterations are performed and / or the longer the calculation time is set, the more likely it is that the resource-integrated state machine model 24 reveals potential DoS attack vulnerabilities.

[0080] Once all desired repetitions and / or iterations are done and / or the calculation time is over, step S14 is performed. For this, the analyzing module 46 is configured for identifying the potential denial-of-service attack vulnerabilities based on the final resource-integrated state machine model 24. The analyzing module 46 may identify the seed inputs 14 that may be used for a resource exhaustion denial-of-service attack. Furthermore, the analyzing module 46 may simulate the corresponding denial-of-service attack in a simulated environment to verify, whether there exist resource depletion vulnerabilities that can be used successfully for a denial-of-service attack.

[0081] The invention also provides a computer program (not shown) comprising instructions which, when the program is executed by a computer, cause the computer to carry out the described computer-implemented method. The invention further provides a computer-readable data carrier (not shown) having stored thereon the computer program.

[0082] Preferred embodiments of the invention may be summarized as follows:

[0083] The overview of proposed system is shown in Fig. 3. The system follows below workflow:

[0084] Step 1 : Prepare materials needed for boosting fuzzing. The materials includes target protocol program needed for testing, initial seeds to boost fuzz, state extractor code to extract protocol state and format parser code to parse the send request messages.

[0085] Step 2: Construct resource integrated state machine. We execute the seed feed to the target program, extract the protocol states it triggers and monitor the resources it consumes under each state transition. The protocol states and resource consumption are used to construct the resource integrated state machine. In the state machine, the node represents the protocol state, the edge represents the state transition and is assigned with an attribute that indicates the most resource consumption under this state transition.

[0086] Step 3: Select a state to fuzz from state machine. To select a state to fuzz, we consider three factors: 1 ) resource consumption, the max resource consumed. 2) active times, the update times for finding new resource consumption. 3) select times, the selected times during fuzzing. For each state transition, we calculate the probability to choose it to fuzz according to the three factors. After selecting the state, get the related seed from seed pool.

[0087] Step 4: Mutate on the selected seed. After getting the selected seed, mutate on it to produce a new test input. Some mutate operators need to choose a region of other seeds, we choose the region according to its resource score.

[0088] Step 5: Execute the mutated seed and update state machine. Feed the mutated seed to the target program to execute, extract the protocol states it triggers and monitor the resources it consumes under each state transition. If a new state transition and / or resource consumption is found, update state machine. Then if fuzzing is not supposed to stop, go to step 3, otherwise, go to step 6.

[0089] Step 6: Analyze the resource-integrated state machine. After fuzzing, the resource-integrated state machine describes the resource consumption status under each protocol state of the target program. We can use the state machine to produce the inputs that may be used to do the resource exhaustion DoS attacks, and simulate the DoS attack in a simulated environment to verify, whether there exist resource depletion vulnerabilities that can be used to conduct successful DoS attacks easily.

[0090] REFERENCE SIGNS

[0091] 10 network protocol program

[0092] 12 network protocol state

[0093] 14 (seed) input

[0094] 16 network protocol transition

[0095] 18 computing resources consumption and / or allocation

[0096] 20 input seed

[0097] 22 seed pool

[0098] 24 resource-integrated state machine model

[0099] 26 fuzzing seed

[0100] 28 data processing device

[0101] 30 initial module

[0102] 32 loop module

[0103] 34 initial seed

[0104] 36 state extractor function

[0105] 37 format parser

[0106] 38 resource-adaptive component

[0107] 40 executor function

[0108] 42 selector function

[0109] 44 mutator function

[0110] 46 analyzing module

Claims

CLAIMS1. A computer-implemented method for identifying potential denial-of-service attack vulnerabilities in a network protocol program (10) installed on a computing device with limited computing resources, the network protocol program (10) including a plurality of network protocol states (12) that can be traversed by corresponding input (14) executed by the network protocol program (10) and prompting the network protocol program (10) to undergo a network protocol transition (16) from one network protocol state (12) to another, each network protocol transition (16) having a consumption and / or allocation (18) of computing resources, the method comprising: a) Preparing an input seed (20) from a seed pool (22), the input seed (20) including a plurality of seed inputs (14) and executing the plurality of seed inputs (14) by the network protocol program (10); b) Extracting the network protocol states (12) which the network protocol program (10) adopts, and the network protocol transitions (16) which the network protocol program (10) undergoes, during execution of the plurality of seed inputs (14), and monitoring the highest computing resources consumption and / or allocation (18) for each undergone network protocol transition (16); c) Representing the network protocol states (12), the undergone network protocol transitions (16), and the corresponding highest computing resources consumption and / or allocation (18) in form of a resource-integrated state machine model (24); and d) Identifying the potential denial-of-service attack vulnerabilities based on the resource-integrated state machine model (24).

2. The method according to claim 1 , further comprising: e) Selecting a network protocol state (12) from the resource-integrated state machine model (24) for fuzzing; f) Fuzzing the selected network protocol state (12) by executing a plurality of seed inputs (14) of a fuzzing seed 26) of the seed pool (22) by the network protocol program (10) while carrying out the step b); and g) Updating the resource-integrated state machine model (24) and / or the seed pool (22) based on results of the step f).

3. The method according to claim 2, further comprising: h) Repeating and / or iterating the steps e) to g) based on the previously updated resource-integrated state machine (24) and / or seed pool (22).

4. The method according to claim 3, characterized in that step e) further comprises: e1 ) Selecting the network protocol state (12) based on an updating time it took previously for extracting new network protocol states (12) and / or new network protocol transitions (16) and / or monitoring new highest computing resources consumptions or allocation (18).

5. The method according to claim 2 to 4, characterized in that step e) further comprises: e2) Selecting the network protocol state (12) based on the maximum highest computing resources consumption and / or allocation (18) among the extracted network protocol transitions (16).

6. The method according to any of the claims 2 to 5, characterized in that step f) and / or step g) further comprise: f1 ) Preparing the fuzzing seed (26) by mutating the seed pool (22) with respect to the selected network protocol state (12) and preferably with respect to computing resources consumption and / or allocation (18); and / or g1 ) Updating the resource-integrated state machine (24) and / or the seed pool (22), when a new network protocol state (12) and / or a new network protocol transition (16) is extracted and / or a new highest computing resources consumption and / or allocation (18) is monitored.

7. The method according to any of the preceding claims, further comprising: i) Adapting the network protocol program (10) for avoiding the identified potential denial-of-service attack vulnerabilities.

8. A data processing device (28) comprising means for carrying out the method of any of the preceding claims.

9. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of the claims 1 to 7.

10. A computer-readable data carrier having stored thereon the computer program of claim 9.

Images