A sketch-based DDoS attack detection method in a high-speed network scenario

Abstract

A DDoD attack detection method based on sketch in a high-speed network scenario, comprising: 1) a host extracts the information of a data packet to generate a flow ID; 2) a difference between a current timestamp and a timestamp of the last record on the corresponding flow in a timestamp array is calculated, if the difference is greater than a time window, a circular queue, the timestamp array and a counter are updated, and if the difference is less than the time window, only the value of the counter needs to be updated; 3) every time period T1, data in the circular queue is traversed, and whether the sum of the data packet counts of each flow in the corresponding time window exceeds a local threshold value is judged; 4) the rate of each flow is calculated according to the uploaded information, and a DDoS attack is detected; and 5) every time period T2, the timestamp array is traversed, the current system timestamp is subtracted from the traversed timestamp, if the difference is greater than the time period T2, the data flow is stored in the sketch and the circular queue, and is reported to the controller, and then the corresponding counter in the sketch and the circular queue are set to 0.

Description

Technical Field

[0001] This invention relates to the fields of network measurement and computer network security, and in particular to a sketch-based DDoS attack detection method in high-speed network scenarios. Background Technology

[0002] In recent years, Distributed Denial-of-Service (DDoS) attacks have become one of the most serious threats facing the Internet. According to Kaspersky's 2020 DDoS Attack Report, the number of DDoS attacks increased by 87.5% compared to 2019. Over the past few decades, much effort has been made to detect DDoS attacks. Numerous intrusion detection and prevention systems have been proposed. However, these systems struggle to strike a good balance between scalability and accuracy. Other methods, such as entropy-based methods, have achieved high accuracy in anomaly traffic detection, but they suffer from the drawback of high computational cost.

[0003] To reduce the overhead of large amounts of data processing, a series of sketch-based DDoS attack detection methods have been proposed. Sketch is a compact data structure that can store large amounts of network traffic data characteristics within a limited space. Due to its low storage overhead, sketch-based anomaly detection methods have been applied to detect DDoS attacks in high-speed networks. However, these sketch-based anomaly detection methods mainly focus on the total number of packets in the flow, ignoring certain DDoS rate change patterns. For example, a Yo-Yo DDoS attack can change the rate of request traffic, causing cloud servers to frequently change resource configurations, resulting in significant economic losses. Simply detecting the number of packets in the traffic is insufficient to accurately detect such DDoS attacks; monitoring changes in the traffic rate is also necessary.

[0004] Therefore, existing technologies have not achieved significant accuracy in predicting DDoS attacks. In order to solve these problems, there is an urgent need to provide an effective attack detection method in the current complex network environment. Summary of the Invention

[0005] To overcome the shortcomings of existing detection methods, this invention proposes a sketch-based DDoS attack detection method for high-speed network scenarios.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] A method for detecting DDoS attacks based on sketch in high-speed network scenarios, the method comprising the following steps:

[0008] 1) When a data packet arrives at the host, the host extracts the five-tuple information of the data packet to generate a stream ID string str. The five-tuple information includes the source IP, source port, destination IP, destination port, and protocol type.

[0009] 2) Based on the extracted stream ID string, obtain the sketch counter, timestamp array, and circular queue index corresponding to the stream. Calculate the difference between the system timestamp of the current data packet arrival and the previous timestamp of the data stream recorded in a one-dimensional array. If the difference is greater than the preset time window, store the counter value of the corresponding stream in the sketch structure into the circular queue corresponding to the stream, update the timestamp recorded in the array to the system timestamp of the current data packet arrival, and then increment the stream counter value by 1. If the difference is less than the preset time window, simply increment the counter corresponding to the stream by 1.

[0010] 3) Each host, based on a preset local threshold, traverses the data in the circular queue every time period T1 to determine whether the sum of the packet counts for each flow in the corresponding time window exceeds the local threshold. If it does, it compiles the total number of packets for the corresponding flow, the number of packets in each time window, and other information into a table and uploads it to the controller. In addition, each host checks whether a flow has been disconnected every large time period T2 and resets the corresponding counters and circular queues for disconnected flows to save resources.

[0011] 4) The controller aggregates information uploaded by all hosts, calculates the rate of each stream based on the uploaded information, and observes the rate changes of each stream over time to detect the presence of a DDoS attack. The rate calculation formula is as follows:

[0012] diff=(q[tail]-q[head]+qSize)mod(qSize)

[0013]

[0014] Where, diff is the difference between the head index and the tail index, q is the circular queue corresponding to a certain flow, tail is the tail index of the circular queue, head is the head index of the circular queue, qSize is the length of the circular queue, w is the size of the time window, x represents the previous x time windows, and f(x) represents the rate of the flow from the latest time window to the previous x time windows, in units of packets per second (PPS).

[0015] Furthermore, the sketch structure described in step 2) is a Count-Min Sketch, consisting of a d-row, w-column two-dimensional array, with each row corresponding to a hash function. i(0≤i≤d-1), the input parameter of the hash function is the stream ID string of each stream, and the output is the index of each stream in the i-th row of the array. That is, in each row, the index of the counter corresponding to each stream is determined by the hash, i.e., index = hash. i (str). The hash function is implemented using the Bob hash function, and the mapping range of each hash function is [0, w-1].

[0016] Furthermore, in step 2), there are a total of w circular queues, each consisting of a one-dimensional array of length qSize. Each element stored in the circular queue is the minimum value among the d counters of the corresponding data stream. The mapping relationship between the data stream and its corresponding circular queue is determined by an index calculated by a hash function. Similarly, the mapping range of this hash function is [0, w-1].

[0017] The timing of storing the minimum value among the d counters of the corresponding data stream into the corresponding circular queue, as described in step 2), depends on a one-dimensional array `timestamps` of length `w`. This array stores the timestamp of the last update of the circular queue for each data stream. The corresponding index of different data streams in the data is obtained by hashing the stream ID, and the mapping range of this hash function is [0, w-1]. When a data packet belonging to a certain data stream arrives at the host, the index of the data stream in the `timestamps` array is first obtained by hashing, which is `i`. Then, `timestamps[i]` is retrieved, and the difference between the current system timestamp and `timestamps[i]` is calculated. This difference is then compared with the time window. If the difference is greater than or equal to the time window, `timestamps[i]` is updated to the current system timestamp, and the number of data packets of that data stream stored in the `sketch` structure is added to the circular queue. Subsequently, the number of data packets of that data stream in the `sketch` structure is incremented by 1. If the difference is less than the time window, only the number of data packets of that data stream in the `sketch` structure is incremented by 1.

[0018] The table uploaded by the host in step 3) contains the following information: the five-tuple information of the data stream, the total number of traffic packets of the data stream, and the total number of data packets recorded by the data stream in qSize time windows.

[0019] The controller described in step 4) is used to summarize the information sent by all hosts to the table. The controller calculates the rate changes of each data stream under different time windows to make a judgment. If the rate change is abnormal, the data stream is determined to be DDoS attack traffic, and corresponding filtering rules are generated and distributed to all hosts to prevent DDoS attacks.

[0020] Furthermore, this method incorporates a data stream disconnection check mechanism. Every relatively large time period T2, the one-dimensional array timestamps is traversed, and the current system timestamp is subtracted from the traversed timestamp. If the difference is greater than the time period T2, then the current data stream has not received any data packets from the host for a long time, and it can be considered that the current data stream has been disconnected. In this case, the data stored in the sketch and circular queue of the data stream is reported to the controller. Then, the corresponding counters in the sketch and the circular queue are set to 0 to promptly clear useless information and facilitate the service for new connections.

[0021] The technical concept of this invention is to add a corresponding circular queue to the Count-Min sketch algorithm, use the timestamp values ​​recorded by a one-dimensional array to collect the total number of data packets in different time windows, then calculate the flow rate in each time window, and use the rate change and total number of data packets to determine whether a DDoS attack exists.

[0022] The main advantages of this invention are: by adding a circular queue to the Count-Min sketch algorithm, traffic feature collection and attack detection were successfully performed, and better detection results were achieved compared with previous methods. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method of the present invention.

[0024] Figure 2 This is a structural diagram of the sketch structure and circular queue of the present invention, where X in the circular queue represents the counter value recorded in the previous time window. Detailed Implementation

[0025] The invention will be further described below with reference to the accompanying drawings.

[0026] Reference Figure 1 A sketch-based DDoS attack detection method for high-speed network scenarios includes the following steps:

[0027] 1) When a data packet arrives at the host, the host extracts the five-tuple information of the data packet to generate a stream ID string str. The five-tuple information includes the source IP, source port, destination IP, destination port, and protocol type.

[0028] 2) Based on the extracted stream ID string, obtain the sketch counter, timestamp array, and circular queue index corresponding to the stream. Calculate the difference between the system timestamp of the current data packet arrival and the previous timestamp of the data stream recorded in a one-dimensional array. If the difference is greater than the preset time window, store the counter value of the corresponding stream in the sketch structure into the circular queue corresponding to the stream, update the timestamp recorded in the array to the system timestamp of the current data packet arrival, and then increment the stream counter value by 1. If the difference is less than the preset time window, simply increment the counter corresponding to the stream by 1.

[0029] 3) Each host, based on a preset local threshold, traverses the data in the circular queue every time period T1 to determine whether the sum of the packet counts for each flow in the corresponding time window exceeds the local threshold. If it does, it compiles the total number of packets for the corresponding flow, the number of packets in each time window, and other information into a table and uploads it to the controller. In addition, each host checks whether a flow has been disconnected every large time period T2 and resets the corresponding counters and circular queues for disconnected flows to save resources.

[0030] 4) The controller aggregates information uploaded by all hosts, calculates the rate of each stream based on the uploaded information, and observes the rate changes of each stream over time to detect the presence of a DDoS attack. The rate calculation formula is as follows:

[0031] diff=(q[tail]-q[head]+qSize)mod(qSize)

[0032]

[0033] Where, diff is the difference between the head index and the tail index, q is the circular queue corresponding to a certain flow, tail is the tail index of the circular queue, head is the head index of the circular queue, qSize is the length of the circular queue, w is the size of the time window, x represents the x time windows to the previous time window, and f(x) represents the rate of the flow from the latest time window to the x time windows to the previous time window, in units of packets per second (PPS). Further, the sketch structure described in step 2) is a Count-Min Sketch, composed of a d-row, w-column two-dimensional array structure, with each row corresponding to a hash function. i (0≤i≤d-1), the input parameter of the hash function is the stream ID string of each stream, and the output is the index of each stream in the i-th row of the array. That is, in each row, the index of the counter corresponding to each stream is determined by the hash, i.e., index = hash. i (str). The hash function is implemented using the Bob hash function, and the mapping range of each hash function is [0, w-1].

[0034] Furthermore, in step 2), there are a total of w circular queues, each consisting of a one-dimensional array of length qSize. Each element stored in the circular queue is the minimum value among the d counters of the corresponding data stream. The mapping relationship between the data stream and its corresponding circular queue is determined by an index calculated by a hash function. Similarly, the mapping range of this hash function is [0, w-1].

[0035] The timing of storing the minimum value among the d counters of the corresponding data stream into the corresponding circular queue, as described in step 2), depends on a one-dimensional array `timestamps` of length `w`. This array stores the timestamp of the last update of the circular queue for each data stream. The corresponding index of different data streams in the data is obtained by hashing the stream ID, and the mapping range of this hash function is [0, w-1]. When a data packet belonging to a certain data stream arrives at the host, the index of the data stream in the `timestamps` array is first obtained by hashing, which is `i`. Then, `timestamps[i]` is retrieved, and the difference between the current system timestamp and `timestamps[i]` is calculated. This difference is then compared with the time window. If the difference is greater than or equal to the time window, `timestamps[i]` is updated to the current system timestamp, and the number of data packets of that data stream stored in the `sketch` structure is added to the circular queue. Subsequently, the number of data packets of that data stream in the `sketch` structure is incremented by 1. If the difference is less than the time window, only the number of data packets of that data stream in the `sketch` structure is incremented by 1.

[0036] The table uploaded by the host in step 3) contains the following information: the five-tuple information of the data stream, the total number of traffic packets of the data stream, and the total number of data packets recorded by the data stream in qSize time windows.

[0037] The controller described in step 4) is used to summarize the information sent by all hosts to the table. The controller calculates the rate changes of each data stream under different time windows to make a judgment. If the rate change is abnormal, the data stream is determined to be DDoS attack traffic, and corresponding filtering rules are generated and distributed to all hosts to prevent DDoS attacks.

[0038] Furthermore, this method incorporates a data stream disconnection check mechanism. Every relatively large time period T2, the one-dimensional array timestamps is traversed, and the current system timestamp is subtracted from the traversed timestamp. If the difference is greater than the time period T2, then the current data stream has not received any data packets from the host for a long time, and it can be considered that the current data stream has been disconnected. In this case, the data stored in the sketch and circular queue of the data stream is reported to the controller. Then, the corresponding counters in the sketch and the circular queue are set to 0 to promptly clear useless information and facilitate the service for new connections.

[0039] This invention, based on the Count-Min sketch algorithm, adds a corresponding circular queue. It uses a one-dimensional array to record timestamp values ​​to collect the total number of data packets in different time windows, then calculates the flow rate within each time window. By analyzing the rate changes and the total number of data packets, it determines whether a DDoS attack exists. This method belongs to the fields of network security and network measurement, and successfully detects DDoS attacks.

[0040] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A DDoS attack detection method based on sketch in high-speed network scenarios, characterized in that, Includes the following steps: 1) When a data packet arrives at the host, the host extracts the five-tuple information from the data packet to generate a stream ID string. The five-tuple information includes the source IP, source port, destination IP, destination port, and protocol type; 2) Based on the extracted stream ID string, obtain the sketch counter, timestamp array, and circular queue index corresponding to the stream. Calculate the difference between the system timestamp of the current data packet arrival and the previous timestamp of the data stream recorded in a one-dimensional array. If the difference is greater than or equal to the preset time window, store the counter value of the corresponding stream in the sketch structure into the circular queue corresponding to the stream, update the timestamp recorded in the array to the system timestamp of the current data packet arrival, and then increment the stream counter value by 1. If the difference is less than the preset time window, simply increment the counter corresponding to the stream by 1. 3) Each host, based on a preset local threshold, iterates through the data in the circular queue every time period T1 to determine whether the sum of the packet counts for each flow in the corresponding time window exceeds the local threshold. If it does, it compiles a table containing information such as the total number of packets for the corresponding flow and the number of packets in each time window and uploads it to the controller. In addition, each host checks whether a flow has been disconnected every time period T2 and resets the corresponding counter and circular queue for the disconnected flow to save resources. Where T2 > T1. 4) The controller aggregates information uploaded by all hosts, calculates the rate of each stream based on the uploaded information, and observes the rate change characteristics of each stream over time to detect whether a DDoS attack is occurring; the rate calculation formula is as follows: in, It is the difference between the head index and the tail index. It is a circular queue corresponding to a certain stream. It is the tail index of the circular queue. It is the head index of the circular queue. It is the length of the circular queue. It is the size of the time window. Indicates moving forward A time window, Indicates the time window from the latest to the previous one. The rate of the stream within a time window, in packets per second (PPS).

2. The DDoS attack detection method based on sketch in a high-speed network scenario as described in claim 1, characterized in that: Step 2) describes a sketch structure as a Count-Min Sketch, consisting of a d-row, w-column two-dimensional array, with each row corresponding to a hash function. , The hash function takes the stream ID string as input and outputs the index of each stream in the i-th row of the array. In other words, the index of the counter corresponding to each stream in each row is determined by the hash function, i.e., index = ... The hash function is implemented using the Bob hash function, and the mapping range of each hash function is [0, w-1].

3. The DDoS attack detection method based on sketch in a high-speed network scenario as described in claim 1, characterized in that: Step 2) There are a total of w circular queues, each consisting of a one-dimensional array of length qSize. The element stored in the circular queue each time is the minimum value among the d counters of the corresponding data stream. The mapping relationship between the data stream and its corresponding circular queue is determined by an index calculated by a hash function, and the mapping range of the hash function is [0, w-1].

4. The DDoS attack detection method based on sketch in a high-speed network scenario as described in claim 3, characterized in that: Step 2) The timing of storing the minimum value among the d counters of the corresponding data stream into the corresponding circular queue depends on a one-dimensional array timestamps of length w. This array stores the timestamp of the last update of the circular queue for each data stream. The corresponding index of different data streams in the data is obtained by hashing the stream ID. The mapping range of this hash function is [0, w-1]. When a data packet belonging to a certain data stream arrives at the host, the index i of the data stream in the timestamps array is first obtained by hashing. Then, timestamps[i] is retrieved, and the difference between the current system timestamp and timestamps[i] is calculated. This difference is then compared with the time window. If the difference is greater than or equal to the time window, timestamps[i] is updated to the current system timestamp, and the number of data packets of that data stream stored in the sketch structure is added to the circular queue. Subsequently, the number of data packets of that data stream in the sketch structure is incremented by 1. If the difference is less than the time window, only the number of data packets of that data stream in the sketch structure is incremented by 1.

5. The DDoS attack detection method based on sketch in a high-speed network scenario as described in claim 1, characterized in that: Step 3) The table uploaded by the host contains the following information: the five-tuple information of the data stream, the total number of traffic packets of the data stream, and the total number of data packets recorded by the data stream in qSize time windows.

6. The DDoS attack detection method based on sketch in a high-speed network scenario as described in claim 1, characterized in that: Step 4) The controller summarizes the information in the table sent by all hosts; the controller calculates the rate change of each data stream under different time windows to make a judgment. If the rate change is abnormal, the data stream is determined to be DDoS attack traffic and corresponding filtering rules are generated and distributed to all hosts to prevent DDoS attacks.

7. The DDoS attack detection method based on sketch in a high-speed network scenario as described in claim 1, characterized in that: There is a data stream disconnection check mechanism. Every time period T2, the one-dimensional array timestamps is traversed. The current system timestamp is subtracted from the traversed timestamp. If the difference is greater than the time period T2, the current data stream has not received any data packets from the host for a long time. It can be considered that the current data stream has been disconnected. The data of the data stream stored in the sketch and circular queue is reported to the controller. Then, the corresponding counters in the sketch and the circular queue are set to 0 to clear useless information in time, so as to facilitate the service of new connections.

Images