A value burst flow detection method based on P4 and sketch

Abstract

The present application relates to a kind of value burst flow detection method based on P4 and Sketch.Break through the detection scheme of traditional based on successive burst, propose value burst flow definition, observe the burst of traffic on the flow level, process the input data packet under the limited memory overhead by the data structure design based on Sketch, detect burst data packet according to the time interval between data packet, and then detect value burst flow. When data packet arrives, each data packet is mapped to a bucket set, but at most to one bucket is operated, under appropriate conditions, can vote according to the voting mechanism based on burst; when each measurement period ends, traverse all buckets, obtain the value burst flow detected by query operation and feedback to control plane. The present application reduces bandwidth overhead and user burden in burst detection process, while only limited memory occupation and low time complexity are generated.

Description

Technical Field

[0001] This invention relates to the fields of network measurement and programmable networks, and specifically to a value burst flow detection method based on P4 and Sketch. Background Technology

[0002] In various production networks, bursts are a common traffic pattern characterized by high flow rates for short periods. Compared to flows with stable rates, while burst flows may have lower rates for most of the time, their flow rates suddenly increase during a burst, with instantaneous rates reaching tens to hundreds of times the average rate. This causes rapid accumulation of switch queues, leading to longer queuing delays and even packet loss. Because bursts can last as short as microseconds, traditional end-to-end measurement schemes for networks struggle to accurately detect them, and their fixed-function data planes offer only limited information and operations. Programmable data planes and the P4 language provide customizable operations and fine-grained network status observation for detecting and handling traffic bursts, offering flexibility and potential for burst detection, analysis, and mitigation.

[0003] Detecting bursts is the first step in mitigating the impact of burst flows on network performance. Research on traffic pattern analysis detects bursts by extracting data flow characteristics, while queue-based micro-burst detection determines the occurrence of micro-bursts by observing port queue lengths in switches. However, existing burst detection techniques employ a successive burst approach, assuming each burst is independent and detecting and reporting it separately. This successive burst approach introduces significant bandwidth overhead between control planes and a high user burden for network administrators, failing to meet the needs of network administrators for network metrics that reflect flows with significant and continuous burstiness. Therefore, we propose the concept and formalized definition of Remarkable BurstFlow and present a Remarkable BurstFlow detection scheme based on P4 and Sketch, reducing bandwidth overhead and user burden in the burst detection process while incurring only limited memory usage and low time complexity. Summary of the Invention

[0004] The purpose of this invention is to propose the concept of value burst flow, and therefore provide a value burst flow detection method based on P4 and Sketch, which reduces the bandwidth requirements between control planes and alleviates the user burden of network administrators in burst detection tasks.

[0005] To achieve the above objectives, the technical solution of this invention is: a value burst flow detection method based on P4 and Sketch, which analyzes the burstiness of traffic at the flow level and proposes a value burst flow definition, detects burst data packets, and then detects value burst flows by the proportion of burst data packets. Furthermore, it deploys a Sketch-based data structure and a burst-based voting mechanism on a programmable data plane. During insertion, the data structure is updated through update operations and the burst-based voting mechanism. During querying, the data structure is directly traversed to obtain the value burst flows within the corresponding measurement period, thus enabling insertion and query operations to be completed in constant time.

[0006] This invention observes the burstiness of traffic at the flow level, detects bursty data packets based on the time interval between data packets, and then detects valuable bursty flows. Through a Sketch-based data structure design, this scheme incurs only limited memory overhead and time complexity, which is beneficial for further hardware deployment. Simultaneously, a burstiness-based voting mechanism improves insertion and query efficiency and the accuracy of valuable bursty flow detection. The method includes the following steps:

[0007] (1) Specify the relevant parameters for the definition of value burst flow as the criteria for judgment and detection;

[0008] (2) Initialize the bucket structure and set the number of bits for each field in the bucket based on information including the stream key length;

[0009] (3) Initialize the data structure and configure higher-dimensional metrics than buckets based on the information provided by the administrator, including memory requirements, such as the number of rows, the number of columns, and the hash function.

[0010] (4) When each data packet arrives and is inserted, the hash function is called in each row to calculate the hash value of the data packet stream key and obtain the set of buckets to which it is mapped;

[0011] (5) Traverse the mapped bucket set and perform update operations. Each bucket records only the information of one flow. Each data packet operates on at most one bucket. If the bucket set does not contain the corresponding flow and is full, a vote is performed based on a burst-based voting mechanism.

[0012] (6) After each measurement cycle ends, all buckets are traversed and a query operation is performed to feed back all detected value burst flow information to the control plane and clear the bucket data accordingly for information storage in the next measurement cycle.

[0013] In one embodiment of the present invention, in step (1), a value burst flow is a burst-based traffic pattern. The time interval between data packets is observed and compared with the average data packet interval within the corresponding measurement period to detect burst data packets. Based on the proportion of the number of burst data packets in the total number of data packets in the corresponding flow, a judgment is made as to whether the corresponding flow is a value burst flow.

[0014] In one embodiment of the present invention, the process of detecting burst data packets involves comparing the time interval between two adjacent data packets with the average data packet interval within the corresponding measurement period. The administrator predefines a parameter β, and when the interval between two adjacent data packets is less than or equal to 1 / β of the average data packet interval, the two data packets are considered to be burst data packets.

[0015] In one embodiment of the present invention, within a measurement period, the administrator predefines a threshold α. When the proportion of the number of burst data packets in the total number of data packets in the corresponding flow exceeds α, the corresponding flow is considered to be a valuable burst flow within the corresponding measurement period.

[0016] In one embodiment of the present invention, each bucket records at most one burst information of a flow at any given time, including the flow key, average inter-packet interval, arrival timestamp of the last packet of the corresponding flow, number of burst packets of the corresponding flow, and total number of packets; since the data structure with the Sketch feature only requires a small and static memory space and generates only limited time complexity in the insertion operation, in steps (2) and (3), a data structure for detecting valuable burst flows is constructed.

[0017] In one embodiment of the present invention, when each data packet arrives, step (4) will obtain a bucket set through hash operation and trigger the update operation described in step (5).

[0018] In one embodiment of the present invention, when a bucket recording the corresponding flow already exists in the bucket set, the data packet updates the corresponding bucket; otherwise, it searches the bucket set to see if there is an empty bucket. When at least one empty bucket exists, the corresponding flow information is inserted into an empty bucket; otherwise, a burst-based voting mechanism is triggered.

[0019] In one embodiment of the present invention, the bucket with the lowest current proportion of burst data packets is selected from the bucket set, and the proportion of burst data packets of the corresponding flow is compared with a predefined threshold. When it is lower than the threshold, the original flow is expelled and new flow information is inserted. Otherwise, the number of burst data packets and the total number of data packets recorded in the corresponding bucket are both decremented by one to reduce the proportion of burst data packets of the corresponding flow.

[0020] In one embodiment of the present invention, when a measurement cycle ends, step (6) performs a query operation on each bucket, feeds back the detected value burst flow to the control plane, and performs the clearing operation described in step (6) according to the information in the bucket. For buckets that have recorded value burst flows, only the burst data packet and total data packet count fields are cleared, and the average data packet interval field is updated. For buckets that have recorded non-value burst flows, all fields are cleared for the next measurement cycle.

[0021] Compared to existing technologies, this invention offers the following advantages: It utilizes the Sketch data structure and the programmability of the data plane to achieve burst flow detection, reducing bandwidth overhead and user burden during burst detection. The solution is simple, flexible in implementation, and well-adaptable to various burst flow detection needs. Attached Figure Description

[0022] Figure 1 This is the overall architecture of the present invention.

[0023] Figure 2 This section describes the update algorithm and the burst-based voting mechanism proposed in this invention. Detailed Implementation

[0024] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] like Figure 1 , 2 As shown, this invention relates to the definition of a traffic pattern, namely the definition of a value burst flow, and discloses a value burst flow detection scheme based on P4 and Sketch. This scheme observes the interval between data packets, detects value burst flows by obtaining the proportion of burst data packets in the flow, and supplements it with an update algorithm, a query algorithm, and a voting mechanism based on the proportion of burst data packets. While reducing bandwidth overhead and user burden, it only generates limited memory usage and low time complexity. Specifically:

[0026] S1. Specify valuable burst flows by configuring relevant parameters. Administrators use the specified threshold parameter β to guide the determination of burst packets, and use the specified burst packet percentage threshold parameter α to determine which flows belong to valuable burst flows.

[0027] S2. Construct a data structure to record burst information of the stream. In the measurement scheme, the internal structure of a single bucket is as follows: Figure 1 As shown, the fields in the bucket are used to record the stream key, average interval between packets, arrival timestamp of the last packet in the stream, number of burst packets in the stream, and total number of packets. Generally speaking, a larger number of rows and columns means a lower probability of hash collisions; the specific design should be configured according to memory requirements.

[0028] S3, Update operation. For example... Figure 2 As shown, each data packet arrival triggers an update operation (see Algorithm 1), which uses hash calculation to obtain the mapped bucket set, where each bucket tracks the burst information of at most one flow. If the flow to which the data packet belongs is already recorded in the bucket set, the burst packet percentage of that flow is updated; if the flow has not yet been recorded and there are empty buckets in the bucket set, the flow information is initialized for an empty bucket.

[0029]

[0030] Furthermore, when the bucket set is full and the flow is absent, a voting mechanism based on the proportion of bursty packets is used (see Algorithm 2). Figure 2 As shown, the process first identifies the flow with the lowest percentage of burst packets in the bucket set, indicating that the flow has the lowest burstiness. If the percentage of burst packets in this flow is below a predefined threshold, the new flow will replace the information in the bucket by clearing the original content and inserting the new flow information; otherwise, the fields recording the number of burst packets and the total number of packets in the bucket will be decremented by one to reduce the percentage of burst packets, thus achieving burstiness-based voting.

[0031]

[0032] S4, Query operation. For example... Figure 1 As shown, querying the non-empty buckets at the end of each measurement cycle can obtain information about the burstiness of the tracked flow.

[0033] Furthermore, at the end of each measurement cycle, the control plane traverses each bucket and compares the proportion of burst packets of the flow recorded in the bucket with the threshold α predefined by the administrator. If the proportion of burst packets exceeds the threshold α, the flow is considered a valuable burst flow, and all valuable burst flow information will be fed back to the control plane for further processing and analysis.

[0034] Furthermore, after the query operation, the information in the bucket should be cleared accordingly for information recording in the next measurement cycle. To provide more accurate and continuous measurement of value bursts, for buckets that have already recorded value burst information, only the burst data packet and total data packet count fields should be cleared, and the average data packet interval field should be updated; for buckets that record non-value bursts, all fields should be cleared for the next measurement cycle.

[0035] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A value burst flow detection method based on P4 and Sketch, characterized in that, This method analyzes the burstiness of traffic at the flow level and proposes a definition of valuable burst flows. It detects burst data packets and then identifies valuable burst flows based on the proportion of burst data packets. A Sketch-based data structure and a burst-based voting mechanism are deployed on a programmable data plane. During insertion, the data structure is updated through update operations and the burst-based voting mechanism. During querying, the data structure is directly traversed to obtain the valuable burst flows within the corresponding measurement period, enabling insertion and query operations to be completed in constant time. The method includes the following steps: (1) Specify the relevant parameters for the definition of value burst flow as the criteria for judgment and detection; (2) Initialize the bucket structure and set the number of bits for each field in the bucket based on information including the stream key length; (3) Initialize the data structure and configure higher-dimensional metrics than buckets based on the information provided by the administrator, including memory requirements, such as the number of rows, the number of columns, and the hash function; (4) When each data packet arrives and is inserted, the hash function is called for each row to calculate the hash value of the data packet stream key and obtain the set of buckets to which it is mapped; (5) Traverse the mapped bucket set and perform update operations. Each bucket records only the information of one flow. Each data packet operates on at most one bucket. If the bucket set does not contain the corresponding flow and is full, then vote according to the burst-based voting mechanism. (6) After each measurement cycle ends, all buckets are traversed and a query operation is performed to feed back all detected value burst flow information to the control plane and clear the bucket data accordingly for information storage in the next measurement cycle. In step (1), a value burst flow is a burst-based traffic pattern. The time interval between data packets is observed and compared with the average data packet interval within the corresponding measurement period to detect burst data packets. The judgment of whether the corresponding flow is a value burst flow is made based on the proportion of the number of burst data packets in the total number of data packets in the corresponding flow. At any given time, each bucket records the burst information of at most one flow, including the flow key, the average data packet interval, the arrival timestamp of the last data packet of the corresponding flow, the number of burst data packets of the corresponding flow, and the total number of data packets. Since the data structure with the Sketch feature only requires a small and static memory space and only generates limited time complexity in the insertion operation, in steps (2) and (3), a data structure for detecting value burst flows is constructed.

2. The value burst flow detection method based on P4 and Sketch according to claim 1, characterized in that, The process of detecting burst packets involves comparing the time interval between two adjacent packets with the average packet interval within the corresponding measurement period. The administrator predefines a parameter β, and when the interval between two adjacent packets is less than or equal to 1 / β of the average packet interval, the two packets are considered to be burst packets.

3. The value burst flow detection method based on P4 and Sketch according to claim 1, characterized in that, Within a measurement period, the administrator predefines a threshold α. When the proportion of burst packets in the total number of packets in the corresponding flow exceeds α, the corresponding flow is considered a valuable burst flow within the corresponding measurement period.

4. The value burst flow detection method based on P4 and Sketch according to claim 1, characterized in that, When each data packet arrives, step (4) will obtain a bucket set through hash operation and trigger the update operation described in step (5).

5. The value burst flow detection method based on P4 and Sketch according to claim 4, characterized in that, If a bucket recording the corresponding flow already exists in the bucket set, the data packet updates the corresponding bucket. Otherwise, it searches the bucket set for an empty bucket. If at least one empty bucket exists, the corresponding flow information is inserted into an empty bucket. Otherwise, a burst-based voting mechanism is triggered.

6. The value burst flow detection method based on P4 and Sketch according to claim 5, characterized in that, Select the bucket with the lowest current burst packet percentage from the bucket set, compare the burst packet percentage of the corresponding flow with a predefined threshold, and if it is lower than the threshold, expel the original flow and insert new flow information; otherwise, decrement the burst packet count and total packet count fields recorded in the corresponding bucket to reduce the burst packet percentage of the corresponding flow.

7. The value burst flow detection method based on P4 and Sketch according to claim 1, characterized in that, When a measurement cycle ends, each bucket is queried in step (6), the detected value burst flows are fed back to the control plane, and the clearing operation described in step (6) is performed according to the information in the bucket. For buckets that have recorded value burst flows, only the burst data packets and total data packet count fields are cleared, and the average data packet interval field is updated. For buckets that have recorded non-value burst flows, all fields are cleared for the next measurement cycle.

Images