Emergency large flow real-time detection method and system capable of reserving flow size information

Abstract

The embodiment of the invention provides a burst large flow real-time detection method and system capable of reserving flow size information, and belongs to the technical field of electric digital signal processing. The method comprises the following steps: acquiring a to-be-detected data stream; performing preliminary filtering on the data stream based on a threshold pair to obtain a potential burst stream in the data stream; detecting the potential burst stream based on a sliding window to determine whether the potential burst stream satisfies a burst feature; and under the condition that the potential burst flow meets the burst feature, recording the potential burst flow based on a key value pair. According to the method and the system, the sudden large flow can be monitored.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital signal processing, and in particular to a method, device, storage medium and processor for processing field information. Background Art

[0002] A burst traffic refers to a large number of packets with the same identifier that appear in the network within a very short period of time (millisecond level). The instantaneous arrival rate of these packets is usually tens or hundreds of times the normal rate, and sometimes even exceeds the network bandwidth. When the arrival rate of burst traffic exceeds the forwarding capacity of the switch, the switch often loses packets frequently due to insufficient buffering. At this time, the packets will fall into a cycle of loss and retransmission, which will further aggravate the congestion of the network and affect the throughput performance of the network. Therefore, in order to effectively resist the impact of burst traffic on the normal operation of the network, the current network urgently needs an efficient and accurate burst traffic detection mechanism.

[0003] However, there are many major challenges in deploying an efficient and performance-guaranteed burst detection module on the switch. The first is the mismatch between massive high-speed network traffic and limited on-chip storage resources. In order for the detection and processing speed of data packets to match the flow rate of the network, the recording and detection of data packets must be performed in on-chip storage (such as SRAM / Cache), but due to the high cost of such storage and process limitations, its capacity specifications are usually extremely small. Considering the actual scale of network traffic, for example, the sampling data of only 1 minute in the public data set CAIDA 2019 already contains 100,000 source address flows, and the limited on-chip storage is basically impossible to complete accurate burst detection of massive network traffic. In addition, there is a one-time pass limitation for the recording and detection of network data packets, that is, as the currently arriving data packet is forwarded to the next-hop switch, the traffic measurement mechanism will not reprocess the data packet when processing the subsequent arriving data packets.

[0004] In response to the above problems, existing related research tends to open up a piece of on-chip storage space specifically for the burst detection module, and divide the traffic appearing in a single measurement cycle into multiple equal-length measurement windows in order, and compare the measurement information under two (or more) adjacent windows to detect bursts. These mechanisms only retain the statistical information of the flow under a short period of time (several sub-flow segments), and fail to save the statistical results of the flow under the complete measurement cycle. In the application scenario of traffic management and control, in order to effectively deal with the burst traffic in the network, in addition to the premise of being able to effectively detect these flows, we also need to use the flow size statistics of the detected flows to reroute these flows, and finally achieve real-time load balancing of the network. Therefore, the detection of burst large flows and the flow size information of the detected burst flows need to be retained at the same time. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a method and system for real-time detection of a burst large flow capable of retaining flow size information. The method and system can realize monitoring of a burst large flow.

[0006] In order to achieve the above object, an embodiment of the present invention provides a method for real-time detection of a burst large flow capable of retaining flow size information, comprising:

[0007] Get the data stream to be detected;

[0008] Preliminarily filtering the data flow based on a threshold value to obtain a potential burst flow in the data flow;

[0009] Detecting the potential burst flow based on a sliding window to determine whether the potential burst flow meets the burst feature;

[0010] When the potential burst flow meets the burst feature, the potential burst flow is recorded based on the key-value pair.

[0011] Optionally, preliminarily filtering the data flow based on a threshold to obtain a potential burst flow in the data flow includes:

[0012] Generate a first counter array and a plurality of first independent hash functions that satisfy uniform distribution;

[0013] Obtaining a flow label for each data packet in the data flow;

[0014] Inputting the flow label into the first independent hash function to obtain a first index in the first counter array;

[0015] Mapping the data packet to the first counter array according to the first index to obtain a corresponding first record value;

[0016] Calculate the minimum value of the first recorded values;

[0017] Determine whether the minimum value of the first recorded value is less than a preset first threshold;

[0018] In the case where it is determined that the minimum value of the first recorded values ​​is less than the first threshold, determining that the data packet does not belong to a potential burst flow;

[0019] When it is determined that the minimum value of the first record is equal to the first threshold, it is determined that the data packet belongs to a potential burst flow.

[0020] Optionally, the potential burst flow includes a plurality of sub-traffic segments, each of which includes at least one data packet;

[0021] Detecting the potential burst flow based on a sliding window to determine whether the potential burst flow meets the burst feature includes:

[0022] generating a plurality of compact data structure segments, wherein each of the compact data structure segments comprises a second counter array and a register array, the second counter array comprising a plurality of second independent hash functions;

[0023] Calculate a second index of the sub-traffic segment to which the current data packet belongs in the compact data structure segment;

[0024] Searching for a compact data structure segment corresponding to the current data packet according to the second index;

[0025] In the compact data structure segment found, inputting the flow label of the current data packet into the corresponding second independent hash function to obtain a third index in the second counter array;

[0026] Mapping the current data packet to the second counter array according to the third index to obtain a corresponding second record value;

[0027] Calculate the minimum value of the second recorded value;

[0028] Determine whether the sum of the minimum value of the second recorded value and the minimum value of the first recorded value is greater than or equal to a preset second threshold;

[0029] When it is determined that the sum of the minimum value of the second recorded value and the minimum value of the first recorded value is greater than or equal to a preset second threshold, a detection operation is performed.

[0030] Optionally, calculating a second index of the sub-traffic segment to which the current data packet belongs in the compact data structure segment includes:

[0031] The second index is calculated according to formula (1):

[0032]

[0033] Among them, r is the second index, t is the index of the sub-traffic segment where the current data packet is located, and h is the index of the compact data structure segment.

[0034] Optionally, when it is determined that the sum of the minimum value of the second recorded value and the minimum value of the first recorded value is greater than or equal to a preset second threshold, performing a detection operation includes:

[0035] Traverse and generate a parameter l within the preset range;

[0036] Taking the current data packet as the center, calculate the first mean of the flow size increments under the most recent l sub-flow segments;

[0037] Taking the current data packet as the starting point, calculating the second mean of the flow size increments under the preset number of sub-flow segments before the current data packet;

[0038] Calculate the ratio of the first mean to the second mean;

[0039] determining a maximum value of the ratio;

[0040] Determining whether the maximum value is greater than a preset third threshold;

[0041] When it is determined that the ratio is greater than the third threshold, it is determined that the potential burst flow meets the burst feature.

[0042] Optionally, calculating the ratio of the first mean to the second mean includes:

[0043] The ratio is calculated according to formula (2):

[0044]

[0045] Among them, α l is the ratio calculated for parameter l, n f,t-j+1 is the flow size increment value of the t-j+1th sub-flow segment, f is the flow label, t is the index of the sub-flow segment where the current data packet is located, h is the index of the compact data structure segment, and n f,t-l-k+1 The value of the flow size increment of the tl-k+1th sub-flow segment.

[0046] Optionally, when the potential burst flow meets the burst feature, recording the potential burst flow based on the key-value pair includes:

[0047] Generate a bucket array and a uniformly distributed third independent hash function, wherein the bucket array includes a plurality of units, each unit includes three fields, namely, a flow label field, a flow size counter field, and a segment index field;

[0048] Inputting the flow label of the data packet into the third independent hash function to obtain a corresponding output value;

[0049] Determine whether the data packet has been recorded in the bucket array according to the output value;

[0050] In the case where it is determined that the data packet has been recorded in the bucket array, the segment index field of the corresponding unit is updated to the information of the sub-flow segment to which the data packet belongs;

[0051] In the case where it is determined that the data packet is not recorded in the bucket array, determining whether there are still vacant units in the bucket array;

[0052] When it is determined that there are still vacant units in the bucket array, the flow label corresponding to the data packet and the information of the sub-flow segment to which it belongs are updated to the flow label field and the segment index field of one of the vacant units respectively, and the corresponding flow size field is summed and updated;

[0053] When it is determined that there are no empty cells in the bucket array, the cell with the smallest value in the segment index field is cleared, and the flow label corresponding to the data packet and the information of the sub-traffic segment to which it belongs are updated to the flow label field and the segment index field of the cleared cell respectively, and the corresponding flow size field is summed and updated.

[0054] Optionally, the corresponding flow size field is summed and updated, including:

[0055] The summation is updated according to formula (3) and formula (4):

[0056]

[0057] Among them, t is the index of the sub-traffic segment where the current data packet is located, is the number of first counters that have a mapping relationship with the current data packet in the first counter array, is an array of registers indexed by j % h, where h is the index of the segment of the compact data structure, The output of the first independent hash function with the flow label of the current packet as input, is the first threshold, is the number of second counters that have a mapping relationship with the current data packet in the second counter array, for the second counter with index j%h, The output of a second independent hash function that takes the current packet's flow label as input.

[0058] On the other hand, the present invention further provides a system for real-time detection of large burst flows capable of retaining flow size information. The system comprises a processor, and the processor is configured to execute any of the above methods.

[0059] Through the above technical scheme, the embodiment of the present invention provides a method and system for real-time detection of large burst flows that can retain flow size information. The method and system adopt a segmented design and use h small-sized compact data structures to record the flow size increment information of each arriving flow under the latest h sub-flow segments, and detect whether the flow meets the burst characteristics by comparing the continuous flow size increment changes of a single flow in multiple time periods in real time. For the detected burst flow, the flow size information of the flow in the multiple time periods maintained will be integrated into the subsequent flow size measurement module and continuously recorded in the remaining time periods thereafter. In addition, the scheme further improves the accuracy of burst large flow detection through the structural design of recording potential burst flows and infrequently occurring flows separately.

[0060] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0062] Figure 1 is a flow chart of a method for real-time detection of a bursty large flow capable of retaining flow size information according to an embodiment of the present invention;

[0063] Figure 2 is a flow chart of a method for determining a potential burst flow according to an embodiment of the present invention;

[0064] Figure 3 is a flow chart of a method for further detecting potential burst flows according to an embodiment of the present invention;

[0065] Figure 4 is a flow chart of a method of detecting operation according to one embodiment of the present invention;

[0066] Figure 5 The figure is a flowchart of a method for recording the potential burst flow based on key-value pairs according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0068] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations. In the embodiments of this application, some existing solutions in the industry such as certain software, components, and models may be mentioned, which should be considered as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0069] like Figure 1 FIG. 1 is a flow chart of a method for real-time detection of large burst flows capable of retaining flow size information according to an embodiment of the present invention. Figure 1 In the method, the method may include the following steps:

[0070] In step S10, a data stream to be detected is obtained;

[0071] In step S11, the data stream is preliminarily filtered based on the threshold to obtain a potential burst flow in the data stream;

[0072] In step S12, the potential burst flow is detected based on the sliding window to determine whether the potential burst flow meets the burst feature;

[0073] In step S13, when the potential burst flow meets the burst feature, the potential burst flow is recorded based on the key-value pair.

[0074] In this Figure 1 In the method shown, step S10 may obtain a data stream to be detected. The data stream may be composed of a plurality of continuous data packets. For each of the data packets, a flow label and an element label may be corresponding.

[0075] Step S11 may be used to preliminarily filter the data stream based on the threshold value to obtain potential burst flows in the data stream. Specifically, in this embodiment, step S11 may further include: Figure 2 In the method shown in Figure 2 In the method for determining the potential burst flow, the method may include the following steps:

[0076] In step S20, a first counter array and a plurality of first independent hash functions satisfying uniform distribution are generated. Specifically, the first counter array may include counters, the specifications of each counter can be bits, and the overflow threshold of the counter is At the beginning of each sub-flow segment, the counter array The values ​​of all counters in can be set to 0. The first independent hash function can represent

[0077] In step S21, a flow label of each data packet in the data flow is obtained;

[0078] In step S22, the flow label is input into a first independent hash function to obtain a first index in a first counter array;

[0079] In step S23, the data packet is mapped to the first counter array according to the first index to obtain a corresponding first record value;

[0080] In step S24, the minimum value of the first record value is calculated. The minimum value can be expressed as

[0081]

[0082] In step S25, it is determined whether the minimum value of the first record value is less than a preset first threshold value;

[0083] In step S26, when it is determined that the minimum value of the first recorded value is less than the first threshold, it is determined that the data packet does not belong to a potential burst flow. In addition, in order to complete the update of the first counter, when it is determined that the minimum value of the first recorded value is less than the first threshold, an increment operation may be performed on the first counter that has a mapping relationship with the minimum value;

[0084] In step S27, when it is determined that the minimum value of the first record is equal to the first threshold, it is determined that the data packet belongs to a potential burst flow.

[0085] After filtering out the potential burst flow in step S11, step S12 can be used to detect the potential burst flow based on a sliding window to determine whether the potential burst flow meets the burst feature, thereby completing further detection of the potential burst flow. Among them, since the potential burst flow is not completely continuous in time series, it includes multiple sub-flow segments, and each sub-flow segment includes at least one data packet. The method for further detection can be in various forms known to those skilled in the art. In one example of the present invention, the method for further detecting the potential burst flow may include: Figure 3 In the steps shown in Figure 3 In the method for further detecting the potential burst flow, the method may include the following steps:

[0086] In step S30, a plurality of compact data structure segments are generated, wherein each compact data structure segment includes a second counter array and a register array, and the second counter data includes a plurality of second independent hash functions. Specifically, each compact data structure segment may include a counter array of counters and one that contains bit register array Second counter array for each compact data structure segment Can include Uniformly distributed independent hash functions Register Array Can be viewed as an array of counters Compressed data structure: When each sub-traffic segment t starts recording detection, the values ​​of all second counters and registers in the compact data structure segment (t%h) may be set to 0.

[0087] In step S31, the second index of the sub-flow segment to which the current data packet belongs in the compact data structure segment is calculated. The specific calculation method of the second index can be various forms known to those skilled in the art. In one example of the present invention, the second index can be calculated using the following formula (1):

[0088]

[0089] Among them, r is the second index, t is the index of the sub-traffic segment where the current data packet is located, and h is the index of the compact data structure segment.

[0090] In step S32, the compact data structure segment corresponding to the current data packet is searched according to the second index;

[0091] In step S33, in the found compact data structure segment, the flow label of the current data packet is input into the corresponding second independent hash function to obtain a third index in the second counter array;

[0092] In step S34, the current data packet is mapped to the second counter array according to the third index to obtain a corresponding second record value;

[0093] In step S35, the minimum value of the second record values ​​is calculated. In order to complete the update of the second counter group, after calculating the minimum value, all second counters whose second record values ​​are equal to the minimum value may be updated by adding 1;

[0094] In step S36, it is determined whether the sum of the minimum value of the second recorded value and the minimum value of the first recorded value is greater than or equal to a preset second threshold;

[0095] In step S37, when it is determined that the sum of the minimum value of the second recorded value and the minimum value of the first recorded value is greater than or equal to a preset second threshold, a detection operation is performed. Specifically, the detection operation may include: Figure 4 In the method shown in Figure 4 In the present invention, the detection operation may include the following steps:

[0096] In step S40, a parameter l is generated by traversing within a preset interval;

[0097] In step S41, taking the current data packet as the center, the first mean of the flow size increments under the most recent l sub-flow segments is calculated;

[0098] In step S42, starting from the current data packet, the second mean of the flow size increment under the preset number of sub-flow segments before the current data packet is calculated. The specific value of the preset number can be in various forms known to those skilled in the art. In this example, the preset number can be preferably h / 2.

[0099] In step S43, the ratio of the first mean and the second mean is calculated. Specifically, in this example, the ratio can also be calculated using the following formula (2):

[0100]

[0101] Among them, α l is the ratio calculated for parameter l, n f,t-j+1 is the flow size increment value of the t-j+1th sub-flow segment, f is the flow label, t is the index of the sub-flow segment where the current data packet is located, h is the index of the compact data structure segment, and n f,t-l-k+1 The value of the flow size increment of the tl-k+1th sub-flow segment.

[0102] In step S44, the maximum value of the ratio is determined;

[0103] In step S45, it is determined whether the maximum value is greater than a preset third threshold;

[0104] In step S46, when it is determined that the ratio is greater than the third threshold, it is determined that the potential burst flow meets the burst feature.

[0105] Step S13 can be used to record the potential burst flow based on the key-value pair when the potential burst flow meets the burst feature, thereby realizing the detection and recording of the burst large flow information. The specific method for recording the potential burst flow based on the key-value pair can be various forms known to those skilled in the art. In one example of the present invention, the method for recording the potential burst flow based on the key-value pair can include: Figure 5 In the steps shown in Figure 5 In the method for recording the potential burst flow based on the key-value pair, the method may include the following steps:

[0106] In step S50, a bucket array and a uniformly distributed third independent hash function are generated, wherein the bucket array includes a plurality of units, each unit includes three fields, namely, a flow label field, a flow size counter field, and a segment index field. Specifically, in this example, the bucket array may include buckets. Each bucket Depend on units, and each unit It contains three fields: the flow label field is used to record the flow detected to have burst characteristics, the flow size counter field is used to record the flow size of the detected flow during the entire measurement period, and the segment index field is used to record the sub-flow segment index of the most recent detection of burst characteristics of the flow.

[0107] In step S51, the flow label of the data packet is input into a third independent hash function to obtain a corresponding output value;

[0108] In step S52, it is determined whether the data packet has been recorded in the bucket array according to the output value;

[0109] In step S53, when it is determined that the data packet has been recorded in the bucket array, the segment index field of the corresponding unit is updated to the information of the sub-flow segment to which the data packet belongs;

[0110] In step S54, when it is determined that the data packet is not recorded in the bucket array, it is determined whether there are still vacant units in the bucket array;

[0111] In step S55, when it is determined that there are still vacant units in the bucket array, the flow label corresponding to the data packet and the information of the sub-flow segment to which it belongs are updated to the flow label field and the segment index field of one of the vacant units respectively, and the corresponding flow size field is summed and updated;

[0112] In step S56, when it is determined that there is no vacant unit in the bucket array, the unit with the smallest value in the segment index field is cleared, the flow label corresponding to the data packet and the information of the sub-flow segment to which it belongs are updated to the flow label field and segment index field of the cleared unit respectively, and the corresponding flow size field is summed and updated. Specifically, the summation update can be performed using the following formulas (3) and (4):

[0113]

[0114] Among them, t is the index of the sub-traffic segment where the current data packet is located, is the number of first counters that have a mapping relationship with the current data packet in the first counter array, is an array of registers indexed by j % h, where h is the index into the compact data structure segment, The output of the first independent hash function with the flow label of the current packet as input, is the first threshold, is the number of second counters that have a mapping relationship with the current data packet in the second counter array, for the second counter with index j%h, The output of a second independent hash function that takes the current packet's flow label as input.

[0115] On the other hand, the present invention further provides a system for real-time detection of large burst flows capable of retaining flow size information. The system comprises a processor, and the processor is configured to execute any of the above methods.

[0116] Through the above technical scheme, the embodiment of the present invention provides a method and system for real-time detection of large burst flows that can retain flow size information. The method and system adopt a segmented design and use h small-sized compact data structures to record the flow size increment information of each arriving flow under the latest h sub-flow segments, and detect whether the flow meets the burst characteristics by comparing the continuous flow size increment changes of a single flow in multiple time periods in real time. For the detected burst flow, the flow size information of the flow in the multiple time periods maintained will be integrated into the subsequent flow size measurement module and continuously recorded in the remaining time periods thereafter. In addition, the scheme further improves the accuracy of burst large flow detection through the structural design of recording potential burst flows and infrequently occurring flows separately.

[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0118] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0121] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0122] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0123] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0125] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for real-time detection of bursty large flows capable of retaining flow size information, characterized in that: The method comprises: Get the data stream to be detected; Preliminarily filtering the data flow based on a threshold value to obtain a potential burst flow in the data flow; Detecting the potential burst flow based on a sliding window to determine whether the potential burst flow meets the burst feature; When the potential burst flow meets the burst feature, the potential burst flow is recorded based on the key-value pair.

2. The method according to claim 1, characterized in that Preliminarily filtering the data flow based on the threshold to obtain a potential burst flow in the data flow includes: Generate a first counter array and a plurality of first independent hash functions that satisfy uniform distribution; Obtaining a flow label for each data packet in the data flow; Inputting the flow label into the first independent hash function to obtain a first index in the first counter array; Mapping the data packet to the first counter array according to the first index to obtain a corresponding first record value; Calculate the minimum value of the first recorded values; Determine whether the minimum value of the first recorded value is less than a preset first threshold; In the case where it is determined that the minimum value of the first recorded values ​​is less than the first threshold, determining that the data packet does not belong to a potential burst flow; When it is determined that the minimum value of the first record is equal to the first threshold, it is determined that the data packet belongs to a potential burst flow.

3. The method according to claim 2, characterized in that The potential burst flow includes a plurality of sub-traffic segments, each of which includes at least one data packet; Detecting the potential burst flow based on a sliding window to determine whether the potential burst flow meets the burst feature includes: generating a plurality of compact data structure segments, wherein each of the compact data structure segments comprises a second counter array and a register array, the second counter array comprising a plurality of second independent hash functions; Calculate a second index of the sub-traffic segment to which the current data packet belongs in the compact data structure segment; Searching for a compact data structure segment corresponding to the current data packet according to the second index; In the compact data structure segment found, inputting the flow label of the current data packet into the corresponding second independent hash function to obtain a third index in the second counter array; Mapping the current data packet to the second counter array according to the third index to obtain a corresponding second record value; Calculate the minimum value of the second recorded value; Determine whether the sum of the minimum value of the second recorded value and the minimum value of the first recorded value is greater than or equal to a preset second threshold; When it is determined that the sum of the minimum value of the second recorded value and the minimum value of the first recorded value is greater than or equal to a preset second threshold, a detection operation is performed.

4. The method according to claim 3, characterized in that Calculating a second index of the sub-flow segment to which the current data packet belongs in the compact data structure segment includes: The second index is calculated according to formula (1): Among them, r is the second index, t is the index of the sub-traffic segment where the current data packet is located, and h is the index of the compact data structure segment.

5. The method according to claim 3, characterized in that: When it is determined that the sum of the minimum value of the second recorded value and the minimum value of the first recorded value is greater than or equal to a preset second threshold, performing a detection operation includes: Traverse and generate a parameter l within the preset range; Taking the current data packet as the center, calculate the first mean of the flow size increments under the most recent l sub-flow segments; Taking the current data packet as the starting point, calculating the second mean of the flow size increments under the preset number of sub-flow segments before the current data packet; Calculate the ratio of the first mean to the second mean; determining a maximum value of the ratio; Determining whether the maximum value is greater than a preset third threshold; When it is determined that the ratio is greater than the third threshold, it is determined that the potential burst flow meets the burst feature.

6. The method according to claim 5, characterized in that Calculating the ratio of the first mean to the second mean includes: The ratio is calculated according to formula (2): Among them, α l is the ratio calculated for parameter l, n f,t-j+1 is the flow size increment value of the t-j+1th sub-flow segment, f is the flow label, t is the index of the sub-flow segment where the current data packet is located, h is the index of the compact data structure segment, and n f,t-l-k+1 The value of the flow size increment of the tl-k+1th sub-flow segment.

7. The method according to claim 1, characterized in that When the potential burst flow meets the burst feature, recording the potential burst flow based on the key-value pair includes: Generate a bucket array and a uniformly distributed third independent hash function, wherein the bucket array includes a plurality of units, each unit includes three fields, namely, a flow label field, a flow size counter field, and a segment index field; Inputting the flow label of the data packet into the third independent hash function to obtain a corresponding output value; Determine whether the data packet has been recorded in the bucket array according to the output value; In the case where it is determined that the data packet has been recorded in the bucket array, the segment index field of the corresponding unit is updated to the information of the sub-flow segment to which the data packet belongs; In the case where it is determined that the data packet is not recorded in the bucket array, determining whether there are still vacant units in the bucket array; When it is determined that there are still vacant units in the bucket array, the flow label corresponding to the data packet and the information of the sub-flow segment to which it belongs are updated to the flow label field and the segment index field of one of the vacant units respectively, and the corresponding flow size field is summed and updated; When it is determined that there are no empty cells in the bucket array, the cell with the smallest value in the segment index field is cleared, and the flow label corresponding to the data packet and the information of the sub-traffic segment to which it belongs are updated to the flow label field and the segment index field of the cleared cell respectively, and the corresponding flow size field is summed and updated.

8. The method according to claim 7, characterized in that And the corresponding flow size fields are summed and updated, including: The summation is updated according to formula (3) and formula (4): Among them, t is the index of the sub-traffic segment where the current data packet is located, is the number of first counters that have a mapping relationship with the current data packet in the first counter array, is an array of registers indexed by j % h, where h is the index into the compact data structure segment, The output of the first independent hash function with the flow label of the current packet as input, is the first threshold, is the number of second counters that have a mapping relationship with the current data packet in the second counter array, for the second counter with index j%h, The output of a second independent hash function that takes the current packet's flow label as input.

9. A real-time detection system for bursty large flows capable of retaining flow size information, characterized in that: The system comprises a processor configured to execute the method according to any one of claims 1 to 8.

Images