Low-overhead burst data stream detection algorithm

Abstract

The invention specifically discloses a low-overhead burst data stream detection algorithm, which filters and identifies an input data stream by using a constructed small stream filter and a burst stream detection table so as to output a real burst stream, and continuously filters a low-frequency stream by setting a time period so as to identify a potential burst stream. The detection precision of the subsequent burst flow is improved; meanwhile, a burst flow detection table is used for tracking potential burst flows, and whether the potential burst flows are real burst flows or not is recognized based on the frequency and the sudden increasing period of each potential burst flow in two adjacent time periods; moreover, a replacement strategy of the burst period and the data stream frequency probability is adopted to eliminate the data stream with lower burst, so that potential burst streams with higher possibility can be stored, and the memory consumption is further reduced. Therefore, the burst data stream detection algorithm provided by the invention has the characteristics of high accuracy, low overhead and high throughput.

Description

Technical Field

[0001] The present application belongs to the field of computer network security technology, and in particular to a low-overhead burst data flow detection algorithm. Background Art

[0002] In early burst flow identification schemes, bursts were defined only as sudden increases in flow frequency while sudden decreases were ignored. However, in some scenarios, it is also important to pay attention to sudden decreases in flow frequency. Currently, BurstSketch is the first Sketch algorithm that focuses on detecting burst flows in high-speed data streams and has achieved good performance. However, the counters used in this scheme need to allocate a large amount of memory in order to record the ID and frequency of each flow, resulting in excessive storage overhead. During storage, some suspicious burst flows are kicked out due to hash conflicts, reducing the accuracy of identification. Therefore, existing solutions for real-time detection of burst flows cannot achieve high accuracy, low memory overhead, and high throughput at the same time. Summary of the invention

[0003] This application is made in view of the above technical problems. The purpose is to provide a burst data flow detection algorithm with high accuracy and low overhead, the algorithm comprising the following steps: S100: Construct a small flow filter and a burst flow detection table, wherein the small flow filter includes counters, the burst detection table is composed of A hash bucket array consisting of hash buckets; S200, based on the small flow filter, filter the low frequency flow in the input data flow and send it to the burst flow detection table, specifically: in the set time period Inside, through An independent hash function maps each input data stream to the corresponding small stream filter In the counters, according to the corresponding Whether the value of a counter reaches a preset filtering threshold value is used to determine whether the data flow is a potential burst flow, thereby filtering the low-frequency flow in the input data flow, wherein, ; S300: judging the data flow based on the burst flow detection table, and then outputting the real burst flow.

[0004] In some embodiments, step 200 specifically includes: setting a filtering threshold of the low-flow filter and time period , in the current time period Inside, through An independent hash function maps each input data stream to the corresponding small stream filter counter and will correspond to The value of each counter is increased by one, if the corresponding The values ​​of the counters have reached the filtering threshold , the data flow is determined to be a potential burst flow and sent to the burst flow detection table, and the data flow information in the small flow filter is cleared; if the corresponding The values ​​of counters do not all reach the filtering threshold , then keep the data stream in the small stream filter until the current time period After completion, reset the values ​​of all counters of the low-flow filter.

[0005] In some implementations, the step S300 specifically includes the following steps: S301, set each hash bucket in the burst detection table to have cells, each of which stores four fields of data flow information ,in, Indicates the identifier of the data stream. Indicates the frequency of the data stream in the previous time period, Indicates the frequency of the data stream in the current time period, Indicates the time period during which the data flow has recently experienced a sudden increase, i.e., the burst period; S302, mapping the data stream input into the burst detection table into a cell of the hash bucket through a hash function, and storing four field information of the data stream in the corresponding cell; S303: Setting the burst threshold , if the cell The value reaches the burst threshold , then calculate the and Ratio ; S304, setting the sudden increase preset value and the maximum allowed burst width ,when When the data flow suddenly increases, it is determined that the data flow has suddenly increased, and the data flow sudden increase period is confirmed. With the current time period Does the difference in value not exceed the maximum allowed burst width? If the data flow increases suddenly With the current time period The difference does not exceed the maximum burst width allowed , the data flow can be reported as a burst flow, and the data flow in the cell is cleared to zero. If the data flow increases sharply With the current time period The difference exceeds the maximum allowed burst width , then the burst cycle Update to the current time period; S305, setting the sudden decrease preset value ,when When the data flow suddenly decreases, it is determined that the data flow has suddenly decreased, and the data flow sudden increase period is confirmed. Whether the difference with the current time period does not exceed the maximum allowed burst width If the data flow increases suddenly With the current time period The difference does not exceed the maximum burst width allowed , the data flow can be reported as a burst flow, and the data flow in the cell is cleared to zero. If the data flow increases sharply With the current time period The difference exceeds the maximum allowed burst width , then Remain unchanged.

[0006] In some implementations, in step S302, mapping the data stream in the input burst detection table to a cell in the hash bucket is specifically as follows: S3021, confirm whether the data stream is stored in the cell of the hash bucket, if yes, increase the frequency of the data stream in the cell corresponding to the hash bucket by one, if no, proceed to step S3202; S3202, confirm whether there are free cells in the hash bucket, if yes, map the data stream to the corresponding free cells and store the four field information of the data stream, if no, proceed to step S3203; S3203. Find the burst cycle in the cell of the hash bucket If there is only one item that is 0, the data stream is combined with the burst cycle If there are multiple items, replace the data stream with the burst cycle. The item with the lowest frequency of 0 is replaced; if there is no sudden increase cycle in the cell of the hash bucket If the item is 0, the data stream is replaced with the data stream item with the largest burst width in the cell.

[0007] In some implementations, the method further includes step S3204, inserting the replaced data stream item whose burst period is not 0 and whose burst width is the largest into an empty cell of an adjacent hash bucket, and storing the corresponding field information of the replaced data stream item.

[0008] In some implementations, the process further includes step S100', checking whether the input data stream is stored in the burst flow detection table. If the data stream is stored, the current frequency value of the data stream in the burst detection table is increased by one. If the data stream is not stored, the data stream is input into the small flow filter for filtering.

[0009] In some implementations, the method further includes step S400 of verifying the output accuracy of the burst flow based on the false positive error rate of the small flow filter and the error limit of the burst flow detection table.

[0010] Compared with the prior art, the present application provides a low-overhead burst data flow detection algorithm, which continuously filters low-frequency flows to identify potential burst flows by setting a time period, thereby improving the detection accuracy of subsequent burst flows; at the same time, a burst flow detection table is used to track potential burst flows, and based on the frequency and burst period of each potential burst flow in two adjacent time periods, it is further identified whether the potential burst flow is a real burst flow; and a replacement strategy of burst period and data flow frequency probability is also used to eliminate data flows with low burstiness, so as to facilitate the storage of more likely potential burst flows, further reducing memory consumption. Therefore, the burst data flow detection algorithm provided by the present application has the characteristics of high accuracy, low overhead and high throughput. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 is a flowchart of a low-overhead burst data flow detection algorithm in some implementation modes of the present application, Figure 2 It is an operation diagram for identifying burst flows based on a burst data flow detection algorithm in some implementation modes of the present application. DETAILED DESCRIPTION

[0013] "Scope" disclosed in the present application is limited in the form of lower limit and upper limit, and a given range is limited by selecting a lower limit and an upper limit, and the selected lower limit and upper limit define the boundary of a special range. The scope limited in this way can be including end values ​​or not including end values, and can be arbitrarily combined, that is, any lower limit can be combined with any upper limit to form a scope. For example, if the scope of 60-120 and 80-110 is listed for a specific parameter, it is understood that the scope of 60-110 and 80-120 is also expected. In addition, if the minimum range values ​​1 and 2 are listed, and if the maximum range values ​​3, 4 and 5 are listed, the following scope can be all expected: 1-3, 1-4, 1-5, 2-3, 2-4 and 2-5. In the present application, unless otherwise specified, the numerical range "ab" represents the abbreviation of any real number combination between a and b, wherein a and b are real numbers. For example, the numerical range "0-5" means that all real numbers between "0-5" are listed in this document, and "0-5" is just an abbreviation of these numerical combinations. In addition, when a parameter is expressed as an integer ≥ 2, it is equivalent to disclosing that the parameter is, for example, an integer of 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, etc.

[0014] If there is no special explanation, the "include" and "comprising" mentioned in this application are open-ended or closed-ended. For example, the "include" and "comprising" may mean that other components not listed may also be included or only the listed components may be included or only the listed components may be included.

[0015] refer to Figure 1 As shown, the embodiment of the present application provides a low-overhead burst data flow detection algorithm, the algorithm comprising the following steps: S100: Construct a small flow filter and a burst flow detection table, wherein the small flow filter includes counters, the burst detection table is composed of A hash bucket array consisting of hash buckets; S200, based on the small flow filter, filter the low frequency flow in the input data flow and send it to the burst flow detection table, specifically: in the set time period Inside, through An independent hash function maps each input data stream to the corresponding small stream filter In the counters, according to the corresponding Whether the value of a counter reaches a preset filtering threshold value is used to determine whether the data flow is a potential burst flow, thereby filtering the low-frequency flow in the input data flow, wherein, ; More specifically, first set the filtering threshold of the small flow filter and time period , in the current time period Inside, through An independent hash function maps each input data stream to the corresponding small flow filter counter and the corresponding The value of each counter is increased by one, if the corresponding The values ​​of the counters have reached the filtering threshold , the data flow is determined to be a potential burst flow and sent to the burst flow detection table, and the data flow information in the small flow filter is cleared; if the corresponding The values ​​of counters do not all reach the filtering threshold , then keep the data stream in the small stream filter until the current time period After the end, reset the values ​​of all counters of the small flow filter; by timely clearing the flows that do not meet the burst conditions at the end of each time period, the probability of misjudgment can be significantly reduced while reducing memory consumption.

[0016] It should be noted that when the input data stream enters the small flow filter for the first time, An independent hash function will randomly map the input data stream to the small flow filter counter until the same input data flow reaches the small flow filter later. An independent hash function will map the input data stream to the corresponding in a counter.

[0017] S300, judging the data flow based on the burst flow detection table, and then outputting the real burst flow; The steps are as follows: S301, set each hash bucket in the burst detection table to have cells, each of which stores four fields of data flow information ,in, Indicates the identifier of the data stream. Indicates the frequency of the data stream in the previous time period, Indicates the frequency of the data stream in the current time period, Indicates the time period during which the data flow has recently experienced a sudden increase, i.e., the burst period; S302, mapping the data stream input into the burst detection table to a cell of the hash bucket through a hash function, and storing four field information of the data stream in the corresponding cell; S303: Setting the burst threshold , if the cell The value reaches the burst threshold , then calculate the and Ratio ; S304, setting the sudden increase preset value and the maximum allowed burst width ,when When the data flow suddenly increases, it is determined that the data flow has suddenly increased, and the data flow sudden increase period is confirmed. With the current time period Does the difference in value not exceed the maximum allowed burst width? If the data flow increases suddenly With the current time period The difference does not exceed the maximum burst width allowed , the data flow can be reported as a burst flow, and the data flow in the cell is cleared to zero. If the data flow increases sharply With the current time period The difference exceeds the maximum allowed burst width , then the burst cycle Update to the current time period; S305, setting the sudden decrease preset value ,when When the data flow suddenly decreases, it is determined that the data flow has suddenly decreased, and the data flow sudden increase period is confirmed. Whether the difference with the current time period does not exceed the maximum allowed burst width If the data flow increases suddenly With the current time period The difference does not exceed the maximum burst width allowed , the data flow can be reported as a burst flow, and the data flow in the cell is cleared to zero. If the data flow increases sharply With the current time period The difference exceeds the maximum allowed burst width , then Remain unchanged.

[0018] In the implementation mode of the present application, the low-frequency flow is continuously filtered by setting a time period to identify the potential burst flow, which effectively improves the detection accuracy of subsequent burst flows; at the same time, the potential burst flow is tracked by using a burst flow detection table, and the frequency and the burst period of each potential burst flow in two adjacent time periods are analyzed and compared, thereby identifying whether the potential burst flow is a real burst flow. Therefore, the burst data flow detection algorithm provided by the present application has the characteristics of high accuracy, low overhead and high throughput.

[0019] In some implementations, in step S302, mapping the data stream in the input burst detection table to a cell in the hash bucket is specifically as follows: S3021, confirm whether the data stream is stored in the cell of the hash bucket, if yes, increase the frequency of the data stream in the cell corresponding to the hash bucket by one, if no, proceed to step S3202; S3202, confirm whether there are free cells in the hash bucket, if yes, map the data stream to the corresponding free cells and store the four field information of the data stream, if no, proceed to step S3203; S3203. Find the burst cycle in the cell of the hash bucket If there is only one item that is 0, the data stream is combined with the burst cycle If there are multiple items, replace the data stream with the burst cycle. The item with the lowest frequency of 0 is replaced; if there is no sudden increase cycle in the cell of the hash bucket If the item is 0, the data stream is replaced with the data stream item with the largest burst width in the cell; S3204. Insert the replaced data stream item whose burst period is not 0 and whose burst width is the largest into the free cell of the adjacent hash bucket, and store the corresponding field information of the replaced data stream item.

[0020] In the above implementation, a replacement strategy of burst period and data flow frequency probability is adopted to eliminate data flows with lower burstiness, so as to store more likely potential burst flows, which can improve recognition accuracy and further reduce memory consumption.

[0021] In some implementations, the process further includes step S100', checking whether the input data stream is stored in the burst flow detection table. If the data stream is stored, the current frequency value of the data stream in the burst detection table is increased by one. If the data stream is not stored, the data stream is input into the small flow filter for filtering.

[0022] In the above implementation, by first detecting whether the input data flow is stored in the burst detection table, the processing load of the small flow filter can be effectively reduced, which has a positive significance for reducing memory overhead and improving throughput.

[0023] In some implementations, the method further includes step S400 of verifying the output accuracy of the burst flow based on the false positive error rate of the small flow filter and the error limit of the burst flow detection table.

[0024] In the above implementation, by calculating the false positive error rate of the small flow filter and the error limit of the burst flow detection table, the accuracy of the algorithm outputting the true burst flow can be effectively proved, wherein the calculation process of the false positive error rate of the small flow filter is as follows: Assume that there is potential bursts and The low-volume filter consists of counters, and use A separate hash function.

[0025] For an input data stream f , first through k Hash functions are mapped to the corresponding small flow filters k In the counter, when the corresponding k counter values ​​reach or exceed the filtering threshold When the input data stream f It will be misjudged as a potential burst flow, resulting in a false positive error.

[0026] There are two cases where each counter has a false positive error, that is, each counter reaches the filtering threshold There are two cases: (1) There is at least one mapped potential burst flow; Assume that the data flow entering the small flow filter is randomly mapped to The probability that any counter among the counters is not mapped by a potential burst flow is 1-1 / W , since the number of potential burst flows is , the probability that any counter has at least one potential burst flow can be expressed as:

[0027] Similarly, the probability that a single data stream is mapped to a specific counter is 1 / W， set up X Indicates the number of data flows mapped to a particular counter. Because each data flow is mapped to a counter independently, this X It follows the binomial distribution, that is

[0028] in, is the number of trials (i.e., the total number of data flows), 1 / W is the probability that a single data flow maps to that particular counter.

[0029] (2) At least λ A mapped data stream; At least λ The probability that a data stream is mapped to a specific counter is X ≧ λ The probability of

[0030] Furthermore, P r( X <λ ) is expressed as:

[0031] Among them, P r (X=i) is the probability that X equals i:

[0032] In the formula, is the binomial coefficient, which represents the number of ways to select i successes (data flows mapped to the counter) from S trials (total number of data flows).

[0033] From this we can see that at least λ The probability that a data flow is mapped to a specific counter can be expressed as:

[0034] In summary, the false positive error rate of the small flow filter is [ P L + P r ( X ≧ λ )] k ,Right now

[0035] The error limit calculation process of the burst flow detection table is as follows: Assume that n data flows enter the burst flow detection table, and the average number of packets per data flow is , is the total number of hash buckets in the burst flow detection table, and the unit of each hash bucket is C , the data flow must first reach the filtering threshold α To enter the burst flow detection table, the burst threshold is For data streams f , set the number of packets to , the estimated frequency is .

[0036] To evaluate any data stream f To solve the frequency estimation error, we first examine all factors that may affect the data stream.

[0037] Case 1: Data packets belong to data flows f In this case, we locate the data flow in the burst flow detection table and increase its count value by 1, so in the estimated data flow f There is no error when the frequency is Case 2: The data packet belongs to a new data flow f', and there are no empty cells in its mapping hash bucket and adjacent hash buckets, where the data flow f is the smallest data flow with a burst period of 0 and the lowest frequency. In this case, the data flow f Will be data flow f' , its estimated frequency will become zero, so the data flow f The frequency estimation error of is equal to its count value; Case 3: The data packet belongs to a new data flow f' , and there are no empty cells in its mapping hash bucket and adjacent hash buckets, where all data flows are recent burst data flows, and data flows f is the earliest burst of data flow. In this case, in order to retain the data flow f' , data flow f will be moved to a new cell in its adjacent hash bucket, so the data stream f There is also no error in the frequency estimate.

[0038] It can be concluded that the data stream frequency estimation error mainly comes from the data stream replacement process in case 2, and the estimation error can be analyzed from the following five conditions.

[0039] (1) A new data stream f' arrive; (2) There are no empty cells in its mapping hash bucket and adjacent hash buckets; (3) Hash Bucket B [ g ( f' )], B [( g ( f' )+1) % N ]and B [( g (f')-1) % N ] stores the data stream f , where B[g(f')] represents the bucket corresponding to the hash value g(f') of the new data stream f', B[g(f')+1] represents the next bucket adjacent to the hash value g(f') of the new data stream f', B[g(f')-1] represents the previous bucket adjacent to the hash value g(f') of the new data stream f', and %N represents the modulo operation; (4) Data flow f The burst period is 0; (5) Data flow f It is the data flow with the lowest arrival rate in these hash buckets.

[0040] The probabilities of the above five situations are deduced below: Probability of case (1): New data stream f' Refers to the data flow that arrives at the burst flow detection table for the first time. Since entering the burst flow detection table must reach the filtering threshold , and the average packet size of each data flow is , so the probability of situation (1) can be expressed as the ratio of the number of data flows to the total number of data packets entering the burst flow detection table, that is: ; Probability of case (2): For a new data stream f' The probability that the mapped hash bucket and the adjacent hash bucket have no empty cells can be approximated by the Poisson distribution. Let the cell of each hash bucket be C ,parameter represents the average number of data flows in each hash bucket, then the probability that there are no empty cells in its mapping hash bucket and adjacent hash buckets is: ; In the formula, Indicates the number of data flows stored in the hash bucket.

[0041] Probability of case (3): Since the data flow entering the burst flow detection table is randomly mapped to N hash buckets, each hash bucket stores the data flow f The probability of is the same, the probability of case (3) can be expressed as: ; Probability of case (4): Since the data flow has to enter the burst flow detection table to record the burst period of the data flow, the probability that the data flow has no burst is , so the data flow f The probability that the burst period is 0 can be expressed as: ; Probability of case (5): Since the unit of each hash bucket is C , so the data flow f is the data flow with the lowest arrival rate among these hash buckets. The probability of case (5) is expressed as: ; Combining the above five conditional probabilities, we can infer the probability of data stream replacement, which can be expressed as:

[0042] Since the frequency of the replaced data stream is lower than the burst threshold β , we can deduce the data flow f The frequency estimation error satisfies , combined with the above probability formula of data flow replacement, the error limit of the burst flow detection table is finally obtained as: .

[0043] The following examples more specifically describe the contents disclosed in the present application. These examples are intended for illustrative purposes only, since various modifications and changes within the scope of the contents disclosed in the present application will be apparent to those skilled in the art.

[0044] like Figure 2 As shown, set the filtering threshold of the small flow filter , the burst threshold in the burst flow detection table , suddenly reduce the preset value 0.5, a sudden increase in the preset value is 2, the current time period is 7, and the maximum burst width allowed , for the field information stored in the first cell of hash bucket Bucket1 For example, Representing data flow Identifier, 150 represents the frequency of the data flow in the previous time period (i.e., the 6th time period), 260 represents the frequency of the data flow in the current time period, and 3 represents the most recent sudden increase period of the data flow (i.e., the data flow has experienced a sudden increase in the 3rd time period).

[0045] (1) Based on this algorithm, the data stream When performing detection, the first cell of hash bucket Bucket1 in the burst flow detection table stores data flow At this time, you only need to change the current time period frequency value of the data stream Just add 1; (2) Based on this algorithm, the data stream When performing detection, since there is no data flow stored in the hash bucket cell of the burst flow detection table , then the data flow Input to the small flow filter, because there is data flow in the small flow filter before The information is converted into data streams through hash functions. Map to the corresponding counter, that is, add 1 to the value in the corresponding counter. At this time, since the value of the corresponding counter does not reach the filtering threshold of 50, no other operation is required; (3) Based on this algorithm, the data stream When testing, because there is data flow in the small flow filter The information is converted into data streams through hash functions. Mapped to the corresponding counter, that is, the value in the corresponding counter is increased by 1. At this time, since the value of the corresponding counter reaches the filtering threshold of 50, the data flow is determined to be For potential data flow, the data flow needs to be Enter the burst flow detection table; search from the burst flow detection table and find that there is an empty cell in hash bucket Bucker2, then the data flow Insert it into the empty cell and store its four field information as ; (4) Based on this algorithm, the data stream When testing, because there is data flow in the small flow filter The information is converted into data streams through hash functions. Mapped to the corresponding counter, that is, the value in the corresponding counter is increased by 1. At this time, since the value of the corresponding counter reaches the filtering threshold of 50, the data flow is determined to be For potential data flow, the data flow needs to be Input burst flow detection table; in the burst flow detection table, firstly use the hash function to convert the data flow Map it to the hash bucket Bucket n and check whether there are any free cells. Since no free cells are found in the hash bucket Bucket n, the data flow with the largest burst width is selected. Replace and store the data stream The field information is , while the data stream Insertion into the adjacent hash bucket is stored in the insertable cell; It should be noted that, here, since there is no data flow item with a burst period of 0 in the hash bucket Bucket n, according to the replacement strategy, the data flow item that meets the second replacement condition, that is, the data flow item with the largest burst width, is found. Therefore, the data flow item in the hash bucket Bucket n is To satisfy the second replacement condition, the data flow Replace with data stream , and then the data stream Insert into the adjacent hash bucket for storage; Until the end of the current time period (i.e., the 7th time period), each data stream in the hash bucket cell is identified and determined, that is, the number of cells in the calculation is and Ratio ; For data streams , ,because Less than the preset value of sudden drop , indicating data flow There was a sudden drop, and at the same time, the data flow Burst Width Less than the maximum allowed burst width 4, so it can be determined that the data flow is a burst flow with a burst width of 3; For data streams , data flow There was a sudden increase in the third time period, and its burst width Less than the maximum allowed burst width of 4, however, the data flow of Greater than the preset value of sudden drop , so the data flow The sudden decrease condition is not met, but the data flow The maximum burst width allowed has been reached and the data stream needs to be The burst period is cleared and the burst is recalculated; For data streams , Data Flow and data flow , there was no sudden increase before the 7th time period, so no other operation was needed; For data streams , data flow There was a sudden increase in the third time period, and its burst width Less than the maximum allowed burst width of 4, however, the data flow of Greater than the preset value of sudden drop , so the data flow The sudden decrease condition is not met, and the data flow The maximum burst width allowed has been reached and the data stream needs to be The burst period is cleared and the burst is recalculated.

[0046] The above is a detailed introduction to a low-overhead burst data flow detection algorithm provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A low-overhead burst data flow detection algorithm, characterized in that: The algorithm consists of the following steps: S100: Construct a small flow filter and a burst flow detection table, wherein the small flow filter includes counters, the burst detection table is composed of A hash bucket array consisting of hash buckets; S200, based on the small flow filter, filter the low frequency flow in the input data flow and send it to the burst flow detection table, specifically: in the set time period Inside, through An independent hash function maps each input data stream to the corresponding small stream filter In the counters, according to the corresponding Whether the value of a counter reaches a preset filtering threshold value is used to determine whether the data flow is a potential burst flow, thereby filtering the low-frequency flow in the input data flow, wherein, ; S300: judging the data flow based on the burst flow detection table, and then outputting the real burst flow.

2. The low-overhead burst data flow detection algorithm as claimed in claim 1, characterized in that: The step 200 specifically includes: setting the filtering threshold of the small flow filter and time period , in the current time period Inside, through An independent hash function maps each input data stream to the corresponding small stream filter counter and the corresponding The value of each counter is increased by one, if the corresponding The values ​​of the counters have reached the filtering threshold , the data flow is determined to be a potential burst flow and sent to the burst flow detection table, and the data flow information in the small flow filter is cleared; if the corresponding The values ​​of counters do not all reach the filtering threshold , then keep the data stream in the small stream filter until the current time period After completion, reset the values ​​of all counters of the low-flow filter.

3. The low-overhead burst data flow detection algorithm as claimed in claim 2, characterized in that: The step S300 specifically includes the following steps: S301, set each hash bucket in the burst detection table to have cells, each of which stores four fields of data flow information ,in, Indicates the identifier of the data stream. Indicates the frequency of the data stream in the previous time period, Indicates the frequency of the data stream in the current time period, Indicates the time period during which the data flow has recently experienced a sudden increase, i.e., the burst period; S302, mapping the data stream input into the burst detection table into a cell of the hash bucket through a hash function, and storing four field information of the data stream in the corresponding cell; S303: Setting the burst threshold , if the cell The value reaches the burst threshold , then calculate the and Ratio ; S304, setting the sudden increase preset value and the maximum allowed burst width ,when When the data flow suddenly increases, it is determined that the data flow has suddenly increased, and the data flow sudden increase period is confirmed. With the current time period Does the difference in value not exceed the maximum allowed burst width? If the data flow increases suddenly With the current time period The difference does not exceed the maximum burst width allowed , the data flow can be reported as a burst flow, and the data flow in the cell is cleared to zero. If the data flow increases sharply With the current time period The difference exceeds the maximum allowed burst width , then the burst cycle Update to the current time period; S305, setting the sudden decrease preset value ,when When the data flow suddenly decreases, it is determined that the data flow has suddenly decreased, and the data flow sudden increase period is confirmed. Whether the difference with the current time period does not exceed the maximum allowed burst width If the data flow increases suddenly With the current time period The difference does not exceed the maximum burst width allowed , the data flow can be reported as a burst flow, and the data flow in the cell is cleared to zero. If the data flow increases sharply With the current time period The difference is greater than the maximum allowed burst width , then Remain unchanged.

4. The low-overhead burst data flow detection algorithm as claimed in claim 3, characterized in that: In step S302, mapping the data flow in the input burst detection table to a cell in the hash bucket is specifically as follows: S3021, confirm whether the data stream is stored in the cell of the hash bucket, if yes, increase the frequency of the data stream in the cell corresponding to the hash bucket by one, if no, proceed to step S3202; S3202, confirm whether there are free cells in the hash bucket, if yes, map the data stream to the corresponding free cells and store the four field information of the data stream, if no, proceed to step S3203; S3203. Find the burst cycle in the cell of the hash bucket If there is only one item that is 0, the data stream is combined with the burst cycle If there are multiple items, replace the data stream with the burst cycle. The item with the lowest frequency of 0 is replaced; if there is no sudden increase cycle in the cell of the hash bucket If the item is 0, the data stream is replaced with the data stream item with the largest burst width in the cell.

5. The low-overhead burst data flow detection algorithm as claimed in claim 4, characterized in that: The method also includes step S3204, inserting the replaced data stream item whose burst period is not 0 and whose burst width is the largest into the free cell of the adjacent hash bucket, and storing the corresponding field information of the replaced data stream item.

6. The low-overhead burst data flow detection algorithm as claimed in claim 5, characterized in that: The method further includes step S100', checking whether the input data stream is stored in the burst flow detection table. If the data stream is stored, the current frequency value of the data stream in the burst detection table is increased by one. If the data stream is not stored, the data stream is input into the small flow filter for filtering.

7. The low-overhead burst data flow detection algorithm as claimed in claim 6, characterized in that: The method further includes step S400 of verifying the output accuracy of the burst flow based on the false positive error rate of the small flow filter and the error limit of the burst flow detection table.

Images