Network flow measurement algorithm based on sampling

Abstract

The invention discloses a network flow measurement algorithm based on sampling. The method comprises the following steps: (1) reasonably configuring parameters of a Counting Bloom Filter structure; (2) judging whether the network flow of an arrival data packet is new flow by using the Counting Bloom Filter structure; (3) if the arrival data packet is judged as the new flow, inserting the new flow into the Counting Bloom Filter, adding 1 to a flow counter, calculating an error probability P of the number of the inserted flow, adjusting the sampling frequency (the formula is as shown in the specification) at the same time, wherein f is the sampling frequency of the entire algorithm, perform random sampling on the new flow according to the adjusted frequency, continuing to process the next packet, repeating the step (2) to continue the circulation; and (4) if it is judged that no new flow arrives, continuing to process the next packet, and repeating the step (2) to continue the circulation. By adoption of the network flow measurement algorithm based on sampling, equal probability sampling of the network flow is realized, the network flow data can be reduced, the feature information of the network flow data can be maintained, and the storage resources of the system are saved.

Description

technical field [0001] The invention relates to the technical field of network flow measurement engineering, in particular to a sampling-based network flow measurement algorithm. Background technique [0002] Due to the continuous improvement of the network link rate and the sharp increase of network data flow, it has become impossible to capture each packet information or flow information for storage and analysis. The introduction of sampling technology has effectively solved this bottleneck problem. Based on sampling Network traffic measurement algorithm has become one of the key research contents in the field of network traffic measurement in recent years. [0003] Although the sampling technology has improved the effectiveness and feasibility of traffic measurement to a certain extent, due to the high-speed network traffic, the increasing complexity and heterogeneity, and the shortage of network processing and storage resources, the basic sampling method has been difficu...

AI Technical Summary

Problems solved by technology

[0003] Although the sampling technology has improved the effectiveness and feasibility of traffic measurement to a certain extent, due to the high-speed network traffic, the increasing complexity and heterogeneity, and the shortage of network processing and storage resources, the basic sampling method has been difficult to meet the reality. Based on the needs of the flow, flow measurement algorithms such as flow-based sampling measurement, sampling and hash combination measurement, and BloomFilter-based sampling measurement are derived.

[0004] The basic sampling method widely has the disadvantage of insufficient sampling of small flow data, resulting in the loss of a large number of flow distribution information, and cannot effectively obtain the real distribution of network flows. Therefore, it is necessary to sample network flows in a more fair manner. The design of the present invention A network flow equal probability sampling algorithm based on Counting Bloom Filter structure

Images