Network data stream statistical method on basis of counting bloom filter

Abstract

The invention discloses a network data stream statistical method on the basis of a counting bloom filter. The method comprises the following steps of: before the counting bloom filter carries out statistics, configuring parameters which comprise an error-tolerant rate, the total number of streams, the number of hash functions and the number of counters; hashing quintuple information of each data packet into the counters at the corresponding positions by the counting bloom filter; and using the minimum values in the corresponding counters as flow of the corresponding streams. The implementation of the network data stream statistical method has the following beneficial effects that under the condition of the preset number of the streams, rapid statistics of the flow and the number of the streams is implemented by a special parameter setting scheme and through using the counting bloom filter which is easy for hardware implementation; and in the high-efficiency hash setting process, overflow of the counters of the counting bloom filter is prevented by the expanded counters and the requirements for stream statistics and rapid storage of the quintuple information under the general conditions in a high-speed network can be met.

Description

technical field [0001] The invention relates to the field of network data flow statistics, in particular to a method for network data flow statistics based on counting bloom filters. Background technique [0002] With the rapid development of today's network, the port rate of OC-48 (2.5Gbps) is very common on high-speed routers and switches, and the rate of the new OC-768 port is as high as 40Gbps. Understanding network behavior characteristics in a high-speed network environment requires network measurement, and flow-level measurement statistics are more and more widely used in network measurement. Flow and traffic statistics can be used for network security detection, traffic accounting, traffic management, traffic engineering, and other network applications. [0003] There are many network flow statistics techniques, and the main pursuit is statistical accuracy, reducing computational complexity and space complexity. For example, using encoded hash statistics, after the...

AI Technical Summary

Problems solved by technology

This statistical method has advantages in space occupation, but the estimation error after sampling is inevitable

For example, the "1out of N" sampling strategy introduced by Cisco in Netflow counts one out of every N flows. This method is simple and easy to implement in hardware, but it deviates from the actual network flow distribution, and the probability of large flows being sampled is high, while short current lower

Images