Network flow identification method based on Hash collision compensation

Abstract

The invention discloses a network flow identification method based on Hash collision compensation, comprising the steps of providing a counter array and a set of Hash function, wherein each Hash function input is a flow mark; measuring each message according to the flow mark of the message during measuring period, utilizing each Hash function in the set of Hash function to generate a Hash value, updating the counter of each Hash value corresponding to the counter array; meanwhile, randomly sampling and recording the flow mark, after the measurement is finished, taking each recorded flow mark as the input of the set of Hash function to generate the Hash value, searching the counter corresponding to the Hash value position in the counter array, taking the minimum value in the set of counters as an initial estimation value of the flow mark rate, then randomly generating a plurality of virtual flow marks, searching the value of the virtual flow mark in the counter array, and taking the median of the values as the flow rate collision compensation value, computing the flow mark rate by subtracting the flow rate collision compensation value from the initial estimation value of the flow mark rate, and outputting the flow mark and the flow rate over the threshold value.

Description

technical field [0001] The present invention relates to network large-flow measurement technology, especially the measurement technology that is beneficial to the large-flow flow record using data flow technology, so that the measurement method can use a small measurement space to detect large-flow data from massive data, and can use Higher accuracy estimates the true flow size of a large flow. The invention belongs to the field of network traffic measurement. Background technique [0002] A small number of large flows in the network account for the vast majority of network traffic. Real-time monitoring and management of these large flows is of great significance for network engineering and improving network transmission performance. Due to the large amount of traffic in the current backbone network, it is generally difficult to search all the traffic in the network traffic to find the large flow. There are mainly two methods for real-time monitoring of network large flow, ...

AI Technical Summary

Problems solved by technology

The advantage of this method is that all message records can be processed in a limited space, but the main defect of this method is that if the array space setting is small, then due to the conflict between different flow identifier hash map positions It is very serious, causing a large number of small streams to be inferred as large streams, and the estimation error of large streams is also very large; if in order to reduce conflict errors, the array space needs to be set very large, which will greatly waste the use of memory space

Images