Network Heavy User Detection via Multistage Filter and Elephant Trap
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional mechanisms for detecting heavy users of network resources are inefficient and inaccurate, leading to false positives and false negatives, and fail to accurately identify the heaviest users and their associated streams.
Innovation Solution
A system combining a counting multistage filter and an elephant trap, where the counting multistage filter uses hash functions and an array to detect heavy users and the elephant trap employs probabilistic sampling and a buffer to remove false positives and negatives, ensuring accurate identification of heavy users and streams.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional mechanisms are used to detect heavy users, then the detection process is simple, but the accuracy is low with false positives and false negatives
Solution Approach 1:
The patent divides the detection system into multiple stages: a counting multistage filter that processes packets through several hashing stages to identify candidate heavy users, and an elephant trap that performs probabilistic sampling to verify candidates. This segmentation allows the system to achieve high detection accuracy by breaking down the complex detection task into manageable stages, each contributing to filtering out false positives and false negatives.
Solution Approach 2:
The patent introduces an intermediary probabilistic sampling mechanism (elephant trap) between the initial packet capture and the final heavy user identification. This intermediary layer samples packets probabilistically to verify candidate heavy users identified by the counting filter, thereby reducing false positives and false negatives without requiring complete inspection of all packets.
2Measurement precision
If all heavy users are detected accurately, then the identification precision is high, but the processing time and resources increase
Solution Approach 1:
The patent applies partial action by using probabilistic sampling in the elephant trap stage rather than examining every packet from candidate heavy users. By sampling a subset of packets probabilistically, the system achieves sufficient accuracy to identify heavy users without the time cost of complete packet inspection, thus balancing precision with detection time.
Solution Approach 2:
The counting multistage filter performs preliminary action by pre-processing all packets through multiple hashing stages to identify candidate heavy users before the more time-consuming verification stage. This preliminary filtering reduces the volume of data that requires detailed examination in subsequent stages, thereby reducing overall detection time while maintaining accuracy.
3Reliability
If probabilistic sampling is used to remove false positives, then the detection accuracy improves, but the system complexity increases
Solution Approach 1:
The patent merges the counting multistage filter and the elephant trap into a unified detection system where both components work together synergistically. The filter identifies candidates and the trap verifies them through probabilistic sampling, with both stages contributing to reducing false positives. This merging allows the system to achieve high reliability while managing complexity through integrated design rather than separate independent systems.
Data Source
AI summary
A device includes a multistage filter and an elephant trap. The multistage filter has hash functions and an array. The multistage filter is operable to receive a packet associated with a candidate heavy network user and send the packet to the hash functions. The hash functions generate hash function output values corresponding to indices in the array. The elephant trap is connected to the multistage filter. The elephant trap includes a buffer and probabilistic sampling logic. The probabilistic sampling logic is operable to attempt to add information associated with the packet to the buffer a particular percentage of the time based in part on the result of the multistage filter lookup. The buffer is operable to hold information associated with the packet, counter information, and timestamp information.