Dynamic Packet Sampling for Elephant Flow Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network technologies face challenges in distinguishing and managing elephant flows from mouse flows, leading to potential quality of service (QoS) degradation for delay-sensitive communications, as they cannot accurately detect large flows in real-time due to uncertainty and resource constraints.
Innovation Solution
A method and system that dynamically adjust the packet sampling rate based on the heavy tailedness of network flow distributions, using a cache to identify and classify flows as either elephant or mouse by computing metrics such as variance, skewness, and kurtosis, allowing for timely detection and differentiation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If packet sampling rate is increased to detect large flows accurately, then detection precision improves, but computational cost and processing overhead increase
Solution Approach 1:
The patent implements dynamic sampling rate adjustment based on heavy tailedness computation. The system starts with an initial sampling rate and iteratively adjusts it based on the computed heavy tailedness metric of observed flows. This dynamic adaptation allows the system to achieve accurate elephant flow detection while minimizing sampling rate (and thus computational cost) under varying network conditions.
Solution Approach 2:
The system changes the sampling rate parameter based on computed heavy tailedness values. By computing heavy tailedness from observed flow data and using it to adjust the sampling rate, the system optimizes the balance between detection accuracy and processing overhead. The sampling rate is increased when heavy tailedness indicates presence of large flows and decreased when traffic is more uniform.
2Reliability
If all packets are sampled to ensure accurate flow detection, then detection reliability improves, but processing speed decreases
Solution Approach 1:
The patent applies partial sampling rather than sampling all packets. By computing heavy tailedness from a subset of sampled packets and using this metric to adjust sampling rate, the system achieves reliable elephant flow detection without the prohibitive cost of processing every packet. The partial sampling approach maintains detection reliability while preserving processing speed.
3Productivity
If sampling rate is decreased to reduce processing overhead, then processing speed improves, but detection precision deteriorates
Solution Approach 1:
The system uses feedback from heavy tailedness computation to adjust sampling rate. By continuously monitoring the heavy tailedness metric computed from sampled flows and using this feedback to increase or decrease sampling rate, the system maintains detection precision adaptively while allowing processing speed to optimize under different traffic conditions.
Data Source
AI summary
In a system for efficiently detecting large/elephant flows in a network, the rate at which the received packets are sampled is adjusted according to the measured heavy tailedness of the arriving traffic, such that the measured heavy tailedness reaches a specified target level. The heavy tailedness is measured using the estimated sizes of different flows associated with the arriving packets. When the measured heavy tailedness reaches and remains at the specified target level, the flows having the largest estimated sizes are likely to be the largest/elephant flows in the network.