Serial Multiple-Stage Filter for Large Flow Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting large flows in networks face challenges such as high memory and resource requirements, and discontinuities in detection due to counter resetting, which can lead to inefficiencies and latency in identifying and managing elephant flows.
Innovation Solution
A network device with a serial multiple-stage filter module that increments counters only after all prior counters in the series have reached their maximum values, allowing for continuous operation without resetting, and includes a lead filter removal module to purge accounting and avoid discontinuities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Duration of action of stationary object
If counters are periodically erased or reset to enable continuous operation, then the system can operate indefinitely, but discontinuities in large flow detection are introduced and additional latency is added
Solution Approach 1:
The counter array is divided into multiple segments or groups, allowing the system to switch between different segments. When one segment's counters are full, the system can transition to another segment without resetting, thereby maintaining continuous detection capability while avoiding the discontinuities caused by periodic resets
Solution Approach 2:
The system discards counters that have reached their maximum values and recovers memory space by reallocating or resetting only those specific counters rather than the entire counter array. This allows continuous operation while minimizing disruption to ongoing flow detection
2Measurement precision
If a counter is provided for each different flow, then accurate flow detection is achieved, but prohibitive amounts of memory and memory access bandwidth resources are required
Solution Approach 1:
Multiple flows are merged and tracked using shared counters. When a counter reaches its maximum value, the system determines that the corresponding flow is a large flow and can be detected without requiring individual counters for every flow. This combining approach reduces memory requirements while maintaining detection accuracy
Solution Approach 2:
Instead of providing exact one-counter-per-flow allocation, the system uses a partial allocation approach where counters are provided for expected flows, and excess flows that exceed counter capacity are identified as large flows through the counter saturation mechanism, reducing overall memory requirements
Data Source
AI summary
A network device to detect large flows includes a card to receive packets of flows. The device includes a large flow detection module including a serial multiple-stage filter module including series filter modules including a lead filter module and a tail filter module. Each filter module includes counters. The serial filter module is to serially increment the counters to reflect the flows, and is to increment counters that correspond to flows of subsequent filter modules only after all counters that correspond to the flows of all prior filter modules have been incremented serially up to maximum values. The serial filter module is to detect flows that correspond to counters of the tail filter module that have been incremented up to maximum values as the large flows. The large flow detection module includes a lead filter removal module to remove the lead filter module from the start of the series.