Elephant Flow Identification Using Hash Tables and Byte Thresholds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network devices face challenges in efficiently identifying and tracking large-data flows, known as 'elephant flows,' amidst mixed flows of varying sizes, which consume significant bandwidth and require effective analytics and load-balancing.
Innovation Solution
The method involves constructing a flow key from packet data, computing a hash value to generate a hashed flow signature, and using a flow hash table to track and identify large-data flows by comparing the signature with table entries, moving flows exceeding a threshold to a large-data flow table for detailed tracking and analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If network devices track all flows to identify large-data flows, then identification accuracy is improved, but device complexity and processing overhead increase
Solution Approach 1:
The patent segments flow tracking into two distinct phases: initial flow identification using hash table lookup, and subsequent large-data flow confirmation using byte count comparison. This segmentation allows the system to process all flows efficiently while applying more rigorous identification only when necessary, thereby maintaining high identification accuracy without proportionally increasing processing overhead for every flow.
Solution Approach 2:
The patent applies different tracking rigor to different flows based on their characteristics. Small flows receive minimal processing through hash table lookup, while flows exceeding the byte count threshold receive enhanced tracking in the large-data flow table. This local quality approach ensures high identification accuracy for elephant flows while minimizing processing overhead for the majority of smaller flows.
2Productivity
If network devices monitor all data flows to distinguish elephant flows, then bandwidth management is improved, but processing time increases
Solution Approach 1:
The patent performs preliminary flow identification using hash table lookup and byte count threshold comparison before committing resources to detailed large-data flow tracking. This preliminary action filters out the majority of flows that do not require extensive monitoring, thereby improving bandwidth management efficiency while minimizing the processing time consumed by full monitoring of all flows.
Solution Approach 2:
The patent applies partial monitoring to all flows through hash table lookup, and excessive (full) monitoring only to flows that exceed the byte count threshold. This partial or excessive action strategy ensures efficient bandwidth management for elephant flows while avoiding the time cost of full monitoring applied uniformly to all flows.
3Measurement precision
If network devices use detailed flow analysis, then load-balancing accuracy is improved, but computational resources are consumed
Solution Approach 1:
The patent applies detailed flow analysis locally only to flows that exceed the byte count threshold and are identified as potential large-data flows. This local quality approach ensures high load-balancing accuracy for elephant flows while minimizing computational resource consumption by avoiding detailed analysis of smaller flows that do not require such precision.
Solution Approach 2:
The patent changes the monitoring parameter from comprehensive flow analysis to threshold-based byte count comparison. This parameter change enables the system to maintain high load-balancing accuracy for significant data flows while dramatically reducing computational resource consumption by using a simpler threshold comparison for initial flow filtering.
Data Source
AI summary
Apparatus, systems and methods may be used to monitor data flows and to select and track particularly large data flows. A method of tracking data flows and identifying large-data (“elephant”) flows comprises extracting fields from a packet of data to construct a flow key, computing a hash value on the flow key to provide a hashed flow signature, entering and/or comparing the hashed flow signature with entries in a flow hash table. Each hash table entry includes a byte count for a respective flow. When the byte count for a flow exceeds a threshold value, the flow is added to a large-data flow (“elephant”) table and the flow is then tracked in the large-data flow table.