Network Flow Tracking via Hash Segmentation for Elephant Flows
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network devices face challenges in efficiently identifying and tracking large-data flows, also known as 'elephant flows,' amidst mixed flows of varying sizes, which are crucial for analytics and load-balancing but require effective bandwidth management.
Innovation Solution
The implementation of a method involving the extraction of flow keys from packet data, computation of hash values to create hashed flow signatures, and the use of hash tables to track and differentiate between large-data flows, with thresholds for moving flows into a dedicated 'elephant' table for detailed tracking and management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all flows are tracked in a single table, then complete flow information is maintained, but the device complexity and processing overhead increase significantly
Solution Approach 1:
The patent divides the flow tracking system into two separate tables: a flow hash table for general flow tracking and an elephant flow table for large-data flow tracking. This segmentation allows the system to handle different types of flows with appropriate tracking mechanisms, reducing overall complexity while maintaining completeness. The flow hash table uses hash-based indexing for efficient lookup, while the elephant flow table specifically tracks flows exceeding a threshold size.
Solution Approach 2:
The patent extracts elephant flows (large-data flows) from the general flow tracking system and places them in a dedicated elephant flow table. This extraction allows the system to apply specialized tracking and management techniques for large flows, improving efficiency without compromising the tracking of smaller flows in the hash table.
2Speed
If hash tables are used for flow tracking, then lookup speed improves, but memory usage increases
Solution Approach 1:
The patent segments flow tracking into two phases: initial tracking in a compact flow hash table using hash values for fast lookup, and subsequent detailed tracking in the elephant flow table. This segmentation allows the system to use memory-efficient hash-based storage for the majority of flows while dedicating more memory only to flows that exceed the elephant threshold.
Solution Approach 2:
The patent changes the tracking parameter from tracking all flows with full details to tracking only elephant flows with full details. By monitoring flow size and transitioning flows to the elephant table only when they exceed a threshold, the system optimizes memory usage by storing detailed information only when necessary.
Data Source
AI summary
Apparatus, systems and methods may be used to monitor data flows and to select and track particularly large data flows. A method of tracking data flows and identifying large-data (“elephant”) flows comprises extracting fields from a packet of data to construct a flow key, computing a hash value on the flow key to provide a hashed flow signature, entering and/or comparing the hashed flow signature with entries in a flow hash table. Each hash table entry includes a byte count for a respective flow. When the byte count for a flow exceeds a threshold value, the flow is added to a large-data flow (“elephant”) table and the flow is then tracked in the large-data flow table.