Network Flow Detection IC Segmentation for Visibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network traffic measurement methods, such as packet sampling and flow caching, face limitations in accurately capturing and analyzing packet flows, particularly in identifying small flows and managing cache memory effectively, leading to incomplete visibility and inaccurate data.
Innovation Solution
Implementing a system that differentiates between large and small packet flows, caching large flows, sampling small flows, and using unique flow estimation to enhance measurement accuracy and reduce memory usage, thereby improving network visibility and provisioning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If packet sampling is used to measure network traffic, then measurement precision is improved, but the quantity of substance (packet data) is reduced
Solution Approach 1:
The patent segments packet flows into large flows and small flows based on bandwidth thresholds. Large flows are cached completely while small flows are sampled, creating different processing paths for different segments of data. This resolves the contradiction by applying complete measurement to significant portions (large flows) and statistical sampling to minor portions (small flows), optimizing both precision and data quantity.
Solution Approach 2:
The patent applies different measurement qualities to different parts of the network traffic. Large flows receive high-quality complete caching, while small flows receive lower-quality sampling. This local differentiation allows the system to maintain measurement precision for critical traffic while reducing overall data processing load.
2Measurement precision
If flow caching is used to store packet flow information, then measurement precision is improved, but device complexity increases due to cache memory management
Solution Approach 1:
The patent segments flow processing into two paths: caching large flows and sampling small flows. This segmentation reduces cache memory management complexity by limiting the cache to only significant flows, while sampling handles the remaining traffic. The segmentation principle directly addresses the complexity issue by reducing the scope of what needs to be cached.
Solution Approach 2:
The patent applies partial action by caching only large flows rather than all flows. This partial caching approach reduces memory requirements and simplifies management while maintaining sufficient measurement precision for the most important traffic patterns. The excessive action of complete flow caching is avoided for small flows that don't require full treatment.
3Loss of information
If all packet flows are cached, then network visibility is improved, but loss of substance increases due to memory constraints
Solution Approach 1:
The patent segments flows into large and small categories, caching only large flows while sampling small flows. This segmentation allows the system to maintain network visibility for the most significant traffic (large flows) while using minimal memory, avoiding the need to store all flow data.
Solution Approach 2:
The patent extracts the essential information from small flows through sampling rather than caching them completely. This extraction approach retains useful statistical data about small flows while avoiding the memory burden of storing all their packet details, resolving the contradiction between visibility and memory usage.
4Measurement precision
If packet sampling rate is increased, then measurement precision is improved, but productivity decreases due to processing overhead
Solution Approach 1:
The patent segments traffic into large and small flows, applying different sampling rates to each. Large flows are processed at lower rates (cached completely), while small flows are sampled at higher rates. This segmentation optimizes the balance between measurement precision and processing efficiency by avoiding excessive sampling of dominant traffic patterns.
Solution Approach 2:
The patent applies local quality by using higher sampling rates for small flows where precision is more critical, and lower processing rates for large flows where volume dominates. This localized optimization maintains measurement precision where needed while improving overall processing efficiency.
Data Source
AI summary
Provided are systems and methods for large flow detection for network visibility monitoring. In some implementations, provided is an integrated circuit. The integrated circuit may be operable to receive packet information describing a packet at a cycle of a clock input. The packet may be associated with a packet flow being transmitted across a network. The integrated circuit may further generate a key using information identifying the packet flow provided by the packet information. The integrated circuit may further read a value for a counter from a counter memory using the key. The integrated circuit may determine whether the packet is associated with a large flow or a small flow using the counter and a packet size provided by the packet information. Upon determining that the packet is associated with a large flow, the integrated circuit may update an entry in a flow memory using the packet information.