Network Monitoring Large Flow Detection Sampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network traffic measurement methods, such as packet sampling and flow caching, face limitations in accurately measuring network traffic due to the high volume of data and the inability to distinguish between large and small flows, leading to incomplete visibility and potential overburdening of network nodes during rerouting.
Innovation Solution
Implementing a system that identifies and caches large flows while sampling small flows, using unique flow estimation to enhance measurement accuracy by correlating data from large and small flow measurements, thereby optimizing network provisioning and attack detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If packet sampling is used to measure network traffic, then the measurement process is simplified and less resources are consumed, but the measurement precision deteriorates due to the selection of only a small portion of packets
Solution Approach 1:
The patent applies local quality by treating large flows and small flows differently. Large flows are cached and measured with flow caching to ensure accuracy, while small flows are measured with packet sampling to save resources. This differentiated approach resolves the contradiction by optimizing measurement precision for critical flows while maintaining overall measurement efficiency.
2Measurement precision
If flow caching is used to store information about all packet flows, then the measurement precision is improved, but the device complexity increases due to the need to store and process large amounts of flow data
Solution Approach 1:
The patent extracts and caches only large flows that meet specific criteria (bandwidth threshold, packet count threshold, or duration threshold) rather than caching all flows. This selective extraction reduces the volume of cached data and processing requirements while maintaining measurement precision for the most significant flows that impact network performance and security.
Solution Approach 2:
The patent applies local quality by providing different measurement qualities for different flows. High-precision flow caching is applied only to large flows where accuracy is critical, while lower-precision packet sampling is used for small flows. This resolves the contradiction by optimizing precision where needed while reducing overall system complexity.
3Ease of operation
If all packet flows are monitored with the same method, then the measurement process is simple, but the ability to detect attacks and optimize provisioning deteriorates due to inability to distinguish significant flows from normal traffic
Solution Approach 1:
The patent applies local quality by implementing different monitoring strategies for large and small flows. Large flows are cached and analyzed in detail to detect attacks and optimize provisioning, while small flows are sampled. This differentiated approach maintains operational simplicity while significantly improving reliability for detecting significant events like DDoS attacks that typically manifest as large flows.
Solution Approach 2:
The patent extracts large flows from the general traffic stream using criteria such as bandwidth thresholds, packet count thresholds, and duration thresholds. By separating and focusing monitoring resources on these extracted large flows, the system improves attack detection capability while maintaining simple operation through automated classification.
Data Source
AI summary
Provided are systems and methods for sampling packets based on large flow detection, for network visibility monitoring. In some implementations, provided is an integrated circuit. The integrated circuit may include large-flow detection logic and a sampling determination logic. The integrated circuit may be operable to received packet information describing a packet a cycle of a clock input. The packet may be associated with a packet flow being transmitted through a network. The integrated circuit may further be operable to determine, using the large-flow detection logic, whether the packet is associated with a large packet flow or a small packet flow. Upon determining that the packet is associated with a small packet flow, the integrated circuit may update the sampling determination logic.