Federated Microburst Detection via Distributed Sensor Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network traffic monitoring systems lack the granularity to detect and diagnose microbursts, which cause temporary network overloads and packet drops even in low-traffic environments, often leading to misdiagnosis and inadequate solutions.
Innovation Solution
A network traffic monitoring system with sensors that detect and analyze packet flows over specified time periods, generating flow summaries that are sent to an analytics engine to identify nodes experiencing microbursts and determine their causes, utilizing an application dependency map to suggest remedies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional packet loss detection methods are used, then packet drops can be identified, but microbursts cannot be effectively detected or diagnosed due to lack of granularity
Solution Approach 1:
The patent segments the monitoring system into distributed sensors deployed at network nodes, each independently analyzing local packet flows. This segmentation enables precise microburst detection at the node level while avoiding the complexity of a centralized system that would need to process all network traffic globally. Each sensor divides the monitoring task into manageable local units.
Solution Approach 2:
The patent adds temporal dimensionality to packet analysis by examining packet arrival patterns over time intervals rather than just counting total packets. This dimensional transformation enables detection of microbursts as temporal anomalies in packet flow patterns, achieving precise detection without requiring complex global state tracking.
2Loss of information
If network monitoring focuses on overall utilization, then general network health can be assessed, but microbursts with low byte-counts are missed
Solution Approach 1:
The patent applies local quality analysis by having each sensor examine the specific characteristics of packet flows passing through its local node, rather than applying uniform global monitoring. This local focus captures microburst patterns specific to each node's traffic patterns while maintaining overall network monitoring efficiency through distributed operation.
Solution Approach 2:
The patent creates local copies of monitoring functionality at each network node through deployed sensors, rather than relying on a single centralized monitoring point. This copying approach ensures that microburst information is captured locally where it occurs, preventing information loss while maintaining scalable system productivity.
3Loss of information
If buffer overflow detection is used on switches, then some microburst indicators can be identified, but sufficient diagnostic information is not provided
Solution Approach 1:
The patent performs preliminary analysis of packet flow characteristics at the sensor level before data reaches the analytics engine. Sensors pre-process packet data to extract relevant features and patterns, providing the analytics engine with prepared diagnostic information that reduces the difficulty of microburst detection and diagnosis while ensuring complete information is captured.
Data Source
AI summary
An example method includes a sensor detecting multiple packets of a flow during a specified total time period (e.g., a reporting time period). The total time period can be subdivided into multiple time periods. The sensor can analyze the detected packets to determine an amount of network utilization for each of the time periods. The sensor can then generate a flow summary based on the network utilization and the flow and send the flow summary to an analytics engine. Multiple other sensors can do similarly for their respective packets and flows. The analytics engine can receive the flow summaries from the various sensors and determine a correspondence between flow with high network utilization at a specific time period and a node or nodes. These nodes that experienced multiple flows with high network utilization for a certain period of time can be identified as experiencing a microburst.