Streaming Telemetry Buffering for Predictive Network Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional pull-based monitoring techniques like SNMP struggle to keep pace with the dynamic nature of modern datacenter networks, leading to gaps in visibility and delayed responses to network conditions, which are inadequate for managing high-bandwidth, high-speed data processing tasks in datacenters.
Innovation Solution
Implementing a telemetry-data monitoring system that includes a cyclic buffer for recording data, detecting anomalies, and using a neural network to predict future issues, with adjustable sampling rates based on trigger conditions, enabling proactive management and optimization of network performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional pull-based monitoring techniques like SNMP are used, then device complexity is reduced and ease of operation is maintained, but measurement precision and response time to network conditions deteriorate
Solution Approach 1:
The patent inverts the traditional monitoring approach by switching from pull-based (SNMP polling) to push-based (streaming telemetry) architecture. Network devices actively push telemetry data to collectors instead of waiting for polling requests, fundamentally reversing the interaction model to achieve continuous real-time visibility without proportional increase in system complexity
Solution Approach 2:
The patent replaces the mechanical polling mechanism with an event-driven streaming telemetry system. Instead of periodic mechanical requests and responses, the system uses continuous data streams pushed by network devices, substituting the old mechanical interaction pattern with a more efficient event-based architecture that provides superior measurement precision
2Speed
If streaming telemetry is implemented for real-time monitoring, then response time and measurement precision improve, but data volume and processing requirements increase
Solution Approach 1:
The patent merges multiple telemetry data streams from various network devices into a unified streaming telemetry system. By consolidating data collection, normalization, and processing into a single architectural framework, the system handles real-time data from multiple sources efficiently without proportionally increasing processing overhead, achieving fast response times while managing data volume through unified processing
Solution Approach 2:
The patent implements periodic sampling of telemetry data at optimized intervals. Instead of continuously streaming all possible data points, the system samples telemetry information at strategically determined periods, maintaining real-time monitoring capabilities while significantly reducing the total volume of telemetry data that needs to be processed and stored
3Measurement precision
If sampling rate is increased to detect future anomalies, then prediction accuracy improves, but use of energy and processing power increase
Solution Approach 1:
The patent implements dynamic sampling rate adjustment based on network conditions and anomaly detection needs. The sampling rate is not fixed but adapts dynamically - increasing when anomalies are detected or predicted, and decreasing during normal operation. This dynamic approach maintains high prediction accuracy when needed while minimizing energy consumption during stable periods
Solution Approach 2:
The patent changes the sampling rate parameter dynamically based on detected patterns and anomaly likelihood. By adjusting this key parameter rather than maintaining a constant high sampling rate, the system achieves high prediction accuracy only when necessary, significantly reducing overall processing energy consumption while maintaining the capability for precise anomaly detection when triggered
Data Source
AI summary
A system for predicting and/or capturing data relating to anomalies in a networking device is provided. In one example, a networking device receives telemetry data, stores the telemetry data in a cyclic buffer, detects an anomaly, and outputs the telemetry data from the cyclic buffer. The telemetry data from the cyclic buffer may be used for training a prediction model. In another example, a trained prediction model analyzes telemetry data sampled at a first rate, predicts a future anomaly, and in response to the prediction of the future anomaly, triggers sampling of the telemetry at a second rate, faster than the first rate.