Telemetry Data Snapshotting for Network Management Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network management systems face challenges in efficiently processing and storing telemetry data from numerous network devices, particularly due to the irregularity of event-driven data, which can lead to increased computational burden and storage requirements.
Innovation Solution
A controller device is configured to snapshot telemetry data differently for periodic and event-driven data, using intent-based graph models to preprocess and filter telemetry parameters, reducing the amount of data stored and processed, and generating implied data to ensure relevant information is stored based on intended network states.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If event-driven telemetry data is collected to reduce data volume, then storage requirements decrease, but the ability to determine state at specific times becomes difficult due to irregularity of events
Solution Approach 1:
The system performs preliminary actions by collecting periodic telemetry data at regular intervals and storing it in advance. This allows the system to later determine the complete state at any specific time by combining the periodic baseline data with event-driven change notifications, resolving the contradiction between reduced data volume and accurate state determination.
Solution Approach 2:
The telemetry data collection is segmented into two complementary approaches: periodic sampling for baseline state information and event-driven collection for changes. This segmentation allows the system to maintain accurate state determination while reducing overall data volume by only storing event information when changes occur.
2Reliability
If more devices stream telemetry data to increase monitoring coverage, then network visibility improves, but computational burden and bandwidth requirements increase
Solution Approach 1:
The system applies partial action by selectively collecting and storing only the necessary telemetry data - periodic baseline data and event-driven change notifications - rather than continuously streaming all data. This reduces computational burden while maintaining adequate monitoring coverage through strategic sampling and change detection.
Solution Approach 2:
The system extracts only the essential information from continuous monitoring - the periodic baseline states and the event-driven changes - separating this critical data from the redundant continuous stream. This extraction reduces bandwidth and computational requirements while preserving monitoring effectiveness.
3Loss of information
If all telemetry parameters are stored to ensure complete data retention, then information completeness improves, but storage requirements and processing load increase
Solution Approach 1:
The system dynamically adjusts what data is stored based on changes in state. Periodic data is stored as baseline, and event-driven data is stored only when changes occur. This dynamic approach ensures complete information retention for actual changes while avoiding storage of redundant unchanged data, reducing overall storage requirements.
Solution Approach 2:
The system changes the storage parameters based on data characteristics - storing periodic data at regular intervals and event-driven data only on changes. This parameter change strategy maintains information completeness for meaningful changes while significantly reducing storage requirements by excluding redundant unchanged data.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A controller device includes a memory and one or more processors coupled to the memory. The memory stores instructions that, when executed, cause the one or more processors to receive, from a set of sensor devices, first telemetry data indicating a first set of changes for telemetry parameters that occur during a first time range. The instructions further cause the one or more processors to determine, using the first snapshot and the first telemetry data, a second snapshot that specifies a first complete state at an end of the first time range. The instructions further cause the one or more processors to determine a second complete state of the telemetry parameters for the second time range based on the second snapshot and second telemetry data indicating a second set of changes for the set of telemetry parameters that occur during a second time range.