gNMI Telemetry Flow Control with Adaptive Sample Intervals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
gNMI services in network management can break down due to issues like server overload, high CPU utilization, and Denial-of-Service attacks, leading to data loss and service disruptions, especially when sample intervals are not adequately managed.
Innovation Solution
Implementing an adaptive flow control mechanism in gNMI that adjusts streaming data rates based on dynamic conditions, using time and iteration parameters to prevent data loss and ensure efficient data delivery across multiple network elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If regular streaming is performed without adaptive flow control, then data is sent at a constant rate, but server overload and CPU exhaustion occur leading to service breakdown
Solution Approach 1:
The system dynamically adjusts the sample interval based on server load conditions. When the server is healthy, it streams data at the configured rate; when overload is detected (CPU threshold exceeded), it increases the sample interval to reduce data rate, thereby adapting the streaming behavior to current system conditions and preventing service breakdown
Solution Approach 2:
The system implements feedback by continuously monitoring server health metrics (CPU utilization) and using this information to adjust the data streaming rate. The health check mechanism provides feedback about server status, which triggers adaptive adjustments to the sample interval, creating a closed-loop control system that maintains reliability under varying load
2Loss of information
If the sample interval is reduced to prevent data loss, then more frequent telemetry data is collected, but server CPU utilization increases leading to overload
Solution Approach 1:
The system uses dynamic sample interval adjustment based on real-time server health assessment. The sample interval is not fixed but adapts to server capacity: when CPU headroom is available, the interval is shorter (more data); when CPU is overloaded, the interval is extended (less data), optimizing the balance between data completeness and processing load
Solution Approach 2:
The system changes the sampling rate parameter adaptively based on server conditions. By modifying the sample interval parameter in response to CPU utilization thresholds, the system adjusts the trade-off between data collection frequency and processing burden, preventing both data loss and CPU exhaustion
3Reliability
If flow control is implemented to prevent server overload, then data loss is reduced, but the system complexity increases
Solution Approach 1:
The system performs self-service by automatically monitoring its own health status and adjusting its data streaming behavior without external intervention. The gNMI server autonomously evaluates its CPU load and modifies the sample interval accordingly, eliminating the need for complex external flow control mechanisms or manual configuration
Solution Approach 2:
The system uses simple feedback loops where health check results directly trigger sample interval adjustments. This feedback mechanism provides reliable flow control through straightforward conditional logic rather than complex algorithms, maintaining service continuity while minimizing added complexity
Data Source
AI summary
Systems and methods provide adaptive flow control of a gRPC Network Management Interface (gNMI)-based telemetry session at a gNMI server. A method includes receiving a telemetry subscription from a gNMI client for telemetry from the gNMI server; determining a sample interval for the telemetry subscription; and, based on the sample interval either performing (1) regular streaming via gNMI of the telemetry subscription or (2) adaptive streaming via gNMI of the telemetry subscription where the sample interval is periodically updated.