Adaptive Network Telemetry Sampling via Stream Processing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional uniform sampling techniques for network telemetry data collection are inadequate as they may miss unusual events during non-sampled intervals, lead to resource exhaustion with high sampling rates, and result in undersampling of important events.

Innovation Solution

The integration of non-uniform sampling with stream processing, which selectively collects data based on network conditions, allowing for adaptive data collection, in-depth analysis, real-time alerting, and automated responses.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If uniform sampling is used for data collection, then implementation is simple, but unusual events may be missed during non-sampled intervals

Engineering Contradiction:
Improveimplementation simplicityVSAvoidevent detection reliability
Core Design Contradiction:
Ease of operationVSReliability

Solution Approach 1:

The patent applies dynamics by transitioning from static uniform sampling to dynamic non-uniform sampling. The sampling rate is dynamically adjusted based on network conditions and event priority, allowing the system to collect data more frequently during critical periods and less frequently during normal operation, thereby resolving the contradiction between implementation simplicity and event detection reliability

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent changes the sampling rate parameter from a fixed uniform value to a variable non-uniform value. By modifying the sampling rate parameter based on detected event types and network conditions, the system achieves both ease of implementation through a straightforward sampling framework and improved reliability through adaptive event detection

Inventive Principle:
Principle #35Parameter changes

2Measurement precision

If high sampling rate is used, then event detection accuracy improves, but resource exhaustion occurs

Engineering Contradiction:
Improveevent detection accuracyVSAvoidresource consumption
Core Design Contradiction:
Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent applies local quality by differentiating sampling rates for different types of network events. Instead of using a uniform high sampling rate for all events, the system uses higher sampling rates only for critical events (such as security incidents or outages) and lower sampling rates for routine events, thereby improving event detection accuracy for important events while reducing overall resource consumption

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent applies partial action by selectively applying high sampling rates only when and where needed. Rather than continuously using maximum sampling rate, the system uses partial sampling (higher rate) only for critical event detection, and standard or reduced sampling for non-critical events, achieving adequate measurement precision for important events while avoiding resource exhaustion

Inventive Principle:
Principle #16Partial or excessive action

3Ease of manufacture

If uniform sampling is used, then data collection is straightforward, but important events are undersampled

Engineering Contradiction:
Improvedata collection simplicityVSAvoidimportant event coverage
Core Design Contradiction:
Ease of manufactureVSQuantity of substance

Solution Approach 1:

The patent changes the sampling rate parameter dynamically based on event priority and type. By modifying the sampling rate parameter in response to detected event characteristics, the system maintains data collection simplicity through a unified sampling framework while ensuring important events receive adequate sampling through parameter adaptation

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS20250150368A1System and method for network telemetry data analysis with stream processing and non-uniform sampling
Publication Date: 2025.05.08 AVIZ NETWORKS INC
  • US20250150368A1 patent drawing
  • US20250150368A1 patent drawing
  • US20250150368A1 patent drawing

AI summary

In some implementations, the method may include providing one or more collectors which periodically request memory utilization from a device. In addition, the method may include receiving, by a stream processor, memory utilization from the one or more collectors. The method may include monitoring, by the stream processor, if an average memory utilization evaluated over a predetermined time crosses a predetermined threshold (for example 5 minutes). Moreover, the method may include sending data downstream to a data sink for persistence. Also, the method may include sending a new sampling strategy to the one or more collectors, if the average memory utilization evaluated over the predetermined time crosses the predetermined threshold.