Application-Specific Network Telemetry With In-Path Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a disconnect between network telemetry data and the specific applications causing the data transmission, making it difficult to correlate application behavior with network operation, especially in large-scale networks with dynamically changing data paths.
Innovation Solution
Insert telemetry metadata, including application and network information, into network data units between layer 4 and layer 3 headers at each hop, allowing real-time correlation of application-specific data flows with network telemetry.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If network telemetry data is collected and aggregated centrally, then network monitoring capability is improved, but the ability to correlate telemetry with specific applications deteriorates
Solution Approach 1:
The patent segments network telemetry data collection by inserting application-specific telemetry metadata into individual network data units at each hop, rather than aggregating all telemetry centrally. This segmentation preserves application correlation information within each data unit while enabling network monitoring.
Solution Approach 2:
The patent nests telemetry metadata within the network data unit structure itself, placing application-specific information inside the packet headers. This allows telemetry data to travel with the application data it describes, maintaining correlation without requiring separate aggregation processes.
2Loss of information
If telemetry metadata is inserted into network data units at each hop, then application-specific correlation is improved, but network overhead increases
Solution Approach 1:
The patent extracts only the essential application-specific telemetry metadata needed for correlation, rather than copying entire data units or excessive information. This extraction approach minimizes the added overhead while preserving the critical application correlation information.
Solution Approach 2:
The patent applies partial action by inserting telemetry metadata into only those network data units that require monitoring, rather than processing every packet uniformly. This selective approach reduces overall network overhead while maintaining application-specific correlation where needed.
3Area of stationary object
If centralized collection of network telemetry is used, then network-wide monitoring is improved, but post-processing complexity increases
Solution Approach 1:
The patent performs preliminary action by pre-correlating application information with network telemetry data at the point of data unit creation and transmission. This preliminary correlation eliminates the need for complex post-processing to match telemetry with applications, as the association is already established within each data unit.
Solution Approach 2:
The patent enables self-service by allowing each network data unit to carry its own application correlation information independently. This eliminates the need for a centralized system to perform complex matching operations, as each unit is self-contained with its own identification data.
4Loss of information
If application instrumentation is made aware of network devices, then telemetry correlation is improved, but system complexity increases
Solution Approach 1:
The patent introduces an intermediary mechanism in the form of standardized telemetry metadata fields that bridge application instrumentation and network devices. This intermediary layer enables correlation without requiring direct awareness or complex integration between applications and network infrastructure, as the metadata serves as a neutral mediator.
Data Source
AI summary
In certain embodiments, a method includes identifying, by a NIC of a node where application sender of an application executes, a network data unit for telemetry metadata insertion based on performing a match operation on information associated with the network data unit, wherein the network data unit is being transmitted from the application sender to an application receiver executing on another node; inserting, based on a first success of the first match operation, telemetry metadata into the network data unit, the telemetry metadata comprising network information and application specific information; stripping, by a last hop device along the data flow path, the telemetry metadata from the network data unit; generating, by the last hop device, a telemetry metadata report using the telemetry metadata; and providing the telemetry metadata report to an application collector executing on the second node.


