Adaptive Telemetry Selective Summarization Metric Reporting

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud computing platforms face issues with monitoring service uptime due to overload when clients are unable to timely upload log data, leading to delayed metrics and potential server crashes, causing visibility problems and extended outages.

Innovation Solution

Implementing adaptive telemetry using cooperative selective summarization metric reporting, where clients switch to transmitting smaller summarization data instead of full raw log data, and switching from a first-in first-out to a last-in first-out transmission mode to prioritize recent log data, allowing the monitoring service to maintain functionality and recover from overload.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If clients transmit full raw log data to the monitoring service, then measurement precision and visibility are improved, but the monitoring service becomes overloaded and uptime deteriorates

Engineering Contradiction:

Improvelog data completenessVSAvoidmonitoring service uptime

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The system dynamically adapts the data transmission mode based on monitoring service health status. Clients switch between raw log data transmission and summarization mode depending on whether the service is overloaded, allowing the system to balance data completeness with service reliability in real-time

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the data format parameter from full raw logs to summarized metrics when the monitoring service detects overload conditions. This parameter change reduces the amount of data transmitted while maintaining essential monitoring functionality, resolving the contradiction between data completeness and service reliability

Inventive Principle:

Principle #35Parameter changes

2Measurement precision

If clients transmit full raw log data continuously, then measurement precision is maintained, but the load on the monitoring service increases causing server crashes

Engineering Contradiction:

Improvemetric accuracyVSAvoidserver overload

Core Design Contradiction:

Measurement precisionVSObject-generated harmful factors

Solution Approach 1:

The system extracts only the essential summary statistics from full log data when transmitting under constrained conditions. By taking out only the critical metrics (count, sum, min, max, avg) rather than transmitting complete raw logs, the system maintains measurement precision for monitoring while reducing the harmful load on servers

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of transmitting complete log data (excessive action), the system transmits partial summarized statistics that are sufficient for monitoring operations. This partial action approach provides just enough information for metric calculation without the excessive data volume that causes server overload

Inventive Principle:

Principle #16Partial or excessive action

3Measurement precision

If the monitoring service processes all incoming log data, then measurement precision is maintained, but productivity decreases due to processing delays

Engineering Contradiction:

Improvedata processing completenessVSAvoiddata processing speed

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system performs partial processing by accepting only summarized data during overload conditions rather than processing all incoming log data. This maintains essential monitoring productivity while avoiding the processing delays that would result from attempting to process complete raw log datasets

Inventive Principle:

Principle #16Partial or excessive action

4Reliability

If clients switch to summarization mode during overload, then monitoring service reliability is improved, but loss of information occurs

Engineering Contradiction:

Improveservice availabilityVSAvoidraw data loss

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

Clients perform preliminary summarization of log data before transmission during overload conditions. By pre-computing summary statistics locally, clients enable the monitoring service to receive essential information without requiring full raw data, thus maintaining service reliability while minimizing information loss through the use of aggregated metrics

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11652755B1Adaptive telemetry using cooperative selective summarization metric reporting

Publication Date: 2023.05.16 AMAZON TECH INC

AI summary

Techniques for adaptive telemetry using cooperative selective summarization metric reporting are described. Upon an impairment event preventing a client from transmitting raw log data to a server, the client switches to generating summarization data based on the raw log data and sends this summarization data to the server. The server may instruct the client which dimensions of data are to be generated and which time apertures apply for the generation of summary statistics. The client may also switch from a first-in first-out (FIFO) transmission mode to a last-in first-out (LIFO) transmission mode for eventually resending the raw log data, providing the server the most current information as timely as possible while leaving older data for later transmission.