Adaptive Telemetry Sampling Rate Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Software products generating large volumes of telemetry data face challenges in reducing data volume without sacrificing visibility into low-volume events, which is typically achieved through sampling but at the cost of losing insights into lightly used features or small user segments.
Innovation Solution
An adaptive, stratified sampling system that calculates optimized sampling rates for different events based on their volume and precision needs, allowing high-volume events to be sampled at lower rates while maintaining higher sampling rates for low-volume events, thus reducing overall telemetry volume without losing critical event tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If broad-based sampling is applied to reduce telemetry volume, then the overall telemetry volume is reduced to a reasonable level, but visibility into low-volume events such as those generated by lightly used features or small segments of the overall population is lost
Solution Approach 1:
The patent applies different sampling rates to different events based on their individual characteristics. High-volume events receive lower sampling rates (e.g., 1%) while low-volume events receive higher sampling rates (e.g., 100%), allowing each event type to be sampled with appropriate quality rather than applying a uniform sampling rate across all events
Solution Approach 2:
The system dynamically adjusts the sampling rate parameter for each event based on its volume characteristics. The volume calculation service computes optimized sampling rates by analyzing event volumes and determining appropriate sampling percentages, then applies these varying parameters to different events to resolve the contradiction between overall volume reduction and low-volume event visibility
Data Source
AI summary
A system for optimizing telemetry volume generated for an application product includes: a volume calculation service and a configuration service coupled to a client device via a network connection. The volume calculation service receiving identification of a particular event that occurs during execution of the application product by the client and calculating an optimized sampling rate for that event in telemetry produced by the client, the optimized sampling rate calculated based on reducing an overall volume of telemetry while still maintaining tracking of the particular event within the telemetry. The configuration service generating a configuration for the client device, the configuration service configuring the client to use the optimized sampling rate to produce telemetry for the event during execution of the application product by the client device.