Intelligent Application Trace Sampling for Storage Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional sampling procedures for application traces in distributed computing systems fail to distinguish between different types of traces, leading to the missed capture of infrequently generated traces that are crucial for performance troubleshooting, resulting in inaccurate representation and inefficient storage management.
Innovation Solution
Implementing intelligent sampling methods that sort application traces by type and duration, adjusting sampling rates accordingly to ensure sufficient capture of low-frequency traces, thereby reducing storage needs while maintaining accurate troubleshooting capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional sampling procedures are used to store a fraction of traces, then storage volume is reduced, but infrequently generated traces are missed and troubleshooting becomes impossible
Solution Approach 1:
The patent segments traces into different categories based on frequency (high-frequency and low-frequency traces) and applies different sampling rates to each segment. This allows the system to store fewer high-frequency traces while capturing enough low-frequency traces for effective troubleshooting, resolving the contradiction between storage volume and troubleshooting capability.
Solution Approach 2:
The patent applies different sampling rates to different types of traces based on their local characteristics. Low-frequency traces receive higher sampling rates while high-frequency traces receive lower sampling rates, optimizing the balance between storage efficiency and the ability to detect performance problems in any trace type.
2Quantity of substance
If conventional sampling procedures are used, then storage costs are reduced, but the representation of trace types becomes inaccurate
Solution Approach 1:
The patent divides traces into segments based on frequency characteristics and applies differentiated sampling strategies to each segment. This ensures that the sampled traces maintain an accurate representation of the original trace distribution, preventing the loss of information about rare but important trace patterns.
Solution Approach 2:
The patent changes the sampling rate parameter dynamically based on trace frequency characteristics. By adjusting the sampling rate parameter according to whether a trace is high-frequency or low-frequency, the system maintains measurement precision while reducing the total number of stored traces.
3Reliability
If all traces are stored for comprehensive analysis, then troubleshooting accuracy is improved, but storage requirements and operational costs increase enormously
Solution Approach 1:
The patent segments the trace population by frequency and applies selective storage based on segment characteristics. Low-frequency traces, which are critical for troubleshooting, are stored at higher rates, while high-frequency traces are stored at lower rates, achieving comprehensive analysis capability with reduced storage requirements.
Solution Approach 2:
The patent applies partial sampling to high-frequency traces (storing only a portion) while applying more extensive sampling to low-frequency traces (storing a larger portion or all traces). This partial/excessive action strategy ensures that storage requirements are reduced overall while maintaining sufficient data for accurate troubleshooting.
Data Source
AI summary
Computer-implemented methods and systems described herein perform intelligent sampling of application traces generated by an application. Computer-implemented methods and systems determine different sampling rates based on frequency of occurrence of trace types and/or frequency of occurrence of durations of the traces. Each sampling rate corresponds to a different trace type and/or different duration. The sampling rates for low frequency trace types and durations are larger than the sampling rates for high frequency trace types and durations. The relatively larger sampling rates for low frequency trace types and low frequency durations ensures that low frequency trace types and low frequency durations are sampled in sufficient numbers and are not passed over during sampling of the application traces. The set of sampled traces are stored in a data storage device.