Adaptive Compression of Systems Management Time Series Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods for systems management data in enterprise computing environments are inefficient due to the dynamic nature of the data, leading to excessive loss of detail and impractical manual configuration of compression settings, especially when dealing with diverse and complex systems.
Innovation Solution
A dynamic compression system that adaptively selects appropriate compression algorithms and settings based on the characteristics of incoming metric data, using a pre-compressor to determine constraints and pass them to a compressor to minimize storage requirements while maintaining data accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If lossless compression is used for time series data, then data accuracy is preserved, but storage overhead increases and compression efficiency decreases
Solution Approach 1:
The patent dynamically adjusts compression parameters including deadband values, compression ratios, and algorithm selection based on the specific characteristics of each time series metric (e.g., CPU utilization patterns, disk usage trends, memory consumption rates). This allows the system to optimize between accuracy and compression efficiency for different data types without applying a uniform approach.
Solution Approach 2:
The patent applies different compression strategies to different metrics based on their individual characteristics. Each system metric (CPU, disk, memory, network) receives a customized compression approach tailored to its specific patterns, ranges, and importance, rather than using a single compression method for all data.
2Quantity of substance
If lossy compression is used to reduce storage requirements, then data volume is reduced, but potentially valuable information is lost
Solution Approach 1:
The patent implements dynamic compression where compression parameters are adjusted in real-time based on the current state and characteristics of the time series data. The system adapts deadband values and compression ratios as metrics change over time, ensuring that information loss is minimized when it matters most while achieving high compression during stable periods.
Solution Approach 2:
The system incorporates feedback mechanisms that monitor compression effectiveness and data quality, allowing it to adjust compression parameters to maintain acceptable information quality while maximizing compression ratios. The feedback loop ensures that compression settings are continuously optimized based on actual data patterns.
3Quantity of substance
If geometric averaging is used to reduce data granularity, then storage space is reduced, but detail of original data is lost
Solution Approach 1:
The patent segments time series data into different compression zones or bands based on value ranges and patterns. Instead of uniformly reducing granularity across all data, the system applies different compression levels to different segments of the data, preserving detail in critical ranges while compressing less important portions.
4Productivity
If manual configuration of compression settings is required for each metric, then compression can be optimized, but system complexity and configuration effort increase
Solution Approach 1:
The patent implements self-configuring compression where the system automatically analyzes each metric's characteristics and determines optimal compression parameters without manual intervention. The compression algorithm itself performs the configuration work by learning from data patterns, eliminating the need for users to manually set deadbands or select compression methods for each metric.
Solution Approach 2:
The patent creates a universal compression framework that handles diverse time series metrics through a single automated system. This multi-functional approach allows the same compression engine to adapt to different data types (CPU, disk, memory, network metrics) using automatic parameter detection, replacing the need for metric-specific manual configuration.
5Ease of operation
If a single compression algorithm is applied to all metrics, then system simplicity is maintained, but compression effectiveness decreases for diverse data types
Solution Approach 1:
The patent implements a universal compression system that automatically adapts to different data types through a single multi-functional algorithm. The system detects metric characteristics and dynamically selects appropriate compression strategies, providing effective compression for diverse metrics (CPU, disk, memory, network) without requiring separate manual configurations for each type.
Solution Approach 2:
The patent employs dynamic algorithm selection where the compression approach changes based on the current metric being processed. The system dynamically adjusts compression parameters and may switch between different compression techniques depending on the data patterns observed, maintaining simplicity while achieving effectiveness across diverse data types.
Data Source
AI summary
A method, system, and medium for compressing systems management information in a historical data store. Dynamically determining the appropriate compression algorithm to apply based on the type of data being compressed and stored. As further input is received for any particular measurement, the appropriate compression algorithm will be automatically selected from the set of available compression algorithms or be defined by a user configuration parameter. The amount of historical data stored with the minimal amount of data loss is optimized by the system dynamically changing the compression algorithm used for the given input data over a particular time span. The system engineer is therefore presented with the pertinent information for monitoring, administrating and diagnosing system activities.