Stream Data Tiering Compression Guided by Storage SLA Objectives
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression technologies lack adaptability and efficiency when moving stream data across various data storage tiers with different Service Level Agreements (SLAs), as they typically rely on static compression algorithms and do not account for dynamic compression objectives across multiple tiers.
Innovation Solution
A context-aware data compression system that dynamically selects lossless compression algorithms based on SLA requirements across different storage tiers, using a server-side compression module with a pool of compressors, which adds or replaces compression metadata to ensure data is compressed and decompressed according to the target tier's objectives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If static compression algorithms are used, then device complexity is reduced, but adaptability to different SLA requirements across storage tiers deteriorates
Solution Approach 1:
The patent implements dynamic compression by maintaining a pool of multiple compression algorithms and dynamically selecting the appropriate algorithm based on the SLA requirements of the target storage tier. The system transitions from static to dynamic compression algorithm selection, allowing adaptability across different storage tiers while managing complexity through automated selection mechanisms.
Solution Approach 2:
The system changes the parameter of compression algorithm selection based on SLA requirements. Different compression algorithms are chosen depending on the specific SLA parameters (compression ratio, speed, resource usage) required by each storage tier, enabling the system to adapt to varying requirements without increasing overall complexity.
2Adaptability or versatility
If multiple compression algorithms are maintained in a pool, then adaptability to different SLA requirements is improved, but device complexity increases
Solution Approach 1:
The compression system performs self-service by automatically selecting the appropriate compression algorithm from the pool based on SLA requirements. The system autonomously evaluates the needs of different storage tiers and chooses the most suitable algorithm without external intervention, managing the complexity of having multiple algorithms through automated decision-making.
Solution Approach 2:
The patent introduces an intermediary mechanism (the compression module with algorithm pool) that mediates between the data compression needs and the SLA requirements. This intermediary manages the complexity of multiple algorithms by providing a unified interface for selection and application, hiding the underlying complexity from the rest of the system.
3Productivity
If compression is applied at server side with metadata management, then data management efficiency is improved, but processing time increases
Solution Approach 1:
The system performs preliminary actions by pre-evaluating and selecting compression algorithms based on SLA requirements before actual data compression. The compression module prepares the appropriate algorithm in advance, and metadata about compression parameters is managed beforehand, reducing the time penalty during actual compression operations.
Solution Approach 2:
The patent implements feedback mechanisms where the system monitors compression performance and adjusts algorithm selection based on observed results. This feedback loop optimizes the balance between data management efficiency and processing time by learning from past compression operations and improving future selections.
Data Source
AI summary
A method includes identifying, by a server module running at a stream data platform, a set of data to be moved to a target data storage tier of the stream data platform, querying, by the server module, service level agreement (SLA) objectives of the target data storage tier, fetching, by the server module, the set of data, selecting, by the server module, a compressor based on the SLA objectives, compressing the set of data using the selected compressor, and storing the set of data in the target data storage tier.