Activity-Based Compression Tiering for Storage Cost and Compute Balance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data storage systems employ a single compression algorithm, leading to suboptimal results in terms of compute time and compression ratio, as different algorithms are better suited for different applications and data types, and there is a need to optimize data storage by considering the level of activity and available resources.
Innovation Solution
A data storage system that evaluates the level of activity of a data set and selects an appropriate compression algorithm, allowing for dynamic compression and recompression based on activity levels and system resources, such as using multiple compression algorithms and adjusting them according to changing access patterns and resource availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single compression algorithm is used, then device complexity is reduced, but compression ratio and compute time performance deteriorate
Solution Approach 1:
The system dynamically selects compression algorithms based on data activity levels rather than using a static single algorithm. The storage system transitions from a fixed compression approach to a dynamic one where the compression algorithm changes according to the activity level of the data set, optimizing both compression ratio and compute time performance.
Solution Approach 2:
The system changes the parameter of compression algorithm selection based on data activity levels. By monitoring and evaluating data activity, the system adjusts which compression algorithm is applied, transforming the compression parameter from a fixed value to a variable that adapts to data characteristics.
2Quantity of substance
If data is compressed with high compression ratio, then storage space is reduced, but compute time increases
Solution Approach 1:
The system adjusts compression parameters based on data activity levels. For inactive data, higher compression ratios are applied to maximize storage space efficiency. For active data, lower compression ratios are used to minimize compute time overhead, thus optimizing the trade-off between storage space and compute time.
Solution Approach 2:
The compression ratio is made dynamic rather than fixed. The system adapts the compression level according to data activity, transitioning from a static compression approach to a dynamic one that optimizes the balance between storage efficiency and processing performance.
3Quantity of substance
If compression is applied to all data, then storage costs are reduced, but system resources are consumed
Solution Approach 1:
The system applies different compression strategies to different data sets based on their activity levels rather than uniformly compressing all data. Inactive data sets receive aggressive compression to maximize storage capacity, while active data sets use lighter compression or none at all to conserve system resources, thus optimizing the balance between storage capacity and resource consumption.
Solution Approach 2:
Instead of applying full compression to all data, the system applies partial compression only where necessary (inactive data). This selective approach reduces unnecessary system resource consumption while still achieving storage cost reduction for the portion of data that benefits most from compression.
4Speed
If flash disks are used instead of ATA or SATA disks, then data retrieval speed is improved, but storage costs increase
Solution Approach 1:
The system changes the parameter of data activity level to determine storage placement. Inactive data is compressed and stored on cheaper ATA or SATA disks, while active data is kept on faster flash disks. This dynamic parameter-based approach optimizes the balance between data retrieval speed and storage cost.
Solution Approach 2:
The system applies different storage media qualities to different data sets based on activity levels. High-performance flash storage is allocated to active data where speed is critical, while lower-cost ATA/SATA storage is used for inactive data where compression compensates for slower speeds, thus optimizing the local quality of storage resources.
Data Source
AI summary
Example embodiments of the present invention relate to methods, systems, and a computer program product for storing data compressed according to a level of activity of a data set. The method includes evaluating a level of activity for a data set and selecting a compression algorithm according to the level of activity of the data set. The data set then may be compressed according to the selected compression algorithm and the compressed data stored in a data storage system.