Compression Tier Selection Using Collected Data Reduction Hints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face challenges in efficiently reducing data size through compression, as existing methods lack a systematic approach to select the most effective compression algorithms for varying data sets, leading to suboptimal compression ratios and resource utilization.
Innovation Solution
The method involves determining and utilizing multiple tiers of compression algorithms based on collected compression information, recommending the use of specific hardware devices for each tier to achieve desired compression ratios, and licensing considerations to optimize data reduction across different data sets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single compression algorithm is used for all data sets, then device complexity is reduced, but compression ratio performance deteriorates
Solution Approach 1:
The patent segments compression algorithms into multiple tiers (first tier with lower complexity algorithms, second tier with higher complexity algorithms that achieve better compression ratios). The system collects compression information for different data sets and determines which tier each algorithm belongs to, allowing selective application based on data characteristics rather than using a single algorithm for all data.
Solution Approach 2:
The system dynamically selects compression algorithms based on collected compression information from I/O operations. The compression tier determination is not static but adapts based on actual performance data, allowing the system to optimize compression ratio for each data set while managing device complexity through intelligent selection rather than fixed configuration.
2Manufacturing precision
If multiple tiers of compression algorithms are implemented, then compression ratio performance is improved, but device complexity increases
Solution Approach 1:
The system performs preliminary actions by collecting compression information for multiple compression algorithms across different data sets before making tier determination decisions. This advance data collection and analysis allows the system to pre-determine which algorithms belong to which tiers, reducing the complexity of real-time decision-making while maintaining high compression ratio performance.
Solution Approach 2:
The patent introduces an intermediary compression information collection mechanism that mediates between the multiple compression algorithms and the tier determination process. This intermediary layer gathers performance data, enables systematic analysis, and facilitates intelligent algorithm selection without requiring direct complex interactions between all algorithms and data sets, thereby managing system complexity.
3Measurement precision
If compression information is collected from multiple data storage systems, then algorithm selection accuracy is improved, but information collection overhead increases
Solution Approach 1:
The system implements a universal compression information collection approach that gathers data from multiple data storage systems simultaneously. The collected compression information serves multiple purposes: determining compression tiers, selecting appropriate algorithms, and optimizing performance across different systems. This multi-functional use of collected information maximizes the value of the time investment while improving selection accuracy through broader data coverage.
Data Source
AI summary
Tiers of compression algorithms may be determined using compression information collected regarding compression ratios achieved for data sets using compression algorithms. Each tier may meet specified criteria regarding expected compression ratios achieved for a specified portion or number of data sets. Compression algorithms of each tier may be implemented by a different hardware device that may include hardware accelerators for the algorithms of the tier. Different tiers, and thus different hardware devices, achieve different levels of compression. A recommendation may be provided using compression information collected, such as from one of the hosts, regarding which hardware device to use for compression. The recommendation may be to purchase a license to use or whether to purchase a particular hardware device for compression. Compression information may be collected by a host that issues tagged I/Os providing a hint regarding what compression algorithm to use for the particular I/O operation data.