Dual-Layer Storage Compression for Low-Latency I/O
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression methods in storage systems consume significant CPU resources and often result in inefficient compression ratios, particularly when using dual compression layers, as the primary compressor in the storage controller and the secondary compressor in the storage drive operate independently without optimal coordination.
Innovation Solution
Implementing a dual-layered data compression system where the primary compressor in the storage controller and the secondary compressor in the storage drive work concurrently, with the primary compressor achieving high compression ratios early in the write path and the secondary compressor providing low-latency compression, thereby improving overall compression performance and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If dual-layered compression is implemented with primary and secondary compressors operating independently, then compression capability is enhanced, but CPU resource consumption increases and compression efficiency deteriorates
Solution Approach 1:
The system dynamically selects which compressor (primary or secondary) to use based on real-time data characteristics and performance requirements. The storage controller can adaptively switch between compression layers, adjusting the compression strategy dynamically rather than using a fixed independent operation mode, thereby optimizing CPU resource utilization while maintaining compression capability.
Solution Approach 2:
The patent introduces a coordination mechanism that acts as an intermediary between the primary and secondary compressors. This mediator optimizes the interaction between the two compression layers, determining when to apply each compressor to avoid redundant CPU operations while maintaining effective compression. The intermediary coordinates the compression workflow to improve overall efficiency.
2Quantity of substance
If primary compressor processes data early in write path, then compression ratio is improved, but latency increases
Solution Approach 1:
The system performs preliminary assessment of data compressibility before initiating full compression processing. The storage controller evaluates data characteristics early in the write path and prepares compression parameters in advance, allowing the primary compressor to work more efficiently on pre-identified compressible data, thereby reducing overall latency while maintaining high compression ratios.
Solution Approach 2:
The patent applies partial compression action by using the primary compressor only when data exhibits high compressibility characteristics. For data that is already highly compressed or incompressible, the system skips or reduces primary compression processing, avoiding unnecessary CPU cycles and latency. This selective approach applies compression effort only where it provides maximum benefit.
3Productivity
If secondary compressor provides low-latency compression, then I/O throughput is improved, but overall compression ratio decreases
Solution Approach 1:
The compression process is segmented into two distinct stages with specialized functions. The primary compressor handles the initial compression pass for maximum ratio, while the secondary compressor provides a faster, lower-ratio compression pass for latency-sensitive operations. This segmentation allows each compressor to be optimized for its specific role, improving overall system throughput while maintaining effective compression through coordinated operation.
Data Source
AI summary
Embodiments for optimizing dual-layered data compression in a storage environment. In a data storage system having a primary compressor implemented in a storage controller and a secondary compressor implemented within a drive-enclosure, the primary compressor is selectively used to perform a first one of a plurality of actions on input/output (I/O) data while a second one of the plurality of actions is performed on the I/O data by the secondary compressor, thereby reducing latency and improving an overall compression performance while processing the I/O data.