NVMeOF Disk Array Cold-Data Recompression During Garbage Collection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems face inefficiencies in data compression due to suboptimal compression algorithms and bandwidth limitations, particularly when trying to improve compression levels in background processes, leading to reduced data reduction ratios and increased read/write bandwidth over the fabric.
Innovation Solution
A Garbage Collection (GC) process identifies storage containers with cold data and determines whether a first compression technique offers a higher level of compression than a second technique, using the more effective method to recompress data, thereby improving data reduction ratios without consuming host CPU cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If a higher compression level is applied to storage containers, then data reduction ratio is improved, but read/write bandwidth over the fabric increases
Solution Approach 1:
The system segments storage containers into hot and cold data portions, applying different compression strategies to each segment. Cold data portions are recompressed with higher compression levels during garbage collection operations, while hot data maintains lower compression levels for faster access, thus improving overall data reduction without proportionally increasing bandwidth usage.
Solution Approach 2:
The system dynamically changes compression parameters based on data characteristics and access patterns. By adjusting compression levels for different storage containers and data portions, the system optimizes the balance between data reduction ratio and bandwidth consumption, applying higher compression only where beneficial.
2Loss of substance
If compression techniques are processed in the background to improve compression levels, then data reduction is enhanced, but host CPU cycles are consumed
Solution Approach 1:
The Smart DAE performs compression operations autonomously using its own compute resources during garbage collection, without requiring host CPU intervention. The system self-manages the recompression of cold data portions, eliminating the need for host CPU cycles while still achieving enhanced data reduction.
Solution Approach 2:
The system performs compression operations as part of the garbage collection process, which naturally occurs during periods of lower system activity. By preliminary identifying cold data portions and scheduling compression during GC operations, the system enhances data reduction without impacting host performance.
3Device complexity
If fixed-size page granularity is used for compression, then storage organization is simplified, but compression efficiency is reduced
Solution Approach 1:
The system segments storage containers into fixed-size pages for organizational simplicity while further segmenting these pages into hot and cold data portions for differential compression. This hierarchical segmentation allows the system to maintain simple fixed-size page structures while applying more efficient compression strategies to specific portions of data based on access patterns.
Data Source
AI summary
A method, computer program product, and computer system for identifying, by a computing device, storage containers that contain cold data. At least a portion of the storage containers may be processed to determine whether a first compression technique will result in a higher level of compression above a threshold level of compression than a second compression technique. The storage containers may be processed using the first compression technique based upon, at least in part, determining that the first compression technique will result in the higher level of compression above the threshold level of compression than the second compression technique.