Inline Compression Thresholding for Small-Write Fragmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage systems employing inline compression face challenges such as increased fragmentation and resource over-commitment due to zero-padding and compression of small data writes, leading to inefficient storage utilization and increased disk traffic.
Innovation Solution
A method where the storage system determines if incoming data is below a predetermined threshold, opting to store it uncompressed if so, thereby avoiding compression and reducing fragmentation, and only compressing data that meets or exceeds the threshold, thus optimizing storage space and reducing computational burden.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If inline compression is performed on small data writes (below block size), then storage efficiency is improved through compression, but storage fragmentation increases and computational burden increases
Solution Approach 1:
The patent changes the parameter of data size threshold to determine whether to apply compression. By comparing the size of incoming data against a predetermined threshold (block size), the system dynamically adjusts the compression parameter - applying compression only when data size exceeds the threshold, and storing uncompressed when below the threshold. This resolves the contradiction by avoiding compression of small data writes that would create fragmentation, while still compressing larger writes that benefit from space savings.
2Ease of operation
If zero-padding is performed to reach block size before compression, then data conforms to file system block requirements, but creates holes and fragmentation when later writes arrive
Solution Approach 1:
The patent extracts the zero-padding operation from the compression process. Instead of padding all data to full block size before compression, the system only processes data that exceeds the threshold. This removes the harmful padding operation that created fragmentation while preserving the necessary block alignment for file system compatibility on actual data writes.
3Quantity of substance
If compression is applied to all incoming data, then storage space is optimized, but computational resources are over-committed and processing time increases
Solution Approach 1:
The patent applies partial compression action by selectively compressing only those data writes that exceed the predetermined threshold size. This partial application of compression avoids the excessive computational burden of compressing every write operation, including small ones that would not benefit from compression, while still achieving storage space optimization on larger writes where compression provides meaningful benefits.
Data Source
AI summary
A technique for performing writes in a storage system that supports data compression tests incoming writes to determine whether the amount of data to be written is less than a predetermined threshold. If so, the storage system avoids compression and stores the amount of data as received without compressing it. If not, the storage system performs compression and stores the incoming data as a compressed extent.