Inline Compression System for Storage Extent Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage systems experience suboptimal compression ratios due to wasted space at the end of each segment, as leftover space is not sufficient to accommodate another compressed extent, leading to inefficient use of storage capacity.
Innovation Solution
Implementing an inline compression system that buffers a stream of block writes in a cache until a certain number are received, then optimizes the packing of compressed data into segments with variable sizes to minimize wasted space, using techniques such as double buffering and distribution algorithms to find the most efficient container configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If data is compressed into segments with fixed size containers, then data organization is simplified, but wasted space increases and compression ratio decreases
Solution Approach 1:
The patent implements dynamic segment sizing where containers can accommodate variable numbers of compressed extents based on actual data needs. The system determines optimal container sizes by calculating total compressed data volume and distributing it across containers with flexible block counts, thereby minimizing wasted space while maintaining organizational simplicity.
Solution Approach 2:
The system changes the parameter of container size from fixed to variable. By allowing containers to have different numbers of blocks based on the compression ratio and total data volume, the system optimizes storage utilization. The patent calculates optimal block distribution by considering the compressed extent sizes and container capacities, adjusting parameters dynamically to reduce waste.
2Ease of manufacture
If fixed-size containers are used for compressed extents, then storage allocation is straightforward, but compression efficiency deteriorates due to insufficient space utilization
Solution Approach 1:
The patent performs preliminary calculations to determine optimal container configurations before actual data storage. The system calculates the total volume of compressed data, determines the number of containers needed, and pre-distributes data blocks across containers to maximize space utilization. This preliminary planning maintains allocation simplicity while improving compression efficiency.
Solution Approach 2:
The system transitions from static fixed-size containers to dynamic containers that adapt their size based on compression requirements. The patent implements a distribution algorithm that dynamically assigns varying numbers of blocks to different containers based on the actual compressed data volume, maintaining ease of allocation while significantly improving space utilization and compression efficiency.
Data Source
AI summary
A technique for efficiently storing compressed data of a storage object in a data storage includes (a) receiving, in a cache buffer, a number, U, of uncompressed blocks of a uniform size, the uncompressed data blocks received in write requests directed to the storage object; (b) compressing the uncompressed blocks of the cache buffer into respective compressed extents; (c) performing an optimization operation including generating a set of distributions of compressed extents among a plurality of containers and searching the set for a distribution having a minimal total amount of storage taken up by its respective plurality of containers, each container having a respective size equal to a respective integer multiple of the uniform size of the uncompressed data blocks; and (d) storing the compressed extents within a plurality of containers in persistent storage in accordance with the distribution having the minimal total amount of storage taken up by its respective plurality of containers.