Inline Compression Control Using Per-Block Entropy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face inefficiencies due to all-or-none back-off features that either compress or skip data blocks without flexibility, compromising storage efficiency and throughput.
Innovation Solution
Implementing per-block entropy calculations to selectively compress data blocks based on their compressibility, allowing for flexible compression decisions that balance throughput and storage efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If inline compression is performed on all blocks, then storage efficiency is improved, but system throughput deteriorates due to increased processing load
Solution Approach 1:
The patent applies local quality by making compression decisions on a per-block basis rather than uniformly compressing all data. Each block is evaluated individually using entropy calculation to determine its compressibility, allowing the system to compress only those blocks that are suitable candidates while leaving others uncompressed, thus optimizing the local compression behavior for each specific data block.
Solution Approach 2:
The patent utilizes parameter changes by calculating entropy values for each data block to dynamically determine compression suitability. The entropy parameter serves as a metric to assess compressibility, and based on this parameter, the system adapts its compression behavior - compressing blocks with favorable entropy characteristics while skipping those that would not benefit from compression, thereby balancing storage efficiency and throughput.
2Productivity
If back-off feature shuts down inline compression to preserve throughput, then system throughput is improved, but storage efficiency deteriorates due to lost compression opportunities
Solution Approach 1:
The patent segments the data stream into individual blocks and evaluates each block independently for compression suitability using entropy calculation. This segmentation allows the system to make fine-grained compression decisions at the block level rather than applying a coarse all-or-nothing back-off approach, enabling selective compression that maintains throughput while preserving storage efficiency.
Solution Approach 2:
The patent implements partial action by performing compression only on a subset of blocks that meet certain entropy-based criteria rather than compressing all blocks or none. This partial compression approach allows the system to achieve meaningful storage efficiency improvements while avoiding the excessive processing load that would occur with universal compression, thus maintaining system throughput.
3Adaptability or versatility
If per-block entropy calculation is performed, then compression decision flexibility is improved, but computational complexity increases
Solution Approach 1:
The patent applies self-service by having each data block essentially evaluate itself through entropy calculation to determine its own compressibility characteristics. This self-assessment mechanism allows blocks to independently identify whether they are suitable for compression without requiring complex external analysis, providing adaptability while keeping the computational approach relatively simple and localized to each block.
Data Source
AI summary
A technique for managing data storage includes generating entropy of blocks on a per-block basis and selectively performing inline compression on blocks based at least in part on their entropy. Entropy of a block provides a rough measure of the block's compressibility. Thus, using per-block entropy enables a storage system to steer compression decisions, e.g., whether to compress and/or how much to compress, flexibly and with high granularity, striking a balance between throughput and storage efficiency.