Storage-Layer Recompression for Already Compressed Data Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems do not effectively achieve additional storage compression savings for already compressed data, as they often rely on lighter-weight compression algorithms that provide limited storage efficiency.
Innovation Solution
Implementing a method where compressed data is decompressed and then recompressed using a heavier-weight compression algorithm at a storage layer, leveraging format information to maintain compatibility and deduplication efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If lighter-weight compression algorithms are used at the application layer, then processing speed and ease of operation are improved, but storage compression efficiency deteriorates
Solution Approach 1:
The compression process is segmented into two distinct layers: application layer compression using lighter-weight algorithms for fast processing, and storage layer compression using heavier-weight algorithms for maximum compression. This segmentation allows each layer to optimize for its specific function without compromising the other.
Solution Approach 2:
The patent introduces an intermediary storage layer that receives compressed data from the application layer and applies additional compression before final storage. This intermediary layer acts as a bridge between the speed-optimized application layer and the efficiency-optimized final storage, enabling both requirements to be met simultaneously.
2Quantity of substance
If heavier-weight compression algorithms are used at the storage layer, then storage compression efficiency is improved, but processing complexity and time consumption increase
Solution Approach 1:
The application layer performs preliminary compression using lighter-weight algorithms before data reaches the storage layer. This preliminary action reduces the amount of data that needs to be processed by the heavier-weight storage layer algorithms, thereby reducing overall processing complexity and time consumption while maintaining high compression efficiency.
3Quantity of substance
If additional compression is applied to already compressed data, then storage consumption is reduced, but deduplication efficiency may be impacted
Solution Approach 1:
The patent applies different compression qualities to different layers of data storage. The application layer uses lighter compression suitable for maintaining data integrity and deduplication efficiency, while the storage layer applies heavier compression optimized for reducing storage consumption. This local quality differentiation ensures that deduplication operations can still function effectively on the first layer while achieving maximum compression in the second layer.
Data Source
AI summary
Techniques are provided for implementing additional compression for existing compressed data. Format information stored within a data block is evaluated to determine whether the data block is compressed or uncompressed. In response to the data block being compressed according to a first compression format, the data block is decompressed using the format information. The data block is compressed with one or more other data blocks to create compressed data having a second compression format different than the first compression format.