Recompressing Existing Compressed Data for Higher Storage Savings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems do not effectively achieve additional storage compression savings for already compressed data, limiting their ability to reduce storage consumption efficiently.
Innovation Solution
Implementing additional compression by decompressing data blocks using format information and then recompressing them with a heavier-weight algorithm at a storage layer, while retaining deduplication benefits and original compression format information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If lighter-weight compression is used at the application layer, then processing speed and ease of operation are improved, but storage compression savings are insufficient
Solution Approach 1:
The compression process is segmented into two distinct stages: first, light-weight compression at the application layer for fast processing, then heavy-weight compression at the storage layer for maximum compression savings. This segmentation allows each compression algorithm to operate optimally in its designated phase without interfering with the other.
Solution Approach 2:
The application layer performs preliminary compression before data is written to storage. This preliminary action reduces the data size initially, and then the storage layer performs additional compression on the already-compressed data, achieving cumulative compression effects while maintaining processing efficiency.
2Quantity of substance
If heavier-weight compression is used at the storage layer, then storage compression savings are improved, but processing time and complexity increase
Solution Approach 1:
By performing light-weight compression at the application layer before storage, the system prepares the data in advance, reducing the amount of data that needs to be processed later. This preliminary action allows the heavy-weight compression at the storage layer to work on already-reduced data, minimizing additional processing time.
Solution Approach 2:
The system applies compression periodically at different layers: application-layer compression occurs when data is prepared for storage, and storage-layer compression occurs when data is written to disk. This periodic application of compression at different stages allows heavy-weight algorithms to be used without continuously impacting processing performance.
3Quantity of substance
If additional compression is applied to already compressed data, then storage efficiency is improved, but deduplication effectiveness may be reduced
Solution Approach 1:
The system segments the compression process into application-layer and storage-layer stages, with deduplication operations performed on the original uncompressed or lightly-compressed data at the application layer before storage-layer compression is applied. This segmentation ensures that deduplication benefits are preserved while still achieving additional compression savings at the storage layer.
Data Source
AI summary
Techniques are provided for implementing additional compression for existing compressed data. Format information stored within a data block is evaluated to determine whether the data block is compressed or uncompressed. In response to the data block being compressed according to a first compression format, the data block is decompressed using the format information. The data block is compressed with one or more other data blocks to create compressed data having a second compression format different than the first compression format.