Recompressing Existing Data Blocks for Higher Storage Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems do not effectively achieve additional storage compression savings for already compressed data, limiting their ability to reduce storage consumption efficiently.
Innovation Solution
Implementing a method where compressed data is decompressed and then recompressed using a heavier-weight compression algorithm at a storage layer, leveraging format information to maintain deduplication savings and manage data blocks effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a lighter-weight compression algorithm is used at the application or file level, then processing speed and ease of operation are improved, but storage compression savings are insufficient
Solution Approach 1:
The compression process is divided into two segments: a first compression algorithm applied at the application or file level for quick processing, and a second heavier-weight compression algorithm applied at the container level for maximum compression savings. This segmentation allows each algorithm to operate in its optimal domain without compromising overall performance.
Solution Approach 2:
The first lighter-weight compression algorithm is applied in advance before the data is written to storage. This preliminary compression reduces the data size before the heavier-weight second compression algorithm is applied during container creation, improving overall efficiency while achieving maximum compression savings.
2Quantity of substance
If a heavier-weight compression algorithm is used, then storage compression savings are improved, but processing time and computational resources increase
Solution Approach 1:
The heavier-weight second compression algorithm is applied during container creation rather than during every read/write operation. This preliminary application of heavy compression during the container setup phase achieves maximum compression savings without impacting the speed of subsequent data operations.
Solution Approach 2:
The compression workload is segmented between two algorithms: the lighter-weight first algorithm handles real-time processing needs, while the heavier-weight second algorithm handles batch processing during container creation. This segmentation distributes the computational burden over time, reducing peak processing demands.
3Reliability
If compressed data blocks are modified or accessed, then data management functionality is maintained, but additional compression operations may impact performance
Solution Approach 1:
The patent introduces a new dimension of compression by applying a second compression algorithm at the container level, which is a different organizational layer than the traditional file-level compression. This dimensional change allows additional compression without interfering with the existing file system operations and data management functionality.
Solution Approach 2:
The second compression is applied in advance during container creation, before any data access or modification operations occur. This preliminary compression ensures that subsequent read/write operations work with already-compressed data, minimizing performance impact during active data management.
Data Source
AI summary
Techniques are provided for implementing additional compression for existing compressed data. Format information stored within a data block is evaluated to determine whether the data block is compressed or uncompressed. In response to the data block being compressed according to a first compression format, the data block is decompressed using the format information. The data block is compressed with one or more other data blocks to create compressed data having a second compression format different than the first compression format.