Merged Block Compression for Faster Sequential Data Reads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Merge compression in data storage systems leads to poor data read performance due to the need for multiple read operations and decompression steps, especially when multiple data blocks are merged and compressed.
Innovation Solution
A method that merges and compresses multiple read requests into a single read operation, reducing the number of read and decompression operations by grouping data blocks based on their locations or sequences, and caching frequently accessed data blocks for direct retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple data blocks are merged and compressed into fewer blocks to save storage space, then storage efficiency is improved, but data read performance deteriorates due to increased decompression operations
Solution Approach 1:
The patent merges multiple read requests into a single read operation by identifying that multiple first data blocks are contained within the same second data block. Instead of performing separate read and decompression operations for each data block request, the system combines these requests and executes a single read operation followed by decompression of the merged data block once, thereby reducing redundant operations while maintaining storage efficiency.
2Reliability
If multiple read requests are processed separately to ensure data accuracy, then data retrieval reliability is improved, but processing time increases
Solution Approach 1:
The patent combines multiple separate read requests into a single merged read request when the target data blocks are contained within the same compressed data block. This merging process maintains data retrieval reliability by ensuring all requested data is obtained through the single operation, while simultaneously reducing processing time by eliminating redundant read and decompression operations.
Solution Approach 2:
The system performs preliminary analysis to determine whether multiple read requests target data blocks within the same compressed block before executing the read operation. This preliminary check enables the system to optimize the read process by merging compatible requests, thereby reducing processing time while maintaining reliability.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
This application relates to a data read method, a data write method, an apparatus, and a system. In this method, consecutive data blocks are selected for merge compression, so that more data blocks appear in a merged and compressed data block during data reading, and especially for a sequential read/write scenario, a plurality of blocks that need to be read at the same time are more likely to appear in a same merged and compressed block. Read requests are merged, to reduce a quantity of read operations, and reduce a disk read amplification ratio and a decompression calculation amount in a read procedure. In addition, whether to perform read, decompression, and re-merge compression is determined based on validity of a data block by using a garbage collection procedure, to reduce read amplification and decompression calculation amount amplification in the garbage collection procedure. During data migration, re-merge compression is performed on adjacent data blocks, to maintain a compression ratio and reduce redundant data.