Compressed File-System Blocks for Low-Amplification Random Reads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods for read/write file systems suffer from random read amplification, leading to low read efficiency and an inability to support data and metadata updates.
Innovation Solution
A data compression method that compresses data blocks using a preset algorithm, establishing a first index for each block to ensure consistent capacity and maintain a mapping relationship, allowing for efficient random reads with minimal amplification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If data is compressed using a fixed-size minimum compressible unit, then compression is simple to implement, but random read amplification occurs and read efficiency decreases
Solution Approach 1:
The patent segments compressed data into fixed-size output blocks and creates separate index structures for each block. This segmentation allows random reads to directly access specific blocks without reading entire compressed files, reducing read amplification while maintaining compression simplicity.
Solution Approach 2:
The patent introduces an index structure as an intermediary between the compressed data and read operations. The index maps logical data blocks to their physical locations in compressed blocks, enabling efficient random access without decompressing entire files, thus improving read efficiency while keeping the compression process simple.
2Productivity
If data is compressed into variable-size blocks, then read amplification is reduced, but device complexity increases
Solution Approach 1:
The patent divides compressed data into fixed-size output blocks with corresponding index entries. This segmentation approach maintains a regular, predictable structure that is easier to manage than variable-size blocks, while still enabling efficient random access by reading only relevant blocks, thus reducing read amplification without significantly increasing complexity.
Solution Approach 2:
The patent changes the parameter of block size from variable to fixed for output blocks. This parameter change simplifies the data structure and indexing mechanism compared to variable-size blocks, while the fixed block size can be optimized to reduce read amplification, achieving a balance between complexity and performance.
3Volume of stationary object
If existing compression methods are used, then storage capacity is reduced, but random read operations require reading more data than necessary
Solution Approach 1:
The patent performs preliminary organization of compressed data into fixed-size blocks with associated index entries during the compression phase. This preliminary action creates a structured format that enables fast random access later, allowing the system to efficiently locate and read only the specific blocks needed for random read operations, reducing read time while maintaining storage efficiency.
Solution Approach 2:
The patent introduces an index structure as an intermediary that stores metadata about compressed block locations and sizes. This index enables the system to quickly locate specific data blocks without scanning through compressed data, significantly reducing random read time while the compression itself maintains efficient storage capacity utilization.
Data Source
AI summary
This application discloses a data compression method and apparatus. The method includes: obtaining m data blocks in a data area of a readable and writable file system; compressing the m data blocks by using a preset compression algorithm, to sequentially obtain n compressed data blocks, where first capacities of the compressed data blocks are the same, and the first capacity represents a quantity of bytes of compressed data that can be included in the compressed data block; and establishing a first index of each of j data blocks corresponding to an ith compressed data block in the n compressed data blocks, and recording a mapping relationship between first indexes and the j data blocks. The first index is used to identify a storage location, in a storage medium, of each data block included in the j data blocks, and attribute information included in each of the j data blocks.