Hybrid Flat Hash Mapping for Low-Latency Block Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing block storage systems face challenges with mapping between uncompressed and compressed block addresses, leading to memory inefficiencies and latency due to large mapping tables that do not fit in memory or result in low throughput.
Innovation Solution
A hybrid design using a flat hash table for address mapping, combined with a set of hash functions and compressed block allocation tables, allows for efficient compression and decompression of data at the block device level, minimizing memory usage and maintaining transparent operation for the host system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large mapping table is used to track compressed block addresses, then mapping accuracy is improved, but memory usage increases significantly
Solution Approach 1:
The patent divides the mapping table into multiple levels: a first mapping table storing compressed block address to physical address mappings, and a second mapping table storing logical block address to compressed block address mappings. This segmentation allows the system to maintain accurate mappings while reducing the memory footprint of any single mapping table, directly resolving the contradiction between mapping accuracy and memory usage.
2Loss of information
If a large mapping table is used to track compressed block addresses, then mapping completeness is improved, but access latency increases
Solution Approach 1:
By segmenting the mapping process into two stages using two specialized mapping tables, the system can complete mappings in two quick lookups rather than one large slow lookup. The first mapping table handles compressed address to physical address translation, while the second handles logical address to compressed address translation, maintaining completeness while reducing access latency.
Solution Approach 2:
The patent pre-calculates and stores compressed block addresses in the first mapping table before they are needed for actual data access. This preliminary action ensures that when data access occurs, the mapping is already prepared and readily available, reducing access latency while maintaining complete mapping information.
3Productivity
If a large mapping table is used, then throughput is maintained, but memory cost increases
Solution Approach 1:
The patent segments the mapping functionality into two dedicated mapping tables, each optimized for its specific purpose. This segmentation allows the system to maintain high throughput by performing two fast, specialized lookups rather than one large, slow lookup, while significantly reducing the total memory cost compared to a single comprehensive mapping table.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A flat hash table includes a plurality of entries, and each entry includes a hash function index and a usage bitmap. A method for block device level compression mapping using the flat hash table includes compressing uncompressed data to compressed data, retrieving an entry of the flat hash table using an uncompressed block address of the uncompressed data, determining a compressed block address of the compressed data by executing at least one hash function and by determining a hash function in the at least one hash function for mapping the uncompressed block address to the compressed block address that corresponds to a space in a block storage device, storing the compressed data to the space that corresponds to the compressed block address, and updating the hash function index of the entry of the flat hash table with an index indicative of the hash function.