Metadata Page Compression With Log-Structured Address Relocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems inefficiently manage metadata (MD) pages, leading to substantial consumption of non-volatile storage due to the inability to leverage compression effectively, especially in log-structured systems where updates result in fixed physical addresses and storage inefficiencies.
Innovation Solution
Implement a log-structured system (LSS) for metadata management that allows for dynamic translation of logical addresses to physical addresses using a translation table (TT), enabling selective storage of MD pages in compressed or uncompressed forms based on access frequency, and storing them in large physical blocks (PLBs) to optimize storage usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If metadata pages are stored in fixed physical addresses in log-structured systems, then update operations are simplified, but storage space is wasted and compression cannot be applied
Solution Approach 1:
The patent segments metadata management into two distinct operational paths: log-based updates for modified pages and direct writes for new pages. This segmentation allows the system to apply compression selectively to log-based updates while maintaining simplicity for new page allocations, thereby reducing overall storage consumption without complicating the update process
Solution Approach 2:
The patent introduces dynamic address allocation where metadata pages can move between log-based storage and direct physical addresses based on their update patterns. Frequently updated pages are routed through the log structure enabling compression, while less frequently updated pages use direct addressing, creating a dynamic system that optimizes storage usage based on actual access patterns
2Quantity of substance
If compression is applied to metadata pages, then storage consumption is reduced, but access speed and retrieval complexity increase
Solution Approach 1:
The patent applies compression selectively only to metadata pages that are frequently updated and benefit from log-based storage, rather than compressing all metadata pages uniformly. This local application of compression ensures that pages where it provides the most benefit are optimized, while pages where compression would hinder performance are left uncompressed
Solution Approach 2:
The patent introduces a translation table as an intermediary layer between logical metadata addresses and physical storage locations. This translation table absorbs the complexity of compression management, allowing the system to maintain simple logical addresses while handling compression decompression operations transparently in the physical layer, thus preserving access speed for applications
3Quantity of substance
If log-based relocation is implemented for metadata updates, then storage optimization is enabled, but system complexity increases
Solution Approach 1:
The patent makes the log structure serve multiple functions: it acts as both an update journal for tracking metadata changes and as a compression storage mechanism. This multi-functionality eliminates the need for separate update logging infrastructure, reducing overall system complexity while enabling both storage optimization and simplified update operations through a single unified mechanism
Data Source
AI summary
In at least one embodiment, processing can include: updating a metadata (MD) page, including recording updates in MD log entries; and flushing the MD log entries. Flushing can include: applying the updates to a first version V1 of the MD page to generate a second version V2 of the MD page, where V1 is stored at physical address PA1; evaluating V2 to determine, in accordance with one or more criteria, whether to store V2 as a selected one of a compressed form or an uncompressed form; and storing V2 at a second physical address PA2 in the selected compressed or uncompressed form, wherein PA2 denotes a different physical address than PA1, and wherein PA2 is included in a physical large block of non-volatile storage having a type attribute that allows storing MD pages in the selected compressed or uncompressed form.