Bin-less Metadata Page Allocator for Log-Structured Storage Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Clustered storage systems face performance overhead due to persistent operations required for allocating and reclaiming logical addresses in hierarchical bin structures, degrading overall system performance.
Innovation Solution
Implementing a 'bin-less' metadata page allocator using a translation table to map logical addresses to physical addresses, allowing for the reclamation and reallocation of logical addresses without updating hierarchical bin structures, thereby avoiding persistent operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hierarchical bin structures are used to manage logical addresses, then address allocation and reclamation can be organized systematically, but persistent operations are required for reading and updating bins which increases storage overhead and degrades performance
Solution Approach 1:
The patent extracts the hierarchical bin structure from the address management system and replaces it with a flat translation table. This removes the complex multi-level bin hierarchy while maintaining address mapping functionality, eliminating the need for persistent operations on bin structures and reducing storage overhead.
Solution Approach 2:
The translation table is organized into multiple segments or pages that can be managed independently. Each segment contains translation entries that map logical addresses to physical addresses, allowing the system to process and update only relevant segments rather than entire bin structures, improving performance while maintaining organization.
2Reliability
If hierarchical bin structures are updated for each logical address allocation or reclamation, then address management remains accurate, but multiple persistent operations are performed which increases overhead and reduces system performance
Solution Approach 1:
The patent merges multiple bin structures into a single translation table that consolidates all logical-to-physical address mappings. This unified structure maintains address management accuracy while eliminating the need for multiple separate persistent operations, as updates can be performed more efficiently in a single centralized location.
Solution Approach 2:
The system uses a translation table that can be copied or replicated across storage nodes, ensuring consistent address management without requiring persistent updates to complex bin structures. This copying mechanism maintains reliability while improving performance by avoiding repeated persistent operations.
3Loss of information
If persistent operations are performed for reading and updating bins during logical address allocation, then address tracking is maintained, but storage capacity overhead increases and overall performance degrades
Solution Approach 1:
The patent replaces the mechanical hierarchical bin structure with a software-based translation table that uses in-memory data structures. This substitution eliminates the need for persistent disk operations to maintain address tracking, as the translation table can be maintained in memory and only requires persistent storage when necessary, significantly improving overall performance while maintaining address tracking accuracy.
Data Source
AI summary
Techniques for providing a “bin-less” metadata page allocator for clustered systems with log-structured metadata storage. The techniques include providing a mapping structure in memory of a storage node of a clustered system. The mapping structure can have multiple layers configured to map logical addresses of metadata pages to physical addresses of storage drives within a storage array. The techniques include providing a translation table configured to translate logical addresses of metadata pages to corresponding current physical addresses of storage drives within the storage array. The techniques include, in response to an allocated logical address of a metadata page no longer being in-use, replacing its corresponding current physical address with a predefined value in the translation table, freeing the logical address, and inserting the freed logical address into a free logical address array. The techniques include allocating the freed logical address from the free logical address array.