Multi-Layer Metadata Redirection for Log-Structured Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern log structure systems face increased latency due to fragmentation of metadata blocks during data relocation processes, which are used to maintain acceptable storage system performance.
Innovation Solution
A multi-layer metadata management process that identifies and remaps redirecting second layer metadata blocks to a target second layer metadata block, adding pointers to redirecting second layer metadata blocks, and limiting the number of redirections to reduce fragmentation and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If garbage collection is performed to create empty chunks for storing user data, then storage system capacity is improved, but metadata block fragmentation increases
Solution Approach 1:
The patent introduces redirecting second layer metadata blocks as intermediary elements that mediate between first layer metadata blocks and target second layer metadata blocks. This intermediary layer allows metadata to be remapped without direct fragmentation, as the redirecting blocks serve as缓冲 (buffer) that manages the mapping relationships while reducing fragmentation effects on data retrieval operations.
2Loss of time
If metadata blocks are remapped to reduce fragmentation, then data retrieval latency is improved, but system complexity increases
Solution Approach 1:
The patent segments the metadata management system into multiple hierarchical layers: first layer metadata blocks, second layer metadata blocks, and redirecting second layer metadata blocks. This segmentation allows complex remapping operations to be distributed across layers, where each layer handles specific mapping tasks, thereby reducing the complexity burden on any single component while improving overall data retrieval efficiency.
Solution Approach 2:
The system performs preliminary remapping actions by identifying redirecting second layer metadata blocks in advance and establishing their mapping relationships before actual data retrieval operations occur. This preliminary organization of metadata mappings reduces the computational overhead during data retrieval, lowering latency while the complexity is managed during the initial setup phase.
3Adaptability or versatility
If the number of redirections in metadata blocks is increased, then storage flexibility is improved, but retrieval performance deteriorates
Solution Approach 1:
The patent extracts the redirection functionality into separate redirecting second layer metadata blocks that are distinct from the primary data storage structure. By taking out the redirection logic into dedicated intermediary blocks, the system achieves storage flexibility through multiple mapping paths while limiting the number of redirections that actual data retrieval operations must traverse, thereby maintaining retrieval performance.
Data Source
AI summary
A method, computer program product, and computing system for identifying one or more first layer metadata blocks that map to one or more second layer metadata blocks. A subset of the second layer metadata blocks that map to another second layer metadata block from the one or more second layer metadata blocks may be identified, thus defining one or more redirecting second layer metadata blocks. For each first layer metadata block of the one or more first layer metadata blocks: One or more first layer metadata blocks that map to the one or more redirecting second layer metadata blocks may be determined and the one or more first layer metadata blocks that map to the one or more redirecting second layer metadata blocks may be remapped to map to a target second layer metadata block.