Multiple Non-Volatile Metadata Logs for Storage Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems with dual node configurations face inefficiencies in space reclamation and storage management due to the use of a single shared non-volatile metadata log, leading to potential space shortages and performance issues, particularly when different in-memory metadata logs have varying ingest rates and flushing rates.
Innovation Solution
Implementing multiple shared non-volatile metadata logs, each optimized for specific metadata update types, allowing for independent management and storage of different metadata update types without overlap, thereby reducing the need for early destaging and volatile memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a single shared non-volatile metadata log is used, then space reclamation becomes complex and volatile memory usage increases, but storage efficiency and recovery performance deteriorate
Solution Approach 1:
The patent divides the single shared non-volatile metadata log into multiple separate non-volatile metadata logs, with each log dedicated to specific metadata update types. This segmentation allows independent management of different metadata types (e.g., data mapping metadata, quality of service metadata), enabling more efficient space reclamation for each log type while reducing overall volatile memory requirements. Each log can be optimized for its specific ingest and flushing rates without being constrained by a unified log structure.
2Ease of operation
If a single shared non-volatile metadata log is used, then space reclamation challenges increase, but device complexity increases
Solution Approach 1:
By segmenting the metadata log into multiple specialized logs, the patent simplifies space reclamation operations within each log while the overall system manages multiple logs through standardized interfaces. Each log type can implement reclamation strategies optimized for its specific characteristics without requiring complex coordination across all metadata types.
Solution Approach 2:
The patent applies different management characteristics to different log types based on their specific requirements. Each non-volatile metadata log can have tailored ingest rates, flushing rates, and reclamation policies suited to its specific metadata update type, rather than applying a one-size-fits-all management approach.
3Productivity
If multiple specialized non-volatile metadata logs are implemented, then storage efficiency and recovery performance improve, but device complexity increases
Solution Approach 1:
The patent organizes multiple non-volatile metadata logs with clear segmentation by metadata update type, where each log handles specific operations (e.g., data mapping, quality of service). This structured segmentation improves recovery performance by allowing targeted reconstruction of specific metadata types while maintaining manageable system complexity through consistent organizational patterns.
Solution Approach 2:
The patent introduces a new dimension of organization by categorizing metadata logs according to update types and assigning them to different non-volatile logs. This dimensional classification enables efficient recovery operations by allowing selective restoration of specific log types while maintaining overall system coherence through standardized management protocols.
Data Source
AI summary
In at least one embodiment, processing can include: receiving a request for a transaction of MD (metadata) updates including a first MD update of a first MD update type of a first set and including a second MD update of a second MD update type of a second set; storing, in a first volatile MD log, the first MD update; storing, in a second volatile MD log, the second MD update; storing, in a first non-volatile MD log, the first MD update; and storing, in a second non-volatile MD log, the second MD update, wherein each MD update of the first volatile MD log and the first non-volatile MD log has a corresponding MD update type of the first set, and each MD update of the second volatile MD log and the second non-volatile MD log has a corresponding MD update type of the second set.