Log-Structured Metadata Architecture for Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face challenges in efficiently managing metadata updates and translation table updates, leading to inefficiencies in storage and retrieval operations.
Innovation Solution
A computer-implemented method and system that record metadata updates in a metadata log, flush these updates, apply them to metadata pages, and update the translation table, while also managing descriptor updates and boot tier updates in a log-structured manner.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If metadata updates are applied immediately to metadata pages, then data consistency is maintained, but storage performance and system responsiveness deteriorate due to frequent I/O operations
Solution Approach 1:
The patent applies preliminary action by recording metadata updates in a log structure before actually applying them to metadata pages. This allows the system to acknowledge writes immediately (improving performance) while deferring the actual page updates (maintaining consistency). The log acts as a buffer that captures updates in advance, enabling batch processing later without losing data integrity.
Solution Approach 2:
The patent introduces an intermediary log structure between the host system and the actual metadata pages. This log serves as a mediator that decouples the immediate write operations from the slower page update operations. The log absorbs write requests and manages the timing of actual page updates, resolving the conflict between immediate consistency and performance.
2Measurement precision
If translation table updates are performed frequently to reflect metadata changes, then address mapping accuracy is improved, but system overhead and processing time increase
Solution Approach 1:
The patent records translation table updates in the log structure before applying them, allowing the system to maintain accurate mapping information in the log while deferring the actual translation table page updates. This preliminary recording ensures that when updates are applied, the address mapping accuracy is maintained without requiring immediate processing.
Solution Approach 2:
The patent implements periodic action by batching translation table updates and applying them at intervals rather than immediately for each metadata change. This periodic application of updates reduces the frequency of expensive I/O operations while maintaining mapping accuracy through the intermediate log structure that tracks all changes.
3Quantity of substance
If metadata pages are updated in-place, then storage space is optimized, but data integrity and recovery capability deteriorate due to potential corruption during updates
Solution Approach 1:
The patent applies the copying principle by maintaining copies of metadata updates in the log structure rather than directly modifying pages in-place. This creates a redundant record of changes that can be used for recovery if corruption occurs. The log serves as a backup copy that preserves the original update intent, enabling recovery even if the target page becomes corrupted during update.
Solution Approach 2:
The patent implements beforehand cushioning by recording updates in the log before applying them to pages. This creates a protective buffer that prevents direct in-place modification, thereby cushioning against potential corruption during the update process. If corruption occurs, the log provides a safe copy to restore from, maintaining data integrity.
4Measurement precision
If the system maintains detailed translation tables for all metadata pages, then address resolution accuracy is improved, but memory usage and system complexity increase
Solution Approach 1:
The patent extracts the detailed translation information from the main system memory into a separate log structure on storage. This removes the burden of maintaining large translation tables in fast memory, reducing system complexity and memory usage. The log retains the detailed mapping information needed for accurate address resolution, but stores it externally where it does not consume precious system resources.
Data Source
AI summary
In at least one embodiment, processing can include: recording, in a metadata (MD) log, a metadata (MD) update to a MD page having a logical address LA1; flushing MD updates, including the MD update, from the MD log; applying at least the MD update to the MD page to generate an updated version; storing the updated version of the MD page at a storage location PA2 different from a first location of the MD page prior to updating; recording, in a translation table (TT) log, a TT update that updates entry E1 of a TT to map LA1 of the MD page to PA2 rather than the first location; flushing TT updates from the TT log; applying at least the TT update to the TT to generate an updated version; and storing the updated TT version at a storage location different from a prior location of the TT prior to updating.