Metadata Object Persistence Latency Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage devices face challenges in maintaining metadata reliability and compliance with command completion time (CCT) specifications due to the large size of metadata objects and the overhead of transferring them to non-volatile memory, which can impact host I/O data transfer rates and increase the risk of data corruption during power loss or high environmental conditions.
Innovation Solution
The metadata object is divided into successive portions, each describing a range of logical addresses, which are transferred to non-volatile memory while maintaining a measured latency, and a journal is used to track updates, allowing for coherent recreation of the metadata object by merging updates from the journal.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the metadata object is transferred to non-volatile memory to ensure data reliability, then data reliability is improved, but host I/O data transfer rates deteriorate due to the large size of metadata objects and transfer overhead
Solution Approach 1:
The metadata object is divided into multiple portions, each describing a different associated range of logical addresses. Each portion is copied in turn to non-volatile memory, allowing the transfer to be performed in smaller, more manageable units that reduce the impact on host I/O performance while still ensuring data reliability.
2Reliability
If the metadata object is copied to non-volatile memory to maintain data reliability, then data reliability is improved, but latency increases beyond acceptable thresholds
Solution Approach 1:
The metadata object is divided into multiple portions that are copied sequentially to non-volatile memory. This segmentation allows the system to maintain a measured latency within a predetermined threshold by controlling the timing and pacing of each portion's transfer, preventing any single large transfer operation from causing excessive latency.
Solution Approach 2:
The metadata object is divided into portions before the copying process begins. This preliminary segmentation prepares the data for efficient transfer, allowing the system to control the timing of each portion's copy operation and maintain latency within acceptable thresholds while still ensuring data reliability.
3Loss of time
If the metadata object is divided into portions and copied sequentially to non-volatile memory, then latency is controlled within thresholds, but the complexity of the persistence operation increases
Solution Approach 1:
A journal data structure is introduced as an intermediary to track changes to the metadata object during and after the copying process. The journal describes changes made to portions after they have been copied to non-volatile memory, providing a systematic way to manage the complexity of the segmented persistence operation while maintaining latency control.
Data Source
AI summary
Apparatus and method for managing metadata in a data storage device. In some embodiments, a metadata object has entries that describe data sets stored in a non-volatile write cache. During an archival (persistence) operation, the metadata object is divided into portions, and the portions are copied in turn to a non-volatile memory at a rate that maintains a measured latency within a predetermined threshold. A journal is formed of time-ordered entries that describe changes to the metadata object after the copying of the associated portions to the non-volatile memory. The journal is subsequently stored to the non-volatile memory, and may be subsequently combined with the previously stored portions to recreate the metadata object in a local memory. The measured performance latency may be related to a specified customer command completion time (CCT) for host commands.