Metadata Change Tracking via Bitmap and Fences in Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In storage systems, transferring metadata from one disk array to another while maintaining continuous host access is challenging, as changes to the source metadata can result in inaccuracies at the destination, leading to application issues and timeouts.
Innovation Solution
The use of a bitmap and two fences to track and manage metadata changes during the transfer process, ensuring that changes are synchronized between the source and destination disk arrays without interrupting host access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If metadata is copied during logical disk transfer while allowing continued host access, then host productivity is maintained, but metadata accuracy at destination deteriorates due to changes in source metadata
Solution Approach 1:
The system performs preliminary actions by creating a consistent snapshot of source metadata before the copy operation begins, and pre-establishes a bitmap to track changes. This allows the copy to proceed from a stable baseline while still permitting host access during the transfer.
Solution Approach 2:
The system implements continuous feedback by monitoring metadata changes during the copy operation through a bitmap that tracks which metadata blocks have been modified. This feedback mechanism allows the system to identify and re-copy only the changed portions, ensuring destination accuracy while maintaining host access continuity.
2Measurement precision
If access to data is locked during entire metadata copy process, then metadata accuracy is guaranteed, but host access productivity deteriorates causing timeouts and application hangs
Solution Approach 1:
The system segments the metadata copy process into multiple independent operations: initial snapshot creation, change tracking via bitmap, selective re-copying of modified portions, and validation. This segmentation eliminates the need for a single long-lived lock, allowing host access to proceed between segments.
Solution Approach 2:
Instead of a continuous lock, the system uses periodic actions: initial metadata copy, periodic bitmap updates to track changes, and periodic validation checks. This periodic approach maintains metadata accuracy through multiple checkpoint validations while allowing host access during the intervals between checkpoints.
3Reliability
If metadata changes are tracked using bitmap and fences, then metadata synchronization is improved, but system complexity increases
Solution Approach 1:
The system uses a bitmap as a simplified copy of the metadata structure's change state, rather than tracking full metadata dependencies. This bitmap copy efficiently captures which blocks have changed without requiring complex dependency graphs or recursive tracking mechanisms.
Solution Approach 2:
The bitmap and fence mechanisms serve as intermediaries between the source metadata, destination metadata, and host access operations. These intermediaries absorb the complexity of change tracking, providing a clean interface that simplifies the overall synchronization logic while maintaining high reliability.
Data Source
AI summary
Embodiments include methods, apparatus, and systems for tracking metadata changes during copying in a storage system. One embodiment includes a method that uses a bitmap and two fences to track changes to metadata while a copy of the metadata is being transferred from a source disk array to a destination disk array.