Non-disruptive Storage Volume Transition via Metadata Tagging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data protection schemes for storage clusters require disruptive changes, necessitating the creation of new volumes and client reconnection to transition between different data protection schemes, which is inefficient and impacts performance.
Innovation Solution
A non-disruptive technique that uses metadata and block services to transition data blocks between different data protection schemes by tagging and deduplicating data blocks, allowing seamless switching between schemes without requiring a new volume or client disconnection, utilizing a metadata layer and block services to manage and store data blocks with associated data protection scheme tags.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a new volume is created with the desired DPS and data is copied from an existing volume to transition between data protection schemes, then the data protection scheme is successfully changed, but the process is disruptive and requires client reconnection
Solution Approach 1:
The volume is segmented into two independent volumes (original volume and new volume) that coexist during the transition period. Each volume can operate with its own DPS independently, allowing the client to access data through either volume without disconnection, thereby eliminating the disruptive reconnection requirement while enabling DPS changes.
2Adaptability or versatility
If data blocks are copied from an existing volume to create a new volume with a different DPS, then the data protection scheme transitions successfully, but storage efficiency is reduced due to duplication
Solution Approach 1:
The original volume and new volume are merged at the client access layer, allowing the client to access data through either volume transparently. This merging approach enables DPS transitions without requiring exclusive use of a single volume, reducing the need for complete data duplication and improving storage efficiency during the transition period.
Solution Approach 2:
Once the new volume with the desired DPS is created and verified, the original volume can be discarded or its resources recovered. This allows the system to temporarily maintain both volumes during transition but then eliminate the redundant original volume, thereby reducing storage space consumption after the DPS change is complete.
3Duration of action of moving object
If the transitioning process is interrupted, then the system can resume from the interruption point, but data consistency may be compromised without proper deduplication
Solution Approach 1:
The block service implements feedback mechanisms by tracking which data blocks have been successfully copied and deduplicated between volumes. When the transitioning process is interrupted and resumed, this feedback information allows the system to identify which blocks need to be reprocessed, ensuring data consistency is maintained while enabling continuous operation from the interruption point.
Solution Approach 2:
The system creates copies of data blocks from the original volume to the new volume during the transition process. By implementing deduplication on these copied blocks, the system ensures that only unique data is stored in the new volume, maintaining data consistency even when the process is interrupted and resumed, while optimizing storage efficiency.
Data Source
AI summary
A technique transitions data blocks of volumes served by storage nodes of a storage cluster from an old data protection scheme (DPS) to a new DPS in a non-disruptive manner. Slice services of the storage nodes forward the data blocks associated with write requests to the block services for storage on storage devices of the nodes. Mapping of volume logical block addresses to block identifiers are contained in slice files, wherein there is a single slice file for each volume. To transition a volume between the old and new DPSs, the slice service tags the data blocks with the new DPS when forwarding new write requests to the block services. In accordance with a background transitioning process, the slice service also retrieves every data block referenced by the to slice file and then resends the data to the block service with the new DPS.