Replication Progress Tracking in UPIT Snapshot Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face challenges in efficiently managing and persistently storing replication progress information during the replication process, particularly in Virtual Storage Area Network (VSAN) environments, due to the large size and unique requirements of this data, which current persistency models cannot adequately address.
Innovation Solution
The implementation of a method using the native Universal Point in Time (UPIT) structure, specifically by determining and replicating different blocks between snapshots, orphaning nodes, and updating tree structures, to effectively manage replication progress information without altering the internal UPIT implementation or persistency model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If replication progress data is stored using existing persistency models, then the replication process can track progress, but the performance and scale requirements are excessively high due to the large size of progress data
Solution Approach 1:
The patent segments the large replication progress data into smaller manageable units by utilizing existing UPIT snapshot structures. Instead of storing progress data as a single large entity, it distributes the information across multiple snapshot references and metadata structures, reducing the burden on persistency services while maintaining complete replication state tracking
Solution Approach 2:
The patent makes the UPIT snapshot structure multi-functional by having it serve both its original snapshot purpose and simultaneously store replication progress information. The same snapshot metadata structures are used to track both data state and replication progress, eliminating the need for separate progress storage infrastructure
2Reliability
If replication progress data is stored in a manner that prevents re-replication upon interruption, then data integrity is maintained, but the storage scale requirements increase significantly
Solution Approach 1:
The patent uses copying by referencing existing UPIT snapshot structures rather than creating duplicate progress storage systems. The replication progress is tracked by creating references to snapshot states that already exist in the storage system, avoiding the need to physically duplicate large amounts of data while maintaining the ability to resume replication from interruption points
3Reliability
If a dedicated progress data container is implemented, then replication progress can be tracked accurately, but the device complexity increases
Solution Approach 1:
The patent merges the replication progress tracking function with the existing UPIT snapshot metadata structures. Instead of creating a separate dedicated progress container, it combines multiple functions (snapshot management and replication progress tracking) into the same data structures, thereby maintaining accurate progress tracking while avoiding additional system complexity
Data Source
AI summary
A method for storing progress data while replicating a snapshot is disclosed. The operations comprise: determining blocks in a storage object that are different between a first and a second snapshots, wherein the first snapshot corresponds to a first tree structure, and the second snapshot corresponds to a second tree structure; and replicating the second snapshot to a remote storage system, comprising replicating all the blocks in the storage object determined to be different between the first and the second snapshots, wherein the replicating further comprises: upon completion of replication of each data slice, determining nodes in the first tree structure that reference only already-replicated blocks; orphaning all such nodes in the first tree structure; and subsequently, updating the first tree structure, comprising merging the first tree structure and the second tree structure, wherein the first tree structure is updated upon completion of replication of each data slice.