Snapshot Replication Using Delta Indices and Global IDs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for replicating snapshots and clones between storage systems often result in inefficiencies due to the transmission and storage of expanded data states, which are larger than necessary, and impose restrictions on the order and subset of snapshots and clones that can be replicated, leading to loss of space efficiency and inflexibility.
Innovation Solution
The solution involves replicating snapshots and clones using minimal information represented as changes or deltas, allowing for any subset or order of snapshots and clones to be replicated while preserving a minimal representation of data and metadata, by utilizing snapshot indices that map logical offsets to physical offsets and leveraging snapshot global IDs to manage dependencies and ordering relationships.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If expanded state of data is transmitted and stored at destination system, then complete data state is available at destination, but transmission and storage efficiency deteriorates due to larger data size
Solution Approach 1:
The patent segments the data representation into two parts: the base snapshot data and the delta changes. Instead of transmitting the entire expanded state, only the incremental delta portions are transmitted and stored at the destination system. This segmentation allows the destination to reconstruct complete data states by applying deltas to the base snapshot, achieving data completeness while minimizing transmission and storage requirements.
2Reliability
If conventional replication methods are used, then data state is replicated at destination, but flexibility deteriorates due to restrictions on order and subset of snapshots and clones
Solution Approach 1:
The patent performs preliminary action by storing snapshot global IDs and their associated delta information at the source system before replication is needed. This pre-prepared metadata structure enables the destination system to reconstruct any desired snapshot or clone state by retrieving the base snapshot and applying the appropriate sequence of deltas, regardless of the replication order or subset selected. This eliminates conventional restrictions on replication flexibility while maintaining accurate data state reproduction.
3Quantity of substance
If delta representation is used instead of expanded state, then space efficiency is improved, but complexity increases due to need to track changes and reconstruct states
Solution Approach 1:
The patent introduces snapshot global IDs as an intermediary element that mediates between the delta representations and the reconstructed data states. Each snapshot global ID serves as a unique identifier that encapsulates the relationship between base snapshots and their corresponding delta sequences. This intermediary simplifies the complexity by providing a straightforward lookup mechanism: the destination system retrieves the base snapshot using the global ID, then applies the associated deltas in sequence to reconstruct the desired state, avoiding complex tracking and reconstruction logic.
Data Source
AI summary
Performing replication of snapshots between storage systems is disclosed. A request to store at a destination system a snapshot data to represent at the destination system a state of a set of data at a first point-in-time is received. A first source system snapshot data that represents at a source system the state of the set of data at the first point-in-time depends on a second source system snapshot data that represents at the source system a state of the set of data at a second point-in-time. The snapshot data to represent at the destination system the state of the set of data at the first point-in-time is determined based on data comprising the first source system snapshot data and a destination system snapshot data that represents at the destination system a state of the set of data at a third point-in-time.