Logical Data Identifier Replication for Deduplicated Snapshots
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing remote replication techniques are inefficient and resource-intensive, particularly when dealing with large volumes of data, as they often require sending entire volumes to replica systems, which can consume significant resources and are not optimized for deduplication.
Innovation Solution
The use of logical data grouping through mediums with identifiers, allowing selective replication of unique data by taking snapshots and utilizing medium maps to identify and transfer only the data that has changed since the last snapshot, along with deduplication techniques to minimize data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If the entire volume is sent from the original system to the replica system, then replication is simple to implement, but it consumes significant resources and network bandwidth
Solution Approach 1:
The patent extracts only the unique data portions from the volume for replication. By using deduplication technology, the system identifies and sends only the data that has changed or is unique to the original system, rather than transmitting the entire volume. This extraction approach significantly reduces network bandwidth consumption while maintaining replication functionality.
Solution Approach 2:
The volume is segmented into multiple data portions or chunks, each with unique identifiers. The system processes and transmits these segments individually, allowing selective replication of only those segments that contain unique data. This segmentation enables efficient bandwidth utilization while preserving the ability to reconstruct the complete volume at the replica system.
2Loss of energy
If selective replication of unique data is implemented, then network bandwidth is optimized, but determining the best approach becomes challenging with increasing data volumes
Solution Approach 1:
The patent uses copy-on-write technology where data portions are copied only when changes occur. Each data portion has a unique identifier that allows the replica system to determine whether data has changed since the last replication. This copying mechanism simplifies the management of selective replication by providing clear indicators of what data needs to be transmitted.
Solution Approach 2:
The system performs preliminary actions by pre-segmenting the volume into data portions with unique identifiers before replication begins. This preliminary organization allows the replica system to quickly identify which portions need replication without complex real-time analysis during the transfer process, reducing the computational complexity of managing selective replication.
3Productivity
If snapshots are used to track changes, then only changed data needs to be replicated, but managing snapshots and anchor mediums increases system complexity
Solution Approach 1:
The patent implements a universal medium identifier system that serves multiple functions: tracking data portions across snapshots, identifying unique data for replication, and managing the snapshot hierarchy. This multi-functional approach consolidates what would otherwise be separate management systems into a single unified mechanism, reducing overall system complexity while maintaining high replication efficiency.
Data Source
AI summary
A system and method for performing replication using mediums. A snapshot ‘S’ is selected for replication, and the anchor medium ‘M’ of S is identified. The replica storage array ‘R’ to which M is being replicated determines a list of medium extents which are available. R sends this list to an original storage array ‘O’ and O generates a list of medium extents which need to be sent to R. Only those medium extents corresponding to M which R does not already have will be sent from O to R. Also, R can obtain medium extents from other source storage arrays during the replication process.