Thin Volume Synchronization for Storage Replication Lag
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data replication systems face challenges in efficiently managing journal lag and validating point-in-time data replicas, leading to increased recovery time objectives and resource consumption, which can result in reduced system performance and potential data corruption.
Innovation Solution
The method involves creating a thin volume associated with a production volume, tracking changes in metadata, and applying these changes asynchronously to a backup storage when the thin volume size is below a threshold, allowing for efficient data replication and reduced resource utilization by avoiding the need for large volumes to account for ongoing changes during replication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is replicated periodically or on change to backup storage, then data protection is provided, but journal lag increases and recovery time objective increases
Solution Approach 1:
The patent creates a thin volume in advance before replication begins, preparing the destination structure ahead of time. This preliminary action allows the replication process to start immediately without waiting for volume creation, reducing the overall recovery time while maintaining data protection through the replication process.
Solution Approach 2:
The patent dynamically adjusts the replication approach by using thin provisioning that allows the volume size to grow as data is replicated. Instead of pre-allocating a large fixed volume, the system adapts the volume size to the actual data being replicated, reducing initial overhead and enabling faster start of replication while still providing complete data protection.
2Reliability
If large volumes are allocated to account for ongoing changes during replication, then data consistency is maintained, but resource consumption increases
Solution Approach 1:
The thin volume automatically manages its own growth as data is replicated. Instead of requiring a pre-allocated large volume, the volume expands self-service style as needed to accommodate the replicated data and ongoing changes, eliminating the need to over-provision storage resources while maintaining data consistency.
Solution Approach 2:
The patent changes the volume allocation parameter from a fixed large size to a dynamic thin-provisioned size that grows with actual data needs. This parameter change allows the system to maintain data consistency through proper replication tracking while consuming only the storage resources actually required, not excessive pre-allocated space.
3Reliability
If synchronous replication is used to reduce journal lag, then data consistency is improved, but system performance decreases
Solution Approach 1:
The thin volume acts as an intermediary structure that facilitates replication without the overhead of traditional large volume allocations. By using thin provisioning as the intermediary layer, the system can implement replication with reduced performance impact while maintaining data consistency through proper metadata tracking and change logging.
Data Source
AI summary
Described embodiments provide systems and methods for synchronizing a production volume and a backup volume of a storage system. A first thin volume is created and associated with the production volume. A first replica of the production volume is generated by copying data from the production volume to a replica volume. During the copying, an I/O request to be written to the production volume may be received. Data from the I/O request is written to the first thin volume and data changed due to the I/O request is tracked in metadata associated with the production volume and the first thin volume. A size of the first thin volume is checked, and when the size of the first thin volume is below a threshold, changes from the first thin volume are applied asynchronously to the backup storage.