Volume Subset Replication Using Change Logs for Flexible Failover
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems require volume-level replication, which limits flexibility in defining failover domains and can result in all applications within a volume having the same recovery point objective (RPO) and recovery time objective (RTO), compromising storage efficiency and application-specific failover needs.
Innovation Solution
Implementing granular replication of volume subsets by defining consistency groups with finer granularity than a full volume, using change logs to update data blocks, and leveraging deduplication and compression to efficiently replicate and maintain data consistency across nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If volume-level replication is used, then storage efficiency and simplicity are improved, but flexibility in defining failover domains and application-specific RPO/RTO control are worsened
Solution Approach 1:
The patent segments volumes into smaller units called consistency groups, which can be independently replicated. This allows fine-grained control over which data is replicated and where, enabling application-specific failover domains while maintaining storage efficiency through selective replication of only necessary data subsets.
2Adaptability or versatility
If granular replication of volume subsets is implemented, then flexibility and application-specific failover control are improved, but system complexity increases
Solution Approach 1:
By segmenting volumes into consistency groups, the system achieves granular replication control without requiring complete system redesign. Existing volume structures are divided into manageable segments that can be independently configured for replication, balancing flexibility with operational simplicity.
Solution Approach 2:
The system pre-establishes consistency groups and their replication configurations before failures occur. Change logs are maintained in advance to track modifications, enabling rapid failover without complex real-time decisions during crisis moments.
3Reliability
If change logs are used to update data blocks, then data consistency and recovery accuracy are improved, but storage space and processing overhead increase
Solution Approach 1:
Instead of replicating entire volumes, the system extracts and replicates only the changed data blocks through change logs. This selective approach maintains data consistency by tracking modifications while minimizing storage space consumption by replicating only necessary changes rather than complete data sets.
Solution Approach 2:
The system discards redundant data by using change logs that only store modifications rather than complete data copies. When recovery is needed, the system recovers data by applying changes from change logs to existing data, reducing storage requirements while maintaining full data consistency and recovery capabilities.
Data Source
AI summary
Data is replicated on a backup node, where the granularity of the replication can be less than a full volume. A data consistency group comprising a subset of data for a volume is defined for a primary node. A set of differences for the data consistency group is sent to a backup node. The backup node creates change logs in response to receiving the set of differences. In response to receiving a request to access a file having data in the data consistency group, the backup node creates a clone of the file. The backup node determines whether an update to a data block of the file exists in the change logs. In response to determining that the update to the data block exists in the change logs, the backup node updates a copy of the data block for the cloned file with data in the change logs.