Automated Consistency Group Scaling via Journal Lag
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data protection systems face challenges in scaling out replica sites due to limitations in load balancing, performance measurement inaccuracies, and constraints on parallel access to virtual volumes, which hinder efficient resource allocation and data consistency in replication systems.
Innovation Solution
A fully automated method that uses replica journal distribution lag as a measure to determine the correct scale for a consistency group, combined with multi-volume journal usage and address partitioning for non-disruptive scale changes, ensuring data consistency and efficient resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If additional volumes are added to journals to provide parallelism, then productivity is improved, but write order fidelity is broken causing data inconsistencies
Solution Approach 1:
The patent segments the address space into multiple partitions, with each partition mapped to a specific journal volume. This allows parallel writes to different partitions without interfering with each other, while maintaining write order fidelity within each partition. The segmentation enables scalability without compromising data consistency.
Solution Approach 2:
The patent introduces a mapping layer that acts as an intermediary between the consistency group and the journal volumes. This mapping layer tracks which partition corresponds to which journal volume, enabling the system to maintain write order fidelity across multiple parallel volumes without requiring complex synchronization protocols.
2Productivity
If virtual infrastructure constraints are overcome to allow parallel access, then productivity is improved, but device complexity increases
Solution Approach 1:
The patent transitions from a single-volume sequential access model to a multi-volume parallel access model by adding the dimension of partitioning. Instead of trying to improve parallel access within the constraints of a single virtual volume, the system creates multiple partitioned volumes that can be accessed in parallel, effectively solving the virtual infrastructure limitation through dimensional expansion.
3Ease of operation
If typical load balancing schemes are used, then ease of operation is improved, but they cannot account for production and replica cluster limitations
Solution Approach 1:
The patent applies local quality by making load balancing decisions specific to each partition and journal volume based on local conditions. Rather than using a generic global load balancing scheme, the system evaluates constraints and performance metrics locally at each replica cluster and adjusts the distribution of IOs accordingly, ensuring that load balancing respects local limitations while maintaining overall system efficiency.
Data Source
AI summary
One example method includes measuring a journal distribution lag of a journal, evaluating a first operational measure at a production site, and evaluating a second operational measure at a replication site operable to communicate with a production site, based on the journal distribution lag, and the evaluation of the first operational measure and the second operational measure, determining a scale out for a consistency group of the replication site, and implementing the scale out of the consistency group.