Node Polling for Consistency Group Formation in Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current disaster recovery systems face challenges in maintaining data integrity and consistency across primary and secondary storage devices, particularly in ensuring that dependent writes are ordered correctly during data mirroring, which is crucial for maintaining data integrity in applications like database systems.
Innovation Solution
The system employs a mode-switching approach between 'Consistency Group in Progress Mode' and 'Normal Transfer Mode' to manage data transfers, using out-of-sync bitmaps and change recording bitmaps to ensure that dependent writes are correctly ordered and transferred to remote sites, with dynamic calculation of time limits based on congestion and data transfer rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system continuously monitors and polls nodes to determine backlog and estimated drain time, then the ability to dynamically adapt to congestion levels and improve consistency group formation success is improved, but the system complexity and overhead of polling operations increases
Solution Approach 1:
The system implements a feedback mechanism where storage controllers periodically poll nodes to obtain backlog information and estimated drain time, then use this feedback to dynamically adjust operational mode between Consistency Group in Progress Mode and Normal Transfer Mode, thereby adapting to changing congestion conditions
Solution Approach 2:
The system dynamically switches between different operational modes based on real-time polling results. When estimated drain time exceeds the time limit, the system transitions from Consistency Group in Progress Mode to Normal Transfer Mode, and vice versa, making the system adaptive to varying network conditions
2Productivity
If the system imposes a strict time limit on consistency group formation, then the productivity and responsiveness of data transfer is improved, but the reliability of maintaining data consistency across all dependent writes deteriorates
Solution Approach 1:
The system dynamically adjusts its behavior based on real-time conditions. A time limit is imposed for consistency group formation, but the system can switch between operational modes depending on whether the time limit is approached or exceeded, balancing responsiveness with consistency requirements
Solution Approach 2:
The system changes operational parameters (mode of operation) based on the relationship between current backlog conditions and the imposed time limit. When estimated drain time is within the time limit, Consistency Group in Progress Mode is used; when exceeded, Normal Transfer Mode is activated
3Reliability
If the system queues writes and uses out-of-sync bitmaps to track transfers, then the reliability of maintaining write order is improved, but the backlog of writes increases and transfer time extends
Solution Approach 1:
The system extracts and processes consistency group formation separately from normal data transfer operations. By using out-of-sync bitmaps to track which writes need to be transferred first, the system can identify and process critical writes in a dedicated consistency group formation process, reducing their impact on overall transfer time
Data Source
AI summary
Provided are a method, system, deployment and program for managing data transfer in consistency group formation. In one embodiment, nodes are periodically polled to determine traffic conditions. An estimated drain time for a node such as the most congested node, for example, is compared to a drain time limit and if within a vicinity of the drain time limit, operations switch to a consistency group formation mode. Other embodiments are described and claimed.