Storage Resync Transfer Using Log Metafiles After Site Failure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The rebuilding of the entire data warehouse during a resync operation in storage environments consumes significant time, exceeding disaster recovery service level objectives, especially when volumes are large, and affects other processes like asynchronous and synchronous replication.
Innovation Solution
A log metafile is used to log virtual volume block number mappings in parallel with data modifications, allowing asynchronous processing by a background scanner, reducing the need to rebuild the entire data warehouse before transferring data modifications, and enabling parallel execution of data and reference phases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire data warehouse is rebuilt during resync operation, then data consistency is ensured, but resync time becomes excessively long exceeding service level objectives
Solution Approach 1:
The patent applies preliminary action by logging virtual volume block number mappings into a log metafile before the resync operation begins. This pre-logging of mapping information allows the system to skip the time-consuming process of rebuilding the entire data warehouse during resync, as the mapping data is already prepared and available when needed.
Solution Approach 2:
The patent extracts only the essential mapping information (virtual volume block number mappings) from the data warehouse reconstruction process and stores it separately in a log metafile. This extraction allows the resync operation to use only the necessary mapping data without reconstructing the entire data warehouse structure, significantly reducing resync time while maintaining data consistency.
2Use of energy by moving object
If sequential processing is used for data warehouse rebuilding, then system resources are conserved, but processing speed becomes too slow for disaster recovery objectives
Solution Approach 1:
The patent segments the data warehouse rebuilding process into two independent parts: (1) logging virtual volume block number mappings into the log metafile, and (2) using these pre-logged mappings during resync. This segmentation allows the mapping logging to be performed separately and in advance, enabling parallel processing during the actual resync operation without requiring full sequential reconstruction of the data warehouse.
Solution Approach 2:
By performing the mapping logging action preliminarily (before resync), the system prepares the necessary data structures in advance. This preliminary action enables the resync operation to proceed much faster since it can directly use the pre-logged mappings without performing the full sequential data warehouse rebuild during the critical recovery window.
Data Source
AI summary
Techniques are provided for performing a resync transfer to recover from a storage site failure. During normal operation of a first site hosting a first volume, data is replicated to a second volume hosted by a second site. If the first site fails, when clients are redirected to the second volume at the second site. When the first site recovers, data modifications made to the second volume are resynced back to the first volume. As part of synchronizing the first volume, a data warehouse is rebuilt at the first site in order to track the location of blocks present on the replication destination. Typically, the data modifications are transferred after the data warehouse is rebuilt, which results in significantly long resync times. The techniques provided herein decrease the resync time by either rebuilding the data warehouse in parallel with resyncing the data modifications or circumvent the need for rebuild.