Storage System Resynchronization via Checkpoint and Snapshot Differencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
During an uncontrolled shutdown of a primary data storage system, data in cache memory that has been acknowledged but not flushed to nonvolatile storage is lost, requiring a full resynchronization with the secondary system, which is inefficient due to the large amount of data transfer involved.
Innovation Solution
Implementing a checkpoint mechanism that regularly pushes data from cache to nonvolatile storage and a time-bound snapshot mechanism to limit data loss to only changes after the last checkpoint, allowing for efficient recovery by sending differences from the secondary system to the primary system during resynchronization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous mirroring is used to guarantee data consistency, then data reliability is improved, but data transfer time and system overhead increase
Solution Approach 1:
The patent segments the data replication process into two distinct phases: synchronous replication for critical data consistency (improving reliability) and asynchronous replication for performance optimization (reducing time overhead). This allows the system to maintain data integrity while avoiding the performance penalty of continuous synchronous operations.
Solution Approach 2:
The patent implements dynamic replication mode selection that can switch between synchronous and asynchronous modes based on system conditions, data criticality, and performance requirements. This dynamic approach allows the system to optimize the balance between reliability and speed in different operational contexts.
2Reliability
If full resynchronization is performed after uncontrolled shutdown, then data recovery completeness is improved, but resynchronization time and network bandwidth consumption increase
Solution Approach 1:
The patent implements preliminary actions during normal operation by maintaining checkpoints and change logs that record the state and modifications of replicated data. When an uncontrolled shutdown occurs, these pre-maintained records enable the system to perform incremental resynchronization rather than complete resynchronization, significantly reducing recovery time and network bandwidth consumption while ensuring data completeness.
3Productivity
If data is kept in cache memory for faster access, then read performance is improved, but data loss risk during uncontrolled shutdown increases
Solution Approach 1:
The patent implements periodic flushing of cache memory to non-volatile storage at predetermined intervals or when cache capacity thresholds are reached. This periodic action ensures that cached data is regularly persisted, reducing the risk of data loss during uncontrolled shutdowns while maintaining the performance benefits of caching for frequently accessed data.
Data Source
AI summary
A method for resynchronizing a first copy of data on a first storage system from a second copy of the data on a second storage system includes, at a regular checkpoint interval, the first storage system pushing data in its cache that were modified prior to a checkpoint time to its nonvolatile storage and saving the checkpoint time to its nonvolatile storage. The method further includes, at a regular snapshot interval greater than the checkpoint interval, the second storage system taking snapshots of the second copy of the data. When the first storage system has an uncontrolled shutdown, the second storage system determines the snapshot closest in time to the last checkpoint time and sends the difference between the last snapshot and the second copy of the data to the first storage system to recover data lost during the uncontrolled shutdown.