Journal Destage Relay for Online Checkpoint Creation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Content addressable storage systems like XtremIO face challenges in maintaining data consistency and metadata integrity during system failures, making it difficult to create reliable system checkpoints for recovery, especially when metadata journals fail to persist correctly.
Innovation Solution
A method is implemented to create an online system checkpoint by configuring a storage system with first, second, and third tables and their respective metadata journals, which map logical addresses to hash digests and physical locations, and manage disk block allocation, allowing for consistent snapshots and destaging of metadata journals while the system is online, ensuring metadata integrity and allowing for transparent recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If metadata journals are persisted to disk during system operation, then data consistency and metadata integrity are improved, but system performance and I/O throughput deteriorate due to additional write operations
Solution Approach 1:
The patent implements preliminary action by creating checkpoints before potential failures occur. The system periodically snapshots the current state of metadata tables and journals to stable storage locations before any failure can compromise data integrity. This advance preparation ensures that even if subsequent write operations fail or corruption occurs, the system can recover to a known good state without losing recent data.
Solution Approach 2:
The patent applies copying by maintaining redundant copies of metadata journals and table states. Multiple copies are stored in different locations (in-memory caches, disk-based journals, and checkpoint snapshots). This redundancy ensures that if one copy becomes corrupted or lost, identical or near-identical copies exist for recovery, thereby improving reliability without requiring every write operation to be persisted immediately to all locations.
2Reliability
If system checkpoints are created offline to ensure consistency, then metadata integrity is improved, but system availability deteriorates due to system downtime
Solution Approach 1:
The patent implements preliminary action by continuously maintaining ready-to-use checkpoints in the background during normal system operation. These checkpoints are prepared in advance and stored in stable locations before any failure occurs. When a failure happens, the system can immediately activate a pre-prepared checkpoint without needing to pause operations or perform recovery procedures, thus maintaining both consistency and availability.
Solution Approach 2:
The patent applies dynamics by making the checkpoint creation process adaptive and continuous rather than static and periodic. The system dynamically adjusts checkpoint frequency and timing based on system state, workload conditions, and failure risks. Checkpoints are created in the background without interrupting I/O operations, and the system can switch between different checkpoint versions dynamically, ensuring both consistency and continuous availability.
3Productivity
If metadata journals are kept in memory for fast access, then I/O performance is improved, but data loss risk increases in case of system failure
Solution Approach 1:
The patent applies copying by maintaining multiple copies of metadata journals across different storage media and locations. In-memory copies provide fast read/write access for active operations, while simultaneous copies are written to disk-based journals and periodically snapshotted to stable storage. This multi-copy strategy ensures that if the system fails before in-memory data is flushed, redundant copies exist for recovery, thereby maintaining both performance and persistence.
Solution Approach 2:
The patent implements preliminary action by proactively writing metadata journal changes to persistent storage locations before the system might fail. Rather than waiting for buffer fills or scheduled flushes, the system periodically and incrementally persists journal data to disk and checkpoint locations in advance. This ensures that even if a sudden failure occurs, the most recent journal state is already safely stored, reducing data loss risk while maintaining in-memory performance.
Data Source
AI summary
A storage system comprising a processor in operable communication with a storage device is configured to have first, second, and third tables and first, second, and third respective metadata journals storing time-based information associated, respectively, with the first, second, and third tables. The first table maps logical addresses associated with LUNs associated with the storage device to respective hash digests computed based on data residing at respective logical addresses. The second table maps a range of hash digests to respective associated physical locations on the storage device. The third table maps disk block allocation on the storage device. An online system checkpoint is created, comprising information relating to the state of the storage system at a point in time, where the first, second, and third, metadata journals are destaged in order so as to destage information stored in the respective journal up to the point in time.