Write-Intent Bitmap Logging for Mirrored Storage Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mirrored storage systems face data inconsistency issues due to factors like network failures, hardware failures, and power outages, particularly when using write-back caches, as data may not be immediately written to disk, leading to inconsistent data across mirrored nodes.
Innovation Solution
Implementing a write-intent log using a bitmap (gate bitmap) to record the intent to write data before it is written across multiple locations, ensuring that the intent is persisted to disk, even across power failures, and using I/O gating to manage data consistency by flagging and storing the gate bitmap on disk.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If write-back cache is used to improve write performance, then write speed is improved, but data consistency across mirrored nodes deteriorates due to power failures or network issues
Solution Approach 1:
The patent implements a write-intent log that records the intent to write data before the actual write operation occurs. This preliminary action ensures that even if a power failure or network issue occurs during the write-back cache operation, the system can recover by checking the write-intent log and completing or aborting the write operation accordingly, thus maintaining data consistency across mirrored nodes while preserving write performance benefits
Solution Approach 2:
The patent introduces a write-intent log as an intermediary mechanism between the write-back cache and the mirrored storage nodes. This intermediary records write intentions and coordinates the actual write operations, ensuring that data consistency is maintained across mirrored nodes while allowing the write-back cache to operate at high speed without immediate synchronization requirements
2Reliability
If synchronous duplication is used to maintain data consistency, then data consistency is improved, but write performance deteriorates due to waiting for all mirrors to be updated
Solution Approach 1:
The patent records the intent to write data in the write-intent log before initiating synchronous duplication to mirrored nodes. This preliminary action allows the system to prepare for consistent data distribution without immediately blocking the write operation, enabling better performance management while ensuring data consistency across all mirrors
Solution Approach 2:
The patent implements dynamic write gating that can adjust the level of synchronization based on system conditions. The write-intent log enables the system to dynamically coordinate write operations across mirrored nodes, allowing for flexible control over when synchronous duplication occurs, thus balancing data consistency requirements with write performance considerations
3Speed
If data is cached in memory for fast access, then read speed is improved, but data persistence deteriorates as cached data may be lost during power failures
Solution Approach 1:
The patent implements a write-intent log that records write operations before they are committed to cached memory. This preliminary action ensures that even if power is lost and cached data is lost, the system can recover by consulting the write-intent log and reconstructing the necessary data, thus maintaining data persistence while preserving the performance benefits of caching
Solution Approach 2:
The write-intent log serves as an intermediary between the cached memory and persistent storage. It records write intentions and coordinates data persistence, ensuring that cached data can be recovered or flushed to persistent storage after power failures, thus maintaining data persistence while allowing fast cached access during normal operation
Data Source
AI summary
Techniques for maintaining mirrored storage cluster data consistency can employ write-intent logging. The techniques can be scaled to any number of mirror nodes. The techniques can keep track of any outstanding I/Os, data in caches, and data that has gone out of sync between mirrored nodes due to link failures. The techniques can ensure that a power failure on any of the storage nodes does not result in inconsistent data among the storage nodes. The techniques may keep track of outstanding I/Os using a minimal memory foot-print and having a negligible impact on the I/O performance. Properly choosing the granularity of the system for tracking outstanding I/Os can result in a minimal amount of data requiring transfer to synchronize the mirror nodes. The capability to vary the granularity based on physical and logical parameters of the storage volumes may provide performance benefits.