Destaging Cached Data in Storage Systems for Crash Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In extreme failure scenarios, such as total crashes, data stored in cache memory is lost due to the inability to transfer data between cache and storage, and metadata inconsistencies can render stored data unusable, making it unclear whether the data represents a valid order-preservation consistency condition crucial for database and application crash consistency.
Innovation Solution
A method and system for operating a storage system that involves recurring data storage, where dirty data is destaged without overwriting superseded data, metadata is retained to indicate the location of superseded data, and a point-in-time is registered for order-preservation consistency, allowing the system to restore to a previous state and maintain data consistency in case of a crash.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in cache memory for fast access, then data access speed is improved, but data loss occurs in case of total crash
Solution Approach 1:
The system performs preliminary actions by repeatedly destaging dirty data from cache to stable storage before a crash can occur. Multiple destaging operations are executed during normal operation, creating intermediate safe copies of data in advance, so that if a crash happens, the system can recover to one of these previous consistent states rather than losing all cached data.
Solution Approach 2:
The invention creates a cushion against data loss by maintaining multiple versions of data in stable storage through repeated destaging. This cushion of previously saved data versions protects the system from total data loss in crash scenarios, allowing recovery to the most recent consistent state before the crash occurred.
2Loss of information
If metadata is updated frequently to track data changes, then data consistency tracking is improved, but metadata loss occurs in case of crash
Solution Approach 1:
Metadata updates are performed as preliminary actions alongside data destaging operations. Each time data is successfully destaged to stable storage, the metadata is also updated to reflect this consistent state. This preliminary updating of metadata ensures that even if a crash occurs, the metadata in stable storage reflects a known good state before the crash.
3Quantity of substance
If data is overwritten to optimize storage space, then storage efficiency is improved, but ability to recover superseded data is lost
Solution Approach 1:
The system implements a discard and recover mechanism where superseded data is discarded from cache to make space for new data, but the superseded data is simultaneously recovered and preserved in stable storage through the repeated destaging process. This allows the system to maintain storage efficiency by freeing cache space while ensuring that superseded data can be recovered if needed for crash consistency.
4Reliability
If multiple destaging operations are performed to ensure data safety, then data consistency is improved, but system complexity increases
Solution Approach 1:
The system employs periodic destaging operations where dirty data is repeatedly staged to stable storage at regular intervals or under specific conditions during normal operation. This periodic action ensures data consistency without requiring complex real-time synchronization mechanisms, as the system simply performs destaging operations periodically rather than continuously managing complex crash recovery protocols.
Data Source
AI summary
Storage system(s) for storing data in physical storage in a recurring manner, method(s) of operating thereof, and corresponding computer program product(s). For example, a possible method can include: upon start of a storage recurrence, destaging dirty data which had been accommodated in the cache memory prior to the start of said storage recurrence thus giving rise to destaged data group, wherein destaging is provided with no overwriting of at least superseded data destaged before starting said storage recurrence while enabling retaining metadata indicative of location of said superseded data in the physical storage space; accommodating data obtained in said cache memory subsequent to the start of said storage recurrence while preventing said data from being destaged during said storage recurrence, thus giving rise to accommodated data group; and registering a point-in-time indicative of successful destaging of the destaged data group, thereby providing an order-preservation consistency indication corresponding to said recurrence.