Storage System Transaction Consistency via Write Plans
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Disk performance issues arise due to caching and out-of-order writing, which complicate maintaining consistency in storage systems, particularly during transactions where data integrity and order are crucial.
Innovation Solution
The solution involves creating logical copies of objects during transactions, updating these copies in memory, and using a data structure to track the intended writes, ensuring that all changes are either successfully written to disk or discarded if inconsistencies occur, with a storage controller managing the writing process to maintain atomicity and consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If disk caches are used to improve disk performance, then writing speed is improved, but data consistency and reliability deteriorate due to out-of-order writes and premature write completion reports
Solution Approach 1:
The system performs preliminary actions by creating a data structure (write plan) that records all intended writes before they are executed. This write plan includes verification data and is stored in non-volatile memory beforehand, establishing a baseline for later consistency verification without blocking the actual write operations to proceed in parallel.
Solution Approach 2:
The system implements feedback by using the stored write plan and verification data to check after write operations whether all data was correctly written to disk in the intended order. This feedback mechanism detects consistency violations caused by cache out-of-order writes and triggers appropriate recovery actions to restore data integrity.
2Loss of time
If the disk controller reports write completion before data is actually transferred to non-volatile memory, then response time is improved, but data integrity deteriorates
Solution Approach 1:
The system introduces an intermediary verification mechanism using write plans and verification data stored in non-volatile memory. This intermediary layer mediates between the disk controller's premature completion reports and the actual data integrity, allowing the system to accept fast write reports while independently verifying correctness through the stored verification data.
Solution Approach 2:
The system performs preliminary recording of the intended write state and verification data before the actual write completion. This preliminary action creates a reference that can be used to verify whether the write actually succeeded, decoupling the response time from the actual verification of data integrity.
3Reliability
If logical copies are created and modified in memory before writing to disk, then transaction atomicity is maintained, but memory usage and system complexity increase
Solution Approach 1:
The system creates logical copies (write plans) of the data to be written, storing them in non-volatile memory along with verification data. These copies serve as a lightweight representation that enables atomicity verification without requiring complex transaction management mechanisms, reducing overall system complexity while maintaining reliability.
Data Source
AI summary
Aspects of the subject matter described herein relate to maintaining consistency in a storage system. In aspects, one or more objects may be updated in the context of a transaction. In conjunction with updating the objects, logical copies of the objects may be obtained and modified. A request to write the updated logical copies is sent to a storage controller. The logical copies do not overwrite the original copies. In conjunction with sending the request, a data structure is provided for the storage controller to store on the disk. The data structure indicates the one or more objects that were supposed to be written to disk and may include verification data to indicate the content that was supposed to be written to disk. During recovery, this data structure may be used to determine whether all of the object(s) were correctly written to disk.