Flash Storage Pseudo-Mirror for Database Log Write Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database log write latency is a significant bottleneck in computer systems, particularly in OLTP environments, due to high disk I/O loads and the need for ensuring data integrity through write-ahead logging, leading to underutilization of resources and performance degradation.
Innovation Solution
Implementing a pseudo-mirror technique using flash storage to asynchronously write database log data to both disk and flash storage, allowing for early confirmation of write operations and reducing latency by utilizing flash storage as a temporary write medium until data is safely written to disk.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is written synchronously to both disk and flash storage, then data integrity is ensured, but write latency increases
Solution Approach 1:
The system performs preliminary writing to flash storage before disk storage completes. The flash storage acts as a temporary buffer that accepts writes immediately, allowing the transaction to proceed while the slower disk write continues in the background. This preliminary action to the faster medium resolves the contradiction by decoupling the timing of write acknowledgments from the slower disk operation.
Solution Approach 2:
Flash storage serves as an intermediary between the database system and the disk storage. It mediates the write operation by accepting data from the database and forwarding it to the disk asynchronously. This intermediary layer allows the database to proceed without waiting for disk completion while ensuring data eventually reaches persistent storage, thus resolving the latency-integrity contradiction.
2Loss of time
If data is written to fast flash storage first, then write latency is reduced, but data integrity risk increases if flash write fails
Solution Approach 1:
The system implements beforehand cushioning by maintaining the disk write operation as a safety buffer. Even though flash storage is written first for speed, the concurrent disk write acts as a cushion against flash failures. If the flash write fails or data is lost, the disk copy serves as a backup, thus cushioning against data integrity risks while preserving the latency benefits of flash-first writing.
Solution Approach 2:
The system changes the timing parameter of write operations to different storage media. Instead of using a single write timing for both flash and disk, it uses asymmetric timing where flash is written immediately and disk is written asynchronously in the background. This parameter change allows the system to optimize for speed on flash while maintaining integrity through the slower disk operation.
3Reliability
If traditional synchronous logging is used, then data integrity is maintained, but system performance degrades
Solution Approach 1:
The logging system is segmented into two independent paths: a fast path to flash storage for immediate acknowledgment, and a slow path to disk storage for persistent reliability. This segmentation allows the system to handle write operations differently based on the storage medium, with the flash path providing high-performance logging and the disk path providing reliable persistent storage, thus resolving the performance-integrity contradiction.
Solution Approach 2:
The system performs preliminary logging to flash storage before completing the full logging cycle to disk. This preliminary action allows transactions to be acknowledged and committed much faster than traditional synchronous disk logging would permit, while the subsequent disk write ensures long-term data integrity. The preliminary flash write resolves the contradiction by providing immediate performance while maintaining eventual reliability.
Data Source
AI summary
Methods, computer-readable media, and computer systems are provided for initiating storage of data on multiple storage devices and confirming storage of the data after the data has been stored on one but not necessarily all of the devices. A storage server receives, from a client, a request to store data. In response to the request, the storage server initiates, in parallel, storage of the data on multiple storage systems. The storage server detects that the data has been stored on any one of the storage systems, such as an auxiliary system, and, in response, indicates, to the client, that the data has been stored. The storage server may flush or discard data on the auxiliary storage system upon detecting that the data has been successfully stored on a target storage system, where the data persists.