Persistent Write Cache Split Writes Reducing Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Sequential redo logs in storage systems can lead to increased latency due to the need for writes to be acknowledged in the order they were written, particularly in scenarios where a small write operation is delayed by a large write operation, even though the small write can complete quickly.
Innovation Solution
Implementing a persistent write cache that allows split write operations, where a parsing header for a data item is written first, followed by its payload, enabling parallel cache write operations and reducing latency by allowing acknowledgment of smaller writes before the completion of larger ones, and utilizing this cache as a cyclic redo log for recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a sequential redo log is used to ensure proper ordering of log entries upon recovery, then data integrity and reliability are improved, but write latency increases because small write operations must wait for large write operations to complete
Solution Approach 1:
The patent segments a large write operation into multiple smaller chunks that can be processed and acknowledged independently. Instead of waiting for the entire large write to complete before acknowledging subsequent small writes, the system processes write requests in smaller units, allowing earlier acknowledgment of small writes while still maintaining recovery integrity through the persistent write cache.
Solution Approach 2:
The patent implements a persistent write cache that pre-stores write data before it is fully committed to the primary storage. This preliminary caching allows the system to acknowledge small write operations sooner, while the persistent cache ensures that data can be recovered in the correct sequence if a failure occurs, effectively decoupling the acknowledgment timing from the actual commit timing.
2Measurement precision
If writes are acknowledged in the order they are written to preserve proper ordering, then recovery accuracy is improved, but parallel write performance deteriorates as small writes must wait for large writes
Solution Approach 1:
By segmenting write operations into manageable chunks and using a persistent write cache, the system can track and reorder segments during recovery while allowing parallel processing during normal operation. This segmentation enables both parallel performance and accurate recovery sequencing.
Solution Approach 2:
The persistent write cache acts as an intermediary layer between the parallel write operations and the sequential recovery process. During normal operation, it accepts parallel writes and returns acknowledgments promptly; during recovery, it provides the sequential ordering information needed for accurate reconstruction, thus mediating between parallel performance and recovery accuracy.
3Stability of the object's composition
If the system waits for large write operations to complete before acknowledging small writes, then sequential consistency is maintained, but overall system throughput decreases
Solution Approach 1:
The persistent write cache performs preliminary storage of write data, allowing the system to acknowledge small writes before large writes complete. The cache maintains the sequential consistency information needed for recovery, enabling early acknowledgment without sacrificing consistency guarantees.
Solution Approach 2:
The patent changes the parameter of when acknowledgments are returned, decoupling acknowledgment timing from write completion timing. By using the persistent write cache to track sequential order, the system can return acknowledgments based on arrival order rather than completion order, thereby improving throughput while maintaining sequential consistency for recovery purposes.
Data Source
AI summary
A storage control system receives a first write request and a second write request following the first write request. The first and second write requests comprise respective first and second data items for storage to a primary storage. First and second cache write operations are performed in parallel to write the first and second data items a persistent write cache. The first cache write operation comprises a split write operation which comprises writing a parsing header for the first data item to the write cache, and writing a payload of the first data item to the write cache. The second cache write operation comprises storing the second data item and associated metadata in the write cache, and waiting for an acknowledgment that the parsing header for the first data item has been successfully stored in the write cache before returning an acknowledgment indicating successful storage of the second data item.