Single I/O Writes in Cloud File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file systems face challenges in reducing write operation latency when hosted in environments where the latency of journal storage media is similar to that of block storage media, such as cloud or commodity server platforms.
Innovation Solution
Implementing a single Input/Output (I/O) write feature that allows for immediate writing of packed block headers and compressed data to disk, followed by asynchronous journaling, thereby reducing write latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional journaling mechanisms are used where journal storage latency is similar to block storage latency, then data consistency is maintained, but write operation latency increases and IOPS efficiency decreases
Solution Approach 1:
The system performs preliminary actions by writing data to the block storage medium first, then asynchronously journaling to the journal storage medium. This reordering of operations allows the write operation to proceed without waiting for journaling completion, reducing latency while maintaining consistency through subsequent verification
Solution Approach 2:
The patent introduces an intermediary mechanism that decouples the data write path from the journaling path. By using separate I/O operations for data and journal, the system avoids the bottleneck of synchronized journaling while maintaining data consistency through the intermediary verification process
2Reliability
If synchronous journaling is performed before acknowledging write operations, then data integrity is ensured, but Input/Output operations per second (IOPS) efficiency deteriorates
Solution Approach 1:
The system performs the data write operation as a preliminary action before initiating journaling. The write operation is acknowledged to the client before journaling completes, allowing high IOPS while data integrity is maintained through asynchronous verification and crash recovery mechanisms
Solution Approach 2:
The patent enables continuous useful action by allowing the write operation to proceed and be acknowledged without interruption for journaling. The journaling process continues asynchronously in the background, maintaining data integrity without blocking productivity-critical operations
Data Source
AI summary
Systems and methods for performing single I/O writes are provided. According to one embodiment, responsive to receipt of a write operation from a client by a file system layer of a node of a distributed storage system and a data payload of the operation having been determined to meet a compressibility threshold, an intermediate storage layer of the node logically interposed between the file system layer and a block storage media is caused to perform a single input/output (I/O) write operation that persists the compressed data payload and corresponding metadata to support asynchronous journaling of the write operation. The single I/O write operation coupled with the use of a new pool file that maintains a list of available blocks for single I/O write operations and a modified node crash recovery approach allows the write operation to be acknowledged to the client while the journaling is performed asynchronously.