Node Cache and Logstore Recovery for Distributed File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed file systems, data writes can be corrupted or lost due to node failures or disconnections before being fully written to stable storage, leading to instability and data loss.
Innovation Solution
Implementing a node cache and logstore system where data is cached in an in-memory node cache and mirrored on stable storage, with mechanisms for updating parent files and managing logstore staleness to ensure data integrity and recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is cached in in-memory node cache for faster access, then write performance is improved, but data integrity deteriorates due to potential loss on node failure
Solution Approach 1:
The patent creates a copy of the cached data in the logstore on stable storage. When data is written to the node cache, it is simultaneously copied to the logstore, ensuring that even if the node cache is lost due to failure, the data can be recovered from the logstore copy.
Solution Approach 2:
The patent performs preliminary action by writing data to the logstore before confirming the write operation is complete. This ensures that if a node fails during the caching process, the data is already safely stored in the logstore and can be recovered without corruption.
2Reliability
If data is immediately written to stable storage, then data integrity is improved, but write performance deteriorates due to slow storage speed
Solution Approach 1:
The patent introduces the node cache as an intermediary layer between the client and stable storage. Data is first written to the fast in-memory node cache, providing quick acknowledgment to the client, while asynchronously flushing to stable storage via the logstore, thus decoupling the performance-critical path from the reliability-critical path.
3Productivity
If node cache is used to buffer writes, then write throughput is improved, but data loss risk increases when nodes crash before flush
Solution Approach 1:
The patent prepares beforehand by maintaining the logstore on stable storage as a cushion or safety net. Before the node cache is flushed to stable storage, the data is already protected in the logstore, providing a cushion against potential data loss from node crashes, power failures, or cache corruption.
4Reliability
If logstore is maintained for every parent file, then recovery capability is improved, but system complexity increases
Solution Approach 1:
The patent makes the logstore a universal component that serves multiple parent files. Instead of creating a separate logstore for each parent file, a single logstore can buffer writes for multiple parent files, reducing the total number of logstores and simplifying the system while maintaining recovery capability for all parent files.
Data Source
AI summary
Various embodiments are directed towards enabling data writes utilizing a node cache and a logstore stored on a stable storage device. A client device may send data to a node for writing to a parent file. The node may cache the received data prior to writing the data to the parent file. Caching the received data may comprise adding the received data to a node cache and to a logstore. In one embodiment, the node cache may include a coalescer that combines data from a plurality of data writes from the client device to the node prior to writing to the parent file. In some embodiments, the logstore may be mirrored logstore on one or more stable storage devices. The parent file may periodically be updated from the node cache data. After the parent file is updated, the node cache data and the logstore data may be purged.