Parallel Journaling in Storage Cluster Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional journaling methods in storage clusters require serialized actions, leading to high IO latency due to waiting for CPU core availability and rescheduling, which limits the efficiency of data processing and recovery in active-active configurations.
Innovation Solution
Implementing parallel journaling by concurrently writing data to a log buffer and sending page descriptors to another node, reducing the number of CPU scheduling events and ensuring consistent recovery in case of failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional serialized journaling is used, then data consistency is maintained, but IO latency increases due to sequential CPU scheduling
Solution Approach 1:
The journaling process is segmented into independent parallel tasks: page descriptor generation, data writing to log buffer, and page descriptor sending to peer node. These segmented operations execute concurrently rather than sequentially, eliminating CPU scheduling waits while maintaining consistency through coordinated completion tracking.
Solution Approach 2:
Page descriptors are generated and sent to the peer node in advance, before the actual data writing completes. This preliminary action allows the system to prepare recovery information ahead of time, reducing the critical path latency while ensuring consistency through acknowledgment verification.
2Loss of time
If parallel journaling is implemented, then IO latency is reduced through concurrent operations, but system complexity increases
Solution Approach 1:
The system implements feedback through acknowledgment signals from the peer node confirming successful page descriptor receipt. This feedback mechanism coordinates the parallel operations without requiring complex inter-process communication, as the acknowledgment serves as both a confirmation and a synchronization trigger for proceeding with data writing.
Solution Approach 2:
Each node independently manages its own journaling operations and recovery process. The parallel journaling mechanism is self-sufficient, with nodes autonomously generating page descriptors, tracking writing status, and initiating recovery without external coordination, thereby reducing overall system complexity despite the parallel architecture.
3Ease of manufacture
If conventional journaling is used, then CPU scheduling overhead is high due to rescheduling, but implementation is straightforward
Solution Approach 1:
The parallel journaling mechanism enables continuous useful action by eliminating CPU scheduling interruptions. Page descriptors are generated and transmitted without being blocked by CPU availability for data writing, and data writing proceeds without waiting for peer node acknowledgment. This continuous execution eliminates rescheduling overhead and maximizes processing efficiency.
Data Source
AI summary
A method, computer program product, and computing system for receiving, via a first node of a plurality of nodes, data for storage in a storage system. The data may be written, via the first node, to one or more data pages of a log buffer of the first node. One or more page descriptors associated with the one or more data pages may be generated via the first node. The one or more page descriptors may be sent to a second node concurrently with the writing of the data to the one or more data pages of the log buffer of the first node.