Disk Unit Journal Replication for Controller-Failure Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data management systems face challenges in maintaining data accessibility and transaction processing when a controller fails, as log files stored on the controller become inaccessible, leading to service disruptions.
Innovation Solution
A data management system with a set of controllers and storage media that utilize persistent memories to write and replicate journal data across multiple nodes, ensuring data integrity and continuity even in the event of controller failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If log files are stored on the controller to reduce latency, then transaction processing speed is improved, but system reliability deteriorates when controller failure occurs
Solution Approach 1:
The system performs preliminary actions by replicating journal data to multiple controllers before failure occurs. Each controller maintains copies of journal data in persistent memory, so when a controller fails, the replicated data is already available to other controllers to continue processing transactions without interruption.
Solution Approach 2:
The system creates copies of journal data across multiple controllers. Each controller stores replicated journal data in its persistent memory, ensuring that if one controller fails, other controllers with identical copies can continue servicing transactions. This copying strategy eliminates the single point of failure while maintaining low-latency access.
2Reliability
If journal data is replicated to multiple nodes, then system reliability is improved, but device complexity increases
Solution Approach 1:
The system implements self-service through automatic failure detection and recovery mechanisms. When a controller fails, other controllers automatically detect the failure condition and continue processing transactions using replicated journal data without requiring manual intervention or complex reconfiguration. The system manages its own reliability through built-in persistent memory replication.
3Reliability
If persistent memory is used to store journal data, then data availability during failure is improved, but loss of information increases for unreplicated data
Solution Approach 1:
The system performs preliminary replication of journal data to multiple controllers before any failure occurs. By maintaining redundant copies in persistent memory across multiple nodes, the system ensures that even if a controller fails, the replicated data is already available to continue processing. This preliminary replication prevents information loss for all transactional data.
Data Source
AI summary
A data management system can include a disk unit and a set of controllers. The disk unit can contain, at least in part, a set of storage media, a first persistent memory, and a second persistent memory. The set of storage media can be configured to implement a storage space. The set of controllers can be configured to write to the storage space and to implement a set of nodes including a first node and a second node. The first node can be configured to generate and write first node journal data to the first persistent memory. The second node can be configured to obtain a failure indication for the first node, obtain the first node journal data from the second persistent memory, and generate and provide a reply to a backend using the first node journal data.