Multi-Controller Journal Replication for Failover Data Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data management systems face challenges in maintaining data accessibility and transaction processing when a controller fails, as log files stored on the controller become inaccessible, leading to service disruptions.
Innovation Solution
A data management system with multiple controllers, each equipped with persistent memory, implements cyclic node replication to write and replicate journal data across nodes, ensuring data availability and continuity even in the event of controller failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If log files are stored on the controller to reduce latency, then transaction processing speed is improved, but system reliability deteriorates when the controller fails
Solution Approach 1:
The patent implements journal replication where each controller maintains a local journal and replicates it to other controllers in the cluster. This copying mechanism ensures that if one controller fails, its journal data is still accessible from replicated copies on other controllers, thus maintaining system reliability while preserving the low-latency benefit of local journal storage
Solution Approach 2:
The patent introduces a coordinator node that manages failover detection and orchestration. When a controller fails, the coordinator detects the failure and coordinates the failover process, acting as an intermediary that ensures reliable transition without compromising the speed of normal operations
2Reliability
If journal data is replicated across multiple controllers, then system reliability is improved, but device complexity increases
Solution Approach 1:
The patent divides the system into distinct functional components: data-bearing controllers that store journal data, coordinator nodes that manage failover detection, and client nodes that handle requests. This segmentation allows each component to have specialized, simplified responsibilities while collectively achieving high reliability through replication
Solution Approach 2:
The patent establishes journal replication as a preliminary action that occurs continuously in the background before failures happen. This proactive replication ensures data is already distributed across controllers, eliminating the need for complex real-time coordination during failure events and simplifying the overall system architecture
Data Source
AI summary
A data management system can include a disk unit and a set of controllers. The disk unit can contain, at least in part, a set of storage media, a first persistent memory, and a second persistent memory. The set of storage media can be configured to implement a storage space. The set of controllers can be configured to write to the storage space and to implement a set of nodes including a first node and a second node. The first node can be configured to generate and write first node journal data to the first persistent memory. The second node can be configured to obtain a failure indication for the first node, obtain the first node journal data from the second persistent memory, and generate and provide a reply to a backend using the first node journal data.