Autonomous Primary-Mirror Reset in Database Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current reset mechanisms in distributed database systems are prone to errors and downtime due to complex external coordination requirements, leading to system instability and user experience disruptions, especially when handling replication faults and communication interruptions between primary and mirror segments.
Innovation Solution
The primary segment is responsible for monitoring health and triggering resets, with the Fault Tolerance Service (FTS) only probing the primary, and the mirror segment disabling reset participation to prevent false fault reporting, employing retry logic for transient communication failures and maintaining replication fault information in local memory for post-reset reporting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If both primary and mirror segments can initiate reset independently, then system responsiveness to faults is improved, but false fault reporting and system instability increase
Solution Approach 1:
The patent applies asymmetry by designating only the primary segment as authorized to initiate resets, while the mirror segment is prohibited from initiating resets even when detecting faults. This asymmetric permission structure prevents the mirror from generating false fault reports that could destabilize the system, while the primary maintains full reset authority to ensure rapid response to actual faults.
Solution Approach 2:
The patent introduces FTS (Fault Tolerance Service) as an intermediary that receives fault reports from the mirror segment but does not allow the mirror to directly initiate resets. Instead, the mirror reports faults to FTS, which then coordinates with the primary segment for reset decisions. This intermediary layer prevents false fault propagation while maintaining system responsiveness.
2Measurement precision
If external coordination is required for resets, then fault detection accuracy is improved, but downtime and complexity increase
Solution Approach 1:
The patent extracts the reset initiation authority from the mirror segment and concentrates it solely in the primary segment. This extraction simplifies the reset coordination mechanism by eliminating the need for complex bidirectional coordination between primary and mirror, thereby reducing reset downtime while maintaining fault detection accuracy through the primary's centralized control.
Solution Approach 2:
The primary segment performs self-service by autonomously initiating resets based on its own fault detection or mirror's fault reports, without requiring complex external coordination protocols. The primary can independently decide and execute resets, reducing coordination overhead and downtime while maintaining system stability.
3Reliability
If mirror reports replication faults independently, then fault detection coverage is improved, but false primary fault reporting increases
Solution Approach 1:
Instead of allowing the mirror to directly report faults to FTS (which causes false primary fault reporting), the patent inverts the reporting relationship: the mirror reports faults to the primary, and the primary then reports to FTS. This inversion ensures that fault information flows through the authorized primary segment, preventing false information while maintaining comprehensive fault detection coverage.
4Reliability
If reset coordination involves multiple nodes, then fault tolerance is improved, but device complexity increases
Solution Approach 1:
The patent segments the reset coordination function by clearly separating responsibilities: the primary segment handles all reset initiation and FTS coordination, while the mirror segment handles only fault detection and reporting to the primary. This segmentation simplifies the overall coordination complexity by eliminating redundant coordination mechanisms while maintaining fault tolerance through the primary-mirror architecture.
Data Source
AI summary
Server supervisor processes in the segment nodes of a database cluster afford transparent reset mechanisms to a fault tolerant service (FTS). FTS probes only primary segments as to their status unless a primary segment does not respond, in which case FTS will probe the mirror of that primary-mirror pair, and change the state of the mirror to primary if the primary is faulty. Only a primary segment to initiate a reset of its mirror segment and then resets itself. If a mirror segment fails, it shuts down and its corresponding primary segment will report the failure to FTS. Transient communication failures are addressed by retry logic on FTS probing, and not reported to FTS. Reset is not initiated by FTS and preserves the segment state in the absence of errors.