Hot Spare Node Recovery for Distributed Database Query Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scale-out systems for distributed database query execution face challenges in fault tolerance and high availability due to node computer failures, leading to disrupted queries and user experience degradation, as they rely on client applications to resubmit queries and do not efficiently handle failures.
Innovation Solution
Implementing a system with hot spare computers that automatically replace failed nodes, allowing for quick and transparent recovery of database query execution by dynamically reallocating query plans and data, thereby minimizing disruption and maintaining system availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If data redistribution is limited to a few node computers using consistent hashing, then system complexity is reduced, but fault tolerance and high availability deteriorate when node failures occur
Solution Approach 1:
The system pre-assigns spare node computers to specific data partitions before failures occur. When a node fails, the pre-assigned spare immediately takes over the failed node's partitions without requiring complex real-time redistribution calculations, thus maintaining fault tolerance while keeping the redistribution mechanism simple.
Solution Approach 2:
Spare node computers act as intermediaries between failed nodes and the system. These spares are pre-positioned to assume specific partitions, serving as a buffer that simplifies the failure recovery process by eliminating the need for complex dynamic redistribution algorithms.
2Device complexity
If the system relies on client application logic to handle query failures and resubmissions, then system complexity is reduced, but user experience and system availability deteriorate
Solution Approach 1:
The distributed database system automatically detects node failures, selects appropriate spare nodes, redistributes data partitions, and resumes query execution without requiring client application intervention. This self-service mechanism maintains simplicity while dramatically improving user experience by eliminating query disruptions.
3Device complexity
If frequent query resubmissions are required after node failures, then system simplicity is maintained, but productivity and user experience deteriorate
Solution Approach 1:
The system maintains continuous query execution by pre-positioning spare nodes and automatically transferring data partitions upon failure. This ensures that query processing continues uninterrupted, eliminating the productivity loss associated with frequent resubmissions while keeping the recovery mechanism simple.
Data Source
AI summary
Herein are acceleration techniques for resuming offloaded execution by replacing a failed computer with a hot spare computer. In an embodiment, a distributed system configures a DBMS, a set of participating computers, and a set of spare computers. The DBMS receives a query of a database. From the query, an offload query plan is generated for distributed execution. The DBMS sends the offload query plan and a respective portion of the database to each participating computer. The distributed system detects that a participating computer failed after the offload query plan was sent. Responsively, the DBMS sends the same offload query plan and same respective portion of the database of the failed computer to a replacement computer from the spare computers. Despite the computer failure, the DBMS receives results of successful distributed execution of the offload query plan that include a result from the replacement computer.