Decentralized Database Index Consistency via Coordinated Checkpoint Roll Calls
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database management systems face performance bottlenecks and troubleshooting difficulties due to the partitioning of databases across multiple servers, particularly in handling high request rates and identifying faulty servers.
Innovation Solution
Implementing a sharded database system with a load balancer that assigns queries to servers within the same server set, simplifying troubleshooting and reducing unnecessary routing, while using checkpoint marker messages and a coordinated checkpoint roll call mechanism to ensure index updates are synchronized across all servers without a centralized manager.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a database is partitioned across multiple servers, then data size and transaction volume capacity are improved, but performance bottlenecks and troubleshooting difficulties worsen
Solution Approach 1:
The database system is segmented into multiple independent server sets, where each server set contains multiple servers that collectively hold a complete copy of the database. This segmentation allows the system to handle large data sizes across distributed servers while maintaining manageable complexity through clear separation of server sets and their associated index structures.
Solution Approach 2:
A load balancer acts as an intermediary component that receives client requests and routes them to appropriate servers within server sets. The load balancer simplifies troubleshooting by providing centralized request routing and monitoring capabilities, reducing the complexity of identifying and diagnosing issues across distributed servers.
2Stability of the object's composition
If a centralized manager is used to coordinate checkpoint roll calls, then consistency across servers is improved, but system complexity and single point of failure risks worsen
Solution Approach 1:
Each server set autonomously performs its own checkpoint roll call process without requiring coordination from a centralized manager. Servers within a set independently track their own checkpoint status and coordinate with each other to ensure consistency, eliminating the single point of failure while maintaining stable index composition across the distributed system.
3Measurement precision
If shard-level load balancers are implemented, then query routing precision is improved, but device complexity and routing overhead worsen
Solution Approach 1:
The patent extracts the load balancing function from individual shard levels and consolidates it at the server set level. Instead of implementing complex shard-level load balancers, the system uses load balancers that operate at the server set level, simplifying the routing logic while maintaining sufficient precision for query distribution across the distributed database system.
Data Source
AI summary
A system and method for maintaining consistency across a decentralized database cluster is presented. When a database server receives an index update message, it adds a message to a queue indicating it has received the message and watches an index update node. Once every server in the cluster has added a message to the queue or a timer has expired, it is known that every active database server has received the message. The update can be performed for each server and the changes committed simultaneously. In such a manner, all database servers in the cluster perform an update at approximately the same time. When a database server is unable to complete an update, it enters a fallen behind mode and performs updates more often, as configured in a configuration database. Other embodiments also are disclosed.