Database Node Metadata Server for High Availability Coherency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for maintaining data coherency across database nodes in high availability systems are impractical due to synchronous approaches that waste time on confirmations and asynchronous approaches that risk transactions being committed without notifying other nodes.
Innovation Solution
A node writes transaction metadata to a shared storage, which is then indirectly sent to other nodes via a metadata server, allowing them to update their local buffers without waiting for confirmations, ensuring that all nodes are aware of the current state through a transaction log and catalog.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous confirmation approach is used to maintain data coherency, then data consistency across nodes is ensured, but system performance deteriorates due to time-consuming confirmation waits
Solution Approach 1:
The patent applies preliminary action by having nodes proactively send notifications to a metadata server when transactions are committed, before other nodes need to query for updates. This allows the metadata server to pre-process and distribute update information, eliminating the need for synchronous confirmation waits while maintaining data coherency through advance preparation of update data.
Solution Approach 2:
The metadata server acts as an intermediary between database nodes, receiving transaction notifications from committing nodes and distributing update information to other nodes. This mediator architecture eliminates direct synchronous confirmation communication between nodes, reducing communication overhead and improving system performance while maintaining data consistency through centralized coordination.
2Productivity
If asynchronous notification approach is used to improve performance, then confirmation wait time is reduced, but reliability deteriorates as transactions may be committed without notifying other nodes
Solution Approach 1:
The patent implements feedback by having nodes send notifications to the metadata server when transactions are committed, creating a feedback loop that tracks the state of all nodes. The metadata server uses this feedback information to determine which nodes need updates and distributes them asynchronously, ensuring no transaction is missed while maintaining high performance through non-blocking communication.
Solution Approach 2:
The metadata server serves as a reliable intermediary that receives all transaction notifications and ensures proper distribution to affected nodes. This intermediary guarantees that even in asynchronous operation, no transaction commitment is lost, as the metadata server maintains the authoritative record of all commits and systematically notifies relevant nodes, resolving the reliability issue while preserving performance benefits.
3Reliability
If direct node-to-node notification is used to maintain data coherency, then data consistency is achieved, but system complexity increases due to multiple confirmation channels
Solution Approach 1:
The patent extracts the complexity of inter-node communication coordination by removing direct node-to-node notification requirements. Instead, each node only needs to communicate with the metadata server, which centralizes the complexity of tracking which nodes need updates. This extraction simplifies the system architecture by eliminating multiple confirmation channels while maintaining data coherency through the metadata server's centralized management.
Solution Approach 2:
The metadata server performs multiple functions: it receives transaction notifications, tracks the state of all database nodes, determines which nodes need updates, and distributes update information. This multi-functional universal component replaces the need for complex peer-to-peer notification logic at each node, reducing overall system complexity while ensuring data coherency through centralized coordination.
Data Source
AI summary
Techniques are disclosed relating to a database system synchronizing a current state of the database system among a plurality of nodes configured to handle requests for data of the database system stored in a distributed storage with one of the plurality of nodes being currently active and the other nodes of the plurality of nodes being currently standby nodes. In various embodiments, the active node receives a request to perform a transaction that includes committing data to the distributed storage. In response to receiving the request, in some embodiments, the active node commits the data to the distributed storage to update the current state of the database system and causes storing of metadata that provides an indication of the commitment in a transaction log stored in the distributed storage.