Distributed Database Commit Timestamps for Ordered Binary Logs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed databases, the chronological order of binary logs generated by different storage nodes is uncertain, leading to inconsistent data ordering and inconsistencies in downstream applications due to direct subscription to these logs from different nodes.
Innovation Solution
A distributed database system with a computing module that controls storage modules to execute transactions, generates commit timestamps for binary logs, ensuring each log is globally ordered, thereby maintaining consistent data ordering across the system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If each storage node independently generates binary logs locally, then the system achieves high concurrency and distributed processing capability, but the chronological order of binary logs across different storage nodes becomes uncertain
Solution Approach 1:
The patent introduces a coordinator node as an intermediary that assigns globally unique transaction IDs and timestamps to transactions across the distributed system. This coordinator acts as a central authority that ensures chronological ordering without blocking the independent binary log generation at each storage node, thus resolving the contradiction between distributed processing capability and data ordering consistency.
Solution Approach 2:
The system performs preliminary actions by having the coordinator node pre-assign transaction IDs and timestamps before transactions are executed at different storage nodes. This preliminary assignment of temporal information ensures that binary logs generated independently at various nodes can be correctly ordered later based on these pre-assigned timestamps, maintaining consistency while allowing parallel execution.
2Speed
If binary logs from different storage nodes are directly subscribed to, then real-time data synchronization is achieved, but incorrect ordering of operations occurs leading to inconsistent data
Solution Approach 1:
The patent introduces a log ordering service as an intermediary between binary log generation and consumption. This service receives binary logs from multiple storage nodes, orders them based on globally unique timestamps assigned by the coordinator, and then delivers them to downstream consumers. This intermediary layer enables real-time synchronization while ensuring correct operational ordering.
Solution Approach 2:
The system replaces the mechanical approach of relying on local clock timestamps at each storage node with a centralized logical timestamping mechanism. The coordinator node issues monotonically increasing transaction IDs and timestamps that serve as a global ordering reference, substituting distributed clock synchronization with a logical clock system that guarantees correct ordering without requiring physical time synchronization.
Data Source
AI summary
A distributed database system includes: a computing module configured to receive a transaction starting request; determine a target transaction and a target storage module corresponding to the target transaction based on the transaction starting request; send an execution instruction to a target storage module; send a commit instruction to the target storage module when the target storage module returns a message that the target transaction is executed successfully, the commit instruction including a commit timestamp of the target transaction; and the target storage module configured to receive the execution instruction, and execute the target transaction according to the execution instruction; return the message that the target transaction is successfully executed to the computing module when the target transaction is successfully executed; and receive the commit instruction, commit the target transaction according to the commit instruction, and generate a binary log according to the commit timestamp of the target transaction.