Distributed Database TSO Sorting for MySQL Binlog Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed databases face challenges in ensuring global orderliness and integrity of transactions during data replication, particularly in scenarios requiring high data consistency, and lack compatibility with MySQL's data replication standards.
Innovation Solution
A data processing method for distributed databases that sorts transactions using a global timestamp TSO, creates local transaction lists, performs multiplexed merge sorting to generate global sorting queues, and merges local transactions with identical timestamps to produce a global binary log, ensuring transaction integrity and compatibility with MySQL's Binlog format.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If distributed databases perform data replication without global timestamp sorting, then data replication speed is faster, but global orderliness and integrity of transactions cannot be ensured
Solution Approach 1:
The patent applies preliminary action by sorting transactions using global timestamps TSO before performing data replication. This pre-sorting ensures that transactions are processed in the correct global order, guaranteeing transaction integrity and consistency across distributed nodes. The sorting operation is performed in advance of the replication process, allowing the subsequent replication to proceed efficiently without needing to re-order transactions during the actual data synchronization.
2Reliability
If distributed databases implement global transaction sorting, then transaction consistency is improved, but system complexity increases
Solution Approach 1:
The patent introduces an intermediary mechanism by using a dedicated global timestamp (TSO) as a mediator for sorting transactions across distributed nodes. Instead of implementing complex multi-node coordination protocols, the system uses TSO as a simple intermediary value that naturally orders transactions globally. This intermediary approach simplifies the sorting logic while ensuring transaction consistency, as each node can independently sort its transactions based on the TSO values without requiring complex inter-node communication for ordering decisions.
3Adaptability or versatility
If distributed databases merge local transactions with identical timestamps, then MySQL compatibility is achieved, but processing time increases
Solution Approach 1:
The patent applies merging by combining local transactions that share identical global timestamps TSO into a single batch operation. This merging strategy achieves MySQL compatibility by producing a unified global binary log that reflects the same ordering semantics as MySQL's Binlog. The processing time overhead is minimized by performing the merge operation efficiently - transactions with the same TSO are grouped together and processed as a unit, rather than individually, which reduces the total number of write operations and improves overall processing throughput.
Data Source
AI summary
A data processing method and system for a distributed database, and a device and a storage medium. The method comprises: sorting transactions in a transQueue by using a global timestamp TSO, creating a local transaction list by means of an XA Event and a waitTrans in the transQueue, and generating first-level sorting queues according to the local transaction list (102); for the first-level sorting queue of each DN, performing multi-path merging and sorting, so as to generate a global sorting queue (104); and for the global sorting queue, merging local transactions that have the same global timestamp TSO, and generating a global binary log Binlog (106).