Distributed Database Transaction Metadata Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed database systems, working nodes often face issues with metadata inconsistency, leading to abnormal transaction processing due to delayed synchronization of metadata modifications, which results in inconsistent user data and metadata visibility during transactions.
Innovation Solution
A database transaction processing method that involves obtaining a transaction timestamp and the current global latest version information before starting a transaction, determining the latest-version metadata, and executing operation statements based on this metadata to ensure timely access to modified metadata across the system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If working nodes obtain metadata from local cache, then transaction processing speed is improved, but metadata consistency deteriorates due to delayed synchronization
Solution Approach 1:
The system performs preliminary actions by obtaining the global latest version information before starting the transaction. This allows the working node to proactively check and synchronize metadata version information in advance, ensuring that the metadata used during transaction execution is the latest version, thus resolving the contradiction between fast local cache access and metadata consistency
Solution Approach 2:
The system implements a feedback mechanism by comparing the local metadata version with the global latest version information. When inconsistency is detected, the working node retrieves updated metadata from the global metadata storage and updates its local cache, ensuring continuous synchronization between distributed nodes while maintaining efficient local access
2Reliability
If working nodes synchronize metadata from global metadata storage, then metadata consistency is improved, but system latency increases
Solution Approach 1:
Instead of synchronizing all metadata continuously, the system performs partial synchronization by only updating metadata when version inconsistencies are detected through the version information comparison. This selective update approach minimizes unnecessary synchronization operations and reduces system latency while maintaining metadata consistency
Solution Approach 2:
The system changes the parameter of metadata synchronization from continuous to event-driven based on version information changes. By using version timestamps as a parameter, the system only triggers synchronization when the global latest version information changes, reducing unnecessary synchronization operations and lowering system latency
3Productivity
If working nodes use local cached metadata, then access efficiency is improved, but transaction accuracy deteriorates due to outdated metadata
Solution Approach 1:
The system performs preliminary verification by obtaining and comparing version information before executing transactions. This preliminary action ensures that the metadata version in the local cache is current, allowing the working node to use local cached metadata with confidence, thus maintaining both access efficiency and transaction accuracy
Solution Approach 2:
The system implements feedback through version information comparison to verify metadata freshness before transaction execution. When the local version matches the global latest version, the working node proceeds with high-speed local access; when mismatched, it retrieves updated metadata, ensuring transaction accuracy without compromising overall access efficiency
Data Source
AI summary
This application discloses a database transaction processing method performed by a computer device. The method includes: in a case of starting a target transaction, obtaining, by a working node in a distributed database system, a transaction timestamp of the target transaction and current global latest version information, the target transaction including at least one operation statement for a target data object, and the global latest version information being version information of latest generated metadata of metadata stored in the distributed database system; determining latest-version metadata of the target data object according to the current global latest version information, and determining to-be-accessed user data of the target transaction according to the transaction timestamp; and executing an operation statement of the target transaction on the to-be-accessed user data based on the latest-version metadata of the target data object.