Master Storage Node Transaction Metadata Push for Read-Only Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed database systems with read-write isolation architecture, a read delay occurs due to the synchronization delay between read-write and read-only nodes, leading to outdated data being read by clients, which is unacceptable for services sensitive to data delay.
Innovation Solution
The master storage node actively pushes transaction status metadata to read-only nodes when a transaction ends, ensuring that the read-only nodes obtain the latest transaction status, thus eliminating the read delay and maintaining data consistency across nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If log replication is used to synchronize data between read-write node and read-only node, then data consistency is maintained, but read delay occurs due to periodic synchronization
Solution Approach 1:
The system performs preliminary actions by pre-calculating and pre-positioning transaction status metadata (including expiration information of data pages) at the read-only node before actual data updates occur. When a read request arrives, the read-only node can immediately determine whether to serve from cache or request fresh data, eliminating the periodic synchronization delay.
Solution Approach 2:
The system implements feedback by having the read-only node send requests to the read-write node to obtain transaction status metadata, and the read-write node responds with updated information including data page expiration status. This feedback loop enables the read-only node to maintain up-to-date cache validity information without continuous polling.
2Device complexity
If periodic synchronization is implemented with fixed time periods, then system complexity is reduced, but data freshness deteriorates with seconds to minutes delay
Solution Approach 1:
The system transitions from static periodic synchronization to dynamic on-demand synchronization. The read-only node dynamically requests transaction status metadata based on actual read requests, and the read-write node dynamically provides updates only when transactions occur. This dynamic approach maintains data freshness without the rigidity of fixed time periods.
3Speed
If read-only node caches data locally, then read performance is improved, but data staleness occurs due to synchronization delay
Solution Approach 1:
The read-only node performs preliminary actions by obtaining and caching transaction status metadata (including data page expiration information) before executing read operations. This preliminary acquisition of validity information allows the node to quickly determine cache freshness without sacrificing read performance.
Solution Approach 2:
The system replaces the mechanical periodic synchronization mechanism with an information-based approach using transaction status metadata. Instead of physically synchronizing data at fixed intervals, the system substitutes this with exchanging compact metadata about data page expiration status, enabling faster and more efficient cache validity verification.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
This application provides a data processing method and apparatus. The method includes: receiving, by a master storage node, information that is about a first transaction and that is sent by a read-write node, where the information about the first transaction is used to request to perform a write operation on first data stored on the master storage node; determining, by the master storage node, the first data based on the information about the first transaction, and executing the first transaction; generating, by the master storage node, first transaction status metadata when the first transaction ends, where the first transaction status metadata includes identification information of expired data and identification information of the first transaction; and sending, by the master storage node, the first transaction status metadata to at least one read-only node. According to the data processing method and apparatus in embodiments of this application, a read delay of a read-only node can be eliminated.