Commit Token Coordination for Sequentially Consistent Read Replicas
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In conventional server-based database management systems, read replicas often lag behind the primary server due to asynchronous updates, leading to inconsistent data across replicas and potential concurrency issues, especially in distributed cloud computing networks.
Innovation Solution
Implementing sequential consistency across a distributed cloud computing network by using a Lamport timestamp (commit token) associated with each request, ensuring all write queries go to the primary database, and read replicas delay read operations if the commit token is outdated, synchronizing with the primary database before servicing requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If asynchronous read replication is used to improve database performance and throughput, then query latency is reduced and load is distributed, but replica lag increases and data consistency deteriorates
Solution Approach 1:
The system performs preliminary actions by checking the commit token timestamp against the current read replica timestamp before allowing read operations. This pre-check prevents inconsistent reads before they occur, resolving the contradiction between allowing asynchronous replication for performance and ensuring data consistency.
Solution Approach 2:
The system implements feedback by continuously monitoring the commit token timestamp from the primary database and comparing it with the read replica's current timestamp. When the primary database commits new transactions, the feedback mechanism triggers updates to read replicas, ensuring they catch up to maintain consistency while allowing asynchronous operation for high throughput.
2Extent of automation
If read replicas are updated independently to enable parallel processing, then system concurrency is improved, but query result consistency across replicas deteriorates
Solution Approach 1:
The commit token acts as an intermediary mechanism that coordinates independent read replica updates. Each replica uses the commit token as a mediator to determine when it should catch up to the primary database, allowing parallel processing while ensuring query result consistency through a single source of truth for the commit token.
Solution Approach 2:
The system changes the parameter of using commit token timestamps to track and synchronize data versions across replicas. By monitoring this temporal parameter, the system enables independent parallel processing of read replicas while maintaining query consistency through timestamp-based coordination.
3Reliability
If the read replica checks for updated commit tokens before each read operation, then data freshness is improved, but query latency increases
Solution Approach 1:
The system performs the commit token check as a preliminary action that is cached and updated asynchronously. The read replica maintains a local copy of the commit token timestamp and only checks for updates when necessary, reducing the overhead of frequent checks while ensuring data freshness.
Solution Approach 2:
The system ensures continuity of useful action by maintaining continuous background synchronization between the primary and read replica databases. The read replica continuously monitors for new commits and updates its local state, so when read operations occur, the data is already fresh without requiring expensive real-time checks.
Data Source
AI summary
Sequential consistency across a distributed cloud computing network is described. A database includes a primary database and multiple read replica databases. Write queries are transmitted to the primary database, and commit tokens are provided to the read replica databases and the clients. Commit tokens are included in requests from clients. If a request for a read operation received at a read replica database does not include a token that is later than a commit token of the most recent update to the read replica database, the read operation is served by the primary database. If a request for a read operation received at a read replica database includes a token that is later than a commit token of the most recent update to the read replica database, the read replica database delays servicing the read update until it receives an update from the primary database with an updated commit token.