Non-blocking Secondary Reads via Snapshot Isolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems face increased read latency and reduced throughput due to the need to wait for batch writes to complete before processing read commands, especially under high write loads, as they typically lock secondary nodes during write operations.
Innovation Solution
Implementing non-blocking secondary reads by generating and utilizing snapshots of data at specific points in time, allowing read commands to be processed from snapshots while write commands are applied, thus avoiding the need for locks and enabling parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the database system locks secondary nodes during write operations to ensure data consistency, then data consistency is improved, but read latency increases and throughput decreases
Solution Approach 1:
The system generates snapshots of secondary nodes before write operations begin. These snapshots capture the state of data at a specific point in time, allowing read operations to proceed on the snapshot data while writes are applied to the current secondary node. This preliminary action of creating a snapshot enables concurrent read and write operations without compromising data consistency.
Solution Approach 2:
The snapshot acts as an intermediary between read and write operations. Instead of reads directly accessing the secondary node that is undergoing write operations, the read operation accesses the snapshot copy. This intermediary mechanism allows both operations to proceed simultaneously without interfering with each other, thus improving throughput while maintaining consistency.
2Reliability
If the database system waits for batch writes to complete before processing read commands, then data consistency is improved, but read latency increases
Solution Approach 1:
The system proactively generates snapshots before write operations commence, rather than waiting for writes to complete before enabling reads. This preliminary snapshot creation allows the system to immediately serve read requests from the snapshot while writes are being applied, eliminating the waiting period and reducing read latency.
Solution Approach 2:
The system dynamically manages multiple snapshots and determines which snapshot to use for read operations based on the current state of write operations. This dynamic approach allows the system to flexibly switch between snapshots and current data, optimizing both consistency and latency performance based on operational conditions.
3Reliability
If the database system uses locking mechanisms during write operations, then data consistency is improved, but system complexity increases
Solution Approach 1:
The system extracts the locking mechanism entirely from the read-write coordination process. Instead of using locks to prevent concurrent access, the system creates isolated snapshot copies that can be freely accessed by read operations while writes proceed independently on the current secondary node. This extraction eliminates the complexity of locking mechanisms while maintaining data consistency.
Solution Approach 2:
The system creates snapshot copies of the secondary node data before write operations. These copies serve as immutable read-only versions that can be accessed concurrently without any locking. By using copying instead of locking, the system simplifies the concurrency control mechanism while ensuring that reads always have a consistent view of the data.
Data Source
AI summary
Described herein are embodiments of a database system. The database system receives a read command to read at least some stored data. The database system may generate a plurality of snapshots of data stored in a first data storage node of a plurality of data storage nodes. The database system may determine whether data is being written to the first data storage node. Responsive to determining that data is not being written to the first data storage node, the database system may process the read command at least in part by reading at least some data stored on the first data storage node. Responsive to determining that data is being written to the first data storage node, the database system may process the read command at least in part by reading at least some data from a snapshot of the plurality of snapshots.