Snapshot-Isolated Scale-Out Database Reads Across Slice Duplicas
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies for shared-nothing databases fail to efficiently handle read operations across multiple nodes without locking and blocking, limiting parallelism and workload balancing.
Innovation Solution
A shared-nothing database system is implemented with slices and duplicas, allowing data distribution across multiple nodes, where each table is divided into slices, with primary and secondary copies of each slice stored on multiple nodes, and read operations are handled by any node with a duplica, while DML operations are restricted to the primary node.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a shared-nothing database system restricts data access to only the node where data is stored, then data access consistency is maintained, but workload balancing and system performance deteriorate
Solution Approach 1:
The patent segments data into slices and further into duplicas (primary and secondary copies), distributing them across multiple nodes. This segmentation allows any node with a duplica to serve read requests, improving workload balancing while maintaining data consistency through the primary duplica's authority over DML operations.
Solution Approach 2:
The patent introduces a coordinator process as an intermediary that manages multi-statement transactions across nodes. The coordinator ensures snapshot isolation and coordinates DML operations, allowing nodes to perform reads without direct coordination while maintaining consistency through the intermediary's control of write operations.
2Productivity
If a shared-nothing database system allows any node to read data from any duplica, then read performance and workload balancing improve, but complexity of managing data versions and consistency increases
Solution Approach 1:
The patent establishes primary and secondary duplicas in advance, with the primary duplica pre-designated as the authoritative source for DML operations. This preliminary arrangement simplifies version management during reads, as nodes can immediately determine which duplica to query without complex coordination.
Solution Approach 2:
The patent implements a feedback mechanism where the coordinator process tracks transaction states and communicates with nodes. When a node needs to read data, it queries the coordinator for the appropriate snapshot version, and the coordinator provides feedback on which duplica contains the correct data version, simplifying version management.
3Reliability
If the system uses locks to ensure data consistency during transactions, then data integrity is maintained, but read operation performance and concurrency deteriorate
Solution Approach 1:
The patent assigns primary duplicas to specific nodes in advance, establishing clear ownership before transactions occur. This preliminary assignment eliminates the need for locks during reads, as nodes can confidently query any duplica knowing the primary's authority over DML operations ensures integrity without blocking reads.
Solution Approach 2:
The patent creates secondary duplicas as copies of the primary duplica, allowing read operations to be performed on these copies without affecting the primary or requiring locks. This copying mechanism enables lock-free reads while maintaining data integrity through the primary's exclusive control of DML operations.
4Reliability
If the system requires the coordinator process to manage all transaction coordination, then transaction consistency is ensured, but the coordinator becomes a performance bottleneck
Solution Approach 1:
The patent segments transaction coordination responsibilities: the coordinator handles high-level transaction management and snapshot isolation, while individual nodes autonomously execute DML operations on their local duplicas. This segmentation reduces coordinator workload and eliminates it as a bottleneck, while maintaining consistency through the coordinated approach to multi-statement transactions.
Solution Approach 2:
The patent pre-assigns primary duplicas to specific nodes before transactions begin, establishing clear ownership and data location information in advance. This preliminary action allows nodes to directly access and modify their local duplicas without coordinator intervention for each operation, significantly improving transaction processing speed while the coordinator maintains overall consistency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A shared-nothing database system is provided in which the rows of each table are assigned to "slices", and multiple copies ("duplicas") of each slice are stored across the persistent storage of multiple nodes. Requests to read data from a particular row of the table may be handled by any node that stores a duplica of the slice to which the row is assigned. For each slice, a single duplica of the slice is designated as the "primary duplica". All DML operations are performed by the node that has the primary duplica of the slice to which the target row is assigned. The changes are then propagated other duplicas ("secondary duplicas") of the same slice.