Shared-Nothing Database Slice Coordination for Read Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Shared-nothing database systems face limitations in workload balancing and data accessibility, as they typically require data to be processed on the same node as its storage, restricting parallelism and efficiency, especially when reading data across multiple nodes without locking or blocking operations.
Innovation Solution
The system introduces the concept of 'slices' and 'duplicas' where rows of a table are assigned to slices and stored across multiple nodes, allowing any node to read data from a slice, with primary and secondary duplicas for each slice, enabling parallel processing and snapshot-based retrieval without locks or blocking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a shared-nothing database system requires data to be processed on the same node as its storage, then data integrity is maintained, but workload balancing and parallelism are restricted
Solution Approach 1:
The database system segments data into slices that can be independently distributed across nodes. Each slice contains a portion of the table rows and can be accessed by any node, breaking the rigid one-to-one mapping between data and processing nodes while maintaining data integrity through controlled access mechanisms
Solution Approach 2:
The system implements universal access where any node can serve as a coordinator for any slice. The coordinator role is not bound to specific nodes but can be dynamically assigned to any node in the cluster, enabling flexible workload distribution and parallel processing across the distributed system
2Adaptability or versatility
If a shared-nothing database system requires data to be processed on the same node as its storage, then node autonomy is preserved, but data accessibility across nodes is limited
Solution Approach 1:
The system introduces slice coordinators as intermediary components that manage access to slices. These coordinators handle read and write operations for their assigned slices, allowing nodes to access data on other nodes through the coordinator mechanism while preserving the shared-nothing architecture and node autonomy
3Manufacturing precision
If read operations obtain locks to ensure data consistency, then data accuracy is maintained, but performance and parallelism are reduced
Solution Approach 1:
The system performs preliminary actions by maintaining version information and transaction logs that allow read operations to retrieve consistent snapshots of data without acquiring locks. The coordinator uses this pre-prepared information to provide accurate data readings while avoiding blocking operations
Solution Approach 2:
The system creates copies of data in the form of snapshot versions that can be read without locking the original data. Multiple readers can access different versions simultaneously, and the coordinator manages version selection to ensure data accuracy while maintaining high read performance
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A shared-nothing database system is provided in which the rows of each table are assigned to "slices", and multiple copies ("duplicas") of each slice are stored across the persistent storage of multiple nodes. Requests to read data from a particular row of the table may be handled by any node that stores a duplica of the slice to which the row is assigned. For each slice, a single duplica of the slice is designated as the "primary duplica". All DML operations are performed by the node that has the primary duplica of the slice to which the target row is assigned. The changes are then propagated other duplicas ("secondary duplicas") of the same slice.