Shared-Nothing Database Slices for Lock-Free Snapshot Reads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Shared-nothing database systems face constraints in workload balancing and data access, particularly when reading data items that require multiple nodes and avoiding locks during read operations.
Innovation Solution
A shared-nothing database system is designed with slices and duplicas, where rows are assigned to slices and stored across multiple nodes, allowing any node to read from duplicas while DML operations are performed on the primary duplica, using delta logs and row heaps for versioning and snapshot-based retrieval without locks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a shared-nothing database system is used with traditional storage architectures, then data consistency and reliability are improved, but workload balancing and data access flexibility deteriorate due to node-specific storage constraints
Solution Approach 1:
The patent segments data into slices and further into duplicas, allowing fine-grained distribution across nodes. Each slice is divided into multiple duplicas that can be independently assigned to different nodes, enabling flexible workload balancing while maintaining data consistency through the slice-dupa hierarchy
Solution Approach 2:
The patent introduces an intermediary mapping layer that decouples logical data organization from physical storage assignment. This mapping mechanism allows the system to maintain reliable data relationships while providing flexible node assignment and workload distribution without direct node-storage coupling
2Ease of operation
If data is stored on node-specific persistent storage in a shared-nothing database, then data access simplicity is improved, but read operation parallelism and performance deteriorate due to limited node access capability
Solution Approach 1:
The patent makes duplicas universal by allowing multiple nodes to access the same dupa for read operations. While DML operations remain node-specific, read operations can be performed from any node that has access to a dupa, enabling parallel read access without compromising data simplicity
Solution Approach 2:
The patent creates multiple copies (duplicas) of data slices across different nodes. These copies enable parallel read access from multiple nodes simultaneously, improving read operation productivity while maintaining data access simplicity through consistent copy structures
3Reliability
If locks are obtained during read operations to ensure data consistency, then data reliability is improved, but operation speed and system performance deteriorate due to blocking and waiting
Solution Approach 1:
The patent performs preliminary actions by maintaining version information and snapshot data in advance. Read operations can retrieve data from snapshots without requiring locks, as the versioning information is prepared beforehand, enabling consistent reads without blocking
Solution Approach 2:
The patent replaces the mechanical locking system with a versioning-based retrieval system. Instead of using locks to ensure consistency, the system uses snapshot versions and version metadata to provide consistent reads without the performance overhead of locking mechanisms
4Adaptability or versatility
If multiple nodes are enabled to perform the same read task, then workload balancing capability is improved, but system complexity increases due to additional coordination and management overhead
Solution Approach 1:
The patent segments data into independent slices and duplicas that can be independently managed and assigned. This segmentation allows workload balancing to operate at the dupa level without requiring complex system-wide coordination, reducing overall system complexity while improving balancing capability
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A shared-nothing database system is provided in which the rows of each table are assigned to "slices", and multiple copies ("duplicas") of each slice are stored across the persistent storage of multiple nodes. Requests to read data from a particular row of the table may be handled by any node that stores a duplica of the slice to which the row is assigned. For each slice, a single duplica of the slice is designated as the "primary duplica". All DML operations are performed by the node that has the primary duplica of the slice to which the target row is assigned. The changes are then propagated other duplicas ("secondary duplicas") of the same slice.