Versioned Hierarchical Data Structures in Distributed Stores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hierarchical storage systems face challenges in maintaining consistent versions of data without sacrificing availability, leading to increased complexity and operational costs due to the need for synchronization protocols that can overwhelm single components.
Innovation Solution
Implementing versioned hierarchical data structures that allow multiple consistency and isolation levels, enabling clients to specify desired performance levels by maintaining multiple versions of data structures and utilizing transaction logs for conflict detection and snapshot isolation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronization protocols are implemented to maintain consistent versions of hierarchical data, then data consistency is improved, but system availability deteriorates due to single component overload
Solution Approach 1:
The patent divides the data store into multiple independent versioned copies distributed across different nodes. Each node maintains its own version of the hierarchical data structure with version identifiers, allowing parallel access without centralized coordination. This segmentation eliminates the single component bottleneck while maintaining consistency through version comparison.
Solution Approach 2:
The patent creates multiple copies of the hierarchical data structure, each with version identifiers. Clients can access any copy to read data and can perform writes to multiple copies simultaneously. The versioning mechanism allows the system to track changes across copies without requiring synchronized updates, thereby maintaining availability while ensuring eventual consistency.
2Productivity
If multiple versions or replicas of data are stored on multiple storage nodes to improve availability, then system availability is improved, but device complexity increases due to synchronization requirements
Solution Approach 1:
The patent pre-assigns version identifiers to each copy of the hierarchical data structure before writes occur. When a client performs a write operation, the system generates a new version identifier and applies it to the affected copies. This preliminary versioning eliminates the need for complex runtime synchronization protocols, reducing operational complexity while maintaining multiple available replicas.
Solution Approach 2:
The patent introduces version identifiers as a new parameter that tracks the state of each data copy. Instead of using complex synchronization protocols to manage replica consistency, the system uses simple version number comparison to determine whether reads should be served from local copies or require coordination. This parameter-based approach dramatically simplifies the complexity of managing multiple replicas.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A distributed data store may maintain versioned hierarchical data structures. Different versions of a hierarchical data structure may be maintained consistent with a transaction log for the hierarchical data structure. When access requests directed to the hierarchical data structure are received, a version of the hierarchical data structure may be identified for processing an access request. For access requests with snapshot isolation, the identified version alone may be sufficient to consistently process the access request. For access requests with higher isolation requirements, such as serializable isolation, transactions based on the access request may be submitted to the transaction log so that access requests resulting in committed transactions may be allowed, whereas access requests resulting in conflicting transactions may be denied.