Prefix Hash Tree Conflict Resolution in Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed Hash Table (DHT) systems with Prefix Hash Trees (PHTs) face challenges in managing write conflicts and ensuring eventual consistency, leading to conflicts that are not adequately addressed in environments prioritizing availability over consistency.
Innovation Solution
A method is introduced to resolve eventual consistency conflicts by creating a prefix hash tree with multiple nodes, aggregating conflicting records, splitting nodes when the record count exceeds a maximum threshold, and using delete-timestamps to manage record deletion, ensuring data integrity and stability in high-availability environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If DHT systems prioritize availability over consistency, then dynamic scaling and system availability are improved, but write conflicts and data consistency deteriorate
Solution Approach 1:
The patent introduces a conflict resolution mechanism that acts as an intermediary between conflicting write operations in DHT systems. When write conflicts are detected (multiple versions of the same record), the system uses timestamp-based validation and selective merging to resolve conflicts, allowing the system to maintain availability while eventually achieving consistency. This mediator approach enables the system to handle concurrent writes without blocking, thus preserving availability while resolving consistency issues.
Solution Approach 2:
The patent implements dynamic conflict resolution strategies that adapt based on the state of the system. The conflict resolution mechanism is not static but dynamically adjusts its behavior based on conflict detection, timestamp comparison, and record merging needs. This dynamic approach allows the system to maintain high availability during normal operation while automatically resolving consistency issues when conflicts are detected, thus balancing both availability and consistency requirements.
2Productivity
If multiple writers act independently on physically separate DHT components, then system scalability and availability are improved, but write conflicts increase
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors for write conflicts across distributed DHT components. When conflicts are detected, the system provides feedback through conflict resolution protocols that compare timestamps and merge records appropriately. This feedback loop enables independent writers to operate concurrently without causing permanent data loss, as the system automatically detects and resolves conflicts, thus maintaining scalability while mitigating the harmful effects of write conflicts.
Solution Approach 2:
The patent prepares for potential write conflicts in advance by implementing timestamp-based versioning and conflict detection mechanisms before conflicts become problematic. Each write operation includes timestamp information, and the system is pre-configured to handle conflicts through selective merging. This beforehand cushioning approach ensures that when independent writers create conflicts, the system is already equipped to resolve them, thus maintaining scalability while preventing data loss from write conflicts.
3Stability of the object's composition
If traditional ACID database systems are used, then transactional integrity and data consistency are improved, but dynamic scaling capability deteriorates
Solution Approach 1:
The patent segments the strict ACID transaction model into more flexible components suitable for distributed systems. Instead of requiring full ACID compliance across the entire distributed system, the patent implements selective consistency mechanisms that provide transactional integrity where needed while allowing independent operation elsewhere. This segmentation enables the system to maintain data consistency for critical operations while achieving dynamic scaling by allowing independent writers to operate on different DHT components without centralized coordination.
4Adaptability or versatility
If PHT nodes store multiple records to support range search, then search functionality is improved, but node size and complexity increase
Solution Approach 1:
The patent implements a nested structure where PHT nodes contain records organized in a hierarchical manner. Leaf nodes store actual data records while internal nodes store routing information and prefixes. This nesting allows the system to support range search functionality efficiently by organizing records in a structured hierarchy, thus providing adaptability for various search operations while managing node complexity through the organized nested structure rather than random or flat storage.
Data Source
AI summary
This invention pertains generally to a method for resolving and managing eventual consistency conflicts when using a Prefix Hash Tree (“PHT”) in which the underlying storage system is a Distributed Hash Table (“DHT”). An embodiment according to the within invention comprises a computer implemented method for resolving eventual consistency conflicts when using a PHT and DHT. The method includes using an algorithm to create a prefix hash tree that relates to a distributed hash table, the prefix hash tree having multiple nodes, wherein the nodes comprise leaf nodes and internal nodes, aggregating all distinct records across all conflicting nodes, wherein conflicting nodes are nodes that have identical data records, and counting records returned to obtain an aggregated record count; splitting the conflicting nodes that have the aggregated record counts that exceeding a max record count, such that the aggregated records are divided by a prefix into each new node, resulting in the consolidation of duplicate data that were in conflicting nodes into a single data record in a single node; and splitting conflicting nodes that are internal nodes regardless of the aggregated record count.