Hash Table Domain Splitting for Atomic Storage Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage systems face challenges in ensuring atomicity and consistency during read-modify-write operations on non-volatile memory, particularly when dealing with unreliable tiers such as RAID over multiple disks, leading to potential data inconsistencies and the need for excessive journal implementations.
Innovation Solution
The method involves splitting a hash table into domains with a spare bucket for each subset, using metadata structures with head and spare bucket pointers that interleave during updates, ensuring atomicity and consistency without the need for lockless updates, and allowing for scalable and efficient expansion of the hash table.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a simple read-modify-write operation is performed on non-volatile memory block devices, then the operation speed is improved, but data consistency cannot be guaranteed under failure conditions
Solution Approach 1:
The patent applies preliminary action by pre-allocating multiple buckets (including spare buckets) and organizing them into domains before write operations occur. Each domain has pre-configured metadata structures with head and spare_bucket pointers. When a write operation occurs, the system can immediately use the pre-prepared spare buckets without needing to perform additional allocation or journaling operations, thus maintaining both speed and consistency.
2Reliability
If excessive journal implementations are used to guarantee atomicity, then data consistency is improved, but device complexity and metadata overhead increase
Solution Approach 1:
The patent segments the hash table into multiple domains, where each domain contains a subset of buckets and has its own metadata structure with head and spare_bucket pointers. This segmentation allows independent management of each domain's consistency without requiring global journaling mechanisms. The segmentation reduces metadata overhead by localizing the consistency management to individual domains rather than requiring system-wide journal structures.
Solution Approach 2:
The patent uses copying by maintaining spare buckets as copies of the bucket structure. Instead of using complex journal implementations that record all modifications, the system simply copies the necessary metadata (head pointer and spare_bucket pointer) to the spare bucket before write operations. This copying approach guarantees atomicity with minimal metadata overhead, as only essential pointer information needs to be replicated rather than full journal entries.
3Reliability
If lockless updates are avoided in favor of traditional locking mechanisms, then data consistency is improved, but productivity and update efficiency decrease
Solution Approach 1:
The patent implements self-service through its domain-based architecture with spare buckets. Each domain independently manages its own consistency using pre-configured head and spare_bucket pointers. When updates occur, the system automatically switches between buckets within the domain without requiring external locking mechanisms. This self-service approach maintains data consistency through the structural design rather than through active coordination, thereby preserving high update efficiency.
Data Source
AI summary
An aspect includes splitting a table of buckets into a fixed number of domains. Each of the domains includes a corresponding subset of the buckets. An aspect also includes providing a spare bucket for each of the subsets of the buckets and providing a metadata structure for each of the domains. The metadata structure includes a head pointer that points to a first bucket of a corresponding subset of the buckets and a spare_bucket pointer that points to the spare bucket of the subset of the buckets. An aspect further includes providing a split-spare bucket pointer that interleaves, during updates to data, among the subset of buckets in the domain. Data subject to the updates is stored in the spare bucket for a corresponding one of the domains. An aspect also includes updating the head pointer and the spare_bucket pointer for corresponding domains in response to updating the data.