3-M Hash Table for Continuous Storage and Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional hash table designs face issues such as slow access speed, high latency, and inefficient storage due to linked lists and collision resolution, as well as the need for rehashing entire tables, which disrupts storage operations and requires significant programming effort and startup/shutdown delays.
Innovation Solution
The 3-Misses (3-M) hash table design, which includes an index file with base slots and rehashed slots, uses direct and indirect records to store data efficiently, reduces cache hostile accesses, and allows individual slot rehashing to maintain continuous storage operations without serialization/deserialization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional open hash table with linked lists is used, then data can be stored with collision handling, but access speed decreases due to multiple random memory accesses
Solution Approach 1:
The hash table is divided into multiple slots, each containing multiple buckets. This segmentation allows the data structure to handle collisions locally within slots and buckets, reducing the need for linked lists and minimizing random memory accesses. The segmentation into slots and buckets provides a hierarchical structure that improves cache locality and access speed while maintaining collision handling capability.
2Quantity of substance
If traditional closed hash table with fixed size entries is used, then storage space is optimized, but access latency increases due to collision resolution requirements
Solution Approach 1:
The hash table implementation uses dynamic slot allocation where slots can be created and expanded as needed. Each slot contains multiple buckets that can dynamically accommodate records. This dynamic structure allows efficient space utilization while reducing collision resolution overhead, as the system can adapt the number of slots and buckets based on the data load, thereby minimizing access latency.
3Adaptability or versatility
If entire hash table rehashing is performed, then storage capacity is expanded, but storage operations are disrupted and startup/shutdown delays increase
Solution Approach 1:
The hash table is segmented into multiple independent slots, each with its own buckets. When expansion is needed, individual slots can be rehashed independently rather than requiring rehashing of the entire table. This segmentation allows storage operations to continue in other slots during rehashing, maintaining productivity and reducing startup/shutdown delays while still achieving storage capacity expansion.
Solution Approach 2:
The system performs preliminary actions by pre-allocating slots and buckets with appropriate capacity. When the hash table needs expansion, new slots are pre-created and gradually populated, allowing continuous storage operations. This preliminary action approach avoids disruptive full rehashing by preparing expansion capacity in advance and transitioning smoothly.
Data Source
AI summary
The present teaching relates to method, system, medium, and implementations for storage management. A hash table is constructed, having an index file having one or more slots, each of which includes one or more buckets. Each bucket stores one or more types of records, including a direct record, an indirect record, and a forwarding record. A direct record stores data directly in a bucket of a slot of the index file. When a storage request is received related to some relevant data, the request is handled based on the constructed hash table.