Dynamic Hash Table Resizing Using Summary Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large hash tables can be time-consuming and processor-intensive to search through, especially when they exceed local memory limits, necessitating slower access memory and increasing search time, and existing solutions do not efficiently support dynamic resizing for read-only or dynamically changing hash tables.
Innovation Solution
A method for dynamically resizing a data structure using a summary table with buckets that can be split, allowing for efficient searching by constructing a summary table with prefix and signature tables derived from values, enabling probabilistic determination of entry existence and reducing search latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If large hash tables are kept in larger or denser memory, then storage capacity is improved, but search time increases due to slower access memory
Solution Approach 1:
The hash table is divided into multiple levels: a first hash table stored in fast memory (DRAM) and a second hash table stored in slower memory. The first hash table contains a subset of entries and serves as a quick lookup table, while the second hash table contains the remaining entries. This segmentation allows the system to take advantage of fast memory access for frequently accessed data while maintaining large storage capacity in slower memory, thereby resolving the contradiction between storage capacity and search time.
2Adaptability or versatility
If hash tables support dynamic insertion and deletion, then adaptability is improved, but searching becomes more processor-intensive
Solution Approach 1:
The system implements dynamic hash tables where the first and second hash tables can be independently resized based on workload requirements. The processor can dynamically adjust the distribution of entries between the two hash tables, adding or removing entries as needed. This dynamic capability allows the system to adapt to changing data sizes and access patterns while maintaining efficient search performance through the multi-level structure, resolving the contradiction between adaptability and processor intensity.
Data Source
AI summary
A computing or storage system constructs a table in memory, and constructs a summary table that summarizes the table. The summary table is for determining whether there is likely an entry for a value in the table. The summary table has buckets pointed to by address fields of values. The first bucket in the summary table is split into a second bucket and a third bucket. Prior to the split, the first bucket is pointed to by a first address field of a first value. After the split, the second bucket and the third bucket are pointed to by the first address field plus one extra bit derived from a remainder of the first value.