Distributed Hash Table Storage via Key Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As data storage needs increase, traditional hash tables face inefficiencies in handling large data sets due to the complexity of operations and the need for larger memory allocation, which affects the utilization of available space, especially when dealing with large key-value combinations like IP addresses and MAC addresses.
Innovation Solution
The method involves splitting a key into multiple sub-keys and distributing them across multiple hash tables, where each sub-key, except the final one, is paired with an identifier pointing to the next sub-key, and the final sub-key is paired with the associated value, allowing for efficient storage and retrieval by using smaller sub-keys and identifiers to access the data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional hash tables are used to store large data sets, then data storage capability is improved, but memory allocation complexity and space utilization efficiency deteriorate
Solution Approach 1:
The patent divides the original key into multiple sub-keys (first sub-key, second sub-key, etc.) and distributes them across multiple hash tables. Each sub-key is stored in a separate hash table with pointers linking them together, allowing the system to handle large data sets by segmenting the key-value pairs across multiple smaller tables rather than one large table, thereby reducing memory allocation complexity while maintaining storage capability.
2Ease of manufacture
If traditional hash tables store complete key-value pairs, then data storage is simplified, but space utilization efficiency deteriorates
Solution Approach 1:
The patent segments the key into multiple sub-keys and stores only the necessary portions in each hash table entry. Each entry contains only the current sub-key and a pointer to the next sub-key, rather than storing complete key-value pairs. This segmentation reduces the space required for each entry while maintaining the ability to retrieve complete data through sequential pointer following, thereby improving space utilization efficiency.
Solution Approach 2:
The patent implements a nested structure where each hash table entry contains a pointer to the next sub-key, creating a linked chain of nested entries. This nested arrangement allows efficient space utilization by storing only essential information in each entry (current sub-key and pointer) while maintaining access to complete data through the nested pointer references, reducing overall space requirements compared to storing complete key-value pairs in each entry.
3Quantity of substance
If hash tables are designed for large key-value combinations, then data storage capacity is improved, but entry width and memory requirements increase
Solution Approach 1:
The patent segments the key into multiple sub-keys of smaller size, with each sub-key stored in a separate hash table entry. This segmentation allows the system to maintain large data storage capacity by distributing data across multiple tables while using smaller entry widths in each individual table, reducing the memory requirements per entry while maintaining overall storage capacity.
Data Source
AI summary
Systems and methods are described that provide for distributively storing and accessing data across multiple hash tables, such that utilization of the hash tables is optimized. In particular, a key associated with a value is split into two or more sub-keys and the sub-keys are inserted into respective hash tables with associated values. For each sub-key except the final sub-key derived from a particular key, the value paired with the sub-key is an identifier that points to the location of the next sub-key and its associated value, which may be stored in the other hash tables. The final sub-key derived from the original key is paired with the value associated with the key, such as an action to be performed. Thus, rather than using a single key (which may be very large) to access or store a particular value, multiple (smaller) sub-keys are used to ultimately access the same value via multiple, smaller hash tables.