Hash Table Lookup via Local Index Signature Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In high-speed packet processing nodes, existing hash table lookup methods require multiple memory accesses, leading to processing delays, as they search through hash chains in external memory for matching entries.
Innovation Solution
An index table stored in local memory is used to predict the location of hash entries by deriving signature patterns from hash keys, allowing the hash table to be accessed only once during a lookup, reducing the number of memory accesses and processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional hash table lookup is used with external memory, then complete hash chain search can be performed, but processing delay increases due to multiple memory accesses
Solution Approach 1:
The hash table is divided into multiple buckets, each containing a subset of hash entries. The index table stores signature patterns for each bucket, allowing the search to be segmented into two phases: first identifying the correct bucket using the index table, then searching only within that bucket. This segmentation reduces the average search space and minimizes memory accesses required.
Solution Approach 2:
Signature patterns are pre-computed and stored in the index table before the actual lookup operation. These signatures represent characteristic bit patterns of hash keys in each bucket. During lookup, the incoming hash key is compared against these pre-computed signatures to quickly identify the target bucket, avoiding the need to search all buckets during the actual query operation.
2Productivity
If hash buckets with hash chains are used, then the search space is reduced, but multiple memory accesses are still required to traverse the hash chain
Solution Approach 1:
An index table is introduced as an intermediary data structure between the hash key and the hash table buckets. This index table contains signature patterns that act as mediators to guide the search process. By comparing the incoming hash key against these intermediate signatures, the system can quickly identify which bucket to search, reducing the complexity of memory access patterns and minimizing the number of memory accesses required.
3Quantity of substance
If the hash table is stored in external memory, then large capacity is achieved, but access time increases due to external memory latency
Solution Approach 1:
The signature patterns are extracted from the full hash keys and stored separately in an index table. This extraction allows the most critical search information (signatures) to be accessed quickly while the full hash table remains in external memory for capacity. The extracted signatures serve as a fast-access guide that reduces the frequency and depth of external memory accesses required for complete lookups.
Data Source
AI summary
The present invention relates to methods and apparatus for performing a lookup on a hash table stored in external memory. An index table stored in local memory is used to perform an enhanced lookup on the hash table stored in external memory. The index table stores signature patterns that are derived from the hash keys stored in the hash entries. Using the stored signature patterns, the packet processing node predicts which hash key is likely to store the desired data. The prediction may yield a false positive, but will never yield a false negative. Thus, the hash table is accessed only once during a data lookup.