Stream Processor Hash Table Lookup Using Buffer Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Stream processors face inefficiencies in data lookups due to hash table collisions, which lead to errors in identifying matching data strings, especially when implementing large FIFO sizes and minimizing power consumption, as content-addressable memory (CAM) solutions are limited by size and power constraints.
Innovation Solution
A method is implemented where a search string is parsed into tokens, each hashed into address and data hashes of shorter lengths, allowing for efficient searching in a hash table with minimized collision errors, by selecting the most frequently pointed locations in the buffer, thereby reducing the likelihood of false positives and improving lookup accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If hash table lookups are used for data storage and retrieval, then power consumption is reduced and FIFO size can be increased, but collision-based errors increase and lookup accuracy deteriorates
Solution Approach 1:
A buffer is introduced as an intermediary layer between the hash table and the data source. The buffer stores candidate locations identified by hash table lookups, allowing for verification and resolution of collision-based errors without requiring additional power-consuming CAM structures. This mediator enables the system to handle collisions gracefully while maintaining low power consumption.
Solution Approach 2:
The patent replaces the mechanical/combinatorial approach of CAM (content-addressable memory) with a computational hashing approach. Instead of using CAM's parallel comparison mechanism which consumes high power, the system uses hash functions to compute addresses and a buffer-based verification process to resolve collisions, substituting a lower-power computational method for a higher-power hardware mechanism.
2Reliability
If CAM solutions are used to eliminate collision errors, then lookup accuracy is improved, but power consumption increases and device size is limited
Solution Approach 1:
The patent uses a buffer as a temporary, low-cost storage structure to hold candidate locations from hash table lookups. Instead of investing in expensive, high-power CAM structures, the system uses a simple buffer that can be cleared and reused, accepting temporary imprecision from hash collisions and resolving them through subsequent verification steps.
Solution Approach 2:
The patent changes the approach from exact matching (CAM) to probabilistic matching with verification (hashing + buffer). By changing the lookup parameter from deterministic to probabilistic, the system can use lower-power hash functions and resolve accuracy issues through multiple sampling and verification passes using the buffer, rather than requiring high-power exact-match hardware.
3Quantity of substance
If larger FIFO sizes are implemented to store more streaming data, then data capacity is increased, but hash table collision frequency increases
Solution Approach 1:
The patent segments the data storage into multiple components: a hash table for initial address computation, a buffer for storing candidate locations, and the main FIFO for data storage. This segmentation allows the system to handle larger FIFO sizes by distributing the lookup process across multiple stages, with the buffer acting as an intermediate stage to manage collision resolution without requiring the entire system to scale uniformly.
Solution Approach 2:
The buffer performs preliminary action by pre-storing candidate locations identified by hash table lookups before final verification. This preliminary storage allows the system to batch process and verify multiple candidates, resolving collisions efficiently even as FIFO size and collision frequency increase, without requiring proportional increases in power consumption or system complexity.
Data Source
AI summary
Responsive to a data lookup in a buffer triggered for a search string, a processor searches for a selection of pairs from among multiple pairs of a hash table read from at least one address hash of the search string and matching at least one data hash of the search string, each row of the hash table assigned to a separate address hash, each of the pairs comprising a pointer to a location in the buffer and a tag with a previous data hash of a previously buffered string in the buffer. The processor identifies, from among the selection of pairs, at least one separate location in the buffer most frequently pointed to by two or more pointers within the selection of pairs. The processor, responsive to at least one read string from the buffer at the at least one separate location matching at least a substring of the search string, outputs the at least one separate location as the response to the data lookup.