Hash Chain Selection for Faster Repeated String Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compressors face inefficiencies in searching for repeated data strings due to the need to search through long or 'bad' hash chains, which are time-consuming and processor-intensive, especially when common substrings lead to lengthy hash collision chains.
Innovation Solution
Implementing look-ahead hash chain matching, where the compressor adjusts the look-ahead offset to select a hash chain that is not bad, thereby avoiding long chains and optimizing the search for repeated data strings by using a substring from a look-ahead offset rather than the current position, and employing chain-select optimizations to prune search lists and prioritize shorter chains.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional hash chain matching is used to search for repeated data strings, then the search can find matches in the input data buffer, but the search becomes time-consuming and processor-intensive when encountering long hash collision chains
Solution Approach 1:
The patent applies preliminary action by examining the spill counter value before initiating a full hash chain search. The compressor checks whether a hash chain is likely to be long (bad) in advance, using the spill counter as a predictor. This allows the system to avoid time-consuming searches on chains that are likely to be inefficient, while still maintaining the ability to find repeated strings when they exist.
Solution Approach 2:
The patent implements dynamics by adaptively selecting between different search strategies based on the spill counter value. When the spill counter indicates a short chain, the compressor performs a traditional hash chain search. When the spill counter indicates a long chain, the compressor switches to an alternative approach (such as searching with a different hash function or skipping the search). This dynamic adaptation optimizes processor cycle usage while maintaining search effectiveness.
2Measurement precision
If the compressor searches through long hash collision chains to ensure all repeated strings are found, then measurement precision is maintained, but productivity decreases due to increased processing time
Solution Approach 1:
The spill counter provides preliminary information about the expected length of hash chains before searching begins. By checking this counter value in advance, the compressor can make informed decisions about whether to proceed with a search, thereby avoiding productivity losses from searching through unnecessarily long chains while still maintaining precision when chains are short.
Solution Approach 2:
The patent changes the parameter of search strategy based on the spill counter value. Instead of always using the same search approach, the system modifies its behavior dynamically - switching between different search methods or abandoning searches entirely based on the spill counter indication. This parameter change allows the system to optimize for productivity when chains are long while maintaining precision when chains are short.
3Device complexity
If the compressor uses a fixed search strategy for all hash chains, then device complexity is low, but adaptability decreases when dealing with varying chain lengths
Solution Approach 1:
The spill counter serves as a preliminary indicator that enables the compressor to adapt its search strategy without requiring complex analysis during the search process itself. The adaptation decision is made in advance based on the spill counter value, maintaining relative algorithmic simplicity while achieving high adaptability to different chain length scenarios.
Data Source
AI summary
In an example, there is disclosed an apparatus, comprising: a data store comprising a hash table having for at least some rows a hash entry indexed by a hash value, and comprising a hash chain of one or more pointers to a history buffer, and a spill counter; and one or more logic elements, including at least one hardware logic element, comprising a data compressor to: inspect a string0 comprising n bytes at position p in a data file; get the spill counter from a hash entry corresponding to string0; inspect a string1 comprising n bytes at p+k, wherein k is a positive integer; get the spill counter from a hash entry corresponding to string1; determine that the spill counter for string1 is less than the spill counter for string0; and search a chain1 (the hash chain of a hash entry corresponding to string1) for a matching string of size at least n+k with an offset of −k.