Numeric String Hashes for Wildcard Search Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current full-wildcard query support in search engines requires expensive index structures and computationally intensive queries due to the need for extensive string comparisons, which hampers computational efficiency.
Innovation Solution
Implementing a K-gram index coupled with a full-text inverted index using numeric string hashes to represent terms, allowing for efficient intersection computations and lookups by comparing integer hash values instead of strings, thereby reducing processing overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If full-wildcard query support is implemented using traditional index structures, then wildcard searching capability is improved, but computational efficiency deteriorates due to expensive string comparisons and intensive query processing
Solution Approach 1:
The patent replaces the traditional mechanical string comparison system with a hash-based numeric comparison system. Instead of comparing actual string characters during wildcard queries, the system computes hash values of K-grams and performs numeric comparisons on these hash values, which are computationally cheaper and faster than string operations.
Solution Approach 2:
The patent transforms the representation of search terms from raw strings to numeric hash values. By changing the parameter representation from character sequences to numeric identifiers, the system enables more efficient computational operations while maintaining the ability to perform wildcard matching through the underlying K-gram structure.
2Adaptability or versatility
If traditional index structures are used for full-wildcard queries, then query functionality is improved, but memory and processor usage increase
Solution Approach 1:
The patent extracts only the essential matching information from full string representations by computing hash values of K-grams. Instead of storing and comparing complete strings, the system extracts numeric hash representations that retain the necessary information for wildcard matching while consuming less memory and processing power.
Solution Approach 2:
The patent uses hash values as temporary, computationally inexpensive representations during query processing. These numeric hash values serve as disposable intermediaries that enable fast comparisons without the overhead of maintaining complex string data structures in memory during query execution.
Data Source
AI summary
Techniques herein improve computational efficiency for wildcard searches by using numeric string hashes. In an embodiment, a plurality of query K-gram tokens for a term in a query are generated. Using a first index, an intersection of hash tokens is determined, wherein said first index indexes each query K-gram token of said K-gram tokens to a respective subset of hash tokens of a plurality of hash tokens, each of hash token of said plurality of hash tokens corresponding to a term found in one or more documents of a corpus of documents. The intersection of hash tokens comprises only hash tokens indexed to all of said plurality of query K-gram tokens by said first index. Using a second index, documents of said corpus of documents that contain said term are determined, said second index indexing said hash tokens to a plurality of terms in said corpus of documents and for each term of said plurality of terms, a respective subset of documents of corpus of documents that contain said each term.