Index-side Stem-based Variant Generation for Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional search engines face difficulties in identifying relevant resources for search queries when exact word matches are not found, due to the complexity of adding multiple synonyms for multi-term queries.
Innovation Solution
A search system augments its index with stem-based variants of words from resources, and modifies queries to include these variants, assigning weights to ensure that resources matching the original terms are prioritized over variants.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple synonyms are added to search queries to handle word variations, then search coverage is improved, but query complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing stem-based variants of terms in the search index during the indexing phase. This eliminates the need to manually expand queries with multiple synonyms at query time, as the variants are already prepared and associated with the original terms in the index structure.
Solution Approach 2:
The patent uses stemming as an intermediary mechanism between the original term and its synonyms. Instead of directly adding multiple synonyms to queries, the system uses stem-based variants as a mediator that captures the semantic relationship between terms, reducing query complexity while maintaining search coverage.
2Measurement precision
If stem-based variants are added to the search index, then resource identification accuracy is improved, but indexing complexity increases
Solution Approach 1:
The patent applies self-service by using the indexing system's own resources to generate and store stem-based variants automatically during the indexing process. The system leverages its existing computational resources and indexing infrastructure to create variants without requiring external intervention or complex additional processing steps.
Solution Approach 2:
The patent changes the parameter representation by transforming original terms into their stem-based variants during indexing. This parameter transformation allows the system to store normalized forms of terms that capture their semantic essence, improving identification accuracy while the indexing process itself handles the complexity through automated parameter conversion.
3Productivity
If query expansion is performed with multiple synonyms, then relevant resource discovery is improved, but processing time increases
Solution Approach 1:
The patent applies preliminary action by pre-generating and storing stem-based variants during the indexing phase, so that when queries are processed, the system can directly retrieve pre-computed variants without performing time-consuming synonym expansion operations at query time.
Solution Approach 2:
The patent replaces the mechanical process of manually expanding queries with multiple synonyms using a computational stemming approach. This substitution uses algorithmic stem generation and index lookup mechanisms that are faster and more efficient than traditional synonym expansion methods.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for index-side synonym expansion. One method includes obtaining a token sequence for a resource and indexing a token in the token sequence. The indexing includes applying one or more stemming rules to the particular token to generate a stemmed form of the token, obtaining a variant of the stemmed form of the token, and storing data associating the resource with both the token and the variant as index terms for the resource in a search engine index.