Adaptive Thesaurus Query Expansion Using Semantic Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional search techniques face challenges in enhancing recall without compromising precision due to the limitations of query expansion methods, such as data glut, false synonyms, and context-free expansions, which result in inaccurate search results.
Innovation Solution
An adaptive thesaurus system that calculates recall gain, expansion independence, and semantic similarity for term pairs to determine whether to store and use them for query expansion, ensuring relevant and precise search results by selecting suitable synonyms based on these measures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If query expansion is performed using conventional thesaurus to enhance recall, then the number of retrieved documents increases, but precision decreases due to inclusion of irrelevant documents
Solution Approach 1:
The patent changes the parameters for selecting synonyms by calculating multiple metrics (recall gain, expansion independence, semantic similarity) instead of using conventional thesaurus entries. This allows dynamic adjustment of synonym selection based on actual search performance and contextual relevance, resolving the contradiction between recall and precision.
Solution Approach 2:
The system implements feedback mechanisms by evaluating the actual impact of synonym expansion on search results. By measuring recall gain and precision changes, the system can adaptively adjust which synonyms to use, ensuring that expansion improves recall without unduly sacrificing precision.
2Reliability
If query expansion uses frequent terms or weak synonyms, then recall increases by retrieving more documents, but data glut occurs overwhelming the user
Solution Approach 1:
The patent introduces expansion independence as a parameter to control the volume of results from query expansion. By selecting synonyms that are independent of frequently occurring terms, the system increases recall without generating excessive results that would create data glut.
Solution Approach 2:
The system applies different quality criteria to different synonyms based on their characteristics. High-quality synonyms with strong semantic similarity and appropriate expansion independence are selected, while low-quality synonyms that would generate excessive results are excluded, maintaining local optimality in synonym selection.
3Adaptability or versatility
If conventional thesaurus is used for domain-specific searches, then general synonyms are available, but precision decreases due to weak or false synonyms not contextually relevant
Solution Approach 1:
The patent calculates semantic similarity as a parameter to measure contextual relevance between query terms and synonyms. This allows the system to adaptively select synonyms that are not only generally applicable but also contextually relevant to the specific search domain and query, resolving the contradiction between general coverage and contextual precision.
Solution Approach 2:
The system dynamically adjusts synonym selection based on the specific query and search context rather than using static thesaurus entries. By evaluating semantic similarity and contextual relevance in real-time, the system adapts to different domains and queries, maintaining both versatility and precision.
Data Source
AI summary
Methods and systems are provided for creating an adaptive thesaurus. A term pair including an index term and an expansion term is received. A recall gain, an expansion independence, and a semantic similarity of the term pair are calculated using a processor. Whether to store the term pair is determined based on the recall gain, the expansion independence, and the semantic similarity. The term pair is stored based on the determination. Methods and systems are provided for searching using an adaptive thesaurus. A search query including a query term is received. An expansion term stored in association with an index term matching the query term in the adaptive thesaurus is retrieved. Using a processor, the search query is expanded using the expansion term based on a recall gain, an expansion independence, and a semantic similarity.