Synonym Detection and Merging in Search Indexes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search technologies face challenges in efficiently indexing and categorizing document content across wide area networks, particularly in supporting searches for synonyms, as they often require multiple keyword searches and may not effectively handle dissimilar content that could be considered synonyms.
Innovation Solution
The method involves detecting and merging synonyms by comparing input documents to existing indexed documents, identifying candidate synonyms based on similarity thresholds, and integrating these synonyms into search indexes to enhance search results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple keyword searches are performed for synonyms, then search completeness is improved, but search time and system complexity increase
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing synonym relationships in the index structure during the indexing phase. When a document is indexed, the system identifies synonyms and creates forward references (lists of synonyms) and backward references (lists of documents containing those synonyms) in advance. This allows the search operation to retrieve all synonym-related documents in a single query rather than performing multiple separate searches, thus improving search completeness while reducing search time.
Solution Approach 2:
The patent merges synonym-related search operations into a unified index structure. By combining multiple synonym searches into a single index lookup operation, the system retrieves all relevant documents containing any of the synonyms simultaneously. This merging of search operations eliminates the need for sequential multiple keyword searches, reducing both search time and system complexity while maintaining complete search coverage.
2Productivity
If synonyms are preprocessed and merged into search indexes, then search efficiency is improved, but indexing complexity increases
Solution Approach 1:
The patent segments the synonym processing task into distinct phases: synonym identification during document analysis, synonym relationship construction, and index structure organization. By dividing the complex preprocessing task into manageable segments, the system can handle synonym processing systematically without overwhelming the indexing operation. Each segment focuses on a specific aspect (identifying synonyms, building relationships, organizing in index), making the overall process more tractable.
Solution Approach 2:
The patent introduces an intermediary data structure (synonym lists and reference tables) that mediates between the raw document content and the final search index. This intermediary layer captures synonym relationships in a structured format that simplifies the indexing process. Instead of directly incorporating complex synonym logic into the index, the system uses this intermediary representation to bridge the gap, reducing indexing complexity while enabling efficient synonym-based searches.
3Measurement precision
If synonym detection and merging is implemented, then search accuracy is improved, but processing overhead during indexing increases
Solution Approach 1:
The patent applies partial action by focusing synonym detection on specific fields or portions of documents rather than processing entire documents uniformly. The system identifies and processes only the relevant parts of documents that are likely to contain synonyms (such as title fields, abstracts, or specific content sections). This selective approach maintains search accuracy for synonym queries while reducing the overall processing overhead during indexing by avoiding unnecessary analysis of entire documents.
Data Source
AI summary
Tools and techniques are described herein for detecting synonyms and merging synonyms into search indexes. The tools provide methods that include receiving input documents for indexing into a search index file. The tools may compare parts of the input documents to parts of other documents already indexed into the search index file. The methods may also evaluate, based on these comparisons, whether the input document and the existing document are sufficiently similar to justify an inference that any dissimilar terms between the input document and the existing document are candidate synonyms. Other methods may include receiving requests to perforin searches that include one or more input keywords. The method then searches for links to synonyms of the input keyword, and returns search results responsive to the input keyword and to the synonyms.