Proximity Keyword Search for Document Corpus Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional keyword search techniques in document collections often return a large number of irrelevant documents due to ambiguities in word meanings and contexts, making it computationally expensive and time-consuming to achieve high precision and recall, especially in applications requiring precise information.
Innovation Solution
The system employs multiple proximity keyword searches with a Boolean proximity search query, using a proximity value between 40 to 60, to isolate relevant documents and filter out irrelevant ones by analyzing the proximity of search terms within the documents, reducing the number of search terms needed and enhancing search efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional keyword search techniques are used to search document collections, then the search system can quickly retrieve documents containing search terms, but the quality and completeness of search results are indefinite and unreliable due to word ambiguity and multiple meanings
Solution Approach 1:
The patent changes the parameter of search proximity by requiring multiple keywords to appear within a specified distance of each other in the document. This proximity parameter filters out irrelevant documents where keywords appear separately, thereby improving search precision while maintaining reliability through consistent application of the proximity criterion
Solution Approach 2:
The patent dynamically adjusts the number of keywords required in the proximity search based on the specific search query and document characteristics. Rather than using a fixed keyword count, the system adapts the keyword requirements to optimize both precision and reliability for different search scenarios
2Reliability
If synonyms and phrases are included in the search query to address word ambiguity, then the probability of finding desired results increases, but the total number of documents retrieved becomes quite large and computationally expensive to generate and analyze
Solution Approach 1:
The patent segments the search process into two stages: first identifying documents with the core keyword proximity matches, then separately considering synonyms and related terms only for the reduced result set. This segmentation prevents the combinatorial explosion that occurs when all synonyms are applied to the entire document collection, thereby maintaining search completeness while improving efficiency
Solution Approach 2:
The patent applies synonym expansion partially rather than comprehensively - using a limited set of high-relevance synonyms rather than all possible synonyms. This partial action approach maintains sufficient search completeness to find desired results while avoiding the excessive computational burden of processing all possible synonym combinations across the entire document collection
3Reliability
If a high degree of precision and high recall are required in search applications such as litigation discovery, then comprehensive keyword coverage is needed, but the synonym list that must accompany the search query generates a large number of irrelevant documents
Solution Approach 1:
The patent performs preliminary proximity-based filtering before applying synonym expansion. By first identifying documents that contain the required proximity matches of core keywords, the system establishes a preliminary filtered set that inherently has higher precision. This preliminary action prevents irrelevant documents from entering the subsequent synonym-matching stage, thereby maintaining both precision and recall while reducing the total number of retrieved documents
Data Source
AI summary
A system and method for searching indexed content data using multiple proximity keyword searches to quickly and efficiently locate subsets of data that are either relevant or irrelevant to an issue of interest to a user.