Document Search Index Segmentation for Noise Word Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional search systems struggle to effectively incorporate noise words into search queries, leading to either reduced search capabilities or unintended results due to their exclusion from keyword indexes, which affects the accuracy and relevance of search results.
Innovation Solution
A computer-implemented method and system that generates and searches a document index by tokenizing keywords, including noise words, and allows users to specify whether to perform a noisy phrase search or a noiseless phrase search, enabling the inclusion or exclusion of noise words based on user input.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If noise words are excluded from keyword index, then storage and processing requirements are reduced, but search capability for phrases including noise words is lost
Solution Approach 1:
The patent segments the keyword index into two separate indices: a first keyword index containing only non-noise words, and a second keyword index containing noise words. This segmentation allows the system to maintain efficient storage and processing by excluding noise words from the primary index while still preserving the ability to search for phrases containing noise words through the secondary index, thereby resolving the contradiction between storage efficiency and search capability.
Solution Approach 2:
The patent introduces an intermediary mechanism that detects when a search query contains noise words and automatically redirects the search to the second keyword index. This intermediary logic layer allows the system to maintain the benefits of excluding noise words from the primary index while still providing comprehensive search capability when needed, effectively mediating between storage efficiency and search versatility.
2Productivity
If noise words are excluded from keyword index, then processing costs are reduced, but search accuracy for phrases including noise words deteriorates
Solution Approach 1:
The patent divides the processing workload by segmenting the keyword index into two separate structures. The first keyword index processes only non-noise words, maintaining high processing efficiency, while the second keyword index handles noise word searches. This segmentation allows the system to achieve both processing efficiency and search accuracy by routing queries to the appropriate index based on whether noise words are present in the search query.
Solution Approach 2:
The patent employs an intermediary detection mechanism that analyzes search queries to determine whether they contain noise words. This intermediary layer routes queries to the appropriate keyword index, ensuring that processing efficiency is maintained for most queries while search accuracy is preserved for queries containing noise words, thus resolving the contradiction between productivity and measurement precision.
3Device complexity
If conventional search systems exclude noise words, then search processing is simplified, but search results become less relevant
Solution Approach 1:
The patent segments the search processing into two distinct paths: one for queries without noise words (using the first keyword index) and one for queries with noise words (using the second keyword index). This segmentation simplifies the overall processing complexity by maintaining a simple primary search path while providing enhanced relevance for queries containing noise words through the secondary path, thereby resolving the contradiction between processing simplicity and result relevance.
Solution Approach 2:
The patent introduces an intermediary query analysis mechanism that determines whether a search query contains noise words and routes it to the appropriate index. This intermediary layer maintains simple processing for the majority of queries while ensuring high relevance for queries containing noise words, effectively resolving the contradiction between device complexity and reliability.
Data Source
AI summary
Systems and methods are provided for document searching. In one implementation, a computer-implemented method provides keyword searching. The method may receive a plurality of noisy keywords for a document collection. A server may generate tokens for a plurality of keywords in the document collection and merge the tokens to create an index. A search query may be received. The search query may include at least one search phrase. For the at least one search phrase, an indication may be received from a user specifying to perform one of a noisy phrase search or a noiseless phrase search. The method may search the index for the at least one search phrase based on the indication received from the user.