Information Search Using Sentence-Based Position Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing information search systems face high processing costs and low accuracy due to large amounts of position information required for adjacency processing and the lack of consideration for word proximity when calculating similarity between multiple words in a query.
Innovation Solution
An information search method that calculates similarity based on the distance and co-occurrence of words within documents, using sentence-based position information to reduce processing costs and improve accuracy by considering proximity and co-occurrence of words.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If word-based position information is stored in the index DB to enable adjacency processing, then search accuracy for compound words is improved, but the amount of position information and processing cost increase significantly
Solution Approach 1:
The patent extracts only the necessary position information (sentence-based positions) from the full word-based position information, retaining only the coarse-grained sentence-level positions rather than all fine-grained word-level positions, thereby reducing data quantity while maintaining search functionality
Solution Approach 2:
The patent segments position information at the sentence level rather than at the word level, dividing the document into sentences and tracking word positions within those sentences, which reduces the overall amount of position data while preserving the ability to detect word adjacency and proximity
2Measurement precision
If adjacency processing is performed for every document containing query words to check compound word formation, then search accuracy is improved, but processing cost increases
Solution Approach 1:
The patent performs preliminary action by pre-calculating and storing sentence-based position information for all words in the index DB during the indexing phase, so that during query processing, the system can directly use this pre-computed information to determine word proximity without performing expensive adjacency processing on every document
Solution Approach 2:
The patent replaces the mechanical adjacency processing approach (iterating through all words in documents to check if query words are adjacent) with a more efficient information retrieval approach using pre-stored sentence-based position data, substituting a computationally intensive process with simpler data lookup and comparison operations
3Device complexity
If TFIDF similarity calculation is used for multiple words in query, then calculation is simple, but search accuracy is low because word proximity is not considered
Solution Approach 1:
The patent merges multiple similarity calculation approaches: it combines TFIDF-based word frequency weighting with sentence-based position proximity measures, integrating both methods into a unified similarity calculation that considers both word frequency and spatial proximity within sentences
Solution Approach 2:
The patent creates a composite similarity score by combining different factors (word frequency, sentence position proximity, co-occurrence information) into a single integrated similarity metric, similar to how composite materials combine different material properties to achieve enhanced performance
Data Source
AI summary
An information search apparatus is provided. The information search apparatus includes: a character string input unit configured to obtain a character string from a client; a character string information search unit configured to obtain information that includes the character string from an index DB; a similarity calculation unit configured to calculate degree of similarity between the character string and searched information; and an output unit configured to output the searched information in descending order of the degree of similarity. In the information search apparatus, the character string information search unit includes a unit configured to, when the input character string contains a plurality of words, search an index DB, based on each word, that stores words and occurrence position information of the words to obtain a distance between occurrence positions of the words, and the similarity calculation unit includes a unit configured to calculate the degree of similarity based on the distance between occurrence positions of the words.