Sentence-Vector Indexing With Transition Matching for Paragraph Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data search methods are not highly accurate due to variations in sentence vectors and transitions within paragraphs, leading to decreased search accuracy when narrowing down similar paragraph candidates.
Innovation Solution
Classify similar sentence vectors into clusters and generate an inverted index associating sentence vectors with offsets, allowing for the specification of similar vectors based on a feature sentence and transition data, thereby enhancing search accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional paragraph-level search is performed by integrating sentence vectors, then search coverage is improved, but search accuracy deteriorates due to variations in sentence vectors and transitions within paragraphs
Solution Approach 1:
The patent segments paragraphs into individual sentences and creates separate vector representations for each sentence. Instead of treating a paragraph as a single integrated unit, the system generates sentence-level vectors and creates an inverted index at the sentence level, allowing for more precise matching while maintaining the ability to retrieve entire paragraphs as results.
Solution Approach 2:
The patent introduces a new dimension of analysis by examining transition data between sentences. It calculates transition vectors that capture the relational structure and flow between consecutive sentences, adding this temporal/structural dimension to the search process. This allows the system to evaluate not just individual sentence similarity but also the coherence and flow of sentence sequences.
2Measurement precision
If sentence-level vector classification is implemented, then search accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent performs preliminary classification of sentence vectors into clusters before the actual search operation. By pre-organizing sentences into semantic clusters and building an inverted index at the sentence level, the system reduces the search space during query processing. This preliminary organization allows for faster and more accurate retrieval without requiring complex real-time computations.
Solution Approach 2:
The patent introduces transition data as an intermediary element that mediates between individual sentence vectors and paragraph-level results. The transition data captures the relational structure between sentences and serves as a bridge, allowing the system to evaluate sentence sequences without directly comparing entire paragraphs. This intermediary representation simplifies the computational process while maintaining accuracy.
3Measurement precision
If transition data specification is performed for multiple similar vectors, then search precision is improved, but processing time increases
Solution Approach 1:
The patent segments the evaluation process by focusing transition data analysis only on candidate sentences that have been pre-identified as similar through vector comparison. Instead of computing transition data for all sentences in a paragraph, the system first filters to relevant candidates using sentence-level vector matching, then applies the more computationally intensive transition analysis only to these narrowed-down candidates.
Data Source
AI summary
A medium storing a program including: classifying vectors of first sentences in a file into each similar vector; generating an inverted index associating a vector of each first sentence with a position of the first sentence on the file; identifying a feature sentence from a second sentences included in a second sentences; specifying similar vectors being vectors similar to a vector of the feature sentence based on the inverted index; specifying, for each similar vector, first transition data indicating transition of vectors at positions before and after the similar vector based on the inverted index; and specifying, from pieces of first transition data obtained by performing the specifying of the first transition data on the similar vectors, transition data similar to second transition data indicating transition of vectors of sentences before and after the feature sentence, to output the transition data as a response of the search query.