Semantic Reverse Search Indexing for Large Corpus Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current indexing techniques fail to capture semantic meaning, resulting in incomplete and inadequate search results in large corpora.
Innovation Solution
The implementation of a semantic reverse search index using machine-learned models that encode search queries and publication titles into a shared semantic vector space, allowing for the identification of closest matches based on semantic proximity, and the use of multiple reverse indexes to enhance search comprehensiveness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional keyword-based indexing techniques are used, then the indexing process is simple and fast, but the search results fail to capture semantic meaning and are incomplete
Solution Approach 1:
The patent transforms the indexing approach by changing the parameter representation from discrete keywords to continuous semantic vectors in a multi-dimensional space. Machine-learned models project both search queries and publication titles into this shared vector space, enabling semantic similarity measurement through geometric distance calculations rather than exact keyword matching.
Solution Approach 2:
The patent introduces semantic vector space as an intermediary representation layer between the search query and the publication corpus. This vector space acts as a mediator that captures latent semantic relationships, allowing the system to bridge the gap between user intent and relevant publications through semantic proximity rather than direct keyword overlap.
2Loss of information
If semantic vector space modeling is implemented, then search results become more comprehensive and relevant, but the computational complexity and processing time increase
Solution Approach 1:
The patent performs semantic vector projection of the entire publication corpus into the shared vector space in advance, before actual search queries are executed. This preliminary indexing creates pre-computed semantic representations that enable fast similarity searches during query processing, avoiding the need to perform complex semantic analysis in real-time for each search.
Solution Approach 2:
The patent segments the search process into distinct phases: (1) projecting the search query into the semantic vector space, (2) retrieving candidate publications based on vector proximity, and (3) ranking results. This segmentation allows the computationally intensive vector projection to be performed once during indexing, while query processing leverages the pre-built semantic structure for efficient retrieval.
Data Source
AI summary
Embodiments of the present disclosure relate generally to semantic indexing to improve search results of a large corpus. Some embodiments, with at least one of the keywords of the search query encoded by a semantic vector in a semantic vector space, identify a plurality of candidate publications in the publication corpus, the plurality of candidate publications encoded by a cluster of a plurality of semantic vectors in the semantic vector space, the identifying based on proximity in the semantic vector space between the at least one of the keywords of the search query and keywords in the plurality of candidate publications, the proximity based on a first machine-learned model that projects the at least one keyword in the search query and the keywords in the plurality of candidate publications into the semantic vector space.