Forward Index Document Ranking via Segmented Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Search engines face inefficiencies in document ranking due to the complexity and cost of using reverse indexes, which are time-consuming and expensive, especially when storing data for complex formulas in reverse indexes.
Innovation Solution
Implementing a forward index as a final stage in the ranking process, which is more time-consuming and costly, but used only for a limited number of documents previously identified as relevant, comprising entries with token streams, context information, static features, and positional data to efficiently rank documents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If reverse indexes are used to identify relevant documents, then document identification is achieved, but the process becomes time-consuming and expensive
Solution Approach 1:
The patent divides the ranking process into multiple stages: a first stage using reverse indexes for preliminary document identification, and a second stage using forward indexes for final ranking. This segmentation allows each index type to be used for its optimal purpose, reducing overall processing time while maintaining accuracy.
Solution Approach 2:
The patent performs preliminary document identification using reverse indexes before applying the more computationally intensive forward index ranking. By pre-identifying relevant documents, the system reduces the number of documents that require expensive forward index processing, thereby reducing total time consumption.
2Adaptability or versatility
If complex formulas are stored in reverse indexes, then ranking functionality is enabled, but storage requirements increase and extraction becomes inefficient
Solution Approach 1:
The patent inverts the traditional indexing approach by introducing forward indexes that store data in the opposite orientation of reverse indexes. Instead of indexing by terms (reverse), the forward index indexes by document position, enabling efficient extraction of positional information without the inefficiencies of reverse index extraction.
Solution Approach 2:
The forward index acts as an intermediary data structure that bridges the gap between reverse index document identification and complex formula evaluation. It provides efficient access to positional and contextual information needed for ranking calculations without requiring inefficient extraction operations on reverse indexes.
3Measurement precision
If forward index is used for all documents, then comprehensive ranking is achieved, but computational cost and time increase
Solution Approach 1:
The patent applies forward index processing only to the subset of documents that were preliminarily identified as relevant by the reverse index, rather than processing all documents in the corpus. This partial action approach maintains ranking accuracy for relevant documents while significantly reducing overall computational cost.
Data Source
AI summary
Methods and computer storage media are provided for generating entries for documents in a forward index. A document and its document identification are received, in addition to static features that are query-independent. The document is parsed into tokens to form a token stream corresponding to the document. Relevant data used to calculate rankings of document is identified and a position of the data is determined. The entry is then generated from the document identification, the token stream of the document, the static features, and the positional information of the relevant data. The entry is stored in the forward index.