Inverted Index Query-Dependent Feature Generation for Real-Time Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search engine technologies face challenges in efficiently ranking digital documents in real-time due to the tradeoff between quality of ranking and processing time, particularly with the use of inverted indexes which require significant resources and are query-independent.
Innovation Solution
A server-based system that leverages query-independent data from an inverted index to generate query-dependent features using a dynamic feature generator, employing algorithms like Neural Networks or decision-tree-based Machine Learning Algorithms to rank documents based on the occurrence of multiple terms within the documents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If query-independent data from inverted index is processed in real-time for ranking documents, then quality of ranking is improved, but processing time and resource requirements increase
Solution Approach 1:
The patent pre-processes and stores query-independent data (such as term frequencies, document lengths, and other statistical features) in the inverted index during document indexing, before any query is received. This preliminary preparation allows the ranking system to quickly retrieve and combine these pre-computed features during real-time query processing without performing heavy computations at query time, thus improving ranking quality while minimizing processing time
Solution Approach 2:
The patent divides the ranking process into separate components: query-independent features (pre-computed and stored in inverted index) and query-dependent features (computed at query time). This segmentation allows the system to leverage pre-prepared data for efficient real-time processing while still incorporating dynamic query-specific information for accurate ranking
2Measurement precision
If query-independent data from inverted index is processed in real-time for ranking documents, then quality of ranking is improved, but resource requirements increase
Solution Approach 1:
The patent performs computationally intensive processing of query-independent data during the document indexing phase, storing results in the inverted index. This shifts resource consumption from query time to indexing time, making query processing more efficient and reducing energy requirements during real-time operations while maintaining high ranking quality
Solution Approach 2:
The patent creates a compact representation of query-independent data in the inverted index that can be quickly copied and used for multiple queries. Instead of re-computing these features for each query, the system retrieves pre-computed values from the index, significantly reducing processing resources required during query execution
Data Source
AI summary
A method and server for ranking digital documents for a query are disclosed. T server hosts a search engine associated with an inverted index. The method includes accessing inverted index for retrieving query-independent data for a first document-term pair and a second document-term pair, where the query-independent data is indicative of (i) a term-specific occurrence of the first term in content of the document and (ii) a term-specific occurrence of the second term in the content of the document. The method includes generating a query-dependent feature using the query-independent data and which is indicative of a group occurrence of the first term with the second term in the content of the document. The method includes generating a ranking feature for the given document based on at least the first term, the second term, and the query-dependent feature, and ranking the given document based on at least the ranking feature.