Proximity Boost Ranking Using Term Distance Standard Deviation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search engines fail to accurately rank documents based on relevance due to their inability to consider the proximity or density of search query terms within documents, leading to biased and inefficient search results.
Innovation Solution
A computer-implemented ranking method that calculates a proximity boost value using the standard deviation of a probability distribution function representing the distribution of terms in a document, which is then applied to a base relevancy score to determine a more accurate relevancy ranking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional Boolean search with hit frequency ranking is used, then the search system is simple and fast, but the ranking precision does not account for term proximity
Solution Approach 1:
The patent changes the ranking parameter from simple hit frequency to a composite score that incorporates proximity information. It introduces a proximity boost factor calculated based on the distance between term occurrences, transforming the ranking metric to account for both frequency and spatial distribution of terms within documents.
Solution Approach 2:
The patent adds a new dimension to the ranking system by considering the spatial proximity of terms along the document axis. Instead of only counting term frequencies, it measures the distance between term occurrences and uses this positional information to generate a proximity boost, effectively moving from one-dimensional frequency counting to two-dimensional frequency-plus-position analysis.
2Measurement precision
If phrase searching is required to consider term proximity, then ranking precision improves, but ease of operation deteriorates as users must explicitly request phrase searching
Solution Approach 1:
The system automatically calculates proximity boost factors without requiring user intervention or explicit phrase search requests. The proximity analysis is performed autonomously on all search results, allowing the system to self-improve ranking accuracy while maintaining simple user interaction. Users simply submit search terms and receive proximity-aware rankings without needing to know or specify proximity preferences.
3Measurement precision
If hit density estimators are used to account for term proximity, then relevant information is better identified, but the estimators are biased and do not use complete information about all hits
Solution Approach 1:
The patent merges multiple sources of information about term occurrences into a unified proximity boost calculation. It combines data from all hit positions within a document, aggregating the distances between all pairs of term occurrences to generate a comprehensive proximity score that utilizes complete hit information rather than relying on biased or partial estimators.
Data Source
AI summary
A method and apparatus are provided for ranking documents according to relevancy scoring. In one implementation, a computer-implemented method is provided for receiving, from a database over a network, a document resulting from a search on a database, the document containing terms that match the search criteria. The method may calculate a standard deviation of a probability distribution function representing a distribution of the terms in the document that match the search criteria. The method may further determine relative distances between the terms in the document that match the search criteria according to the standard deviation. The method may further calculate a proximity boost value using the relative distances, and apply the proximity boost value to a base relevancy score of the document to determine a relevancy ranking. The document may then be ranked according to the relevancy ranking.