Website Quality Evaluation via Document-Query Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current internet search engines face challenges in accurately ranking search results due to limited user feedback data for specific documents, leading to inconsistencies in relevance and quality assessment.
Innovation Solution
The method involves receiving and partitioning document-query pairs based on various parameters such as information retrieval scores, query length, and user feedback data, aggregating user feedback data across partitions to evaluate website properties like quality, and using these evaluations to improve document ranking algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If user feedback data is collected and aggregated over time for ranking adjustment, then the ranking accuracy improves, but the reliability of ranking for individual documents with scarce feedback deteriorates
Solution Approach 1:
The patent combines feedback data from multiple documents within the same website into aggregated website-level feedback. This merging allows the system to leverage collective user behavior patterns across all documents from a website, enabling reliable quality assessment even when individual document feedback is sparse. The aggregated website feedback then informs ranking decisions for individual documents through transfer learning.
Solution Approach 2:
The system performs preliminary evaluation of website quality by aggregating feedback data across all documents from a website before using this evaluation to rank individual documents. This preliminary action creates a baseline quality assessment that can be applied to individual documents even when their own feedback history is insufficient, improving ranking reliability from the outset.
2Measurement precision
If document ranking is based on individual document feedback data, then the specificity of ranking is improved, but the robustness deteriorates due to limited data availability
Solution Approach 1:
The patent implements a two-level feedback aggregation strategy where feedback is aggregated at the website level for robust quality assessment, while still maintaining the ability to apply this assessment to individual documents locally. This allows the system to benefit from abundant aggregated data for reliability while preserving document-specific ranking adjustments for specificity.
3Quantity of substance
If website quality evaluation is performed using aggregated feedback from all documents, then the data sufficiency improves, but the computational complexity increases
Solution Approach 1:
The patent segments the feedback aggregation process into two distinct stages: (1) website-level aggregation where feedback from all documents is combined to evaluate overall website quality, and (2) document-level application where the website quality evaluation is applied to individual documents. This segmentation reduces computational complexity by performing the expensive aggregation operation once at the website level rather than repeatedly for each document.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for evaluating website properties by partitioning user feedback data are disclosed. In one aspect, a method includes receiving a plurality of document query pairs (D-Qs) associated with a website, partitioning the plurality of D-Qs into one or more groups according to values for a partition parameter associated with the plurality of D-Qs, evaluating a property parameter of the website based on aggregated user feedback data of the D-Qs included within at least one of the one or more groups, and providing the evaluated property parameter as an input for ranking documents from the website as result documents for searches.