Search Relevance Estimation via User Satisfaction Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search engine optimization mechanisms inadequately interpret clickthrough data, as they assume perceived relevance equals actual relevance and fail to account for user satisfaction, leading to inaccurate ranking of search results.
Innovation Solution
A user satisfaction model is employed to determine the utility or despair associated with each clicked URL, influencing the relevance estimation by processing satisfaction data to improve the interpretation of clickthrough data, thereby providing a more accurate context for document relevance in search rankings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If clickthrough data is treated as a strong indicator of relevance, then optimization mechanisms can be simplified, but the accuracy of relevance estimation deteriorates
Solution Approach 1:
The patent segments the single clickthrough signal into multiple independent features: click position, dwell time, number of clicks, and satisfaction indicators. Each segment provides a different aspect of user behavior, collectively improving relevance estimation accuracy without requiring complex optimization mechanisms.
Solution Approach 2:
The patent introduces satisfaction data as an intermediary between clickthrough data and relevance estimation. This mediator translates raw click behavior into meaningful relevance signals by incorporating user satisfaction assessments, thereby improving accuracy without directly complicating the optimization mechanism.
2Device complexity
If perceived relevance is assumed to equal actual relevance, then the interpretation of clickthrough data becomes simpler, but the accuracy of search result ranking deteriorates
Solution Approach 1:
The patent transforms the static assumption of relevance into a dynamic measurement process. Instead of assuming perceived relevance equals actual relevance, the system continuously measures multiple behavioral dimensions (click position, dwell time, satisfaction) that dynamically reflect actual relevance, improving ranking accuracy without excessive complexity.
Solution Approach 2:
The patent changes the parameters used to measure relevance from a single binary click indicator to multiple continuous parameters including dwell time duration, click position in results list, and satisfaction scores. These parameter changes enable more accurate differentiation of actual relevance while maintaining manageable interpretation complexity.
3Device complexity
If all clicks are treated as providing user satisfaction, then the processing of clickthrough data becomes simpler, but the accuracy of relevance interpretation deteriorates
Solution Approach 1:
The patent applies partial action by selectively weighting different click characteristics rather than treating all clicks equally. Clicks with longer dwell times, appropriate position in results, and associated satisfaction indicators receive higher weights, while brief or poorly-positioned clicks receive lower weights, improving accuracy without requiring complete reprocessing of all data.
Data Source
AI summary
The subject disclosure is directed towards using a satisfaction model's prediction as to whether a user was satisfied or dissatisfied in satisfying a search goal to help estimate the relevance of a URL/document that was returned and clicked by the user. The clickthrough data for a search goal session is processed by either a utility model or a despair model based on whether the satisfaction model indicated that the search goal session ended with the user satisfied or dissatisfied, respectively. The utility model distributes a utility value to each clicked URL, while the despair model distributes a despair value to each clicked URL. The utility value and despair value of each query-URL pair may be used as corresponding feature data for learning a search ranker.