Ranking Signal Exploitation Detection in Search Results
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search systems are vulnerable to exploitation of ranking signals by resource publishers to artificially inflate page rankings, leading to lower-quality resources being prioritized in search results.
Innovation Solution
A system that detects and adjusts for anomalous ranking signals by determining an expected information retrieval score and using a Z-Score to flag domains exploiting these signals, adjusting rankings to demote outlier domains and promote resources based on typical values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If ranking signals are used to prioritize resources in search results, then search efficiency and relevance are improved, but the system becomes vulnerable to exploitation by resource publishers who artificially manipulate rankings
Solution Approach 1:
The system implements feedback by monitoring ranking signal values across multiple queries and resources, comparing actual values against expected values, and using this information to detect anomalies. The feedback loop continuously adjusts rankings by demoting resources with anomalously high signal values, thereby maintaining ranking integrity while preserving search efficiency.
Solution Approach 2:
The patent introduces an intermediary detection layer that sits between the ranking signal generation and the final search result presentation. This intermediary component analyzes ranking signal values, identifies anomalies through comparison with expected values, and adjusts rankings accordingly, preventing manipulated resources from appearing in search results without eliminating the underlying ranking signals.
2Reliability
If the system adjusts rankings to demote exploited resources, then ranking integrity is improved, but the complexity of the search system increases
Solution Approach 1:
The system changes parameters by introducing expected value thresholds and anomaly detection metrics into the ranking process. Instead of fundamentally redesigning the search system, it modifies existing ranking parameters by comparing actual ranking signal values against expected values and adjusting rankings based on detected anomalies, thereby maintaining simplicity while improving integrity.
Solution Approach 2:
The patent applies partial action by focusing anomaly detection on specific ranking signals that show anomalously high values rather than analyzing all ranking signals comprehensively. This selective approach detects exploitation without requiring complete system overhaul, balancing ranking integrity improvement with system complexity management.
Data Source
AI summary
Disclosed implementations for detecting exploitation of ranking signals used to provide search results. An expected value for a ranking signal is determined based on a plurality of resources responsive to a query. A residual value is determined by aggregating a difference between the expected value and an information retrieval score for the ranking signal across a domain, wherein the domain includes at least one of the plurality of resources. Responsive to determining the residual value is indicative of an exploit, adjust a ranking of a resource associated with the domain in a search result page, the resource responsive to a second query based on the ranking signal.