Robust PageRank with Supporting Set for Spam Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search engine algorithms, such as PageRank and Hubs and Authorities, are vulnerable to manipulation by web spammers who artificially inflate page ranks through link structure engineering, making it difficult to effectively identify and filter out spam pages without costly content analysis during the crawling stage.
Innovation Solution
A system that determines a contribution vector for each page in a web graph, identifying a supporting set of pages that significantly contribute to the page's rank, and uses unsupervised or supervised learning features based on this set to label pages as spam or non-spam, while a robust rank system limits individual contributions to prevent rank inflation, employing mechanisms like threshold-based contribution limiting and approximate contribution vector generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If web spammers create link farms to artificially inflate page ranks, then the page rank of spam pages increases, but the accuracy of search results deteriorates
Solution Approach 1:
The patent segments the contribution of linking pages to the target page's rank by introducing a supporting set that identifies a small number of pages contributing most to the rank. This segmentation allows the system to distinguish between legitimate concentrated contributions and artificial link farm contributions, thereby maintaining measurement precision while reducing spam impact.
Solution Approach 2:
The patent applies partial action by considering only the top-k contributing pages (supporting set) rather than all linking pages. This partial consideration is sufficient to detect spam patterns while being computationally efficient. The robust rank then uses this partial information to limit the maximum contribution from the supporting set, preventing excessive rank inflation from link farms.
2Measurement precision
If content analysis is performed to detect web spam, then spam detection accuracy improves, but processing time increases
Solution Approach 1:
The patent performs preliminary action by computing the supporting set and robust rank during the crawling stage before content analysis. This preliminary structural analysis provides early spam detection signals based on link topology, allowing the system to identify suspicious pages without yet investing in costly content analysis, thereby reducing overall processing time while maintaining detection accuracy.
Solution Approach 2:
The supporting set acts as an intermediary between the link structure and content analysis. It translates the complex link topology into a simplified representation that captures spam patterns, serving as a bridge that enables efficient detection without requiring full content analysis of all pages.
3Measurement precision
If the number of incoming links to a page increases, then the page rank improves, but vulnerability to spam manipulation increases
Solution Approach 1:
The patent applies local quality by differentiating between legitimate and spammy incoming links through the supporting set concept. Instead of treating all incoming links uniformly, the system identifies the local structure of top contributors and applies different weighting through the robust rank function. This allows the system to maintain high page ranks for legitimate concentrated links while resisting manipulation from link farms.
4Measurement precision
If the number of outgoing links from linking pages decreases, then the target page's rank increases, but exploitation by spammers becomes easier
Solution Approach 1:
The patent applies preliminary anti-action by introducing the robust rank function that limits the maximum contribution from the supporting set before spam exploitation can occur. This preemptive measure counteracts the vulnerability to low-outgoing-link exploitation by capping the influence of any single linking page, thereby maintaining rank calculation accuracy while preventing spam exploitation.
Data Source
AI summary
The claimed subject matter provides a system and/or a method that facilitates reducing spam in search results. An interface can obtain web graph information that represents a web of pages. A spam detection component can determines one or more features based at least in part on the web graph information. The one or more features can provide indications that a particular page of the web graph is spam. In addition, a robust rank component is provided that limits amount of contribution a single page can provide to the target page.