Web Page Analysis Using Mixture of Random Walks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current web page classification methods treat hyperlink-based and word-based representations as separate systems, failing to effectively combine multiple graph representations for comprehensive web page analysis, such as clustering and spam detection.
Innovation Solution
A system that represents web pages as a directed graph and generates similarity graphs based on features, using random walks to create a Markov mixture for web page analysis, allowing for clustering, classification, and spam detection across multiple graph representations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple graph representations are used for web page analysis, then the comprehensiveness and accuracy of analysis results improve, but the system complexity increases
Solution Approach 1:
The patent combines multiple graph representations (hyperlink graph and content-based similarity graphs) into a unified framework using random walk techniques. The mixture of random walks integrates information from different graph types, allowing the system to leverage both structural and content-based features simultaneously, thereby improving analysis accuracy while managing complexity through a coherent mathematical framework.
2Device complexity
If separate systems are used for hyperlink-based and word-based web page classification, then the system design is simpler, but the analysis comprehensiveness deteriorates
Solution Approach 1:
The patent creates a universal random walk framework that can process multiple types of graph representations simultaneously. The mixture of random walks serves as a multi-functional mechanism that handles both hyperlink-based structural information and content-based semantic information within a single analytical system, eliminating the need for separate classification systems while preserving information from all sources.
Data Source
AI summary
A collection of web pages is modeled as a directed graph, in which the nodes of the graph are the web pages and directed edges are hyperlinks. Web pages can also be represented by content, or by other features, to obtain a similarity graph over the web pages, where nodes again denote the web pages and the links or edges between each pair of nodes is weighted by a corresponding similarity between those two nodes. A random walk is defined for each graph, and a mixture of the random walks is obtained for the set of graphs. The collection of web pages is then analyzed based on the mixture to obtain a web page analysis result. The web page analysis result can be, for example, clustering of the web pages to discover web communities, classifying or categorizing the web pages, or spam detection indicating whether a given web page is spam or content.