Web Page Recognition Using Logistic Regression Probability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing web page recognition methods are inefficient due to the diversity of web page content, leading to incorrect judgments of safe web pages as spam, especially when distinguishing between spam and non-spam web pages is difficult.
Innovation Solution
A web page recognition method utilizing a logistic regression model and a naive Bayesian model to calculate weights of segmented words and web page features, determining the category of a web page by comparing weights in two predetermined categories, thereby improving recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a key word library is used for web page recognition, then the recognition process is simple and fast, but the recognition accuracy deteriorates due to diversity of web page content and false positives
Solution Approach 1:
The patent transforms the recognition approach from discrete keyword matching to continuous probability calculation. By computing probability values based on multiple features (keywords, links, images, layout) and applying threshold comparison, the system achieves both efficiency and improved accuracy. The probability value represents the likelihood of a web page being spam, allowing nuanced classification beyond simple keyword presence.
2Measurement precision
If multiple features are analyzed to improve recognition accuracy, then the recognition precision improves, but the system complexity increases
Solution Approach 1:
The patent divides the web page analysis into distinct feature segments: keywords in content, links, images, and layout elements. Each segment is independently analyzed and assigned a probability value. This segmentation allows comprehensive analysis without creating a monolithic complex system, as each feature can be processed separately and then integrated through probability combination.
Solution Approach 2:
The patent merges multiple feature analysis results into a unified probability value. By combining probabilities from keywords, links, images, and layout features, the system achieves comprehensive recognition accuracy. The merging is done through probability calculation that synthesizes all feature contributions into a single decision metric.
Data Source
AI summary
Disclosed is a web page recognizing method, which includes obtaining a weight of each segmented word of a web page to be recognized; acquiring, according to the weight of each segmented word of a web page to be recognized, weights of the web page to be recognized in two predetermined web page categories through calculation by using a logistic regression model established in advance; and taking a web page category having a greater weight as a category of the web page to be recognized. Also disclosed is a web page recognizing apparatus. The present disclosure recognizes a web page more accurately, especially for a web page in which key words are difficult to distinguish.