Web Block Classification Using Linguistic Features for Search Noise Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Search engine services face difficulties in identifying the primary topic of a web page due to the presence of noise information, such as advertisements, which can lead to incorrect keyword mapping and irrelevant search results.
Innovation Solution
A classification system that trains a classifier to differentiate between information and non-information blocks on a web page using linguistic and layout features, allowing for accurate identification of primary content and noise, employing machine learning techniques like support vector machines and neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional keyword extraction methods are used on entire web pages, then all keywords including noise information are captured, but the accuracy of representing the primary topic deteriorates
Solution Approach 1:
The web page is divided into multiple functional blocks (content blocks, advertisement blocks, navigation blocks, etc.) using HTML structure analysis and layout features. Keywords are then extracted separately from each block type, allowing the system to focus on content blocks for primary topic representation while excluding noise from advertisement and navigation blocks.
Solution Approach 2:
The patent extracts and removes noise information by identifying and excluding specific block types ( advertisements, navigation bars, legal notices) from the keyword extraction process. A classification system with trained classifiers distinguishes between information-bearing blocks and noise blocks, extracting keywords only from relevant blocks to improve topic representation accuracy.
2Loss of information
If the entire web page content is analyzed for keyword extraction, then comprehensive coverage is achieved, but the ability to identify primary topic deteriorates due to noise information
Solution Approach 1:
The web page is segmented into functional blocks using HTML structure analysis, CSS layout features, and content characteristics. This segmentation allows the system to identify which blocks contain primary topic information versus which blocks contain noise, thereby maintaining information completeness from relevant blocks while excluding noise blocks from analysis.
Solution Approach 2:
Different quality standards are applied to different blocks based on their function. Content blocks are analyzed with comprehensive keyword extraction to capture primary topic information, while advertisement blocks and navigation blocks are either excluded or analyzed with different criteria. This local differentiation of analysis quality improves overall primary topic identification accuracy.
3Productivity
If a simple classification approach is used for web blocks, then processing speed is maintained, but the accuracy of distinguishing information from noise deteriorates
Solution Approach 1:
The system performs preliminary classification of web blocks using multiple features (HTML structure, layout position, content characteristics) before keyword extraction. Classifiers are pre-trained on labeled data to recognize patterns of information blocks versus noise blocks. This preliminary action enables efficient filtering that maintains processing speed while achieving high classification accuracy through ensemble methods and feature integration.
Solution Approach 2:
The classification system uses multiple parameters and features (block position, size, HTML tags, text density, link density) rather than a single simple criterion. By changing from a simple classification approach to a multi-parameter approach with trained classifiers, the system achieves higher accuracy in distinguishing information from noise while maintaining productivity through efficient feature extraction and classification algorithms.
Data Source
AI summary
A classification system trains a classifier to classify blocks of the web page into various classifications of the function of the block. The classification system trains a classifier using training web pages. To train a classifier, the classification system identifies the blocks of the training web pages, generates feature vectors for the blocks that include a linguistic feature, and inputs classification labels for each block. The classification system learns the coefficients of the classifier using any of a variety of machine learning techniques. The classification system can then use the classifier to classify blocks of web pages.