Web Information Extraction Using HTML Tag Ruler Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for extracting web information from web pages, such as keyword complete match, keyword fuzzy match, and regular expression algorithms, suffer from high time complexity and poor extensibility, particularly when handling large amounts of data.
Innovation Solution
A method and device that utilize HTML tags as 'tag ruler' elements to generate a tag ruler from an HTML text, segment web information according to matched tags, and extract relevant information segments efficiently, allowing for flexible and accurate information extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If keyword complete match method is used to extract web information, then extraction accuracy is improved, but time complexity increases significantly when handling large amounts of data
Solution Approach 1:
The patent segments web information extraction into multiple stages: first using HTML tag structure to identify potential information regions, then applying keyword matching only within those segmented regions. This segmentation approach maintains extraction accuracy while reducing the overall search space and time complexity when handling large amounts of web data.
Solution Approach 2:
The patent performs preliminary action by first parsing HTML tags and identifying structurally significant regions before conducting keyword matching. This preliminary structuring of the data based on HTML grammar rules prepares the information in advance, allowing subsequent keyword extraction to be performed more efficiently on already-organized segments rather than raw text.
2Difficulty of detecting and measuring
If regular expression algorithms are used for web information extraction, then pattern matching capability is improved, but extensibility deteriorates when dealing with diverse web page formats
Solution Approach 1:
The patent applies universality by using HTML tag structure as a universal framework that can handle diverse web page formats. Instead of creating separate regular expressions for each format, the system uses the common HTML grammar structure to universally identify information regions across different page types, then adapts keyword extraction to specific contexts within that universal framework.
Solution Approach 2:
The patent changes parameters by switching from fixed regular expression patterns to flexible HTML tag-based region identification combined with context-aware keyword matching. This parameter change allows the system to adapt to different web page formats by adjusting which tags are considered significant and how keywords are extracted within those regions, rather than relying on rigid pattern matching.
3Loss of information
If web information is extracted by analyzing contents of web files, then information completeness is improved, but processing complexity increases
Solution Approach 1:
The patent applies the taking out principle by extracting and utilizing the existing HTML tag structure as a guiding framework for information extraction. Instead of analyzing entire web file contents from scratch, the system takes out and leverages the pre-existing structural information embedded in HTML tags to identify and extract relevant content regions, thereby reducing processing complexity while maintaining information completeness.
Data Source
AI summary
A method for extracting web information includes: selecting a number of Hypertext Markup Language, HTML, tags as tag ruler elements to generate a tag ruler from an HTML text of a web page according to sequence of the HTML text; matching the HTML text with the tag ruler elements in the tag ruler according to the sequence of the tag ruler elements in the tag ruler, segmenting web information according to matched HTML tags and saving web information segments and location information of HTML tags enclosing the web information segments in the HTML text; and determining location of HTML tags containing web information needed by a user in the HTML text, extracting web information segments corresponding to the web information needed by the user from the saved web information segments.