Webpage Information Extraction Using Filtering Characteristic Strings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current DOM tree-based webpage information extraction methods are inefficient due to the need to analyze entire webpage contents, including irrelevant information, which reduces extraction efficiency.
Innovation Solution
A method and apparatus that extract target information from webpage source code using preset filtering characteristic strings and tag matching modes, segmenting the source code and generating page information in a specified format, thereby improving efficiency and relevance for mobile terminals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If DOM tree-based webpage information extraction method is used, then webpage information can be extracted, but extraction efficiency is low due to analyzing entire webpage contents including irrelevant information
Solution Approach 1:
The patent segments the webpage source code into multiple parts using regular expressions to identify and extract only the relevant content sections (such as article body, title, author) while excluding irrelevant parts (advertisements, navigation menus, footers). This segmentation approach divides the large quantity of webpage content into manageable segments, analyzing only the necessary portions to improve extraction efficiency.
Solution Approach 2:
The patent extracts only the target information (article content, title, author, publication time) from the webpage source code using predefined regular expression patterns. By taking out only the relevant information and discarding the rest, the system reduces the amount of data that needs to be processed and analyzed, thereby improving extraction efficiency without sacrificing information quality.
2Reliability
If DOM tree-based webpage information extraction method is used, then webpage information can be extracted, but system resource usage increases
Solution Approach 1:
The patent extracts only the essential information elements (title, author, content, publication time) using regular expressions, avoiding the overhead of building and traversing the entire DOM tree structure. This extraction approach maintains information accuracy by focusing on key elements while significantly reducing memory allocation and processing resources required.
Solution Approach 2:
The patent uses lightweight regular expression patterns instead of heavy DOM tree structures. The regular expressions are simple, disposable patterns that can be quickly compiled and executed without requiring complex object management, memory allocation for node structures, or iterative traversal overhead, thereby reducing system resource usage while maintaining extraction reliability.
Data Source
AI summary
A method is provided for extracting page information. The method includes obtaining to-be-processed source code of a webpage; based on a preset filtering characteristic string and a preset tag matching mode, extracting target information from the source code; and generating page information in a specified format based on the extracted target information.


