Webpage Information Extraction Using Filtering Characteristic Strings

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current DOM tree-based webpage information extraction methods are inefficient due to the need to analyze entire webpage contents, including irrelevant information, which reduces extraction efficiency.

Innovation Solution

A method and apparatus that extract target information from webpage source code using preset filtering characteristic strings and tag matching modes, segmenting the source code and generating page information in a specified format, thereby improving efficiency and relevance for mobile terminals.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If DOM tree-based webpage information extraction method is used, then webpage information can be extracted, but extraction efficiency is low due to analyzing entire webpage contents including irrelevant information

Engineering Contradiction:
Improvewebpage information extraction efficiencyVSAvoidamount of webpage content to be analyzed
Core Design Contradiction:
ProductivityVSQuantity of substance

Solution Approach 1:

The patent segments the webpage source code into multiple parts using regular expressions to identify and extract only the relevant content sections (such as article body, title, author) while excluding irrelevant parts (advertisements, navigation menus, footers). This segmentation approach divides the large quantity of webpage content into manageable segments, analyzing only the necessary portions to improve extraction efficiency.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent extracts only the target information (article content, title, author, publication time) from the webpage source code using predefined regular expression patterns. By taking out only the relevant information and discarding the rest, the system reduces the amount of data that needs to be processed and analyzed, thereby improving extraction efficiency without sacrificing information quality.

Inventive Principle:
Principle #2Taking out (Extraction)

2Reliability

If DOM tree-based webpage information extraction method is used, then webpage information can be extracted, but system resource usage increases

Engineering Contradiction:
Improveinformation extraction accuracyVSAvoidsystem resource usage
Core Design Contradiction:
ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent extracts only the essential information elements (title, author, content, publication time) using regular expressions, avoiding the overhead of building and traversing the entire DOM tree structure. This extraction approach maintains information accuracy by focusing on key elements while significantly reducing memory allocation and processing resources required.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent uses lightweight regular expression patterns instead of heavy DOM tree structures. The regular expressions are simple, disposable patterns that can be quickly compiled and executed without requiring complex object management, memory allocation for node structures, or iterative traversal overhead, thereby reducing system resource usage while maintaining extraction reliability.

Inventive Principle:
Principle #27Cheap short-living objects (Disposable)

Data Source

PatentUS10515142B2Method and apparatus for extracting webpage information
Publication Date: 2019.12.24 ALIBABA GROUP HOLDING LTD
  • US10515142B2 patent drawing
  • US10515142B2 patent drawing
  • US10515142B2 patent drawing

AI summary

A method is provided for extracting page information. The method includes obtaining to-be-processed source code of a webpage; based on a preset filtering characteristic string and a preset tag matching mode, extracting target information from the source code; and generating page information in a specified format based on the extracted target information.