Webpage Content Extraction via Image Histogram Analysis

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for extracting webpage content are inaccurate due to frequent changes in webpage layout and label attributes, affecting user reading experience.

Innovation Solution

A method and device that determine the starting and ending positions of body content in a webpage snapshot using a histogram statistical array and rendering tree, allowing for precise extraction of content without relying solely on label attributes.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If body content extraction is performed based on label attributes and button label positions, then extraction can be achieved, but extraction accuracy deteriorates due to frequent changes in webpage layout and attributes

Engineering Contradiction:

Improveextraction capabilityVSAvoidextraction accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent introduces image processing techniques as an intermediary method to locate body content. Instead of directly relying on label attributes (which change frequently), the system uses image processing to identify visual characteristics of body content regions, creating a stable intermediary layer that bridges the gap between extraction capability and accuracy.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the mechanical/structural approach (relying on HTML label attributes and document structure) with an optical/image processing approach. By using image processing to analyze visual characteristics such as background colors, text density, and layout patterns, the system achieves more stable and accurate body content extraction that is not affected by changes in HTML attributes.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Ease of manufacture

If extraction relies on label attributes, then implementation is simple, but accuracy is affected by frequent layout changes

Engineering Contradiction:

Improveimplementation simplicityVSAvoidextraction stability

Core Design Contradiction:

Ease of manufactureVSReliability

Solution Approach 1:

The patent changes the extraction parameters from HTML attribute-based (label text, class names) to image-based parameters (pixel color distributions, text density, spatial patterns). This parameter transformation makes the extraction process independent of HTML structure changes, thereby improving reliability while maintaining implementation simplicity through standardized image processing algorithms.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS10970464B2Method, device for processing webpage content and storage medium

Publication Date: 2021.04.06 ZTE CORP

AI summary

A method, device for processing web content and a storage medium, the method comprising: determining, in a histogram statistical array of a webpage snapshot, a starting position and an ending position of a longest continuous image along a longitudinal axis direction; determining, based on a resolution of the webpage snapshot, a first starting position and a first ending position of a body in the webpage snapshot along the longitudinal axis direction in a range from the starting position to the ending position in the longitudinal axis direction; determining, based a rendering tree of the webpage snapshot, a second starting position and a second ending position of the body in the webpage snapshot in the longitudinal axis direction in a range from the first starting position to the first ending position.