Web page cleaning method based on theme

A page and theme technology, applied in the field of theme-based Web page cleaning, can solve problems such as low efficiency, high cost, and complex algorithms, and achieve the effects of improving cleaning efficiency, reducing difficulty, and reducing complexity and difficulty

Inactive Publication Date: 2013-12-11
BEIJING UNIV OF TECH
View PDF2 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] Aiming at the problems of complex algorithm, low efficiency and high cost in the prior art, the pres

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Web page cleaning method based on theme
  • Web page cleaning method based on theme
  • Web page cleaning method based on theme

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] The present invention will be further described below according to the accompanying drawings and specific embodiments.

[0022] The flowchart of the method of the present invention is as figure 1 shown, including the following steps:

[0023] Step 1, determine the subject area: Manually collect and select multiple sample pages for analysis, extract keywords, determine the subject range of the page content by keywords, and finally determine the subject area.

[0024] Since this method is based on the theme, the determination of the page theme can greatly improve the accuracy and efficiency of cleaning.

[0025] This implementation selects tourism as the theme. Tourism information webpage is a kind of webpage with strong development value. It has the characteristics of stable structure, large amount of information, high practical value and wide demand. Considering the above-mentioned characteristics, if the traditional page cleaning work is performed on such a web page...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the field of computer science and technology, and relates to a web page cleaning method based on a theme. The web page cleaning method comprises the steps that the field of the theme is determined; a sample page is collected, the web page structure of the sample page is analyzed, and a page element tree structure is determined; an extracted content label boundary is determined, regular expression matching rules are defined according to the extracted content label boundary, and an extraction rule template is formed by a plurality of extraction rules; a page to be cleaned is cleaned and preprocessed to remove noisy contents easy to recognize; the page is cleaned through the generated extraction rule template to acquire a cleaned page. Because useful information is directly extracted from the page, the web page cleaning method greatly reduces page cleaning complexity and difficulties. Because web page labels are complex, diversified and locally fixed, required labels can be directly matched and extracted according to a regular expression to reserve useful information and automatically remove useless information, and accordingly the purpose of page cleaning can be effectively achieved.

Description

technical field [0001] The invention belongs to the field of computer science and technology, and relates to a theme-based method for cleaning Web pages. Background technique [0002] Resource information on the Internet today is still concentrated in various Web pages. Because Web pages are rich in content and complex in structure, sometimes we may only care about a small part of the content blocks in the page. How to remove a large amount of useless information in different web page structures, and only obtain the required data to provide people with better search services has become an important task in the process of web resource processing. Page cleaning is for to achieve this goal. [0003] The current general page cleaning method is mainly divided into three steps: the first step is to remove irrelevant information such as style sheets, scripts, and comments in the page. The second step divides the page into blocks, including image blocks, text blocks, link blocks, ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 沈琦宋清明张猛汤艳
Owner BEIJING UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap