Automatic extraction method of web page information based on WEB structure

A technology for automatic extraction of web page information, applied in the Internet field, can solve the problem that applications cannot directly parse and utilize massive information, and achieve the effect of good extraction, noise reduction, and wide versatility.

Inactive Publication Date: 2010-06-09
XI AN JIAOTONG UNIV
View PDF0 Cites 67 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Due to the lack of description of the data itself, various HTML tags only tell the browser how to display the information it describes, but do not contain clear semantic information...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Automatic extraction method of web page information based on WEB structure
  • Automatic extraction method of web page information based on WEB structure
  • Automatic extraction method of web page information based on WEB structure

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] see figure 1 , the present invention includes the following modules: HTML source code acquisition module, DOMTree generation module, data record set extraction module, information extraction template generation module and intelligent classification module, first the HTML source code acquisition module obtains the HTML source code through the webpage URL; Then the DOMTree generation module corrects the wrong tags in the HTML, and filters the noise tags at the same time, and then generates a preprocessed page DOMTree; then extracts the data record set in the page according to the DOMTree characteristics of the page, and according to certain rules. The noise record set is filtered out, the acquired record set is analyzed, and the detailed information contained in it is obtained; at the same time, the extracted data record set is handed over to the information extraction template generation module for processing, and the information extraction template is generated; finally,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an automatic extraction method of WEB structured information, belonging to the technical field of the internet. The method adopts an automatic data recordset extraction method and an automatic information block content identification extraction method and comprises the following steps of: firstly acquiring an HTML code of a page through a URL; preprocessing the HTML code to eliminate errors and irrelevant marks, and generating a DOMTree of the page; then extracting similar information blocks on the page; finally extracting a text template, and classifying the content of information items of the extracted information blocks. The invention aims at providing an extraction method of WEB information capable of being used universally and automated within a certain limit and can obviously enhance the extraction precision and the automation degree.

Description

Technical field: [0001] The invention relates to the technical field of the Internet, in particular to a method for universally and automatically extracting WEB structured information. Background technique: [0002] With the continuous development of the Internet, the number of Web pages has increased significantly, and the network has become a huge, distributed and shared information resource. A lot of information is included in the vast web, how to help people quickly extract effective information has become a very important issue. [0003] Most of the data currently appears in the form of HTML. Due to the lack of description of the data itself, various HTML tags only tell the browser how to display the information it describes, but do not contain clear semantic information. It is a semi-structured data, which makes the page described by HTML only It is suitable for human browsing, and the application program cannot directly analyze and utilize the massive information on...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 管晓宏高峰周亚东赵俊舟薛峰
Owner XI AN JIAOTONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap