Webpage information extracting method and system

Abstract

The invention discloses a webpage information extracting method and system. The webpage information extracting method comprises the following steps of: analyzing a webpage to be extracted as a document object model (DOM) tree, and obtaining a template corresponding to the webpage to be extracted; traversing the DOM tree according to webpage division fineness defined by the template, and dividing the corresponding webpage into content blocks; and outputting content and type information of the content blocks according to an output rule defined by the template. With the adoption of the webpage information extracting method and system provided by the invention, the precision of extracting webpage information can be improved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a web page information extraction method and system thereof. Background technique [0002] Web information extraction refers to extracting target information from web pages, which is a basic link in search engines. Web pages themselves are structured data, so there are many features that can be used to extract information from them. For example, a browser can render the source code of a web page into a beautiful web page through the structural information in the web page. For a search engine, it is not only necessary to use structured information to extract the information, but also to perform further processing on the extracted information, such as classifying and labeling the extracted content. [0003] In the face of massive and ever-changing data, the mainstream of current web page extraction methods is automatic extraction and classification based on machine learning. M...

AI Technical Summary

Problems solved by technology

[0004] For a large site, there may be multiple sets of templates, and the internal structure of the same template is also very complicated, which brings great difficulties to the configuration of the template, which will lead to possible omissions of web page information

Moreover, local changes of site templates are relatively frequent. The configured templates need to be changed frequently to ensure the accuracy of network element information extraction, and it is not easy to monitor the changes of site templates.

It can be seen that the existing web page information extraction methods are not convenient for detecting web page information, and the processing effect on web pages of certain types of sites is not very good

Especially for vertical search, it is necessary to accurately extract the information in the webpage, and it is necessary to accurately classify the extracted information types, and the existing automatic extraction methods through machine learning become difficult to apply, or the effect is poor

[0005] It can be seen that the existing web page information extraction technology needs to improve the accuracy rate of web page extraction for relatively large sites, and the recall rate of extracted information also needs to be improved

Images