Automatic generating method of wrapper of complex page

Abstract

The invention discloses an automatic generating method of a wrapper of complex pages. The method comprises the followings steps: (1) acquiring two HTML page documents based on the same template to generate an HTML Tag tree; (2) acquiring a minimum region DS containing a data record set; (3) acquiring initial data record (DR) from the minimum region; (4) recording the layout combination relation of the DR according to the initial data record, determining aggregation relation of extraction items according to the similarity of characteristic items, carrying out semantic annotation on entities in the same aggregation block in combination with the knowledge of the field, and recombining a new data record according to the semantic relation among entities; (5) generating the extraction rule of each aggregation block according to the position relation of the generated data record in step (4) in the HTML Tag tree, and then constructing the wrapper. The invention can extract the true data record rule from the complex pages through the analysis of the structural relation of the HTML Tag tree, thereby automatically constructing the wrapper with high extraction accuracy rate.

Description

technical field [0001] The invention relates to a method for identifying information of a Web page, in particular to an automatic generation method of a wrapper for extracting deep web page data information applied to complex pages. Background technique [0002] Most of the Web pages on the Internet are presented in the form of HTML, and the characteristics of HTML enable any organization and individual to publish information with various contents and rich forms on the Web according to their own ideas. The semi-structured or even unstructured state of Web data makes Web pages only suitable for human browsing, and it is not conducive to applications to directly parse and utilize the massive and valuable information on the Web. On the other hand, with the rapid development of the Internet and e-commerce, "information explosion" has become an obstacle for people to obtain information effectively. Therefore, it becomes more realistic and urgent to use computer to extract Web in...

AI Technical Summary

Problems solved by technology

The current wrappers mainly have the following disadvantages: (1) The development and use of wrappers requires high skills and manual participation, and it takes a lot of time to study the structure of the web pages to be extracted

This approach does not take advantage of large-scale web data integration

(2) Since the wrapper is closely related to a specific data source, if the designer of the webpage changes the layout of the original webpage, the existing wrapper may become invalid

Images