Automatic generation method for wrapper on the basis of DOM (Document Object Model) tree abstraction

Abstract

The invention relates to an automatic generation method for wrapper on the basis of DOM (Document Object Model) tree abstraction. The method comprises a wrapper generation stage and a structural dataabstraction stage. The wrapper generation stage comprises the following steps that: a user inputs a webpage set to obtain a DOM tree set; then, combining DOM trees, traversing the DOM trees to combinethe subnodes with the same tag, carrying out path feature annotation on each node, and finally, converting the DOM tree set into a combination tree set; carrying out abstraction operation on the combination tree set to obtain an abstraction tree, and storing the abstraction tree into a database; and according to a configuration document, determining the path feature of the structural data in thecombination tree, and writing corresponding processed path features into a document to generate the wrapper. The structural data abstraction stage comprises the following steps that: analyzing an extracted target webpage into the DOM tree, matching with the abstraction tree to determine whether the target webpage is a type webpage corresponding to the wrapper or not; and reading a path in the configuration document to extract the target DOM tree. By use of the method, the wrapper can be automatically generated, and in addition, the method exhibits good accuracy and time performance.

Description

technical field [0001] The invention belongs to the field of cloud computing, and in particular relates to a method for automatically generating wrappers based on DOM tree abstraction. Background technique [0002] Traditional wrappers are defined manually, and different wrappers need to be made for different types of pages, so the maintenance of wrappers is a big overhead. Once the original page style changes, the original wrappers need to be redefined . Therefore, the current mainstream research trend is the automatic generation of wrappers, and this application proposes a feasible technology for automatic generation of wrappers based on DOM tree abstraction. This technology mainly consists of two parts: first, the DOM tree abstraction of the target type webpage; second, the feature acquisition and positioning of the target node. By using this technology, wrappers can be automatically generated for various types of web pages. Experiments are carried out on 5 websites, a...

AI Technical Summary

Problems solved by technology

[0002] Traditional wrappers are defined manually, and different wrappers need to be made for different types of pages, so the maintenance of wrappers is a big overhead. Once the original page style changes, the original wrappers need to be redefined

Images