A web page information extraction method and device

Abstract

The embodiment of the present invention provides a method and device for extracting webpage information, including: determining the candidate elements and text information of the webpage according to the DOM tree of the webpage; determining the candidate location information of the webpage according to the screenshot of the displayed page of the webpage; The text information of each candidate element determines the first probability of each candidate element as a target extraction element and the second probability of each candidate position as a target extraction position; determine the target from each candidate element and each candidate position according to the first probability and the second probability Extracting elements and target extraction positions; performing information extraction on the webpage according to the candidate elements determined as target extraction elements and the candidate positions determined as target extraction positions. By extracting the text information and candidate position information of the candidate elements of the web page, the neural network model and the spatial probability distribution model are used to locate the extraction position, which improves the positioning accuracy and fault tolerance rate of the web page information extraction, and realizes the automatic extraction of web page information.

Description

Technical field [0001] Embodiments of the present invention relate to the field of Internet technology, and more particularly to a web information extraction method and apparatus. Background technique [0002] With the rapid growth of information in the Internet, the Internet page has become the most important way for human acquisition knowledge and information. Traditional search engine technology can raise the web page according to the user query, improve the efficiency of information retrieval. However, for a large number of results of search engine feedback, manual investigation and screening are required. With the explosive growth of information, this information search method has been difficult to meet the needs of people to control information resources, and the emergence of knowledge map technology provides new ideas for information retrieval issues. Knowledge map technology returns to users in a graphic manner, which is the foundation and bridge of intelligent language r...

AI Technical Summary

Problems solved by technology

[0003] Embodiments of the present invention provide a method and device for extracting webpage information, which are used to solve the problem of low efficiency and accuracy of the method of relying on manually marking the DOM tree to locate the location of webpage information extraction

Images