Webpage information extraction method

Abstract

The invention discloses a webpage information extraction method, in particular a method for extracting concept attributes from a network encyclopedia data source and processing the concept attributes. The method comprises the following steps of: constructing an example list, and extracting candidate attributes of examples in the list from a multi-source heterogeneous data source; performing synonymic induction on the extracted attributes, and putting synonymic attributes in the same set; sub-classifying the induced attributes; analyzing the corresponding attribute value types of the classified attributes; and recommending the attributes and corresponding attribute value type information to a user, or storing the attributes and the corresponding attribute value type information into a structured database. By adoption of the scheme of the invention, high-quality concept attribute information can be extracted from a webpage, a knowledge base can be better constructed, and other natural language processing tasks such as extraction of attribute values, text classification and classification of query logs in a search engine can be better performed.

Description

technical field [0001] The invention provides a method for extracting web page information, in particular to a method for extracting concept attributes from a network encyclopedia data source and processing them. Background technique [0002] Today, with the explosive growth of Internet texts, how to organize information and represent knowledge reasonably and effectively, and establish a good knowledge base so that people can quickly and quickly obtain the knowledge they want from massive web pages is a very important task. research work. In the construction of knowledge base, concepts and attributes are the core elements of knowledge representation. A concept is an object that reflects objective things and their unique attributes, and an attribute is a description of the characteristics of a concept. From attribute information, a more comprehensive understanding of the characteristics of a concept can be obtained. Therefore, in the automatic construction of knowledge base...

AI Technical Summary

Problems solved by technology

[0004] However, there is a problem in these current methods, that is, only some candidate attributes are extracted, and the extracted attributes are not processed in the later stage, which leads to the rough granularity of the extracted candidate attributes and the low accuracy. The expression of polysemous words is relatively poor in quality, and can only be added to the knowledge base after manual selection

And these methods do not evaluate the attributes, because some attributes are closely related to the target concept, and some are relatively weak. Selecting the closely related attributes can be beneficial to the classification of concepts

Images