Entity attribute extraction method and system oriented to open web page

Abstract

The invention provides an entity attribute extraction method and system oriented to an open web page. The method includes the steps: extracting a text of the open web page to obtain a candidate text set of a target entity; extracting the value of a target entity attribute from the candidate text set in a rule-based or statistical-based mode according to the frequency of the target entity attribute in a training text set. Extraction accuracy and recall rate of the entity attribute of the open web page can be improved, and the method is independent of a web page structure and can adapt to change of open web page types.

Description

technical field [0001] The invention relates to the technical field of data mining, in particular to an entity attribute extraction method and system for open webpages. Background technique [0002] Open webpages refer to unstructured Internet webpages with unfixed data sources and containing a variety of network data, such as blogs, forums, news, chat records, emails, etc. It's all unpredictable. With the development of network technology, especially the rapid development of Internet and Intranet technology, open web pages have their own characteristics of flexible structure, while the number of open web pages is rapidly increasing, it also brings difficulties to understand the text: [0003] 1. The text structure is not fixed, and there is no specific context grammar; [0004] 2. The scope of keywords is not fixed, and the subject areas involved are diverse; [0005] 3. The length of the text is not fixed, and there is a large gap in the amount of contextual information...

AI Technical Summary

Problems solved by technology

[0005] 3. The length of the text is not fixed, and there is a large gap in the amount of contextual information;

[0006] 4. The data source is not fixed and the language phenomenon is complex

[0009] First, the text structure of open web pages is not fixed, and entities and their descriptions have no fixed rules to follow, and most of them are in free text, which is not easy to extract and analyze;

[0010] Second, the traditional rule-oriented attribute extraction method, the rule definition is rigid, too dependent on the context grammar, and the matching efficiency is low;

[0011] Third, the data sources of open web pages are not fixed, the language phenomenon is complex, and common rules are difficult to cover. Traditional rule-based attribute extraction does not support nested matching of rules;

[0012] Fourth, the traditional statistics-based entity attribute extraction method, the preparation of training data is too manual, the efficiency is not high, and the accuracy and recall rate are low;

[0013] Fifth, traditional attribute extraction is mostly limited to a certain field or discipline, and the system cannot be directly transplanted to other fields or disciplines for use. It lacks universal correlation features and is not easy to transplant and expand

Images