General text information extraction method and system

Abstract

The invention provides a general text information extraction method. According to the method, a finite number of regular expressions are written to extract an original corpus; a finite proportion of corpus is divided from the extracted corpus to serve as a training corpus; an extraction model is constructed for the training corpus through an automatic mode induction method; and extraction is performed through the extraction model. The invention furthermore provides a general text information extraction system which is suitable for service scenes with different requirement levels and beneficialfor cultivating engineers meeting the corresponding requirements and forming "assembly line" operation of text key information extraction.

Description

technical field [0001] The invention relates to a general text information extraction method and system. Background technique [0002] With the continuous development of big data technology, massive and heterogeneous "unstructured documents" that were difficult to use, such as various web pages and office documents, have gradually gained attention. The demand for analysis and mining of unstructured documents However, the existing mainstream big data technology mainly solves the storage and calculation problems of massive data, and the mining and analysis of unstructured data has not been widely used and has not formed a stable technical route. [0003] The existing extraction of key information in text, the so-called "text key information extraction", refers to the automatic extraction of structured information from unstructured "unstructured documents" according to specific business needs. There are two types of schemes that are more mainstream: [0004] 1. Extraction bas...

AI Technical Summary

Problems solved by technology

[0010] Since the pattern can be automatically induced by the algorithm (forming an "extraction model"), the method based on the "pattern induction" has the advantage of strong adaptability. The extraction of key information that is difficult to observe specific rules) has a strong extraction ability; its disadvantages are:

[0014] The rule-based method has a stable extraction effect and does not require manual labeling, but it is too restrictive and the matching range is relatively small, so it is not suitable for text extraction without a fixed template;

[0015] The pattern automatic induction method has a large matching range, but it needs to prepare a lot of manually labeled corpus in advance, and the extraction effect is unstable and the accuracy is difficult to predict. It is not suitable for scenarios with strict requirements on the extraction accuracy.

Images