Bidirectional recursive neural network-based enterprise abbreviation extraction method

Abstract

The invention relates to the field of natural language processing, in particular to a bidirectional recursive neural network-based enterprise abbreviation extraction method. The method comprises the steps of serializing to-be-processed texts through word segmentation processing; selecting a certain number of the to-be-processed texts to perform manual annotation, segmentally annotating enterprise names in the to-be-processed texts as starting parts, keyword parts, industrial parts and organization form parts, and annotating data except the enterprise names as unrelated parts; inputting annotated training samples into a bidirectional recursive neural network to train the bidirectional recursive neural network; extracting word sequences belonging to the enterprise names through prediction of the bidirectional recursive neural network, and further extracting fields belonging to the keyword parts of the names as enterprise abbreviations; and establishing a corresponding enterprise abbreviation database. Therefore, powerful technical support is provided for related information analysis of informal texts.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular to a method for extracting enterprise abbreviations based on a bidirectional recursive neural network. Background technique [0002] With the rapid development of the Internet, a large amount of public web data has been generated, which has also spurred various emerging industries based on big data technology, such as Internet medical care, Internet education, corporate or personal credit investigation, etc. The rise of these Internet industries is inseparable from the analysis of a large amount of information and data, and the value of information analysis lies in sharpness and accuracy. Sharp analysis requires timely and rapid discovery of new information; but most of the data obtained directly from web pages are very Structured, in order to use these data, data cleaning has become the place where companies spend the most time and energy. In data cleaning, the extractio...

AI Technical Summary

Problems solved by technology

To use conditional random fields, you first need to design and construct feature templates based on the characteristics of the entity to be recognized. Feature templates include first-order words or multi-order phrases with a specified window size context, word prefixes, suffixes, part-of-speech tags and other status features; feature templates The construction is very time-consuming and labor-intensive, and the feature templates set manually are often only based on the characteristics of some samples, which has poor versatility; the recognition results are highly dependent on the feature templates, and usually only local context information can be used. The use is also independent of each other. The prediction cannot rely on longer historical state information, nor can it use longer future information feedback to correct possible historical errors; the prediction process is time-consuming and laborious, and the prediction results are difficult to achieve global optimality

In addition, due to the randomness and irregularity of the company's abbreviation, it is difficult to extract the company's abbreviation using the traditional CRF method

Images