Named entities recognition method based on bidirectional LSTM and CRF

Abstract

The invention discloses a named entities recognition method based on bidirectional LSTM and CRF. The named entities recognition method based on the bidirectional LSTM and CRF is improved and optimizedbased on the traditional named entities recognition algorithm in the prior art. The named entities recognition method based on the bidirectional LSTM and CRF comprises the following steps: (1) preprocessing a text, extracting phrase information and character information of the text; (2) coding the text character information by means of the bidirectional LSTM neural network to convert the text character information into character vectors; (3) using the glove model to code the text phrase information into word vectors; (4) combining the character vectors and the word vectors into a context information vector and putting the context information vector into the bidirectional LSTM neural network; and (5) decoding the output of the bidirectional LSTM with a linear chain condition random field to obtain a text annotation entity. The invention uses a deep neural network to extract text features and decodes the textual features with the condition random field, therefore, the text feature information can be effectively extracted and good effects can be achieved in the entity recognition tasks of different languages.

Description

technical field [0001] The invention relates to a named entity recognition method, in particular to a named entity recognition method based on bidirectional LSTM and CRF. Background technique [0002] Named Entities Recognition (NER) is a basic task of Natural Language Processing (NLP). Its purpose is to identify named entities such as names of people, places, and organizations in the input text. [0003] In the field of named entity recognition, existing technologies can be divided into two categories. One is based on dictionaries and rules, which construct phrase dictionaries for high-frequency words according to the frequency of occurrence of phrases. For words that can be retrieved in dictionaries, directly It is identified as a named entity; or according to the composition rules of the phrase, for example, the name of an institution usually includes location and function information, etc., and the phrase that meets the corresponding rules is directly marked. The other ...

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to provide a named entity recognition method based on bidirectional LSTM and CRF, which effectively solves the problems of low efficiency and low accuracy of traditional named entity recognition in practical applications

Images