Chinese name recognition method based on recurrent neural network

Abstract

The invention provides a Chinese name recognition method based on a recurrent neural network. The method includes the steps of S1, corpus pretreatment; S2, word vector training, wherein a word2vec tool is used for word vector training; S3, Chinese name recognition model training, wherein data obtained after processing in S1 and word vectors obtained after training in S2 are used for training a neural network model; S4, name recognition and aftertreatment, wherein the model obtained after modeling in S3 carries out name recognition on test corpuses, names recognized by the model are subjected to aftertreatment through a context rule and a diffusion algorithm, and finally names are obtained. By means of the method, the complexity of feature selection during Chinese name recognition can be effectively lowered, rich syntax and grammar information included in Chinese texts is fully utilized through the word vectors, the generalization ability of the model is improve accordingly, Japanese names and foreign transliteration names are recognized at the same time, and the extent of Chinese name recognition is widened.

Description

technical field [0001] The invention relates to the fields of natural language processing, deep learning, named entity recognition and the like, in particular to a recognition method applicable to Chinese names, Japanese people and foreign transliterated names in Chinese texts. Background technique [0002] With the rapid development of Internet technology and the rapid expansion of new information, the need to extract useful information from massive data is becoming more and more urgent. How to quickly and effectively obtain useful information and knowledge from large-scale, unstructured language texts has become a research hotspot in the field of natural language processing. Compared with English and other languages, Chinese information lacks separation marks, which increases the difficulty of named entity recognition. But named entity recognition has an important impact in areas such as information extraction, machine translation, and text classification. In the named e...

AI Technical Summary

Problems solved by technology

In addition, the current Chinese name recognition system mainly recognizes Chinese names, but less involves Japanese names, foreign transliterated names, and minority transliterated names. The breadth of Chinese name recognition needs to be improved urgently.

Images