Chinese named entity recognition method suitable for multiple fields

Abstract

The invention belongs to the field of named entity recognition, and particularly relates to a Chinese named entity recognition method suitable for multiple fields. The method comprises the following steps: carrying out specific field classification on a Chinese named entity data set; adopting a sample learning method to sample the data after field classification to obtain a data set, and inputting the data set into a shared code presentation layer of the model; obtaining the probability distribution of the field to which the data belongs through a domain classifier, enabling the expert layer of each field to extract the unique characteristics of the field, enabling the public expert layer to synthesize the characteristics of experts of each field according to the probability distribution of the field to which the public expert layer belongs, and inputting the characteristics extracted by each expert layer into the corresponding CRF layer to obtain an entity recognition result. According to the method, the multi-task learning technology is applied to the field of Chinese named entity recognition, data of different domains are independently regarded as a training task, and a specific multi-expert model structure is designed to extract unique features and common features of the domains, so the different domains assist one another, and the recognition effect is improved.

Description

technical field [0001] The invention belongs to the fields of deep learning, transfer learning, natural language processing, and named entity recognition, and specifically relates to a Chinese named entity recognition method applicable to multiple fields. Background technique [0002] Named entity recognition technology is a key technology in the field of natural language processing and the basis of other natural language processing applications. It aims to extract entity fragments that people care about from text, such as person names, organization names, place names, etc. At present, Chinese named entity recognition for a single domain has achieved good performance. [0003] With the in-depth application of natural language processing technology and the development of various industries in society. There are also more and more types of texts, such as radio conversations, TV news, Internet blogs, etc. The named entities defined in different domains are also different. Ho...

AI Technical Summary

Problems solved by technology

However, the current problems are: single-domain named entity recognition requires a large amount of labeled data, and most fields require professional labeling, which is expensive; when the domain transfer occurs in the training set and test set, the performance will drop significantly; in order to get usable The effect takes a lot of time to retrain the model

Images