BERT-BiGRU-IDCNN-CRF named entity identification method based on attention mechanism

Abstract

The invention discloses a BERT-BiGRU-IDCNN-CRFnamed entity recognition method based on an attention mechanism. The method comprises the steps: training a BERT pre-training language model through large-scale label-free prediction; on the basis of the trained BERT model, constructing a complete BERT-BiGRU-IDCNN-Attention-CRF named entity recognition model; constructing an entity recognition training set, and training the complete entity recognition model on the training set; inputting an expected material to be subjected to entity recognition into the trained entity recognition model, and outputting a named entity recognition result. According to the method, feature vectors extracted by the BiGRU and IDCNN neural networks are combined, the defect that local features are ignored in the process of extracting global context features by the BiGRU neural network is overcome, and meanwhile, the attention mechanism is introduced, and weight allocation is performed on the extracted features, so that the features playing a key role in entity recognition are enhanced, irrelevant features are weakened, and the recognition effect of named entity recognition is further improved.

Description

technical field [0001] The invention belongs to the field of named entity recognition, and in particular relates to a BERT-BiGRU-IDCNN-CRF named entity recognition method based on an attention mechanism. Background technique [0002] Named entity recognition (NER) is one of the basic tasks in the field of natural language processing, which is to identify instances that embody concepts in text, that is, entities, such as names of people, places, and institutions. Named entity recognition has been widely used in tasks such as information extraction, question answering systems, intelligent translation, and knowledge graph construction. [0003] The methods for naming recognition are mainly divided into the following three categories: [0004] The first type of method is based on dictionaries and rules. This method first constructs dictionaries or rule templates artificially, and performs named entity recognition through matching. [0005] The second category is a method based...

AI Technical Summary

Problems solved by technology

[0008] 1. The first stage is to adopt a method based on dictionaries and rules. The disadvantage of this method is that it relies on the manual construction of rule templates by linguists, which is not only labor-intensive, but also has subjective factors and is prone to errors. Different fields not portable

[0009] 2. The method based on statistical machine learning still requires a lot of manual participation in feature extraction and relies heavily on corpus

[0010] 3. The disadvantage of BiLSTM-CRF, the mainstream model based on deep learning methods, is that the word vector of the word embedding layer cannot represent polysemy, which affects the recognition effect of the lower layer. In addition, when BiLSTM and BiGRU extract global context features , will ignore some local features

Images