Named entity recognition method based on deep learning

Abstract

The invention discloses a named entity recognition method based on deep learning. The named entity recognition method comprises the following steps: 1) constructing a one-to-one synonym dictionary; 2) selecting a data set, and defining an entity class set; 3) performing comparative learning pre-training on a BERT model; 4) performing fine tuning training on the named entity recognition task to obtain an encoder and a classifier; 5) collectively referring the rest sentences to be processed in the test set as sentences S1; 6) inputting the sentence S1 into the encoder to obtain a word embedding vector set and a sentence vector u; 7) selecting a text segment in the sentence S1 based on the span, and constructing a word embedding vector of the text segment; (8) replacing the text segments selected in the sentence S1 in the step (7) with a synonym dictionary to obtain a sentence S2; 9) processing the sentence S2 by using the encoder to obtain a sentence vector v; and 10) calculating span_em, and performing classification through the classifier to obtain an entity set C. The method is high in recognition accuracy.

Description

technical field [0001] The invention belongs to the technical field of computer natural language processing, and relates to a named entity recognition method based on deep learning. Background technique [0002] The purpose of named entity recognition is to identify entities with a certain meaning, category or part of speech in the text, such as names, countries, emotional words, subject proper nouns, phone numbers, etc. With the rise of big data technology, named entity recognition has been more and more widely used in knowledge graphs, data analysis, intelligent data processing and other fields. [0003] Early named entity recognition methods were generally based on rules. After the gradual progress of deep learning, many named entity recognition methods based on deep learning have emerged in recent years. However, the current named entity recognition method based on deep learning is difficult to recognize fuzzy entities such as nested entities, and the accuracy rate need...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a named entity recognition method based on deep learning, which solves the problem in the prior art that it is difficult to identify nested entities by using the entity relationship extraction method, resulting in inaccurate entity classification

Images