Entity identification method and device

Abstract

The embodiment of the invention discloses a named entity recognition method, which comprises the following steps: when an entity in a to-be-recognized text needs to be recognized, obtaining a word vector of a word segmentation entry in the to-be-recognized text; and according to the word vector of the word segmentation entry and the entity recognition model, determining a first score of each typeof label corresponding to the word segmentation entry; Respectively calculating a first matching score between the feature vector of the word segmentation entry and the label vector of each type of label, wherein the first matching score reflects the possibility that the word segmentation entry has each type of label; according to the first score and the first matching score, obtaining a second score of each type of label corresponding to the word segmentation entry, and adding the score of the label of the word segmentation entry in the entity dictionary on the basis of the first score. By combining the first matching score on the output layer of the entity recognition model and adding the score of the label of the word segmentation entry on the basis of the first score, the influence ofthe entity dictionary on calculation of the score of each type of label is enhanced, so that the calculated score is more accurate, and more entities can be recalled.

Description

technical field [0001] The present application relates to the field of text processing, and in particular to an entity recognition method and device. Background technique [0002] Named Entity Recognition (NER) refers to the recognition of entities with specific meaning in text. NER is the basis of various natural language processing technologies such as information extraction, information retrieval, machine translation, and question answering systems. Whether or not entities in text can be accurately identified has a great impact on the processing effect of natural language processing technologies. [0003] Due to the large number of entities and possible constant updates, the entities included in the text to be recognized may be Out of Vocabulary (OOV for short) entities in the training corpus, and it is difficult for the training corpus to cover all entities. To do this, entities need to be identified in combination with entity dictionaries. At present, when identifying...

AI Technical Summary

Problems solved by technology

[0004] Since this method combines the entity dictionary before the word vector is input to the recognition network model, the features related to the entity dictionary are in the input layer of the recognition network model, so that the impact of the entity dictionary on the score output of the output layer is very small, and it is difficult to use the entity dictionary. For the calculation of the score, the calculated score is not accurate enough, which affects the recall of entities by entity recognition

Images