Small sample named entity recognition method based on data enhancement and active learning

Abstract

The invention discloses a small sample named entity recognition method based on data enhancement and active learning, and aims to improve the F1 value of early recognition unlabeled data of an active learning method, and the method is effective for various query strategies. According to the technical scheme, the method comprises the steps of firstly, constructing an active named entity recognition system combined with data enhancement; and preparing a data set required for training the named entity recognition model. The active learning module, the data annotation module and the data enhancement module train a named entity recognition model in the named entity recognition module and annotate and enhance data in a multi-round circulation mode. And the trained named entity recognition module performs named entity recognition on the text in the test data pool T to obtain a predicted tag sequence. According to the method, the effect of quickly improving named entity recognition when the annotation data participating in training in the early stage is few is achieved, and compared with an original named entity recognition method under active learning, F1 values of various query strategies are all improved.

Description

technical field [0001] The invention relates to the field of named entity recognition, in particular to a small-sample named entity recognition method based on data enhancement and active learning. Background technique [0002] Natural language refers to languages ​​such as Chinese, English, Spanish, French, German, etc. As other languages ​​​​used by people in daily life, they play an important role in human communication. Natural language is a language that naturally arises with the development of human society, rather than a language deliberately created by humans. Natural language processing is to use the computing power of computers to process information such as shape, sound, and meaning of human natural language, that is, to input, output, identify, analyze, Understand, generate operations, and process this information. Realizing information exchange between man-machine or machine-to-machine is an important issue of common concern in the world of artificial intellig...

AI Technical Summary

Problems solved by technology

However, the query strategy is not universal. The query strategy needs to be determined in advance and cannot be adjusted during the training process.

If the F1 value of the named entity recognition model trained by the query strategy on a data set improves, it may not improve or even decline after changing the data set.

At the same time, active learning relies on multiple rounds of training, so the early (that is, the early stages of multiple rounds) training is usually less labeled data, resulting in the name entity recognition model being limited by the labeled data, and the F1 value is slowly improved.

Images