A Small Sample Named Entity Recognition Method Based on Data Augmentation and Active Learning

Abstract

The invention discloses a small-sample named entity recognition method based on data enhancement and active learning. The purpose is to improve the F1 value of the early recognition of unlabeled data by the active learning method, and is effective for various query strategies. The technical solution is to first construct an active named entity recognition system combined with data enhancement; prepare the data set required for training the named entity recognition model. The active learning module, the data labeling module, and the data enhancement module train the named entity recognition model in the named entity recognition module in a multi-round cycle and label and enhance the data. The trained named entity recognition module performs named entity recognition on the text in the test data pool T, and obtains the predicted label sequence. The present invention realizes the effect of rapidly improving named entity recognition when there are few labeled data participating in training in the early stage, so that compared with the original named entity recognition method under active learning, F1 values ​​of various query strategies are all improved.

Description

technical field [0001] The invention relates to the field of named entity recognition, in particular to a small sample named entity recognition method based on data enhancement and active learning. Background technique [0002] Natural language refers to Chinese, English, Spanish, French, German and other languages. As other languages ​​that people use every day, they play an important role in human communication. Natural language is a language naturally produced with the development of human society, rather than a language specially created by human beings. Natural language processing is to use the computing power of computers to process the shape, sound, meaning and other information of human natural language, that is, to input, output, identify, analyze, Understand, generate actions, and process this information. The realization of information exchange between humans and machines or between machines and machines is an important issue of common concern in the global arti...

AI Technical Summary

Problems solved by technology

However, the query strategy is not universal. The query strategy needs to be determined in advance and cannot be adjusted during the training process.

If the F1 value of the named entity recognition model trained by the query strategy on a data set improves, it may not improve or even decline after changing the data set.

At the same time, active learning relies on multiple rounds of training, so the early (that is, the early stages of multiple rounds) training is usually less labeled data, resulting in the name entity recognition model being limited by the labeled data, and the F1 value is slowly improved.

Images