Improved Transformer + CRF (Content Recognition Function)-based method for identifying old rattle named entity

Abstract

The invention discloses an improved Transformer + CRF-based method for recognizing a named entity of Lambda language, and belongs to the field of recognition of small languages in natural language processing. According to the method, an improved Transformer model is adopted to encode the Lao language, and then a CRF (Conditional Random Field) model is adopted to decode the Lao language to obtain an optimal annotation sequence. The method comprises the following steps: firstly, taking Transformer as a character encoder to carry out character encoding on a single character of the Lambda language; splicing the character-level vector obtained by encoding with a pre-trained word vector and carrying out position encoding; wherein the input signal is used as the input of a Transformer encoding part; through training of a multi-head attention layer and a feedforward neural network layer in sequence, finally, an obtained vector set is input into a CRF model fused with old-age naming and geographical name institution name linguistic features for named entity recognition training, and a named entity recognition model fused with old-age language rules is obtained. Compared with a mainstream BLSTM + CRF named entity recognition method, the recognition effect of the method is improved.

Description

technical field [0001] The invention relates to a Lao language named entity recognition method based on an improved Transformer+CRF, which belongs to the field of small and medium language recognition in natural language processing. Background technique [0002] Named entity recognition is a fundamental task in natural language processing. Currently, commonly used methods for named entities include rule-based and dictionary-based methods, statistical-based methods, hybrid model-based methods, and deep learning-based methods. Due to the lack of labeled corpus in Lao language, deep models such as RNN, CNN, BiLSTM, and BiGRU are currently mainly used in the research of Lao language named entity recognition. The neural network model trains itself and extracts features. With the introduction of the Transformer model based on the attention mechanism with higher parallel efficiency, people have also begun to use the Transformer+CRF model to complete the task of natural language p...

AI Technical Summary

Problems solved by technology

However, the characteristics of Lao language make it difficult to summarize the language rules of Lao language, and the research corpus of Lao language is extremely scarce. At present, there are few studies on Lao language in the field of named entity recognition. Therefore, the recognition accuracy of Lao language is still very high. room for improvement

Images