Thai and Burmese part-of-speech tagging method for fusing word-syllable pairs by using a local multi-head attention mechanism

Abstract

The invention relates to a Thai and Burmese part-of-speech tagging method for fusing word-syllable pairs by using a local multi-head attention mechanism, and belongs to the field of natural language processing. The method comprises the following steps: preprocessing a Thai or Burmese text data set; selecting word-syllable pair features as model input in a windowing mode; learning context features from the word-syllable pair sequence by using a local multi-head attention mechanism; and finally, modeling a part-of-speech dependency relationship through a conditional random field, and predicting a part-of-speech tag. Experimental results for part-of-speech tagging data sets of Thai and Burmese show that compared with a current optimal model, the method integrates syllables as morphological features of words, facilitates learning of context features of unknown words, and relieves the influence of wrong tagging of the unknown words on the performance of the model. Moreover, a local multi-head self-attention mechanism is adopted, so that the model can obtain richer local dependence features, and a better tagging result is obtained in a part-of-speech tagging task.

Description

technical field [0001] The invention relates to a part-of-speech tagging method for Thai and Burmese texts using a local multi-head attention mechanism to fuse word-syllable pairs, and belongs to the technical field of natural language processing. Background technique [0002] Part-of-speech tagging is to judge the part of speech of each word in a given sentence, which belongs to one of the basic tasks in the field of natural language processing (NLP). Part-of-speech tagging can improve the accuracy of syntactic analysis , thus facilitating the improvement of many NLP tasks. [0003] Early part-of-speech tagging methods mainly include rule-based and statistical machine learning. The rule-based part-of-speech tagging method has the problems of incomplete rule customization and rule conflicts. At present, methods based on statistical machine learning mainly include Support Vector Machine SVM, Hidden Markov Model HMM and Conditional Random Fields CRFs. This kind of method us...

AI Technical Summary

Problems solved by technology

[0006] The present invention provides a part-of-speech tagging method for Thai and Burmese texts using a local multi-head attention mechanism to fuse word-syllable pairs, so as to tag Thai and Burmese and other Southeast Asian phoneme-syllable languages, and solve the problem of low-frequency speech and unknown words The problem of poor part-of-speech tagging effect

Images