Sentence classification method based on LSTM and combining part-of-speech and multi-attention mechanism

Abstract

The invention discloses a sentence classification method based on LSTM and combining part-of-speech and a multi-attention mechanism. The method comprises steps: each sentence is converted into a semantic word vector matrix and a part-of-speech word vector matrix which are based on continuity and density in an input layer; learning context information of words or part-of-speech in the sentences ina shared bidirectional LSTM layer respectively, connecting learning results of each step in series, and outputting the learning results; a self-attention mechanism and a point multiplication functionare adopted in the self-attention layer to learn important local features at all positions in the sentence from the semantic word vector sequence and the part-of-speech word vector sequence respectively, corresponding semantic attention vectors and part-of-speech attention vectors are obtained, and the semantic attention vectors and the part-of-speech attention vectors are constrained through theKL distance; carrying out weighted summation on the output sequence of the bidirectional LSTM layer in the merging layer by utilizing the obtained semantic attention vector and part-of-speech attention vector to obtain semantic representation and part-of-speech representation of the sentence, and obtaining final sentence semantic representation; And finally, prediction and classified output are carried out through an MLP output layer.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular to a sentence classification method based on LSTM combined with part-of-speech and multi-attention mechanism. Background technique [0002] Sentence classification has always been a research hotspot in the field of Natural Language Processing (NLP). In recent years, with the wide application of deep learning in NLP, many scholars have successively proposed various sentence classification methods based on the Long Short-Term Model (LSTM), and in many sentence classification corpora such as Stanford Twitter Sentiment (STS), Stanford Sentiment Treebank binary classification (SSTb2) and quintuple classification (SSTb5), TREC, IMDB, etc. have all achieved better results than traditional machine learning methods. Compared with the convolutional neural network CNN, LSTM can better describe the context information and long-term dependencies of text sequence data, and effectively ...

AI Technical Summary

Problems solved by technology

However, these studies are mainly focused on the attention at the content

Images