Text classification method based on attention mechanism

Abstract

The invention provides a text classification method based on an attention mechanism. According to the method, a neural network model based on DenseNet is used; before a neural network is trained, semantic coding is utilized to initialize the weight of a convolutional filter of DenseNet, so that the network identifies important semantic information before training, and effective position information of each sentence can be captured in a convolutional layer; and feature extraction is performed on the text information through a local attention intensive connection module. Compared with the prior art, the text classification method has higher feature extraction capacity, the content of the text information is reserved, the effect is remarkable when multiple text classification tasks including sentiment classification, theme classification and the like are included, and the classification accuracy is effectively improved.

Description

technical field [0001] The invention belongs to the field of text classification in natural language processing, and proposes a text classification method based on an attention mechanism. Background technique [0002] With the rapid growth of information data, natural language processing has developed rapidly in the wave of artificial intelligence technology. Text classification is the basic task of natural language processing, which is used in various fields of natural language processing, such as information retrieval, information filtering and semantic analysis. However, in the face of the massive short text information emerging in the era of big data, how to divide new fields The category to which it belongs, its training samples are often insufficient. Deep learning (Xu Yilong, Li Wenfa, Zhou Chunjie. A review of natural language processing based on deep learning. Network Application Branch of China Computer Users Association, 2018.] can not only realize the automation...

AI Technical Summary

Problems solved by technology

For the complexity of text data, using DenseNet is only a simple feature extraction of text information by increasing the number of network layers, which will not only lead to redundant network parameters, but even network degradation.

Images