Document classification method based on hierarchical multi-attention network

Abstract

The invention discloses a document classification method based on a hierarchical multi-attention network. The method comprises the following steps of utilizing a Bi-GRU sequence model for carrying outword-sentence and sentence-to-document modeling on the document; using Bi-GRU sequence model to encode each word, obtaining the context information in the sentence, and using the Soft attention to carry out the weight distribution on each word; for the process from the sentences to the document, introducing the CNN attention, and obtaining the local relevant characteristics between the sentencesin the window by utilizing a CNN model, so that the attention weight of each sentence is further obtained. Modeling can be carried out from words to sentences and from sentences to documents accordingto document characteristics, and the hierarchical structure of the documents is fully considered. Meanwhile, aiming at the word level and the sentence level, different attention mechanisms are respectively adopted to properly distribute the weights of the related contents, so that the document classification accuracy is improved.

Description

technical field [0001] The invention belongs to the field of natural language processing technology and sentiment analysis, in particular to a document classification method based on a hierarchical multi-attention network. Background technique [0002] Text classification is one of the important topics in the field of natural language processing. With the continuous improvement of the amount of data and the computing power of hardware, the theory and method of text classification are playing an increasingly important role and have received widespread attention. Early text classification research was mainly based on the method of knowledge engineering system, requiring experts in a certain field to customize classification rules for texts in this field, but this method requires a lot of manpower to expand or modify the rules and do a lot of maintenance work. Later, with the development of machine learning technology, text classification methods based on machine learning grad...

AI Technical Summary

Problems solved by technology

We mentioned many classic models above, which have achieved remarkable results in many text classification problems. However, for document classification, these models still have the following problems: 1. Ignore the document from word to sentence , the hierarchical structure information from sentence to document, and directly use each word vector as the input of the deep network; 2. Use a single attention mechanism to determine the contribution weight of each part in the document, and do not fully consider the transition from word to sentence, sentence to document The characteristics of the two parts make it impossible to effectively use the internal structure information of the document

Images