Text classification method based on deep multi-task learning

Abstract

The invention provides a text classification method based on deep multi-task leaning. The method comprises the steps that by means of a recurrent neural network obtained through other task training and by combining the learning ability of a convolutional neural network, additional document representation is obtained, that is to say, a large amount of external information is introduced, semantic representation of a document is extended, and the problem that training data is insufficient is effectively solved. Accordingly, compared with a traditional multi-task leaning method, the convolutional neural network is used for conducting feature extraction on bottom-layer features of an auxiliary task, the features of other tasks can be utilized for being effectively transferred to the current task, and the performance of text classification is improved.

Description

technical field [0001] The present invention relates to the technical field of natural language processing, and more particularly, to a text classification method based on deep multi-task learning. Background technique [0002] With the development of the Internet, there are more and more demands for tasks such as topic identification, spam identification, and sentiment analysis, which are all based on text classification. The goal of text classification is to give some documents and their corresponding class labels as a training set, and learn a classifier through an algorithm that can predict the class labels of documents without labels in the test set. [0003] There are many text classification algorithms based on deep neural networks, including recurrent neural networks, convolutional neural networks, recurrent convolutional neural networks, and the combination of these networks with attention mechanisms, memory modules, etc. These neural networks have achieved good re...

AI Technical Summary

Problems solved by technology

The neural network parameters are large in scale and the training data is small, so the problem is that it is easy to overfit, and the generalization ability on the test set is reduced.

There are many methods to improve the overfitting problem, such as parameter regularization, batch normalization, etc., but it does not essentially solve the problem of insufficient training data.

Images