Text classification method and system fusing self-attention mechanism and deep learning

Abstract

The invention belongs to the technical field of text classification, and particularly relates to a text classification method and system fusing a self-attention mechanism and deep learning, and the method comprises the steps: obtaining a to-be-classified text data set, and carrying out the preprocessing; and carrying out classification processing on the preprocessed to-be-classified text data set by utilizing a trained deep learning model, wherein the deep learning model comprises an ERNIE pre-training module used for extracting sentence-level word vector representation in a to-be-classified text data set, a BiLSTM module used for extracting context information of each word according to the sentence-level word vector representation, a DPCNN module which is used for deeply extracting the context information of each word according to the sentence-level word vector representation and the context information of each word, an attention mechanism module which is used for extracting a text depth information distribution weight according to the sentence-level side vector representation and the context information of each word, and a Softmax module which is used for carrying out classified output according to the text depth information distribution weight. The text classification recognition effect is improved by combining an attention mechanism and each model.

Description

technical field [0001] The invention belongs to the technical field of text classification, in particular to a text classification method and system integrating self-attention mechanism and deep learning. Background technique [0002] With the continuous development of artificial intelligence, more and more researchers pay attention to natural language processing. More and more researchers apply machine learning and deep learning methods to text classification, which has gradually become a popular trend in text classification. The application of text classification in machine learning is mainly reflected in the following four aspects: (1) Logistic regression algorithm, which is used in most cases to judge the probability of its category; (2) Naive Bayesian algorithm, which uses advanced mathematics (3) random forest algorithm, which selects some decision trees in a random way, and then builds them into the forest for classification, thereby improving the accuracy of classif...

AI Technical Summary

Problems solved by technology

[0003] To this end, the present invention provides a text classification method and system that integrates self-attention mechanism and deep learning, which overcomes the shortcomings of obtaining semantic associations between contexts and the inability of text information to penetrate well in the existing technology Into the deep learning neural network and then affect the text classification recognition effect

Images