Text classification method, classifier and system based on key information and dynamic routing

Abstract

The invention discloses a text classification method, classifier and system based on key information and dynamic routing, and belongs to the technical field of text classification. The method comprises the steps that a to-be-classified text is preprocessed, and keywords in the to-be-classified text are extracted; inputting the preprocessed to-be-classified texts and the keywords into an embedded layer of a pre-training language model BERT to obtain pre-classification output; inputting the pre-classified output into BERT for depth coding to obtain a re-classified output; in BERT, pooling operation in the CNN is replaced by dynamic routing to build a downstream classifier, and pre-classification output and re-classification output are respectively input into the classifier. According to the method, the output feature of the embedded layer in the BERT and the final output feature of BERT depth coding are input into the downstream classifier for two times of classification prediction, and the final classification result is obtained through weighted addition of the two prediction results, so that the shallow feature expression in the BERT is fully utilized; and the classification accuracy is greatly improved while the internal structure of the BERT is not changed.

Description

technical field [0001] The invention relates to the technical field of text classification, in particular to a text classification method, classifier and system based on key information and dynamic routing. Background technique [0002] With the popularization of smart phones and the development of Internet technology, online news media, social media platforms, online live streaming, self-media platforms and other applications that change people's lifestyles have sprung up, and the amount of text data generated by them has also increased. A blowout growth trend. Facing the increasing mass of text data, how to automatically classify it to help people quickly and accurately obtain the information they need or are interested in is an important research problem. The text classification task has begun to receive extensive attention from researchers and has become a An important task in natural language processing. Text classification is the process of automatically determining ...

AI Technical Summary

Problems solved by technology

The emergence of the pre-trained language model BERT has brought the research of text classification tasks into a new stage. Most of the research work is directly carried out in the downstream of BERT to build the text classification model, which has achieved quite good classification results.

However, the existing methods still have shortcomings: (1) directly input the text to be classified into the model for prediction, without using the key information in the original text for auxiliary classification, resulting in a waste of key information in the text; (2) in the BERT The downstream network simply piles up other neural network models to build the question answering model, only using the high-level features of the text to be classified after BERT's deep encoding, and not using the low-level features inside BERT, resulting in a waste of BERT's internal features; (3) Existing classification models use a large number of pooling operations in convolutional neural networks (CNN), which loses part of the feature information in the model, making the network unable to learn more advanced thinking

Images