Multi-label text classification method based on statistics and pre-trained language model

Abstract

The invention discloses a multi-label text classification method based on statistics and a pre-training language model. The multi-label text classification method comprises the following steps: S1, preprocessing training corpora needing to be classified; S2, establishing a label acquisition model based on a statistical method and a language model; S3, processing the obtained label data; S4, establishing a multi-label classification model based on a pre-training language model, and performing model training by utilizing the obtained label data; S5, performing multi-label classification on the text data to be classified by using the trained multi-label text classification model. According to the method, a statistical method and a pre-trained language model label obtaining method are combined, the ALBERT language model is used for obtaining the semantic coding information of the text, a data set does not need to be manually labeled, and the label obtaining accuracy can be improved.

Description

technical field [0001] The invention relates to a multi-label text classification method based on statistics and a pre-trained language model. Background technique [0002] Since 2013, the deep learning theory based on neural network has made great progress, and has been widely used in the fields of image and natural language processing, and has derived many research and application directions. Text classification is one of the most important tasks in natural language processing, and it has many applications in real life, such as public opinion monitoring, tag recommendation, information search, etc. Traditional single-label text classification algorithms are difficult to solve the problem of text diversity in real-life scenarios. Multi-label text classification has become a popular research direction in natural language processing text classification tasks. [0003] The current multi-label text classification methods are mainly divided into two categories: [0004] The fi...

AI Technical Summary

Problems solved by technology

[0007] (1) The multi-label text classification method based on traditional machine learning requires manual design of features, which is very time-consuming and labor-intensive, and the quality of features has a great impact on the classification effect

[0008] (2) Most of the existing deep learning-based methods use CNN, RNN, etc. to extract semantic information. Although good results can be achieved, there is still a certain gap compared to using pre-trained language models to extract semantic information.

[0009] (3) Both of the above two methods require a large-scale labeled data set, especially the multi-label text classification method based on deep learning, which puts forward higher requirements for the label accuracy and scale of the training data set, while For many application domains, the acquisition cost of large-scale high-accuracy labeled datasets is often very high

Images