Multi-label microblog text classification method based on semi-supervised learning

Abstract

The invention discloses a multi-label microblog text classification method based on semi-supervised learning, and relates to the field of natural language processing. The method comprises the following steps: firstly, preprocessing an original microblog text, and labeling a small amount of texts; generating augmented data of the annotation data set by using reverse translation, generating augmented data of the unannotated data set by using synonym replacement and random noise injection, guessing and generating a pseudo tag of the unannotated data by using a classifier, and forming a new training set together with the augmented annotation data set; converting the multi-label classification task into a plurality of dichotomy tasks, training a semi-supervised microblog text classification model, randomly extracting two samples from a new training set each time during training, generating a new sample in a text hidden space by using a sample mixing technology, calculating a loss value, and updating network parameters; and finally, classifying the microblog texts by comprehensively using a plurality of trained classifiers. The invention has an important application value for fine-grained information extraction of the microblog texts.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular to a method for multi-label classification of microblog texts through a semi-supervised model. Background technique [0002] Social media generates massive amounts of data all the time, and there is a lot of information such as disaster information and emotional information that can be mined. However, social media data has the characteristics of low information fragmentation density and many irregular terms. With a huge amount of data and a variety of discussion topics, if you only rely on manual information extraction for text classification, the cost of data analysis will be too high. If you use dictionaries or rules to quickly filter text, you will also face various network languages ​​such as text ambiguity. Problems of sexuality and colloquialism. Compared with using manual or vocabulary and rules to filter the required information from massive text data, deep learni...

AI Technical Summary

Problems solved by technology

However, deep learning often requires high-quality labeling of a large amount of data before the neural network can be fully trained, and data labeling requires expensive costs.

Images