Text classification method, text classifier and storage medium for unbalanced data set

Abstract

The invention discloses a text classification method, a text classifier and storage medium for an unbalanced data set, wherein the method comprises the following steps of: acquiring the data set usedfor training a classification model; determining whether each text data is a plurality of samples or a few samples according to the category information marked by the text data; calculating the ratiobetween the number of the plurality of samples and the number of the few samples to obtain an unbalance ratio; carrying out pre-processing on the text data to obtain a corresponding sample point to map into a vector space; updating the data set after the interpolation sample is obtained based on the preset interpolation strategy, the unbalance rate and each sample point; training the classification model using the updated data set as a training sample set; acquiring the text data to be tested, and introducing the text data to be tested into the classification model after finishing the trainingto classify so as to obtain the category of the text data to be tested as a classification result. According to the invention, the few samples and a boundary region thereof can be enlarged, and the classification effect of the model can be effectively improved.

Description

technical field [0001] The invention relates to the technical field of text information, in particular to a text classification method for an unbalanced data set, a text classifier and a storage medium. Background technique [0002] With the advancement of science and technology and the popularization and application of the Internet, in the face of massive text data, such as comments published or fed back by users, the automatic text classification technology has emerged as the times require. [0003] At present, machine learning methods are gradually being applied to text classification technology. Traditional machine learning methods are implemented based on data balance. However, in practical applications, text classification methods are less effective due to the uneven distribution of most data. . There are two ways to deal with the problem of data imbalance, one is the algorithm level, and the other is the sampling level, so as to achieve class balance and improve the ...

AI Technical Summary

Problems solved by technology

[0004] The main purpose of the present invention is to provide a text classification method, text classifier and storage medium for unbalanced data sets, aiming to solve the limitations of traditional classification methods in the face of unbalanced data sets, thereby improving the accuracy and efficiency of text classification technical issues

Images