Classification method of short text

Abstract

The invention relates to a classification method of short text. The classification method is characterized in that a hyperplane cuts two classes of samples, then the geometric distance between each multiclass sample and the hyperplane is calculated, multiple subdomains are divided according to the geometric distance, each subdomain is endowed with unique weight, the weight of the subdomains decreases gradually along with the increasing of the distance from the subdomains to the hyperplane, sub-sampling is performed on data according to the weight in a sub-sampling stage, and obtained sampled samples are imported into an SVM algorithm to perform classification. By the classification method, the problems of high-dimension sparsity and class imbalance in text classification can be solved effectively.

Description

Technical field [0001] The invention relates to a short text classification method, which belongs to the field of machine learning and data mining. Background technique [0002] In recent years, big data and artificial intelligence technologies have developed rapidly, and speech and image recognition, natural language processing, and knowledge graphs have become hot research areas. Text Categorization is the most typical problem in the field of machine learning and data mining. It has many classification algorithms, such as naive Bayes algorithm K nearest neighbor algorithm (K-NN), neural network algorithm and support vector machine (Support Vector Machine, SVM), etc. SVM is a representative classification method with strong generalization ability based on statistical learning theory. It aims at minimizing structural risk and overcomes the problem of dimensionality disaster through the introduction of kernel functions, and has become a classic candidate for text classification p...

AI Technical Summary

Problems solved by technology

However, in real application scenarios, there is an obvious problem of unbalanced class distribution in text datasets, especially the natural high-dimensional sparse nature of short texts, which poses challenges to existing classification algorithms

Images