Short text labeling method, system and device for large-scale classification system

Abstract

The invention belongs to the field of text classification, particularly relates to a short text labeling method, system and device for a large-scale classification system, and aims to solve the problem that the short text labeling system for the large-scale classification system is low in stability under the condition of limited data. The method comprises the steps that a first short text information set to be classified is acquired, and preprocessing is carried out based on a forward maximum matching segmented word and a word2vec word vector representation technology to obtain a second shorttext information set; based on a rule-based classification method and a supervised neural network classification method, perform binary classification on a second short text information set, then perform short text filtering, perform first-level and second-level classification labels of each short text based on the same classification method, and perform third-level and fourth-level classificationlabels of each short text based on a label propagation method of semi-supervised learning. According to the method, the stability of the short text label system oriented to the large-scale classification system is ensured under the condition of limited data.

Description

technical field [0001] The invention belongs to the field of text classification, and in particular relates to a short text labeling method, system and device for a large-scale classification system. Background technique [0002] With the widespread use of new Internet platforms such as Internet official media and WeChat public accounts We Media, Weibo, Tieba, etc., it is of great significance to automate the labeling system for short texts published on these platforms. With the rapid expansion of text information, how to effectively organize and manage this information, and quickly, accurately and comprehensively find the information users need is a major challenge in the field of information science and technology. As a key technology to process and organize a large amount of text data, the automatic labeling of short text can solve the problem of information clutter to a large extent, conveniently and accurately locate the required information and divert information. As ...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the above-mentioned problems in the prior art, that is, in order to solve the problem of low stability of the short text label system for large-scale classification systems under the condition of limited data, the first aspect of the present invention proposes a large-scale classification-oriented A systematic short text automatic labeling method, the method comprises the following steps:

Images