Text classification method and text classification device

Abstract

The invention relates to a text classification method and a text classification device. The method comprises the following steps: an NLP (Natural Language Processing) pre-processing step, wherein analysis of a natural-language processing method is carried out on user dialogue text to obtain a word set and semantic labeling results about the user dialogue text; a multi-dimensional-feature selectionstep, wherein combination is carried out for the word set and the semantic labeling results according to a plurality of rules to obtain a vectorized characterization form of semantic information contained by the user dialogue text; and a classification step, wherein probability estimation values are calculated for user dialogue classes obtained by the multi-dimensional-feature selection step. According to the text classification method and the text classification system of the invention, the advantages of counting and a deep-learning method can be integrated, and a customer demand-oriented text classification solution can be realized through multi-dimensional-feature selection.

Description

technical field [0001] The invention relates to text processing technology, in particular to a text classification method and a text classification device. Background technique [0002] At present, the implementation schemes of text classification technology are mainly divided into statistical learning methods and deep learning methods. The former is mainly based on the feature selection method. The word and sentence-level features of the text are selected through indicators such as TF-IDF, PMI, and chi-square value, and the feature vector representing the text is obtained, and the feature vector is obtained by machine learning. The probability of each label is used as the final classification standard; the latter is mainly based on model construction, using the discrete information of the text as input, through the serial and parallel structure of the multi-layer neural network, supplemented by the back propagation algorithm to update the network weight , directly get the ...

AI Technical Summary

Problems solved by technology

[0003] However, both statistical learning methods and deep learning methods have their flaws

The former's over-reliance on feature selection leads to limited improvement space for subsequent algorithms, and discretized features often lead to problems such as data sparseness and semantic gaps; the latter has a black-box structure, and end-to-end learning methods are difficult to generalize and are difficult to Difficult to achieve domain transfer, extremely dependent on the size of training data

Images