Text classification system and method based on deep learning

Abstract

The invention discloses a text classification system and method based on deep learning. The system comprises a text source acquisition module, a text source input module, a standardization processing module, a text preprocessing module, a training text module, a test text module, a feature extraction dimension reduction unit, a feature weight calculation unit, a training text feature weight calculation unit, a training text feature vector, a test text feature weight calculation unit, a test text feature vector and a classification module. The system and method are characterized in that the text preprocessing module comprises a new word adding unit, a word segmentation unit, an useless word clearing unit and a full-text index establishing unit; the classification module comprises a classification judgment unit and a classifier; the dimensionality reduction effect on high-dimensional text data is achieved through the system and the method based on deep learning; a text is subjected to word segmentation, and useless words are cleared, so that more accurate features can be extracted, and the classification accuracy is improved.

Description

technical field [0001] The present invention relates to the technical field of text classification, in particular to a text classification system and method based on deep learning. Background technique [0002] With the rapid development of network technology, massive information resources exist in the form of text. People urgently hope to quickly and effectively find the content they are interested in from the explosive wave of information. As an important research direction of information processing, text classification is a common method to solve text information discovery. Deep learning is an unsupervised feature learning and feature hierarchy learning method. Unsupervised learning methods generally reconstruct the original input data. The feature learning method for feature extraction has been popular in the field of machine learning in recent years. Its essence is to learn more abstract advanced features by using a large amount of training data and constructing a netw...

AI Technical Summary

Problems solved by technology

[0003] In order to be able to apply the neural network algorithm in deep learning to text classification, the first thing to do is to represent the text in a form that can be easily processed by a computer. However, for massive data, the high dimensionality of features brings many problems to text classification. It cannot meet people's needs for obtaining useful knowledge, and the existing processing methods have certain shortcomings. Therefore, we propose a text classification system and method based on deep learning to improve the existing technology

Images