Twitter sentiment classification text processing optimization system based on word embedding

Abstract

The invention belongs to the field of text processing, and particularly relates to a Twitter sentiment classification text processing optimization system based on word embedding, which comprises the following modules: a text processing module, a word embedding module, a Twitter embedding module and a model training and verification module. The text processing module uses a traditional text processing mode to eliminate the influence of stop words on tweet classification; the word embedding module is used for carrying out word embedding on a specific Twitter context by using a skip gram model; the Twitter embedding module is used for averaging m n-dimensional word vectors to obtain an n-dimensional tweet vector, and comparing the performances of three aggregation methods, namely the sum of the n-dimensional tweet vector, the weighted average of the n-dimensional tweet vector and k most important word vectors selected and added according to the importance of words; and the model training and verification module carries out model training and verification by using a 10*2 re-nested cross validation mode.

Description

technical field [0001] The invention belongs to the field of text processing, and in particular relates to a word embedding-based Twitter sentiment classification text processing optimization system. Background technique [0002] Current text processing steps are often performed using off-the-shelf routines and pre-built word dictionaries without optimization for the domain, application, and context. Some keywords containing Twitter colloquial words, emoji, and hashtags, which are usually removed because they are not available in traditional literary corpora, play a large role in Twitter sentiment classification. [0003] Reasons for problems or defects: At present, in the text processing stage of Twitter text sentiment analysis, because of the traditional method of using a pre-built word dictionary, some colloquial key information is filtered. Contents of the invention [0004] Aiming at the above-mentioned problem that some colloquial key information is filtered, the pr...

AI Technical Summary

Problems solved by technology

[0004] Aiming at the above-mentioned problem that some colloquial key information is filtered, the present invention provides a system capable of alleviating the negative impact of traditional methods on emotions

Images