A pre-processing method for feature verification of text sentiment analysis

Abstract

The invention discloses a preprocessing method of text sentiment analysis characteristic verification. The preprocessing method comprises the following steps: preprocessing an original training set to obtain preprocessing information, and then, carrying out characteristic verification and characteristic selection on preprocessing information, wherein the step of preprocessing the original training set to obtain the preprocessing information comprises the following specific steps: determining the summary of the original training set, determining the summary of an original characteristic vector set, and expanding original data so as to construct integrated processing information. The preprocessing method has the active effects that analysis information is generated from two aspects of training set and characteristic vector, guarantees the rich degree of the information of an evaluation conclusion, and is simultaneously favorable for improving the accuracy of the whole sentiment analysis flow. The preprocessing method exhibits good universality and expandability, and can perform a good effect on various different modeling and implementation sentiment analysis algorithms.

Description

technical field [0001] The invention belongs to the field of text sentiment analysis, and in particular relates to a pre-processing method for feature verification of text sentiment analysis. Background technique [0002] Existing text classification feature selection and verification schemes have achieved good results in content domain classification, but they have the following problems when applied to the field of sentiment analysis: [0003] 1. For general applicable scenarios, the business field of sentiment analysis is not deep enough, especially the emotional information on the Internet has an obvious unbalanced corpus distribution, and it is easy to misclassify when artificially constructing a sentiment analysis corpus. These two issues are generally insufficiently considered; [0004] 2. The general verification and comparison basis for different feature extraction algorithms lacks the best practice design for practical applications. For example, in the commonly u...

AI Technical Summary

Problems solved by technology

[0003] 1. For general applicable scenarios, the business field of sentiment analysis is not deep enough, especially the emotional information on the Internet has an obvious unbalanced corpus distribution, and it is easy to misclassify when artificially constructing a sentiment analysis corpus. These two issues are generally insufficiently considered;

[0004] 2. The general verification and comparison basis for different feature extraction algorithms lacks the best practice design for practical applications

[0006] 1. The accuracy rate of most existing Chinese sentiment analysis algorithms is low, and there is a lack of feature verification or feature selection schemes that can guide algorithm improvement. For example, according to the results of the 5th Chinese Tendency Analysis and Evaluation Symposium COAE2013, the accuracy rate is generally 60% %about;

[0007] 2. Text information is represented by feature vectors. However, due to the lack of recognized best practices in the field, the modeling and algorithms of sentiment analysis are also varied. Therefore, the recognition and verification scheme for feature vectors of sentiment analysis also needs to consider the respective characteristics of these algorithms and models , such as commonly used bag of words, n-gram, word2vec and other models

Images