A public opinion tendency identification method for training sample category distribution imbalance

Abstract

The invention discloses a public opinion tendency identification method for training sample category distribution imbalance. The method comprises the steps of firstly collecting the vocabularies related to the concerned public opinion field as public opinion hot words to create a lexicon; crawling a comment data set from a public opinion information source and divided into a training set and a test set; then classifying the public opinion tendency of the training set manually, and for the problem of class imbalance, adopting a bootstrap learning method for supplementing processing; extractingfeatures of each type of training samples, training an algorithm model by adopting naive Bayes, a support vector machine, a decision tree and other algorithms, classifying test set data by using the trained model, and identifying public opinion tendency according to a classification result. The methods of bootstrap learning, feature vector construction and classification model training all adopt atime-sensitive weighting method for weighting, so that the public opinion tendency reflected by the methods is more timely. The public opinion tendency identification method solves the problem of inaccurate classification caused by imbalance of training data, and improves the accuracy of public opinion tendency identification and the timeliness of public opinion analysis.

Description

technical field [0001] The invention belongs to the technical field of natural language processing and machine learning, and relates to a method for analyzing the tendency of public opinion by using a machine learning algorithm, in particular to a method for identifying the tendency of public opinion aimed at the unbalanced distribution of training sample categories. Background technique [0002] The current Internet penetration rate is growing rapidly, the number of updated news on the Internet is very large, and the resulting impact on public opinion is also very large. It is under this situation that the analysis technology of public opinion tendency was born, aiming at analyzing the public opinion generated on the Internet. Reviewers' tendentious attitudes and attitude changes are screened in a timely manner, thereby helping regulatory authorities to detect changes in public opinion in a timely manner and build a civilized and harmonious public opinion environment. [00...

AI Technical Summary

Problems solved by technology

[0003] When using general-purpose machine learning algorithms to analyze public opinion tendencies, problems such as class imbalance of training data, timeliness of text release, and timeliness of public opinion often lead to a large deviation between the effect of tendency recognition and the actual tendency

Currently, no effective solution has been proposed

Images