Key new word finding method based on multidimensional word and sentence characteristics and sentiment analysis

Abstract

The invention provides a key new word finding method based on multidimensional word and sentence characteristics and sentiment analysis. The key new word finding method comprises the following steps:step 1, grasping needed comments from an e-commerce platform; step 2, pre-processing the comments; step 3, carrying out word segmentation on the comments by adopting an NLPIR word segmentation tool; step 4, constructing and filtering repeated strings of a text subjected to the word segmentation by utilizing the multidimensional word and sentence characteristics; step 5, manually screening accuratenew words of a training sample; step 6, counting digital combinations and new class combinations of the new words; step 7, adjusting a filtering threshold value in step 4 according to a counting result and adding the new class combination and filtering to obtain a repeated string set of the tested sample; step 8, adding a user dictionary for optimizing the word segmentation; step 9, carrying outdependence grammar analysis on a word segmentation result by utilizing LTP (Long Term Potentiation); step 10, carrying out sentiment marking on governing words in a dependency relation by utilizing aCRF++ tool; step 11, taking the repeated strings which are sentiment words or the repeated strings with the governing words which are the sentiment words as the key new words. The key new word findingmethod provided by the invention is used for carrying out new word mining on a lot of comment texts of certain e-commerce products to obtain the new words which play a key role in the comments, so that the word segmentation accuracy is improved and a good foundation is provided for the mining and analysis work of the texts.

Description

technical field [0001] The invention relates to the field of electronic commerce, and specifically designs a method for discovering new key words based on multi-dimensional sentence features and sentiment analysis. Background technique [0002] Word segmentation technology is a very important research content in the field of Chinese natural language processing. New word discovery is an important part of optimizing word segmentation. Because in Chinese information processing, unlike English and other western languages, there are fixed separators between words, so word segmentation is usually a necessary step at the beginning of Chinese information processing tasks. Words that are not included in the word segmentation tool dictionary encountered in the word segmentation task (that is, unregistered words, the new words referred to in this article are unregistered words) will significantly affect the performance of word segmentation. Work has important meanings. However, in re...

AI Technical Summary

Problems solved by technology

The unsupervised method needs to set the threshold of effective statistics. The statistics of word and sentence features mainly include the word itself, part of speech, left and right entropy, mutual information, and TF / IDF. This method has a wide range of applications, but the threshold is difficult to set and accurate. low rate

Images