Multilingual text data sorting treatment method

Abstract

The invention discloses a self-learning sorting method relating to multilingual data treatment, comprising the steps of extracting candidate emotion words by a first seed word Chinese or foreign language 'very', filtering stop words, and automatically obtaining a stop word list from a language database; simultaneously carrying out support or opposing clustering on the emotion words and emotion texts by a second seed word 'good' and a third seed word 'bad' or foreign languages 'good' and 'bad'; building an emotion classifier by semi-supervised learning, training the initial classifier by selecting convinced samples from a clustering result, and selecting new samples to be added into a training set by fusing emotion scores of the texts and the posterior probability of the classifier. According to the sorting method, the method facing multilingual opinion analysis is irrelevant with languages, a machine translation system and a large-scale bilingual dictionary are not needed, the emotion classifier is directly learned on a target language, the resource dependence is the smallest, and for each target language, only three seed words are needed and other priori knowledge is not needed.

Description

technical field [0001] The invention relates to the field of data processing, and relates to the tendency analysis of massive text data, in particular to an automatic emotion classification method for multilingual (unfamiliar with languages). Background technique [0002] With the rapid development of the Internet and the acceleration of the globalization process, the information resources provided by the Internet are multilingual. According to the survey data of Nielsen Net Ratings, a global standard Internet user survey and analysis authority, in the nine years from 2000 to 2008, the growth rate of Internet usage in various languages ​​around the world reached 305.5%. The multilingual nature of Internet resources and the difference in familiarity between users' native language and non-native language inevitably bring about language barriers for users to use network information. [0003] The Internet is quietly affecting people's living habits. With the continuous emergenc...

AI Technical Summary

Problems solved by technology

Multilingual sentiment analysis would be difficult to do without machine translation systems or compiled bilingual dictionaries

[0017] (2) The performance of multilingual sentiment analysis is not satisfactory

[0018] First, machine translation-based methods lose accuracy in cross-lingual sentiment analysis

Secondly, most works based on bilingual dictionaries do not consider the context dependence of emotional words when selecting translation words

Finally, the emotional expression of different languages ​​varies greatly, and there is information loss when the model derived from the original space is transferred to the target language space

Images