Social media text denoising method based on space-time burst features

Abstract

The invention discloses a social media text denoising method based on space-time burst features, belongs to the field of data processing, and aims to solve the problem of poor text classification effect in the prior art. An improved Ripley's K function is used for measuring whether time-space points where words appear have aggregation or not, noise words and event words are distinguished, meanwhile, in order to reduce the influence of a Ripley's K function threshold value l of the words on a result and reduce word misjudgment, a graph regularization algorithm is introduced, and accuracy of word validity judgment is improved by fusing relevance information between the words.

Description

technical field [0001] The invention belongs to the field of data processing, in particular to a text denoising technology. Background technique [0002] With the widespread use of social media, tens of billions of text messages are published on Twitter, Facebook, Instagram, Weibo and other social media every day, and these messages contain the most complete and most time-sensitive types of information. By extracting and analyzing this information, we can accomplish many valuable things. [0003] Text is one of the important forms for users to express content in social media, so social media text data contains a lot of valuable information, and these data are also the input of many social media data mining tasks. However, due to the openness of social media, most of the text information in social media is a description of personal life and personal emotions, and these texts usually do not contain valuable information. [0004] Social media text denoising aims to identify a...

AI Technical Summary

Problems solved by technology

The effect of these methods is very dependent on the quantity and quality of the training set, and the scalability is also poor (it is difficult to extend the classification of one task to other tasks)

In practice, the labeling of training sets generally needs to be done manually, so it is very difficult to obtain a large number of high-quality training sets; at the same time, due to the characteristics of social media text data such as strong sparsity and irregular expressions, many texts It is difficult to extract the classification features accurately, which leads to the poor classification effect of the existing methods

Images