Similarity calculation-based junk comment detection method

Abstract

The invention provides a similarity calculation-based junk comment detection method. The method comprises the following steps of 1, performing data acquisition: capturing webpages of a forum, e-commerce and the like related to a specified product by utilizing a network crawler, then extracting comment data in the webpages, and storing the comment data in a database; 2, performing false comment detection: calculating a time difference T between commenting time and purchase time, and if T is less than cargo transport time, determining that a comment is a false comment; 3, performing repeated comment detection: performing word segmentation on each comment, then calculating the similarity between the comments, and when the similarity between the comments is higher than a threshold, determining that one comment and another comment similar to the one comment are repeated comments; 4, performing data processing: performing processing including syntactic analysis, emotion annotation, feature word extraction and the like on the comments, and constructing a product feature dictionary according to a product specification; and 5, performing unrelated comment detection: calculating whether a comment target of each comment is a target product and features thereof or not, and detecting out the comments unrelated to the target product. The invention provides a new junk comment detection method.

Description

[0001] Technical field: [0002] The invention belongs to the field of user comments of natural language processing, in particular to a spam comment detection method based on similarity calculation. [0003] Background technique: [0004] With the advent of the mobile Internet era and the increasingly perfect construction of the Internet of Things, online shopping has gradually become a new and important way of consumption. More and more user-generated content (user-generated content) appears in online applications. Most consumers After shopping, they will post their own shopping experience, usage experience and opinions on products on the e-commerce platform. On the one hand, according to a 2011 survey report by Cone Corporation of the United States, 64% of users will refer to existing user reviews before purchasing, and the reviews published by users will have a certain impact on the consumption behavior of potential consumers and have commercial value; On the one hand, user...

AI Technical Summary

Problems solved by technology

[0006] The uneven quality of user comments brings confusion and even misleading to consumers who browse reviews and companies that obtain product-related information by researching reviews. At the same time, it increases the cost of information mining and reduces the accuracy of automatic mining tools.

Images