A text topic clustering algorithm based on natural language processing

Abstract

The invention discloses a text topic clustering algorithm based on natural language processing. Firstly, a Chinese corpus is formed according to the invention. Secondly, data preprocessing is carriedout for the Chinese text in order to reduce the computational cost of the subsequent algorithm. Then, a novel Chinese word segmentation and feature vectorization algorithm is implemented to transformthe lexical items of comment text from lexical text space to vector space. Then, the dimension of the generated text lexical vector space is reduced to realize the transformation from the text lexicalvector space to the text theme space. Finally, the text topic clustering will be carried out according to the generated text topic model to get the attention of the commenting user to a certain commodity, and finally give some improvement directions for a certain commodity, in order to make the product more and more close to the needs of most users, such as the quality or price needs to be improved and so on.

Description

technical field [0001] The present invention relates to the field of natural language processing, more specifically, to a text topic clustering algorithm based on natural language processing. Background technique [0002] At present, there are mainly the following shortcomings in the traditional text clustering algorithm: [0003] (1) The accuracy of the current Chinese text word segmentation algorithm is not high; [0004] (2) The accuracy of the current text topic model construction algorithm is not high; [0005] (3) The current text topic clustering algorithm cannot well remove the influence of historical records on current decision-making, that is, it cannot slowly forget too old evaluation texts like humans, resulting in the excavated user concerns deviating from the user's latest The focus of attention, which in turn causes businessmen, such as smart home equipment operators, to make mistakes in improving ideas, causing serious economic losses. [0006] The method ...

AI Technical Summary

Problems solved by technology

[0003] (1) The accuracy of the current Chinese text word segmentation algorithm is not high;

[0004] (2) The accuracy of the current text topic model construction algorithm is not high;

[0005] (3) The current text topic clustering algorithm cannot well remove the influence of historical records on current decision-making, that is, it cannot slowly forget too old evaluation texts like humans, resulting in the excavated user concerns deviating from the user's latest The focus of attention, which in turn causes businessmen, such as smart home equipment operators, to make mistakes in improving ideas, causing serious economic losses

[0006] The method most similar to the present invention has Zhang Wanshan et al. (Zhang Wanshan, Xiao Yao, Liang Junjie, etc. Web text clustering method based on theme [J]. Computer Application, 2014,34(11):3140-3143.) in the above In the completed research, aiming at the problem that the traditional Web text clustering algorithm does not consider the topic information of Web text, resulting in low accuracy of multi-topic Web text clustering results, a topic-based Web text clustering method is proposed.

Images