Semantic analysis based text clustering system and method

Abstract

The invention provides a semantic analysis based text clustering system and method. Texts are clustered according to semantic analysis, the workload of a clustering algorithm is reduced, the working efficiency is improved, and the clustering result is more ideal. The semantic analysis based text clustering system provided by the invention comprises a preprocessing module, a semantic analysis module, a vector generation module and a clustering module which are connected sequentially, wherein the preprocessing module is used for performing Chinese word segmentation and stop word filtration on the texts; the semantic analysis module is used for performing semantic similarity analysis and characteristic item weight calculation, extracting characteristic items of keywords and normalizing the texts, a body and an entity dictionary are arranged in the semantic analysis module, the body is used for performing semantic analysis on the texts, the entity dictionary is used for performing entity extraction on the texts, the basic composition unit of the body is concept, and concepts form a concept tree which forms the body; and a vector space model is arranged in the vector generation module, and the vector generation module is used for vectoring the texts.

Description

technical field [0001] The invention relates to the technical field of text information processing, in particular to a text clustering system and method based on semantic analysis. Background technique [0002] In recent years, with the large-scale popularization of the network and the improvement of enterprise informatization, various resources have grown explosively. However, most of the information is stored in text databases. For such semi-structured or unstructured data, The means to obtain specific content information are weak, resulting in difficulty in information search and low utilization rate of information. As a result, research on text mining, information filtering and information retrieval has reached an unprecedented climax. Fast and high-quality text clustering technology can form a large amount of text information into a small number of meaningful clusters, and make the text information in the same cluster have a high degree of similarity, while the texts i...

AI Technical Summary

Problems solved by technology

[0002] In recent years, with the large-scale popularization of the network and the improvement of enterprise informatization, various resources have grown explosively. However, most of the information is stored in text databases. For such semi-structured or unstructured data, The means to obtain specific content information are weak, resulting in difficulty in information search and low utilization rate of information

Images