Topic detecting system based on atlas model

Abstract

The invention discloses a topic detecting system based on an atlas model. The topic detecting system comprises a preprocessing module, an atlas structure constructing module and a subgraph detecting module. The preprocessing module is used for preprocessing a corpus test set. The atlas structure constructing module is used for constructing an atlas structure for expressing an original model, the vertex of the atlas structure is a lexical item of a corpus file, the edges of the atlas structure are the relevance among word pairs, different weights are given to the relevance among the word pairs according to whether words are entity words or not, whether dependence relationships exist among the word pairs or not and the distances among the word pairs, and an undirected graph with the entity words as the atlas center is constructed. The subgraph detecting module is used for processing relevancy atlases to obtain vocabulary sets corresponding to all subgraphs, and then ranking and screening are conducted to obtain final results of all subtopics. By means of the topic detecting system based on the atlas model, implicit topics in a discrete text set can be automatically obtained, and the topic detecting system based on the atlas model can be applied to characteristic space dimensionality reduction, relevancy calculation, semantic extension and other related field of natural language processing.

Description

technical field [0001] The present invention relates to technical fields such as information extraction, shallow semantic analysis, feature space dimensionality reduction, named entity recognition, dependency syntax parsing, clustering algorithm, undirected graph model, and specifically a graph based on the use of entity words and syntactic information A topic detection system that models to detect hidden topics in discrete text collections. Background technique [0002] Shallow semantic analysis has important applications in the field of natural language processing. It is necessary to consider the implicit semantics of documents when judging the relevance of documents. How to find similar concepts or topics in discrete documents is a hot issue in text mining (Text Mining) research. . As one of the important technologies of shallow semantic analysis, topic model is one of the more effective models, and it also has important applications in other fields of machine learning. ...

AI Technical Summary

Problems solved by technology

However, from a semantic point of view, these methods do not take into account the semantic information carried by the vocabulary, especially the entity words themselves, and the dependency information that exists in the context of the sentence where the vocabulary is located.

Images