Multi-document topic discovery method based on two-level clustering

Abstract

The invention discloses a multi-document subject discovery method based on two-layer clustering. The multi-document subject discovery method comprises the following steps: S1 using a plurality of documents as input, pretreating each document, i.e. the documents are broken up into clauses, and the clauses are broken up into words, so as to obtain a noun group and a verb group in a multi-document group, and performing emantic disambiguation processing on polysemes in the noun group and the verb group; S2 respectively performing word clustering analysis on the noun group and the verb group which are output in the step S1 according to word similarity by adopting an improved OPTICS algorithm, extracting semantic concepts, and establishing vector space models on the clauses according to the semantic concepts; S3 performing clustering analysis on the clauses by using an improved K-medoid algorithm, so as to obtain a subject. Inner semantic relations between words are extracted by the multi-document subject discovery method, and the problem of non-orthogonality among feature items when feature vectors of the clauses are established is solved.

Description

technical field [0001] The invention relates to the research field of two-layer clustering, in particular to a multi-document topic discovery method based on two-layer clustering. Background technique [0002] In terms of sentence representation for multi-document topic discovery, the general technology mainly uses sentence segmentation, and uses word frequency vectors or TF-IDF vectors based on word segmentation results to represent sentences. Usually, the distribution of words in the semantic space is not uniform, so the "skew" between the feature components in the traditional vector space model will have a negative impact on sentence clustering. In the density-based sentence clustering algorithm, the general radius parameter needs to be specified in advance, which also brings inconvenience to the topic discovery of multiple documents. Contents of the invention [0003] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the pri...

AI Technical Summary

Problems solved by technology

Usually, the distribution of words in the semantic space is not uniform, so the "skew" between the feature components in the traditional vector space model will have a negative impact on sentence clustering

In the density-based sentence clustering algorithm, the general radius parameter needs to be specified in advance, which also brings inconvenience to the topic discovery of multiple documents

Images