Discriminant text clustering method and system based on minimum normalized information distance

A text clustering and discriminant technology, which is applied in the field of discriminant text clustering methods and systems, can solve problems such as underfitting models, poor clustering models, and insufficiency, and achieve the effect of overcoming overfitting

CN110955773APending Publication Date: 2020-04-03UNIV OF SCI & TECH OF CHINA
3 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Current Assignee / Owner
Publication Date
2020-04-03

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a discriminant text clustering method and system based on a minimum normalized information distance, and the method comprises the steps: carrying out the vectorization of a text data set which comprises a plurality of texts, wherein each text comprises a plurality of keywords; aiming at the vectorized text data set, initializing a model parameter set; calculating and updating the parameter set by a gradient descent method through the minimum normalized information distance; setting a termination condition and outputting a final parameter set; and designing a discriminant text clustering algorithm by utilizing the final parameter set to realize text clustering. The invention provides the discriminant text clustering method and system based on the minimum normalized information distance. Aiming at the model selection problem of the existing discriminant clustering algorithm, the invention provides the method for using normalized information measure as a target function, so the algorithm has automatic model selection capability, and the capability of obtaining a better clustering result under the condition that the initial model order of manual selection is unreasonable is improved.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the fields of natural language processing and text mining, in particular to a discriminative text clustering method and system based on minimum normalized information distance. Background technique

[0002] Existing text clustering mostly uses the k-means algorithm, and the method of maximizing mutual information (or its variants) is mostly used in the discriminative clustering algorithm. These methods are likely to cause the model order (number of clusters, such as K) of K-means is always equal to the initial value, which makes this type of algorithm not have the ability of automatic model selection, so the model order of the final clustering result is largely determined by humans. However, it is difficult for people to give the most reasonable model order in text clustering, and a large or small model order can easily lead to poor clustering results.

[0003] The existing discriminative clustering algorithm based on maximum ...

Examples

Embodiment Construction

[0052] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0053] Aiming at the model selection problem existing in the existing discriminant clustering algorithm, the present invention proposes a method of using normalized information measure as the objective function, so that the algorithm has the ability of automatic model selection, thereby improving the algorithm's unreasonable initial model order artificially selected. The ability to obtain better clustering results under certain circumstances.

[0054] Data clustering is to divide the collection of data objects into multiple different classes or clusters. The similarity between data objects in each cluster is higher than that of objects in other clusters. In text processing, customer group grouping And image segmentation a...