Deep semi-supervised text clustering method based on pairwise constraint and cluster guidance

Abstract

The embodiment of the invention provides a deep semi-supervised text clustering method based on pairwise constraint and cluster guidance, which belongs to the technical field of data processing, and specifically comprises the following steps: preprocessing and vectorizing target text data to obtain a multi-dimensional vector; learning hidden layer features of the target text data according to the multi-dimensional vector, and inputting the hidden layer features into a preset algorithm for clustering to obtain an initial clustering center; calculating clustering loss; generating pairwise constraint loss by using cross entropy; calculating the cluster distribution loss of all the label clusters and all the label-free clusters; and calculating a joint loss function according to the reconstruction error, the clustering loss, the pairwise constraint loss and the cluster distribution loss, and obtaining a clustering result when a preset condition is reached according to iteration of the joint loss function. Through the scheme disclosed by the invention, the supervision information in the label is fully mined, the guide learning between the label and the label-free cluster is enhanced, the robustness of the deep semi-supervised clustering model is improved, and meanwhile, the text clustering precision is improved.

Description

technical field [0001] The embodiments of the present disclosure relate to the technical field of data processing, and in particular to a deep semi-supervised text clustering method based on pairwise constraints and cluster guidance. Background technique [0002] At present, in the face of the information overload problem caused by the continuous expansion of social networks, how to rationally organize and efficiently utilize the value information in the text has become a focus in the field of data mining, and the use of clustering technology to integrate and block can assist The in-depth mining, analysis and prediction of massive texts is also an important prerequisite for key knowledge discovery and extraction. It has important applications in many aspects such as entity classification, sentiment analysis, intelligent recommendation, and automatic question answering. [0003] Existing clustering methods such as Kmeans and other clustering methods are prone to cause the "cu...

AI Technical Summary

Problems solved by technology

[0006] In view of this, the embodiments of the present disclosure provide a deep semi-supervised text clustering method based on pairwise constraints and cluster guidance, which at least partially solves the problems of poor clustering performance and robustness in the prior art

Images