Clustering method for high dimensional data based on Bayes mixed common factor analyzer

Abstract

The invention discloses a clustering method for high dimensional data based on a Bayes mixed common factor analyzer. The method comprises the following steps: firstly, a model of a Bayes mixed common factor analyzer is built for to-be-clustered high dimensional data; secondly, posteriori distributions of various random variables of the model are subjected to inference, and statistics relevant to the random variables can be obtained; and finally, categories which each dimensional datum belongs to can be obtained through judgment, and the clustering process can be completed. According to the invention, the built Bayes mixed common factor analyzer model has strong flexibility; as the method is based on the inference procedure of Bayes criterion, the phenomenon of overfitting and a dimensionality disaster can be prevented effectively; the method can automatically adjust an optimal structure of the model according to the high dimensional data, so that optimal category data can be confirmed automatically to finish clustering smoothly while performing dimensionality reduction, and excellent clustering performance and computational efficiency can be obtained.

Description

Technical field [0001] The invention involves a clustering method based on the Bayesian hybrid public factor analyzer, which is a processing method and application technology in high -dimensional data. [0002] Background technique [0003] With the continuous development of collection and storage technology, high -dimensional and ultra -high -dimensional data has continued to emerge.For example, tens of thousands of face images commonly common in image retrieval and document search and the inevitable high -vitamin vector, voice and audio signal processing of hundreds of thousands of web texts, voice and audio signalsPerform high -vitamin expression data in cluster analysis, and so on.Obviously, the higher the number of dimensions (the more attributes of the object), you can more comprehensively portray the described objects and better distinguish the object.However, when the data sample is not large, the excessive dimension inevitably has a severe challenge to the processing of ...

AI Technical Summary

Problems solved by technology

However, MFA-based methods for high-dimensional data processing, especially when used for clustering, still have limitations

First of all, in MFA, because each mixture component has a different factor loading matrix, the overall number of parameters of the model is large, and the existing MFA is based on the maximum likelihood criterion for model inference and parameter estimation, so in high When the number of samples of dimensional data is not large, overfitting problems are prone to occur; secondly, and most importantly, in most cases in the application of data clustering, the number of categories is unknown in advance, if set too high Or too low, will affect the accuracy of the final clustering results, and for high-dimensional data, this problem will become more difficult, how to adaptively determine the optimal clustering based on high-dimensional data while reducing dimensionality The number of categories, so as to obtain better clustering performance, is a difficult problem and key point in high-dimensional data clustering techniques and methods

Images