Dimensionally reduction method based on deep Pearson embedment

Abstract

The invention discloses a dimensionally reduction method based on a deep Pearson embedment, comprising the following steps: firstly, constructing a deep Pearson embedded network, wherein a cost function of the deep Pearson embedded network is the sum of Pearson relevant coefficients of row vectors in a similarity matrix between high dimension samples and a euclidean distance matrix between low dimension embedment samples; secondly, initializing network parameters by means of random or non-supervision pretraining; and finally, supervising and fine-tuning the entire network in a manner of storing data local correlation structures. According to the dimensionally reduction method disclosed by the invention, as the Pearson relevant coefficients are adopted to evaluate relevancy of the raw vectors in the similarity matrix of a high dimension sample pair and the euclidean distance matrix of a low dimension embedment sample pair, and negative correlation is required for the relevancy as far as possible, a smaller Euclidean distance in a low-dimensional space is ensured for samples of greater similarity in a high-dimensional space. The dimensionally reduction method based on deep learning and the Pearson relevant coefficients disclosed by the invention can be widely applied to solving a visualization task, a classification task and the like in the field of machine learning.

Description

technical field [0001] The invention relates to a dimensionality reduction method of deep Pearson embedding, which can be applied to solving tasks such as machine learning, pattern recognition and visualization and classification in the field of computer vision. Background technique [0002] Dimensionality reduction transforms high-dimensional observation data into a low-dimensional feature space through linear and nonlinear mapping under the assumption that low-dimensional data can maintain the essential structure of high-dimensional spatial data. Linear dimensionality reduction methods can discover the geometric structure of embedded linear subspaces from high-dimensional data spaces, such as principal component analysis and linear discriminant analysis. The linear dimension reduction method is simple to implement and has high computational efficiency, but its assumption of linearity or approximately linearity in the embedded subspace of the high-dimensional data space is ...

AI Technical Summary

Problems solved by technology

Local Linear Embedding (Local Linear Embedding, LLE), Laplacian Eigenmap (Laplacian Eigenmap, LE), and t distribution Stochastic Neighborhood Embedding (t-SNE) and other manifold learning methods are well maintained. The manifold geometric structure of the data has a good effect on dealing with the data of nonlinear distribution, however, it faces the problem of "out-of-sample"

In general, a transformation matrix can be learned explicitly through a linear approximation strategy, but this leads to approximation errors and increases the computational cost

t-SNE solves the "out-of-sample" problem through the parameterization of the neural network, but because t-SNE uses the heavy-tailed student distribution in the embedding space to construct the neighbor probability, it cannot preserve the data well when the embedding dimension is greater than 3. local structure

In recent years, the development of deep neural networks has made it possible to learn more complex nonlinear mappings, such as autoencoders (Anto-Encoder, AE), but it mainly focuses on maximizing the variance of low-dimensional spatial data, which makes it very difficult. It is difficult to preserve the local structure information of the data like the manifold learning method

Therefore, people improve AE by adding regularization items so that they can learn the manifold geometry of the data, such as Laplacian Auto-Encoder (LAE), Hessian regularized sparse autoencoder (Hessian regularized Sparse Auto-Encoder, HSAE), etc., but they take a lot of time to train the entire network

Images