Medical dataset characteristic dimension reduction method based on subspace learning

Abstract

The invention discloses a medical dataset characteristic dimension reduction method based on subspace learning. The method comprises the following steps of constructing an original high-dimension datamatrix X and a label column according to a to-be-analyzed medical dataset; constructing a most optimized target function, and solving a Lagrangean function thereof; according to the original high-dimension data matrix and the label column, calculating global discriminating information and local discriminating information; iterating and solving a conversion matrix Q until the target function is convergent or reaches a highest cycling number-of-times, thereby obtaining a dimension-reduced data matrix; training a model according to the calculated conversion matrix, calculating an AUC value evaluation dimension reduction matrix and classification accuracy. Compared with an existing characteristic dimension reduction method of the medical dataset, the method according to the invention is advantageous in that the local discriminating information and the global discriminating information of data are simultaneously used for performing dimension reduction; the method is suitable for the characteristic dimension reduction problem in a common scale, and relatively high classification accuracy is realized when the characteristic scale of the data is far higher than the sample scale.

Description

technical field [0001] The invention belongs to the field of big data technology and machine learning, in particular to a method for reducing the dimensionality of medical data set features based on subspace learning. Background technique [0002] Dimensionality Reduction aims to transform high-dimensional data into low-dimensional data. The emergence of feature dimensionality reduction technology is due to the fact that in machine learning problems generated in practical application scenarios, a large amount of complex high-dimensional data will be generated. The running time of most data analysis tasks increases at least linearly with the increase of data dimensions, and storing and analyzing high-dimensional data requires a large amount of computer storage resources and a lot of computing time. And many data mining and machine learning tasks, such as classification, clustering and regression, only achieve good results in low-dimensional spaces, and it will be very diffic...

AI Technical Summary

Problems solved by technology

When the number of samples is much smaller than the feature dimension, the LDA method may have a singular matrix and cause the calculated eigenvectors to be inaccurate

On the contrary, LPP maintains the linear structure of the node neighborhood by constructing the adjacency graph of sample points and then calculating the weight, but it does not consider the importance of global discriminant information, so the classification effect is not good.

In addition, some existing feature dimensionality reduction methods, such as PCA and LDA, are parameterless feature dimensionality reduction methods, and both assume that the distribution of sample points obeys the Gaussian distribution, so they are very sensitive to outliers

[0005] At present, the problem of feature dimensionality reduction for high-dimensional data is modeled as an optimization problem, and its solution method often involves eigenvalue decomposition. However, some literature points out that the optimal solution of some problems cannot be solved by eigenvalue decomposition.

Images