Robust principal component analysis method for application scenarios with large amounts of unsupervised noisy data

Abstract

The present invention discloses a robust principal component analysis method for application scenarios with a large amount of unsupervised noise data. By adaptively assigning a weight of 0 or 1 to each sample to discard outliers and calculating the average value through the selected normal samples, the resulting projection space is more reasonable and representative for most samples. In order to maintain the local smoothness of the sample distribution, the present invention learns the affinity relationship of the samples in the subspace, maintains the local smoothness of the sample distribution, and finally proposes an iterative algorithm to solve the model. Based on robust principal component analysis and combined with adaptive outlier detection, the present invention can be applied to multiple fields such as image recognition, data compression, pattern recognition and classification, machine learning, statistics and data analysis.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning and data mining, and particularly relates to a robust principal component analysis method. Background Art

[0002] In today's data-driven research, effectively analyzing correlations in high-dimensional data has become a hot topic. High-dimensional data often contains numerous noisy and redundant features, which can lead to storage burdens, computational costs, and interference with the preservation of internal structure. The core idea behind many dimensionality reduction algorithms, whether used for feature selection or extraction, remains to be based on learning global or local manifold structures.

[0003] Several robust methods have been proposed to improve the robustness of principal component analysis to outliers. One of the most representative distance-based methods is the l1-norm, followed by the F-norm. Generally speaking, the use of the 1-norm includes minimizing the reconstruction error and optimizing the 1-norm variance in the projection space, such as l1-PCA and PCA-l1. Although the l1-norm-based method has good performance, it also has the disadvantage of being difficult to solve and cannot correctly describe the geometric structure of the data, while the F-norm can effectively utilize the inherent spatial structure embedded in the data. However, robust principal component analysis always has the "out-of-sample problem", that is, the previous and current training data need to be recalculated for new samples, resulting in high computational complexity. In addition, robust principal component analysis ignores the relationship between reconstruction error and low-rank representation, resulting in high computational complexity.

[0004] Neighbor-based dimensionality reduction methods that preserve local structure are generally more robust than neighbor-free dimensionality reduction methods that preserve global structure. It is worth noting that adaptive neighborhood-based dimensionality reduction methods use adaptive similarity graphs as one of their components. A simple example is the Adaptive Neighbors Using Correlograms (PCAN) as part of projected clustering, which considers adaptive neighbors and uses the number of clusters as a predefined parameter. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, the present invention provides a robust principal component analysis method for application scenarios with a large amount of unsupervised noise data. By adaptively assigning a weight of 0 or 1 to each sample to discard outliers and calculating the average value through the selected normal samples, the resulting projection space is more reasonable and representative for most samples. In order to maintain the local smoothness of the sample distribution, the present invention learns the affinity relationship of the samples in the subspace, maintains the local smoothness of the sample distribution, and finally proposes an iterative algorithm to solve the model. Based on robust principal component analysis, the present invention combines adaptive outlier detection and can be applied to multiple fields such as image recognition, data compression, pattern recognition and classification, machine learning, statistics and data analysis.

[0006] In specific engineering practice, this dimensionality reduction algorithm has higher accuracy when facing a large amount of unsupervised noisy data. The original high-dimensional space contains redundant information and noise information, which will introduce errors in practical applications and affect the accuracy. Dimensionality reduction can extract the essential structure of the data, reduce the errors caused by redundant information and noise information, and improve the accuracy in the application.

[0007] The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0008] Step 1: Assume S∈R n×n It is a diagonal matrix with only 0 or 1 values, used to filter outliers; X∈R d×n is the sample matrix, m is the deviation, U∈R d×n is the projection matrix, V∈R n×p is the sample matrix after dimensionality reduction, n represents the number of samples, d represents the number of dimensions before dimensionality reduction, and p represents the number of dimensions after dimensionality reduction;

[0009] Define the model as:

[0010]

[0011] st|s|0=k,s ii ∈{0, 1}, U T U=I,V T V=I

[0012] Among them L S =D S -(G T +G) / 2 is the Laplace matrix, matrix G∈R n×n is the similarity matrix of the graph, the degree matrix D S The i-th element of is ∑ j (g ij +g ji ) / 2,s ii The i-th element of the diagonal matrix representing the outlier filtering, xi Represents the i-th element of the sample matrix, v i represents the i-th element of the matrix after dimensionality reduction, λ represents the non-negative regular term coefficient, k represents the number of normal samples, and I represents the identity matrix;

[0013] Step 2: Model solution;

[0014] Step 2-1: Input the original data set X, and preset the non-zero number k of the outlier screening matrix and the coefficient λ of the local manifold structure; obtain the Laplace matrix L of the data set X S , initialize the outlier screening matrix S, and solve the minimization problem minTr(V T L S V) to initialize the reduced-dimensional matrix V;

[0015] Step 2-2: Continuously iterate and solve each parameter through alternating optimization;

[0016] When V is fixed, by solving Update the projection matrix U;

[0017] When both U and V are fixed, Where 1 represents a column vector with all values 1;

[0018] When s, m, U and V are fixed in the previous iteration, by solving To update S, by solving To update V.

[0019] Preferably, the Laplace matrix L of the data set X is obtained S This is done through the K-NN method.

[0020] The beneficial effects of the present invention are as follows:

[0021] 1. The present invention discards outliers by adaptively assigning a weight of 0 or 1 to each sample and calculates the average value through the selected normal samples. The resulting projection space is more reasonable and representative for most samples.

[0022] 2. The present invention learns the affinity relationship of samples in the subspace and maintains the local smoothness of the sample distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a basic flow chart of the method of the present invention.

[0024] Figure 2These are the clustering index comparison charts of the method of the present invention and other dimensionality reduction algorithms in different dimensions on the letter dataset, (a) F-score index comparison chart, (b) ACC index comparison chart, (c) ARI index comparison chart, and (d) NMI index comparison chart. DETAILED DESCRIPTION

[0025] The present invention will be further described below with reference to the accompanying drawings and examples.

[0026] The present invention proposes a new unsupervised dimensionality reduction model (joint dynamic manifold and alternating binary weights update learning for feature extraction, JMBWL), which combines dynamic manifold and alternating binary weights update to adaptively eliminate anomalies by discarding outliers by adaptively assigning a weight of 0 or 1 to each sample and calculating the average value through selected normal samples. In order to maintain the local smoothness of the sample distribution, the present invention learns the affinity relationship of samples in the subspace, maintains the local smoothness of the sample distribution, learns local reconstruction relationships and low-dimensional data embedding. Finally, an iterative algorithm is proposed to solve the model. Based on robust principal component analysis and combined with adaptive outlier detection, the present invention can be applied to multiple fields such as image recognition, data compression, pattern recognition and classification, machine learning, statistics and data analysis.

[0027] A robust principal component analysis method for a large amount of unsupervised noise data application scenario, assuming S∈R n×n Diagonal matrix values are only 0 or 1 to filter outliers, X∈R d×n Sample matrix, m is the deviation, U∈R d×n The projection matrix, and V∈R n×p To reduce the dimensionality of the sample matrix, first define the model as:

[0028]

[0029] st|s|0=k,s ii ∈{0, 1}, U T U=I,V T V=I

[0030] Among them L S =D S -(G T +G) / 2 is the Laplace matrix, matrix G∈R n×n is the similarity matrix of the graph, and the i-th element of the degree matrix DS is ∑ j (g ij +g ji ) / 2.

[0031] The steps to solve the model are as follows:

[0032] Step 1: Input the original data set X and preset the non-zero number k of the outlier screening matrix and the coefficient λ of the local manifold structure. The Laplace matrix L of the data set X is obtained by the K-NN method. S , initialize the outlier screening matrix S, and solve the minimization problem minTr(V T L S V) to initialize the reduced-dimensional matrix V.

[0033] Step 2: Based on the idea of alternating optimization, iterate and solve each parameter. When V is fixed, by solving To update the projection matrix U, when U and V are fixed, Similarly, when other values are fixed, by solving To update S, by solving To update V. Specific embodiment:

[0035] like Figure 1 As shown in FIG, the present invention provides a robust principal component analysis method for application scenarios with a large amount of unsupervised noisy data. The method mainly consists of two main steps: initializing various parameters and alternate iterative optimization. The basic implementation process is as follows:

[0036] 1. Initialize various parameters

[0037] First, the Laplace matrix L of the sample matrix X is solved by the K-NN method S , and initialize outlier screening

[0038] The matrix s is as follows:

[0039] s=I∈R n×n (2)

[0040] Initialize the reduced-dimensional sample matrix V by solving the following problem, which can be simply solved iteratively via SVM decomposition:

[0041]

[0042] 2. Alternating Iterative Optimization

[0043] Once the center vector m is initialized, it must be further transformed not only to update m itself, but also to update the U matrix and V matrix, which converts the vector m into a function that contains both U and V parameters. Expanding the first term of formula (1):

[0044] Tr(-XS1m T -m1 T SXT +m1 T S1m T +2UV T S1m T ) (4)

[0045] Calculate the partial derivative of formula (4) with respect to m:

[0046]

[0047] The solution of the above formula is:

[0048]

[0049] Since the number of non-zeros in the s matrix is k, the above formula corresponds to:

[0050]

[0051] If s and m are fixed, by substituting formula (7) into problem formula (1), we can get the following result:

[0052]

[0053] Where P = kS - S11 T S, I k ∈R n×n is the identity matrix, where all diagonal entries are k.

[0054] Now we can solve U and V alternately. When V is fixed, the solution to problem (8) is as follows:

[0055]

[0056] Similarly, when U is fixed, the problem (8) becomes:

[0057]

[0058] When the (XPV) matrix is constant, the compact SVD method can easily and effectively converge to the problem equation (9):

[0059] (1) Initialization. Randomly initialize U∈R d×n Let U^TU=I n .

[0060] (2) Update M∈R d×p ←XPV.

[0061] (3) Calculated by compact SVD method in S∈R p×p ,

[0062] (4) Update

[0063] (5) Iteratively execute steps (2)-(4) until convergence.

[0064] When the (1kP+λL) and (1kPXTU) matrices are constants, the GPI method can easily and effectively converge to the problem equation (10):

[0065] (1) Initialization. Randomly initialize α and V∈R n×p , so that V T V=I p By the power method, is a positive definite matrix.

[0066] (2) Update

[0067] (3) Calculated by compact SVD method in S∈R p×p ,

[0068] (4) Update

[0069] (5) Iteratively execute steps (2)-(4) until convergence.

[0070] When the projection matrices U, v, and m are all fixed, equation (1) is simplified to:

[0071]

[0072] st|s|0=k,s ii ∈{0, 1}

[0073] The task is to find the corresponding Sort by assigning 1 to the first k smallest sequence numbers and 0 to the rest.

[0074] like Figure 2 As shown, there are clustering index comparison diagrams of the method of the present invention and other dimensionality reduction algorithms in different dimensions on the letter dataset, (a) F-score index comparison diagram, (b) ACC index comparison diagram, (c) ARI index comparison diagram, and (d) NMI index comparison diagram.

Claims

1. An image recognition method based on robust principal component analysis for a large amount of unsupervised noisy data application scenario, characterized by: The steps include: Step 1: Assumptions It is a diagonal matrix with only 0 or 1 values, used to filter outliers; is the sample matrix, It's a deviation. is the projection matrix, is the sample matrix after dimensionality reduction, represents the number of samples, represents the number of dimensions before dimensionality reduction, Indicates the number of dimensions after dimensionality reduction; Define the model as: in is the Laplace matrix, the matrix is the similarity matrix of the graph, the degree matrix No. The elements are , The first diagonal matrix representing the outlier filter elements, The first elements, Represents the first matrix after dimensionality reduction elements, represents the non-negative regularization coefficient, represents the number of normal samples, represents the identity matrix; Step 2: Model solution; Step 2-1: Input the original data set X and preset the non-zero number k of the outlier screening matrix and the coefficient of the local manifold structure ; Obtain the Laplace matrix of the data set X , initialize the outlier screening matrix S and solve the minimization problem by SVD decomposition To initialize the reduced dimension matrix V; Step 2-2: Continuously iterate and solve each parameter through alternating optimization; when V When fixed, by solving Update the projection matrix U ; When both U and V are fixed, , where 1 represents a column vector with all values 1; When s, m, U and V are fixed in the previous iteration, by solving To update S , by solving To update V .

2. The image recognition method based on robust principal component analysis for a large amount of unsupervised noise data application scenario according to claim 1, characterized in that: The Laplace matrix of the data set X is obtained This is done through the K-NN method.

Images