A fast and robust unsupervised dimensionality reduction algorithm based on adaptive bipartite graph
Through the adaptive two-part graph and anchor point strategy, similar graphs are constructed, combined with the new distance criterion and PCA dimensionality reduction, the existing unsupervised dimensionality reduction algorithm has solved the problem of high complexity and incomplete structure retention on large-scale data sets, and the rapid and robust dimensionality reduction effect is achieved.
Patent Information
- Application Number
- CN202210796500.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-06
- Publication Date
- 2025-08-08
- Estimated Expiration
- 2042-07-06
AI Technical Summary
Existing unsupervised dimensionality reduction algorithms have high run time complexity on large-scale datasets, are susceptible to noise and outliers, and are difficult to preserve the local manifold and global structure of the data at the same time.
Adaptive two-part graph method is adopted, and similar graphs are constructed using anchor strategy, combining new distance criterion and PCA dimensionality reduction, and solving the projection matrix through iterative optimization to achieve fast and robust unsupervised dimensionality reduction.
Effectively reduce the computational complexity and reduce the impact of outliers, and can preserve the local and global structure of the data at the same time, improving clustering accuracy and robustness.
Smart Images
Figure CN115329837B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of data dimensionality reduction and machine learning, and specifically relates to a fast, robust, unsupervised dimensionality reduction algorithm based on an adaptive bipartite graph. Background Art
[0002] Data dimensionality reduction plays a crucial role in fields such as machine learning, computer vision, and data mining. Existing dimensionality reduction algorithms can be categorized as supervised, semi-supervised, and unsupervised. Both supervised and semi-supervised methods require guidance from category labels. However, category labels require manual labeling and are not easily obtained. Therefore, research on unsupervised dimensionality reduction (UDR) algorithms that do not require label information and can directly learn the internal structure of the data is highly valuable.
[0003] Graph-based Unified Dimension Reduction (UDR) algorithms have attracted widespread attention due to their ability to efficiently construct relationship graphs between samples to capture statistical information about objects or data. Existing graph UDR algorithms fall into two categories: the first can be unified into a two-step general framework for graph embedding. Specifically, a similarity graph that captures the structural information of a dataset is first constructed in a high-dimensional space, followed by a low-dimensional representation based on the graph. These two steps are performed independently, and once the similarity graph is constructed, it remains immutable. This makes the performance and efficiency of dimensionality reduction entirely dependent on the quality of the graph. However, noise and irrelevant features in the original space can degrade the graph's quality. The second approach combines these two steps into a unified framework to adaptively construct a graph and perform low-dimensional learning. Projection directions are learned and the graph is constructed simultaneously. However, existing graph-based UDR methods still have significant challenges. First, these methods often learn similarity graphs based on pairwise relationships between samples, resulting in a runtime complexity proportional to the square of the number of samples. This limits their applicability to large-scale datasets. Second, they use a squared distance criterion based on the l2 norm to calculate distances between data points, which amplifies the interference of outliers in the graph construction process. Third, most methods only retain a single structure in the local or global part of the data and cannot better and more completely preserve the statistical or geometric characteristics of the data. Summary of the Invention
[0004] In response to the defects and shortcomings in the prior art, the present invention provides a fast robust unsupervised dimensionality reduction algorithm based on adaptive bipartite graphs (FRUDR-ABG), which uses an anchor point strategy to construct a bipartite graph describing statistical or geometric information, and can quickly and adaptively update the graph and learn low-dimensional representations. In addition, a new distance criterion is used to deal with the negative impact of outliers. In order to more completely retain structural information, practical strategies are added to the algorithm to achieve the purpose of simultaneously retaining the local manifold and global structure of the original data. In this algorithm, structural learning and dimensionality reduction are carried out simultaneously.
[0005] In order to achieve the above tasks, the present invention adopts the following technical solutions:
[0006] A fast and robust unsupervised dimensionality reduction algorithm based on adaptive bipartite graph, characterized by comprising the following steps:
[0007] Step 1: Dataset preprocessing:
[0008] The dataset includes sample data (for dimensionality reduction) and label information (for testing). Before data processing, mapminmax or NormalizeFea normalization is performed, depending on the dataset. Mapminmax normalization converts each row of the matrix into the interval [-1, 1]. NormalizeFea normalizes rows or columns based on the number of rows. For datasets with a large number of features, PCA preprocessing is performed by retaining 92% of the information to simplify calculations.
[0009] The mathematical definition of mapminmax is Y = (YMAX-YMIN)*(X-XMIN) / (XMAX-XMIN)+YMIN. Where X is the preprocessed data point matrix, which is defined as a d*n matrix, that is, X = [x1, x2, ...x n ]∈R d*n , where d is the dimension of the sample and n is the number of samples. YMIN and YMAX are the expected minimum and maximum values of each row, and XMIN and XMAX are the maximum and minimum values of the training data.
[0010] If all the data in a row are the same, then XMAX=XMIN and the divisor is 0, then the data remains unchanged.
[0011] NormalizeFea is defined as follows: if row = 1, normalize each row of the data to have unit norm; if row = 0, normalize each column of the data to have unit norm.
[0012] Step 2: Establish an adaptive dimensionality reduction and similarity graph construction model and define the objective function for calculating the projection matrix from high-dimensional space to low-dimensional space:
[0013]
[0014]
[0015] Among them, the sample matrix X=[x1,x2,…x n ]∈R d*n , d is the number of features, n is the number of data points, and each column represents a sample. k is the kth vector representing the point matrix M. ijis defined as the i-th data point x i The similarity between the kth representative point. S={s ik}∈R n×p It is made by ik The weight matrix in the similarity graph composed of W∈R is referred to as the similarity matrix. d×d′ (d′<d) represents the projection matrix, and d′ represents the dimension of the projection subspace. ||W T x i -m k ||2 is based on l 2，1 The norm distance calculation criterion is used to calculate the value of the penalty term. γ is the balance parameter of the penalty term. t =XX T Tr(W T S t W) is The trace ratio form of . Constraint 0≤s ik ≤1, Represents x i The sum of the similarity weights in the cluster of similar data points is 1; and it is assumed that the closer the local distance is, the higher the similarity between the sample pairs is, that is, the corresponding s ik The bigger.
[0016] Step 3: Calculate the objective function to obtain the global optimal solution of the mapping matrix in the dimensionality reduction model;
[0017] Step 4: The global optimal solution of the mapping matrix obtained in step 3 is used to project the sample data into the low-dimensional subspace using the following formula:
[0018] X_projected = X′×W
[0019] Where, X=[x1,x2,…x n ]∈R d*n is the sample data matrix; X_projected = [x1, x2, ...x n ]∈R d′*n (d′<d) is the low-dimensional representation of the sample data X in the projection subspace. W∈R d×d′ (d′<d) represents the projection matrix.
[0020] Finally, the reduced dataset was trained 20 times with random initialization, compared to the label information in the original dataset. The projection results were then clustered using the K-means method with the Euclidean distance metric. The model was evaluated using the clustering accuracy (ACC) and normalized mutual information (NMI).
[0021] According to the present invention, the implementation of step 2 includes the following sub-steps:
[0022] Specifically, the step 2 is described step by step as follows:
[0023] Step 2.1: A natural way to first calculate the projection matrix from the high-dimensional space to the low-dimensional space is to optimize the following formula (1):
[0024]
[0025]
[0026] Among them, for the i-th data point x i , s ij is defined as the jth data point x j and x i The similarity between them.
[0027] Step 2.2: To prevent the problem (1) from generating trivial solutions during the solution process, that is, only the nearest data point can be x i The neighbors of are 1 in similarity, while the other data points are not. A penalty term is added to Equation (1) to solve this problem:
[0028]
[0029]
[0030] Here, γ is the balancing parameter for the penalty term. The addition of a second penalty term unifies graph construction and dimensionality reduction within a single framework. The similarity graph construction minimizes the dispersion within the graph and maximizes the similarity entropy, adaptively optimizing the similarity matrix and projection matrix. The constraints here ensure that all pairs of data points are correlated. However, this approach requires extensive computation and consumes significant resources in practice and is not applicable to all datasets.
[0031] Step 2.3: It is proposed to select some representative anchor points (representative points) and use the pairwise relationship between sample points and anchor points to learn the similarity matrix, which will reduce the amount of calculation and shorten the calculation time.
[0032] The anchor matrix of the low-dimensional subspace M = [m1, m2, ..., m p ]∈R d′×p (p<n) is proposed, then the similarity matrix is redefined as S={s ik}∈R n×p , s ik Represents the similarity between the i-th sample and the k-th anchor point. The model can be re-expressed as:
[0033]
[0034]
[0035] Step 2.4: In order to reduce the impact of outliers on graph construction, a distance criterion between data points based on the l2 norm is proposed to calculate the distance between sample points and anchor points and learn the corresponding weights. This is shown below:
[0036]
[0037]
[0038] Where, Essentially multiple matrices l 2，1 The sum of the norms.
[0039] Step 2.5: Finally, PCA dimensionality reduction is added to the model to eliminate noise and redundant features in the global data.
[0040] The goal of PCA is to maximize Convert it into trace form: Tr[W T XX T W], and then integrate it into the model to obtain:
[0041]
[0042]
[0043] At this point, the model simultaneously learns the global and local geometric structures of the original data and solves the optimal projection matrix to map the samples into a low-dimensional subspace.
[0044] The model proposed above is a constrained nonlinear optimization problem. Solving it directly is very complex, so it is solved using an iterative optimization method. This involves fixing two variables to solve for the other variable, and then iterating the optimization until the objective function converges.
[0045] Specifically, the implementation of step 3 includes the following sub-steps:
[0046] Step 3.1: Fix W and M and update S (similarity learning)
[0047] Using the Lagrange multiplier method and according to the KKT condition, the optimal S can be verified.
[0048] Step 3.2: Fix S and update W and M (iterative solution)
[0049] For the convenience of solving, assume And let u ik =q ik s ik , then formula (5) is converted to:
[0050]
[0051] At this time, there are three variables in the formula, W, M and U, and the solution for the three variables is divided into three steps.
[0052] (1) Fix W and U to solve M (solve the representative point matrix)
[0053] Let Eq.(6) be k Taking the derivative and setting it to 0, we get:
[0054]
[0055] (2) Fix M and U to solve W (solve the projection matrix)
[0056] When the variables M and U are fixed, we solve for W. Formula (6) can be simplified to
[0057]
[0058] Where, Where A and B are diagonal matrices, and the diagonal elements are and and This is a trace ratio problem related to the projection matrix W. An efficient algorithm based on the Newton-Raphson method is used to obtain a globally optimal solution to the trace ratio problem.
[0059] (3) Fix W and M to solve U (solve the similarity matrix)
[0060] Because uu ik =q ik S ik , and s ik It has been solved in 3.1, so solving for Q can lead to solving for U.
[0061] The fast and robust unsupervised dimensionality reduction algorithm based on adaptive bipartite graphs of the present invention is a new unsupervised dimensionality reduction algorithm that can efficiently reduce the dimensionality of data. Compared with the existing technology, it brings the following beneficial technical effects:
[0062] This algorithm utilizes an anchor point strategy to construct a bipartite graph describing statistical or geometric information, enabling rapid and adaptive graph updates and learning of low-dimensional representations. Furthermore, a novel distance criterion is used to mitigate the negative impact of outliers. To more fully preserve structural information, the algorithm incorporates practical strategies to simultaneously preserve the local manifold and global structure of the original data. Clustering results on a benchmark dataset demonstrate the effectiveness and robustness of this approach. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] Figure 1A schematic diagram of the innovative idea of converting the construction process of the original space paired sample point graph into the construction of the original space sample points and low-dimensional subspace representative point graph is given;
[0064] Figure 2 The clustered ACC results of the dermatology dataset are plotted as the parameters change;
[0065] Figure 3 The clustered ACC results of the heart data set are plotted as the parameters change;
[0066] Figure 4 The clustering ACC results of the USPS_subset dataset vary with the parameters;
[0067] Figure 5 The clustered NMI results of the dermatology dataset are plotted as the parameters change;
[0068] Figure 6 The clustering NMI results of the heart data set are plotted as the parameters change;
[0069] Figure 7 The clustered NMI results of the USPS_subset dataset vary with the parameters;
[0070] Figure 8 The clustering ACC results of the data sets AR, control, and heart change with the dimension reduction dimension;
[0071] Figure 9 The clustering NMI results of the data sets AR, control, and heart change with the dimensionality reduction dimension;
[0072] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. DETAILED DESCRIPTION
[0073] The innovative ideas of this application are as follows Figure 1 As shown in Figure 3, the construction process of the paired sample point graph in the original space is converted into the graph of the sample points in the original space and the representative points in the low-dimensional subspace.
[0074] First, the datasets used in the fast unsupervised dimensionality reduction algorithm based on adaptive bipartite graphs in this application are introduced. They are the datasets control, dematology, heart, gene dataset CLL_SUB_111, face dataset AR and handwritten digital image dataset USPS_subset in the UCI machine learning library. CLL_SUB_111 is a B-cell chronic lymphocytic leukemia dataset. The USPS database contains 9298 grayscale handwritten digital images scanned from envelopes by the United States Postal Service, and the inventors selected 1854 of them for experiments. The AR face database contains more than 4000 color face images of 126 people (70 men and 56 women), including frontal images of faces with different facial expressions, lighting conditions and occlusions. In order to improve computational efficiency, some images are selected and cropped to 16X16 pixels, and pixel values are used as features.
[0075] This embodiment provides a fast robust unsupervised dimensionality reduction algorithm based on adaptive bipartite graph (FRUDR-ABG), which specifically includes the following steps:
[0076] Step 1: Dataset preprocessing:
[0077] The dataset includes sample data (for dimensionality reduction) and label information (for testing). Before processing, either mapminmax or NormalizeFea normalization is performed, depending on the dataset. Mapminmax normalization converts each row of the matrix into the interval [-1, 1]. NormalizeFea normalizes rows or columns based on the number of rows. In addition, for datasets with a large number of features, PCA preprocessing is performed by retaining 92% of the information to simplify calculations.
[0078] The mathematical definition of mapminmax is Y = (YMAX - YMIN) * (X - XMIN) / (XMAX - XMIN) + YMIN. Here, X is the preprocessed data point matrix, which is defined as a d*n matrix, that is, X = [x_1, x_2, ... x_n] ∈ R^(d*n), where d is the dimension of the sample and n is the number of samples. YMIN and YMAX are the expected minimum and maximum values for each row, and XMIN and XMAX are the maximum and minimum values of the training data. If the data in a row is all the same, then XMAX = XMIN, and the divisor is 0, the data is unchanged. Y is the normalized matrix.
[0079] NormalizeFea is defined as follows: if row = 1, normalize each row of the data to have unit norm; if row = 0, normalize each column of the data to have unit norm.
[0080] Advantages of this step: The reason for normalizing the data first is that different rating indicators often have different dimensions and dimensional units, which can affect the results of data analysis. To eliminate the dimensional effects between indicators, data normalization is required to ensure comparability between data indicators. After data normalization, the raw data is at the same order of magnitude, making it suitable for comprehensive comparative evaluation. The reason for using PCA for datasets with a large number of features is that some small sample datasets have a far greater number of features than the number of samples, which places a heavy burden on subsequent calculations. Therefore, PCA is performed on these datasets to remove noise and redundant features.
[0081] Step 2: Establish an adaptive dimensionality reduction and similarity graph construction model and define the objective function for calculating the projection matrix from high-dimensional space to low-dimensional space:
[0082]
[0083]
[0084] Among them, the sample matrix X=[x1,x2,…x n ]∈R d*n , where d is the number of features, n is the number of data points, and each column represents a sample. k is the kth vector representing the point matrix M. For the i-th data point x i , s ij is defined as the i-th data point x i The similarity between the kth representative point. S={s ik}∈R n×p It is made by ik The weight matrix in the similarity graph composed of W∈R is referred to as the similarity matrix. d×d′ (d′<d) represents the projection matrix, and d′ represents the dimension of the projection subspace. ||W T x i -m k ||2 is based on l 2，1 The norm distance calculation criterion is used to calculate the value of the penalty term. γ is the balance parameter of the penalty term. t =XX T Tr(W T S t W) is The trace ratio form of . Constraint 0≤s ik ≤1, Represents x i The sum of the similarity weights in a cluster of similar data points is 1. We assume that the closer the local distance is, the higher the similarity between the sample pairs, that is, the corresponding s ik The bigger.
[0085] In this step, the objective function is defined as follows:
[0086] Step 2.1: A natural way to first calculate the projection matrix from high-dimensional space to low-dimensional space is to optimize the following formula (1)
[0087]
[0088]
[0089] Among them, for the i-th data point x i , s ij is defined as the jth data point x j and x i The similarity between them.
[0090] Step 2.2: To prevent the problem (1) from generating trivial solutions during the solution process, only the nearest data point can be x i The neighbors of are 1 in similarity, while the other data points are not. Adding a penalty term to the problem (1) solves this problem:
[0091]
[0092]
[0093] Here, γ is the balancing parameter for the penalty term. The addition of a second penalty term unifies graph construction and dimensionality reduction within a single framework. The similarity graph construction minimizes the dispersion within the graph and maximizes the similarity entropy, adaptively optimizing the similarity matrix and projection matrix. The constraints here ensure that all pairs of data points are correlated. However, this approach requires extensive computation and consumes significant resources in practice and is not applicable to all datasets.
[0094] Step 2.3: Select some representative anchor points (representative points) and use the pairwise relationship between sample points and anchor points to learn the similarity matrix, which will reduce the amount of calculation and reduce the calculation time. The anchor matrix M of the low-dimensional subspace is [m1, m2, ..., m p ]∈R d′×p (p<n) is proposed, then the similarity matrix is redefined as S={s ik}∈R n×p , s ik represents the similarity between the i-th sample and the k-th anchor point, then the model can be re-expressed as the following formula (3):
[0095]
[0096]
[0097] Step 2.4: In order to reduce the impact of outliers on graph construction, a distance criterion between data points based on the l2 norm is proposed to calculate the distance between sample points and anchor points and learn the corresponding weights.
[0098]
[0099]
[0100] Essentially multiple matrices l 2，1 The sum of the norms.
[0101] Step 2.5: Finally, PCA dimensionality reduction is added to the model to remove noise and redundant features in the global data. The goal of PCA is to maximize the following form:
[0102]
[0103] Convert it into trace form:
[0104] Tr[W T XX T W] (6)
[0105] Then integrate it into the model to obtain:
[0106]
[0107]
[0108] At this point, the model can simultaneously learn the global and local geometric structure of the original data. The goal is to use such a model to solve the optimal projection matrix to map the sample into the d′-dimensional subspace.
[0109] Advantages of this step: The model construction process is described step by step, providing a clear understanding of the modifications and mathematical representations of the proposed method based on previous models. The resulting model not only reduces time complexity but also mitigates the impact of outliers on the graph construction process while preserving both global and local data structures.
[0110] Step 3: Calculate the objective function to obtain the global optimal solution of the mapping matrix in the dimensionality reduction model.
[0111] The model proposed above is a constrained nonlinear optimization problem. Solving it directly is very complex, so it is solved through an iterative optimization method. In the model, there are three variables to be solved: the projection matrix W, the anchor point (representative point) matrix M, and the similarity matrix S. Model (7) is solved through iterative optimization, that is, fixing two of the variables to solve the other variable, and then iterating the optimization until the objective function converges.
[0112] The solution process includes the following sub-steps: 1. Fix W and M and solve S; 2. Fix S and solve W and M.
[0113] 3.1: Update S (Similarity Learning)
[0114] Fixing W and M and solving for S, the model becomes
[0115]
[0116]
[0117] Let D ik =||W T x i -m k ||2, use the Lagrange multiplier method and calculate s ik Taking the derivative to make it equal to 0, we get
[0118]
[0119] 1≤k≤p, 1≤i≤n,
[0120] and
[0121]
[0122] Combining the two formulas, we can get:
[0123]
[0124] According to the KKT condition, it can be verified that the optimal S should be:
[0125]
[0126] 3.2: Update W and M (iterative solution)
[0127] When the variable S is fixed, the model will be transformed into the following form:
[0128]
[0129] For the convenience of solving, assume And let u ik =q ik sik , then the above formula is converted into the following formula (14):
[0130]
[0131] At this point, there are three variables in the formula: W, M, and U. Solving the three variables is divided into three steps:
[0132] Step (1): Fix W and U to solve M (solve the representative point matrix)
[0133] After fixing W and U, M becomes the only variable, and Eq. (14) is used to calculate m k Taking the derivative and setting it to 0, we get:
[0134]
[0135] M k Expressed as To facilitate subsequent calculations.
[0136] Step (2): Fix M and U to solve W (solve the projection matrix)
[0137] When the variables M and U are fixed, solve for W: Equation (14) becomes:
[0138]
[0139] make: in addition, Then we have:
[0140]
[0141] make:
[0142] Where A and B are diagonal matrices, and the diagonal elements are and and
[0143] make: Then formula (18) is simplified to the following trace ratio form:
[0144]
[0145] This is a trace ratio problem related to the projection matrix W. An efficient algorithm based on the Newton-Raphson method is used to obtain a globally optimal solution to the trace ratio problem.
[0146] Step (3): Fix W and M to solve U (solve the similarity matrix)
[0147] Because u ik =q ik sik , and s ik It has been solved in 3.1, so solving for Q can lead to solving for U.
[0148]
[0149] Advantages of this step: The model obtained in step 2 is a constrained nonlinear optimization problem. Solving it directly is very complex and cannot obtain a global optimal solution. Therefore, step 3 uses an iterative optimization method to solve the above problem to obtain a global optimal solution and the optimal projection matrix.
[0150] Step 4: The global optimal solution of the mapping matrix obtained in step 3 is used to project the sample data into a low-dimensional subspace using the formula X_projected = X′×W.
[0151] Where X = [x1, x2, ... x n ]∈R d*n is the sample data matrix.
[0152] X_projected = [x1, x2, ... x n ]∈R d′*n (d′<d) is the low-dimensional representation of the sample data X in the projection subspace. W∈R d×d′ (d′<d) represents the projection matrix.
[0153] Finally, the reduced dataset was trained 20 times with random initialization, compared to the label information in the original dataset. The projection results were then clustered using the K-means method with the Euclidean distance metric. The model was evaluated using the clustering accuracy (ACC) and normalized mutual information (NMI).
[0154] The fast and robust unsupervised dimensionality reduction algorithm based on adaptive bipartite graph of this embodiment is applied to this specific data set to demonstrate its more prominent clustering effect.
[0155] (1) Table 1 below is a brief description of all datasets.
[0156] Table 1: Detailed information of the dataset
[0157]
[0158] (2) Evaluation criteria:
[0159] The performance of the algorithm is evaluated using two metrics: clustering accuracy (ACC) and normalized mutual information (NMI). The larger the ACC and NMI, the better the performance.
[0160] (3) Parameter settings:
[0161] The dimension of the dimensionality reduction subspace of all algorithms takes values in the range of [1, 3, 5, ... fea_num-1] (fea_num is the number of features of each data set).
[0162] The number of nearest neighbors for LLE is set to [2, 4, 6, 8, 10, 12]. LPP and NPE use heat kernel to characterize the relationship between data points in high-dimensional space. In the proposed algorithm, the penalty parameter γ is set to [10 -4 , 10 -3 ,…,10 5 ] The representative points are an innovation in this paper. Their value plays an important role in the algorithm. The adjustment range is (where N is the number of samples and c is the number of categories).
[0163] All the above parameters were tested using the grid search method.
[0164] (4) Result analysis
[0165] First, to illustrate the impact of parameters on dimensionality reduction performance, we conducted a parameter sensitivity experiment on the penalty term's balance parameter γ and the number of representative points p in the low-dimensional subspace. Three representative datasets were selected for testing. The experiments were performed on the c-1 dimension of the corresponding dataset, where c is the number of categories.
[0166] Figures 2 to 7 The clustered ACC results of the dermatology dataset are given as the parameters change ( Figure 2 ); Cluster ACC results of heart data set as parameters change ( Figure 3 ); Cluster ACC results of USPS_subset dataset vary with parameters ( Figure 4 ); Cluster NMI results of dermatology data set with parameter changes ( Figure 5 ); Cluster NMI results of heart data set as parameters change ( Figure 6 ); Cluster NMI results of USPS_subset dataset as parameters change ( Figure 7 ). As can be seen from Figures (2) to (7), the parameters have a great influence on the results. For heart and USPS_subset, the algorithm's ACC and NMI are slightly sensitive to parameter changes. The reason for this change is that a too small γ will lead to a trivial solution during the solution process, while a too large γ will make the penalty term dominate and ignore the local distance input of the original data. For dermatology, ACC and NMI are not sensitive to parameter changes. This shows that different parameters for different data sets will have different experimental performance. Therefore, how to choose the parameters to achieve the best results depends on the data.
[0167] Secondly, we conduct an experimental analysis from the perspective of the impact of the dimension of the dimensionality reduction subspace on the results, and select the results of the three data sets AR, control, and heart for presentation. Figure 8 and Figure 9 The clustering ACC results of the data sets AR, control, and heart are plotted as the dimensionality is reduced ( Figure 8 ) and the clustering NMI results of the data sets AR, control, and heart as the dimensionality is reduced ( Figure 9 ).
[0168] As can be clearly seen from the figure, the ACC results for NPE, LPP, MN-PCA, and PCA are stable across all datasets. In contrast, LLE and FRUDR-ABG fluctuate significantly with changes in dimensionality. In most cases, although this method is significantly affected by dimensionality, it maintains a high clustering accuracy. When the subspace dimensionality is low, the algorithm's clustering performance is low, likely because too low a dimensionality causes excessive loss of feature information. As dimensionality increases, the clustering accuracy of the fast, robust, unsupervised dimensionality reduction algorithm based on adaptive bipartite graphs presented in this embodiment fluctuates significantly. Because different subspace dimensions contain different discriminant information, increasing dimensionality allows for more comprehensive data mining, but also introduces more redundant information. The variation graph for NMI differs significantly from that for ACC, particularly on the AR and control datasets, where NPE achieves poor performance on NMI. While our algorithm exhibits significant fluctuations, it still outperforms the other five compared algorithms. This suggests that the fast, robust, unsupervised dimensionality reduction algorithm based on adaptive bipartite graphs proposed in this embodiment can obtain more effective information.
[0169] Finally, the clustering performance of the proposed adaptive bipartite graph-based fast and robust unsupervised dimensionality reduction algorithm was compared with five other unsupervised dimensionality reduction algorithms on six benchmark datasets. Tables 2 and 3 report the detailed accuracy (ACC) and normalized mutual information (NMI) and their standard deviation results.
[0170] Table 2
[0171] All features PCA MN-PCA LLE LPP NPE ours control 57.54±02.55 67.58±01.62 67.98±01.39 55.30±00.15 67.91±00.27 77.45±00.00 82.31±04.43 dermatology 81.33±06.72 85.29±05.40 87.24±02.57 92.62±00.00 92.38±00.12 87.10±00.00 93.47±07.02 heart 54.88±00.30 54.98±00.41 56.77±00.00 58.23±02.60 51.49±00.00 55.12±00.00 66.62±00.33 AR 12.27±00.46 12.55±00.36 12.32±00.58 01.15±00.00 12.12±00.57 12.38±00.00 17.49±00.48 USPS_subset 69.55±00.46 69.56±00.51 69.65±00.83 16.24±00.00 70.37±00.62 68.16±00.00 80.45±00.00 CLL_SUB_111 37.84±00.00 37.84±00.00 37.84±00.00 47.48±00.42 53.56±04.40 38.74±00.00 48.87±00.65
[0172] Table 3
[0173]
[0174]
[0175] As shown in the table, the fast and robust unsupervised dimensionality reduction algorithm based on adaptive bipartite graphs presented in this example outperforms other algorithms on all datasets. This demonstrates that using anchor points and sample points to represent similarity relationships does not result in missing information and can effectively represent overall similarity relationships. This algorithm achieves superior results compared to PCA-based algorithms, demonstrating that simultaneously capturing both local and global relationships preserves information more completely than a single structure alone. Furthermore, this algorithm avoids the errors introduced by predefined graphs and is more stable than methods such as LLE and LPP.
[0176] It should be noted that the above embodiments are preferred examples, and the present invention is not limited to the above embodiments. Any additions to technical features and equivalent transformations made on the basis of the technical solution of this application should fall within the protection scope of the technical solution of the present invention.
Claims
1. A fast and robust unsupervised dimensionality reduction method based on adaptive bipartite graph, characterized by: include: Step 1: Dataset preprocessing: The dataset includes sample data for dimensionality reduction and label information for testing. The dataset is an image dataset. Before preprocessing, map minmax or NormalizeFea normalization processing is performed depending on the dataset. Among them, map minmax normalization processing processes each row of the matrix into the interval [-1, 1]; NormalizeFea normalization processing normalizes rows or columns according to the number of rows. In addition, for datasets with too many features, PCA preprocessing is performed by retaining 92% of the information to simplify calculations. The mathematical definition of mapminmax is: Y=(YMAX-YMIN)*(X-XMIN) / (XMAX-XMIN)+YMIN; Among them, X is the preprocessed data point matrix, which is defined as a d*n matrix, that is, X=[x1,x2,…,x n ]∈R d ×n , where d is the number of features of the sample data, n is the number of sample data; YMIN and YMAX are the minimum and maximum values of each row, and XMIN and XMAX are the maximum and minimum values of the training data; If all the data in a row are the same, then XMAX=XMIN and the divisor is 0, then the data remains unchanged. The NormalizeFea is defined as follows: if row=1, normalize each row of the data to have unit norm; if row=0, normalize each column of the data to have unit norm; Step 2: Establish an adaptive dimensionality reduction and similarity graph construction model and define the objective function for calculating the projection matrix from high-dimensional space to low-dimensional space: Among them, the sample matrix X=[x1,x2,…,x n ]∈R d×n , where d is the number of sample data features, n is the number of sample data points, and each column represents a sample; m k is the kth vector of the anchor matrix M of the low-dimensional subspace, M=[m1,m2,…,m p ]∈R d'×p , p <n; For the i-th data point x i ,s ik is defined as x i The similarity between the k-th anchor point; S = {s ik }∈R n×p It is made by ik The weight matrix in the similarity graph is referred to as the similarity matrix; W∈R d×d' represents the projection matrix, d′ represents the dimension of the projection subspace, d′<d; ||W T x i -m k ||2 is based on l 2,1 The norm distance calculation criterion is used to calculate; γ is the balance parameter of the penalty term; S in the denominator t =XX T , Tr(W T S t W) is Trace ratio form; constraint Represents x i The sum of the similarity weights in the cluster of similar data points is 1; and it is assumed that the closer the local distance is, the higher the similarity between the sample pairs is, that is, the corresponding s ik The bigger; Step 3: Calculate the objective function of the projection matrix to obtain the global optimal solution of the mapping matrix in the dimensionality reduction model; Step 4: The global optimal solution of the mapping matrix obtained in step 3 is used to project the sample data into the low-dimensional subspace using the following formula: X_projected=X T ×W Where, X T is the transpose of matrix X; X_projected = [x1, x2, ..., x n ]∈R d'×n It is the low-dimensional representation of the sample data X in the projection subspace; W∈R d×d' represents the projection matrix, d′<d; Finally, the label information in the original dataset was compared, and the reduced-dimensional dataset was randomly initialized and trained 20 times. The projection results were then clustered using the K-means method with Euclidean distance metric. The model was evaluated using clustering accuracy and normalized mutual information.
2. The method according to claim 1, wherein The implementation of step 2 includes the following sub-steps: Step 2.1: A natural way to first calculate the projection matrix from high-dimensional space to low-dimensional space is to optimize the following problem (1) Among them, for the i-th data point x i ,s ik is defined as the jth data point x j and x i similarity between Step 2.2: To prevent the problem (1) from generating trivial solutions during the solution process, that is, only the nearest data point is x i The neighbors of are 1 in similarity, while the other data points are not. A penalty term is added to formula (1) to solve this problem: Where γ is the balancing parameter of the penalty term. The addition of the second penalty term can unify graph construction and dimensionality reduction into one framework. In the process of constructing a similarity graph, the discreteness within the graph can be minimized and the similarity entropy can be maximized, and the similarity matrix and projection matrix can be adaptively optimized. The role of the constraint is to ensure that all pairs of data points are correlated. Step 2.3: It is proposed to select some representative anchor points and use the pairwise relationship between sample points and anchor points to learn the similarity matrix to reduce the amount of calculation and shorten the calculation time. The anchor matrix M of the low-dimensional subspace is M = [m1, m2, …, m p ∈ R d'×p is proposed, where p < n, and the similarity matrix is redefined as S = {s ik} ∈ R n×p , where s ik represents the similarity between the i-th sample and the k-th anchor point; the model can be re-expressed as Step 2.4: In order to reduce the impact of outliers on graph construction, a distance criterion between data points based on the l2 norm is proposed to calculate the distance between sample points and anchor points and learn the corresponding weights; as shown below: Essentially multiple matrices l 2,1 Sum of norms; Step 2.5: Finally, PCA dimensionality reduction is added to the model to eliminate noise and redundant features in the global data; The goal of PCA is to maximize Convert it into trace form: Tr[W T XX T W], and then integrate it into the model to obtain: At this point, the model simultaneously learns the global and local geometric structures of the original data and solves the optimal projection matrix to map the samples into the d'-dimensional subspace.
3. The method according to claim 1, wherein: The implementation of step 3 specifically includes the following sub-steps: 3.1: Fixed W and M update S Using the Lagrange multiplier method, according to the KKT condition, the optimal S can be verified; 3.2: Fixed S Update W and M For the convenience of solving, assume that: And u ik =q ik s ik , then formula (5) is converted to: At this point, there are three variables in the formula: W, M, and U. Solving the three variables is divided into three steps: Step (1) fix W and U to solve M, that is, solve the representative point matrix: Let Eq.(6) be k Taking the derivative and setting it to 0, we get: Step (2) fix M and U to solve W, that is, solve the projection matrix: When variables M and U are fixed and W is solved, formula (6) is simplified to: in, A and B are diagonal matrices, and the diagonal elements are and and This is a trace ratio problem related to the projection matrix W. An efficient algorithm based on the Newton-Raphson method is used to obtain the global optimal solution to the trace ratio problem. In step (3), fix W and M to solve U, that is, solve the similarity matrix: Because u ik =q ik s ik , and s ik It has been solved in 3.1, so solving for Q can lead to solving for U.