Multi-core subspace clustering method based on variance-covariance subspace distance

Abstract

The invention relates to the field of machine learning and pattern recognition, and particularly discloses a variance-covariance subspace distance-based multi-kernel subspace clustering method, which comprises the following steps of: S1, acquiring a high-dimensional data matrix, calculating a variance-covariance matrix and second-order statistical information thereof, defining a preliminary VCSD kernel value and obtaining a VCSD kernel matrix through exponential transformation and normalization, constructing a multi-core pool of r base cores; s2, constructing an optimized objective function based on non-negative matrix factorization, and solving to obtain a local affinity feature map of a block diagonal structure under constraint conditions; s3, capturing high-order feature association through a matrix method, and converting tensor optimization into matrix operation; s4, solving the target function by adopting an alternating optimization strategy, reducing the calculation complexity, and generating a target affinity graph; and S5, clustering the target affinity spectrum, and outputting a result. According to the technical scheme provided by the invention, the problems of missing statistical characteristics, complex calculation and nuclear noise interference of the existing method are solved, and the clustering precision and efficiency are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning and pattern recognition, and particularly to a multi-kernel subspace clustering method based on variance-covariance subspace distance. BACKGROUND

[0002] The multi-kernel subspace clustering method has important application value in high-dimensional nonlinear data processing scenarios and can be widely used in practical tasks such as face recognition and image clustering. Unlike the adaptation limitations of traditional single-kernel clustering methods to complex data structures, multi-kernel subspace clustering (MKSC) can more efficiently capture the nonlinear characteristics of data by fusing the information of multiple base kernel functions, and has shown significant advantages in small and medium-sized high-dimensional data clustering.

[0003] Among existing high-dimensional data clustering techniques, multi-kernel subspace clustering is one of the research hotspots. Different MKSC improvement methods have been proposed by multiple research teams. In 2024, Zhao et al. proposed the pure kernel graph fusion tensor subspace clustering (PKGT) method, which constructs a block diagonal structure of local affinity feature graph through non-negative matrix factorization, and achieves excellent performance. In 2021, Zhang et al. proposed the low-rank robust multi-kernel subspace clustering (LRMKSC), which uses a weighted truncated Schatten-p norm to impose a low-rank constraint. In 2023, Wang et al. developed the late fusion multi-kernel subspace clustering and proxy graph refinement (LFMKC-PGR) method, which constructs a self-expression matrix for each base kernel.

[0004] In terms of subspace distance measurement, in 2023, Karami et al. proposed the variance-covariance subspace distance (VCSD), which evaluates feature importance by calculating the variance-covariance matrix norm of deleted features, and performs outstandingly in the field of unsupervised feature selection. There are also neural network-based methods such as deep subspace clustering, which can handle complex data associations but rely on a large amount of labeled data.

[0005] These MKSC methods all adopt a hierarchical framework of "base kernel capture - consistent kernel optimization - affinity graph update". These methods divide the clustering task into three processing levels: first, at the highest level, different nonlinear characteristics of data are captured through multiple base kernel functions (linear kernel, polynomial kernel, Gaussian kernel, etc.); then, at the intermediate level, a weight distribution strategy is designed, and an optimal consistent kernel is learned through an optimization algorithm to fuse the information of different kernel matrices; finally, at the bottom level, the learned consistent kernel is used to iteratively update the affinity graph, which is used for subsequent spectral clustering. This hierarchical processing framework enables MKSC methods to effectively handle complex nonlinear data structures and effectively handle nonlinear data in specific scenarios.

[0006] However, although these methods improve the performance of single-core methods through multi-core fusion, there are still many limitations: the similarity measure of traditional MKSC methods depends on Euclidean distance, principal angle, etc., lacks statistical consideration of data variance distribution and covariance relationship between features, and has limited feature expression ability and sensitivity to noise; methods such as PKGT require explicit tensor operations when capturing high-order feature correlations, with a high computational complexity of O(n³), which is difficult to adapt to large-scale data; most MKSC methods aim to optimize the optimal consistent kernel, rather than directly constructing the optimal affinity graph, and indirect optimization can introduce kernel noise to continuously interfere with the clustering process. At the same time, although VCSD is effective in feature selection, it has not been applied to multi-core subspace clustering, and the application of statistical enhancement techniques is still a blank; deep subspace clustering faces the realistic limitations of high computational complexity, long training time and dependence on labeled data.

[0007] Therefore, how to construct a multi-core subspace clustering framework that integrates statistical information, has low computational complexity and can avoid the interference of kernel noise has become a key and difficult point in the field of high-dimensional data analysis. SUMMARY

[0008] The present application provides a multi-core subspace clustering method based on variance-covariance subspace distance to effectively solve the problem of capturing high-order feature correlation through matrix method and effectively improve the clustering accuracy and efficiency.

[0009] To achieve the above-mentioned purpose, the present application provides a multi-core subspace clustering method based on variance-covariance subspace distance, comprising: S1, obtaining a high-dimensional data matrix , constructing a VCSD kernel matrix and a multi-core pool: calculating the variance-covariance matrix and second-order statistical information of the data, defining the preliminary VCSD kernel value between samples and performing exponential transformation and normalization processing to form the VCSD kernel matrix, and simultaneously constructing a multi-core pool containing r base kernel matrices by fusing linear kernel, polynomial kernel and Gaussian kernel; S2, constructing a local affinity feature graph based on the feature matrix after non-negative matrix factorization kernel mapping : taking the base kernel matrix in the multi-core pool as the input, constructing an optimization objective function, and solving to obtain a local affinity feature graph with block diagonal structure under the constraints of non-negativity, symmetry and row randomness ; S3, capturing high-order feature correlation through matrix method: introducing representation matrix , similarity matrix and global consistency matrix Z, constructing a comprehensive objective function containing reconstruction error, local consistency, global consistency and VCSD regularization term, and converting the original tensor optimization to matrix operation; S4. Solve the objective function using an alternating optimization strategy: update while keeping other variables constant. G^(v) and Z reduce computational complexity and generate the target affinity graph; S5. Perform spectral clustering on the target affinity map and output the final clustering results.

[0010] Preferably, in S1, the specific steps for defining the initial VCSD kernel value between samples and performing exponential transformation for normalization include: S11. Calculate any two samples and Difference vector - The initial VCSD core value is defined as follows: ; In the formula, It is a variance-covariance matrix. This is second-order statistical information. ∈[0,1] represents the parameters that balance first-order and second-order statistical information. For two samples and The difference vector; S12. Perform an exponential transformation and normalization on the initial kernel value to obtain the positive definite VCSD kernel matrix: .

[0011] Preferably, in S1, the r basic kernel matrices included in the multi-core pool are: VCSD kernel matrix, linear kernel matrix, polynomial kernel matrices of different degrees, and Gaussian kernel matrices of different bandwidths.

[0012] Preferably, in S2, the optimization objective function and constraints of the local affinity feature map are as follows: ; ; In the formula, For the first A basis kernel matrix, where Tr represents the trace operation and 1 represents a vector of all 1s. For the first A local affinity feature map.

[0013] Preferably, the In the consistency index matrix The matrix is ​​an approximately symmetric projection matrix, and satisfies the following conditions: .

[0014] Preferably, in S3, the comprehensive objective function is as follows: ; In the formula, is a VCSD base matrix, λ1, β, γ, ρ are regularization parameters, λ1∈[0.1, 10], {β∈[0.01, 1]}, γ∈[0.001, 0.1], ρ∈[0, 1], and ||·||F is the Frobenius norm.

[0015] Preferably, in S4, the specific steps of the alternating optimization strategy include: S41, fixing G^(v) and Z, solving the optimal solution of through eigenvalue decomposition; S42, fixing G^(v) and Z, updating by projecting G^(v) to the simplex constraint space; S43, fixing G^(v) and Z, updating the value of Z by weighted average calculation; S44, repeating S41-S43 until the objective function converges, and the number of iterations does not exceed 15 times. Preferably, in S4, the computational complexity is reduced from O(n 3 ) to O(rn 2 ), where r is the number of base kernels in the multi-core pool, and r∈[5, 20], and n is the number of samples. Preferably, in S5, the spectral clustering processing is based on eigenvalue decomposition of the target affinity graph, and the evaluation indexes of the clustering result include accuracy ACC, normalized mutual information NMI and purity Purity.

[0016] Preferably, the high-dimensional data matrix

[0017] is derived from a face recognition data set, including Yale, ORL, AR, Umist, orlraws10P and warpPIE10P.

[0018] Therefore, the present application proposes a multi-core subspace clustering method based on variance-covariance subspace distance, which has the following beneficial effects: (1) Enhance the discriminant ability of similarity measurement: fuse the variance-covariance statistical characteristics, provide a solid statistical basis for similarity evaluation, can effectively capture the variance distribution and covariance relationship between features of data, improve the problem of limited feature expression ability of traditional measurement method, and reduce the sensitivity of model to noise.

[0019] (2) Improve the calculation efficiency and scene adaptability: through the matrix high-order correlation capture mechanism, replace the explicit tensor operation with efficient matrix operation, significantly reduce the computational complexity, break through the bottleneck of large-scale data processing, and make the method more widely adapt to high-dimensional big data clustering scene. (2) Improve the calculation efficiency and scene adaptability: through the matrix high-order correlation capture mechanism, replace the explicit tensor operation with efficient matrix operation, significantly reduce the computational complexity, break through the bottleneck of large-scale data processing, and make the method more widely adapt to high-dimensional big data clustering scene.

[0020] (2) Improve the calculation efficiency and scene adaptability: through the matrix high-order correlation capture mechanism, replace the explicit tensor operation with efficient matrix operation, significantly reduce the computational complexity, break through the bottleneck of large-scale data processing, and make the method more widely adapt to high-dimensional big data clustering scene.

[0021] ​(3) Enhance the robustness of clustering process: based on non-negative matrix factorization to construct the local affinity feature map with block diagonal structure, avoid the traditional method through optimization of consistent kernel indirect update affinity map path, from the root to reduce the continuous interference of kernel noise on clustering process, improve the reliability of clustering results.

[0022] The technical solutions of the present application will be further described in detail below by means of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0023] Fig. 1 is the system architecture diagram of the multi-kernel subspace clustering method based on variance-covariance subspace distance of the present application; Fig. 2 is the matrix high-order correlation capture mechanism schematic diagram of the multi-kernel subspace clustering method based on variance-covariance subspace distance of the present application. DETAILED DESCRIPTION

[0024] In order to make the technical solutions, advantages and purposes of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below. The described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the described embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the present application.

[0025] Unless otherwise defined, the technical terms or scientific terms used in the present application shall be understood as the usual meaning understood by those skilled in the art in the field of the present application.

[0026] As Figs. 1-2 shown, the multi-kernel subspace clustering method based on variance-covariance subspace distance provided by the present application comprises: S1, obtaining a high-dimensional data matrix , constructing VCSD kernel matrix and multi-kernel pool: calculating the variance-covariance matrix and second-order statistical information of the data, defining the preliminary VCSD kernel value between samples and performing exponential transformation and normalization processing to form the VCSD kernel matrix, and simultaneously fusing linear kernel, polynomial kernel and Gaussian kernel to construct a multi-kernel pool containing r base kernel matrices; the r base kernel matrices contained in the multi-kernel pool are specifically: VCSD kernel matrix, linear kernel matrix, polynomial kernel matrix of different degrees and Gaussian kernel matrix of different bandwidths. In S1, the specific steps of defining the preliminary VCSD kernel value between samples and performing exponential transformation and normalization processing include: S11, calculating the difference vector and between any two samples - The initial VCSD core value is defined as follows: ; In the formula, It is a variance-covariance matrix. This is second-order statistical information. ∈[0,1] represents the parameters that balance first-order and second-order statistical information. For two samples and The difference vector; S12. Perform an exponential transformation and normalization on the initial kernel value to obtain the positive definite VCSD kernel matrix: ; High-dimensional data matrix It comes from face recognition datasets, including Yale, ORL, AR, Umist, orlraws10P, and warpPIE10P; S2, Feature matrix after nonnegative matrix factorization kernel mapping Furthermore, a local affinity feature map is constructed: using the base kernel matrix in the multi-kernel pool as input, an optimization objective function is constructed, and under constraints of nonnegativity, symmetry, and row randomness, a local affinity feature map with a block diagonal structure is obtained by solving the problem. The specific objective function and constraints for optimizing the local affinity feature map are as follows: ; ; In the formula, For the first A basis kernel matrix, where Tr represents the trace operation and 1 represents a vector of all 1s. For the first A local affinity feature map, in which... In the consistency index matrix The matrix is ​​an approximately symmetric projection matrix, and satisfies the following conditions: ; S3. Capturing high-order feature correlations through matrix methods: Introducing representation matrices Similarity matrix Together with the global consistency matrix Z, we construct a comprehensive objective function that includes reconstruction error, local consistency, global consistency, and VCSD regularization terms, transforming the original tensor optimization into matrix operations. The comprehensive objective function is as follows: ; In the formula, is the VCSD base matrix, λ1, β, γ, ρ are regularization parameters, λ1∈[0.1, 10], {β∈[0.01, 1]}, γ∈[0.001, 0.1], ρ∈[0, 1], and ‖·‖F is the Frobenius norm; S4, solving the objective function by using an alternating optimization strategy: updating , G^(v) and Z while fixing other variables, reducing the computational complexity, generating the target affinity graph; the computational complexity is reduced from O(n 3 ) to O(rn 2 ), wherein r is the number of base kernels in the multi-core pool, and r∈[5, 20], and n is the sample quantity; In S4, the specific steps of the alternating optimization strategy include: S41, fixing G^(v) and Z, and solving the optimal solution of by eigenvalue decomposition; S42, fixing and Z, and updating G^(v) by projecting it into a simplex constraint space; S43, fixing and G^(v), and calculating the updated value of Z by weighted average; S44, repeating S41-S43 until the objective function converges, and the number of convergence iterations is not more than 15 times; S5, performing spectral clustering processing on the target affinity graph, and outputting the final clustering result.

[0027] In S5, the spectral clustering processing is based on eigenvalue decomposition of the target affinity graph, and the evaluation indexes of the clustering result include accuracy ACC, normalized mutual information NMI and purity Purity.

[0028] Embodiment one, as shown in Figs. 1-2 , the multi-core subspace clustering method based on variance-covariance subspace distance provided by the application is tested in actual application on the face recognition data sets ORL and Umist, and the clustering performance and calculation efficiency are verified.

[0029] The specific steps are as follows: S1, data preparation and VCSD kernel matrix construction: Load the ORL and Umist face recognition data sets, wherein the ORL contains 40 classes (10 images per person), the Umist contains 20 classes (1-20 images per person), normalize the size of all images to 32x32 pixels and flatten them into 1024-dimensional feature vectors, and form a high-dimensional data matrix (n=400 in ORL, n=564 in Umist); Calculate the variance-covariance matrix of the data and its second order statistics , set the balance parameter = 0.5, calculate the preliminary VCSD kernel value between samples according to the formula , and then perform exponential transformation and normalization according to the formula to generate the VCSD kernel matrix; Meanwhile, linear kernel, quadratic polynomial kernel, cubic polynomial kernel, Gaussian kernel with bandwidth 0.1, and Gaussian kernel with bandwidth 0.5 are fused to construct a multi-kernel pool r = 6 containing 6 base kernel matrices.

[0030] S2, local affinity feature map learning: Taking the 6 base kernel matrices in the multi-kernel pool as input, based on the non-negative matrix factorization theory, an optimization objective function is constructed according to the formula under the constraints of non-negativity ≥ 0, symmetry = , row randomness 1=1, 6 local affinity feature maps with block diagonal structure are obtained ; Among them, as an approximate symmetric projection matrix of the consistency class indicator matrix satisfies , which directly avoids kernel noise interference.

[0031] S3, matrix high-order correlation capture: Introducing the representation matrix , the similarity matrix and the global consistency matrix Z, according to the formula: ; An integrated objective function is constructed, including reconstruction error term, local consistency term =1.0, global consistency term =0.2, regularization term γ=0.01 and VCSD regularization term =0.5, , which converts the original tensor optimization problem into matrix operation.

[0032] S4, alternating optimization and target affinity map generation: An alternating optimization strategy is used to solve the objective function, fixing and Z, and obtaining the optimal solution through eigenvalue decomposition; fixing and Z, updating by projecting it into the simplex constraint space; fixing and , updating Z by weighted average; after 12 iterations, the objective function converges, and the computational complexity is O(n3 ) to O (6n 2 ), and generate the final target affinity graph.

[0033] S5, spectral clustering and performance evaluation: Eigenvalue decomposition is performed on the target affinity graph, the first k eigenvectors are taken, k is the number of categories, K-means clustering is performed, the clustering result is output, and the accuracy ACC, normalized mutual information NMI and purity Purity are used to evaluate the performance.

[0034] The results show that: on the Umist data set, the three indicators all reach 1.0, that is, perfect clustering; on the ORL data set, ACC=0.96, NMI=0.92, Purity=0.95, the accuracy is improved by 8.3% compared with the PKGT method, and the calculation efficiency is improved by 42%. At the same time, the parameter sensitivity test shows that when ∈[0.5,2.0] and β∈[0.1,0.3], the model performance fluctuates less than 3%, and the robustness is excellent.

[0035] Therefore, the application provides a multi-kernel subspace clustering method based on variance-covariance subspace distance, which provides solid statistical support for similarity measurement by fusing variance-covariance subspace distance, effectively captures the variance distribution of data and the covariance relationship between features, and improves the discriminant ability of clustering; with the help of matrix high-order correlation capture mechanism, explicit tensor operation is replaced by efficient matrix operation, which significantly reduces the calculation complexity and adapts to large-scale high-dimensional data processing scenarios; based on non-negative matrix decomposition, a local affinity feature graph with block diagonal structure is directly constructed, which avoids the kernel noise interference caused by indirectly updating the affinity graph by optimizing the consistent kernel in the traditional method, enhances the model robustness, and maintains stable performance in a wide parameter range, has good adaptability to different types of high-dimensional data, and can be widely applied to face recognition, image clustering and other practical tasks.

[0036] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the application and not to limit them, although the application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that: it can still modify or equivalently replace the technical solutions of the application, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the application.

Claims

1. A multi-kernel subspace clustering method based on variance-covariance subspace distance, characterized in that, include: S1. Obtain the high-dimensional data matrix Constructing the VCSD kernel matrix and multi-core pool: Calculating the variance-covariance matrix of the data. and its second-order statistical information Define the initial VCSD kernel value between samples and perform exponential transformation and normalization to form the VCSD kernel matrix. At the same time, integrate linear kernel, polynomial kernel and Gaussian kernel to construct a multi-kernel pool containing r basic kernel matrices. S2, Feature matrix after nonnegative matrix factorization kernel mapping Furthermore, a local affinity feature map is constructed: using the base kernel matrix in the multi-kernel pool as input, an optimization objective function is constructed, and under constraints of nonnegativity, symmetry, and row randomness, a local affinity feature map with a block diagonal structure is obtained by solving the problem. ; S3. Capturing high-order feature correlations through matrix methods: Introducing representation matrices Similarity matrix Together with the global consistency matrix Z, we construct a comprehensive objective function that includes reconstruction error, local consistency, global consistency, and VCSD regularization terms, thus transforming the original tensor optimization into matrix operations. S4. Solve the objective function using an alternating optimization strategy: update while keeping other variables constant. G^(v) and Z reduce computational complexity and generate the target affinity graph; S5. Perform spectral clustering on the target affinity map and output the final clustering results.

2. The multi-kernel subspace clustering method based on variance-covariance subspace distance according to claim 1, characterized in that, In S1, the specific steps for defining the initial VCSD kernel value between samples and performing exponential transformation for normalization include: S11. Calculate any two samples and Difference vector - The initial VCSD core value is defined as follows: ； In the formula, It is a variance-covariance matrix. This is second-order statistical information. ∈[0,1] represents the parameters that balance first-order and second-order statistical information. For two samples and The difference vector; S12. Perform an exponential transformation and normalization on the initial kernel value to obtain the positive definite VCSD kernel matrix: 。 3. The multi-kernel subspace clustering method based on variance-covariance subspace distance according to claim 1, characterized in that, In S1, the r basic kernel matrices contained in the multi-core pool are: VCSD kernel matrix, linear kernel matrix, polynomial kernel matrices of different degrees, and Gaussian kernel matrices of different bandwidths.

4. The multi-kernel subspace clustering method based on variance-covariance subspace distance according to claim 1, characterized in that, In S2, the optimization objective function and constraints of the local affinity feature map are as follows: ； ； In the formula, For the first A basis kernel matrix, where Tr represents the trace operation and 1 represents a vector of all 1s. For the first A local affinity feature map.

5. The multi-kernel subspace clustering method based on variance-covariance subspace distance according to claim 4, characterized in that, The In the consistency index matrix The matrix is ​​an approximately symmetric projection matrix, and satisfies the following conditions: .

6. The multi-kernel subspace clustering method based on variance-covariance subspace distance according to claim 1, characterized in that, In S3, the comprehensive objective function is as follows: ； In the formula, Let be the VCSD fundamental matrix, λ1, β, γ, and ρ be regularization parameters, λ1∈[0.1,10], {β∈[0.01,1]}, γ∈[0.001,0.1], ρ∈[0,1], and ∥·∥F be the Frobenius norm.

7. The multi-kernel subspace clustering method based on variance-covariance subspace distance according to claim 1, characterized in that, In S4, the specific steps of the alternating optimization strategy include: S41. Fix G^(v) and Z, and solve using eigenvalue decomposition. The optimal solution; S42, Fixed And Z, project G^(v) onto the simplex constraint space for update; S43, Fixed And G^(v), calculate the updated value of Z by weighted average; S44. Repeat S41-S43 until the objective function converges, with the number of convergence iterations not exceeding 15.

8. The multi-kernel subspace clustering method based on variance-covariance subspace distance according to claim 1, characterized in that, In S4, the computational complexity is reduced from O(n 3 ) decreased to O(rn) 2 ), where r is the number of base kernels in the multi-kernel pool and r∈[5,20], and n is the number of samples.

9. The multi-kernel subspace clustering method based on variance-covariance subspace distance according to claim 1, characterized in that, In S5, spectral clustering is based on eigenvalue decomposition of the target affinity graph. The evaluation metrics for the clustering results include accuracy (ACC), normalized mutual information (NMI), and purity.

10. The multi-kernel subspace clustering method based on variance-covariance subspace distance according to claim 1, characterized in that, The high-dimensional data matrix The datasets are derived from face recognition datasets, including Yale, ORL, AR, Umist, orlraws10P, and warpPIE10P.

Images