A multi-view subspace clustering method and device

Abstract

A multi-view subspace clustering method and device can fully utilize the complementary information between multi-view data and ensure the clustering structure consistency of the self-representation matrix, which has a great improvement in clustering performance. The method includes: (1) obtaining the multi-view feature matrix #imgabs0# of the original data set; (2) for a given multi-view data X v , according to the low-rank self-representation subspace clustering method, the representation coefficient matrix Z of each visual data v Low-rank constraint and rank structure consistency constraint are performed respectively to construct a multi-view subspace clustering model based on low-rank matrix decomposition and rank structure consistency constraint; (3) For the constructed multi-view subspace clustering model, the alternating direction multiplier method is used to solve the optimization problem to obtain a multi-view self-representation coefficient matrix with consistent rank structure; (4) The multi-view self-representation coefficient matrix is ​​fused to obtain the correlation matrix W, and spectral clustering is performed on the correlation matrix W to obtain the final clustering result.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a multi-view subspace clustering method and a multi-view subspace clustering device. Background Art

[0002] Multi-view data refers to data features from multiple different perspectives that describe the same sample. Multi-view data is becoming increasingly common in many practical applications. For example, in video surveillance systems, multi-camera systems record human activities from multiple angles; internet news includes text, images, and audio and video; and for image data, different features can be used to describe images, such as SIFT features, LBP features, and HOG features. Multi-view data describes the same sample from different perspectives and can provide complementary information. Cluster analysis is an important data mining method, but traditional clustering methods fail to maximize the utilization of the complementary information in multi-view data. Directly applying single-view clustering methods to multi-view data does not yield good results. Therefore, multi-view subspace clustering has been proposed. Its goal is to classify multi-view samples that are approximately sampled from the same sub-low-dimensional space into the same cluster. This method integrates the complementary information from multi-view data while ensuring consistency as much as possible.

[0003] In recent years, various multi-view subspace clustering methods have been proposed. Because self-representative subspace clustering methods have a sound theoretical foundation and are robust to noise, some multi-view subspace clustering methods incorporate the self-representative nature of data into multi-view clustering. Based on the assumption that data is linearly separable, the so-called self-representative nature of data means that a data sample can be represented by a linear combination of other data samples from the same subspace (i.e., the same class). The complementarity principle and the consistency principle are two important principles in multi-view learning. The key to multi-view subspace clustering methods based on self-representation lies in how to utilize multi-view data features to learn the self-representative coefficient matrix that fuses multi-view information. For example, to better mine the complementary information of multi-views, Diversity-induced Multi-view Subspace Clustering (DiMSC) uses the Hilbert-Schmidt independence criterion to impose diversity constraints on the multi-view representation coefficient matrices. Low-rank Tensor Constrained Multi-view Subspace Clustering (LTMSC) employs low-rank tensor constraints to explore higher-order relationships between multi-view representation coefficient matrices. In addition to complementary information, mining the consistency of multi-view data is also important. Consistent and Specific Multi-view Subspace Clustering (CSMSC) encodes the consistency and complementarity information of each view into a common representation coefficient matrix and a view-specific representation coefficient matrix. In addition, some multi-view subspace clustering methods fuse multi-view information at the feature level. For example, Latent Multi-view Subspace Clustering (LMSC) assumes that the multi-view data comes from a latent feature space, and then applies the classic single-view Low Rank Representation (LRR) subspace clustering method on the latent features to obtain a common self-representation coefficient matrix.

[0004] While the clustering results of the aforementioned methods have been significantly improved, most multi-view self-representational subspace clustering methods assume that the multi-view data share the same self-representational coefficient matrix to explore consistent information in the multi-view data. In fact, this is unreasonable for multi-view clustering because data from different views should have view-specific self-representational properties, that is, the self-representational coefficient matrices should have different elements. However, a reasonable assumption should be that the self-representational coefficient matrices of data from different views have the same data distribution or clustering structure, rather than being completely element-wise identical. Summary of the Invention

[0005] In order to overcome the defects of the existing technology, the technical problem to be solved by the present invention is to provide a multi-view subspace clustering method, which can make full use of the complementary information between multi-view data and ensure the consistency of the clustering structure of the self-representation matrix, thereby greatly improving the clustering performance.

[0006] The technical solution of the present invention is: the multi-view subspace clustering method comprises the following steps:

[0007] (1) Obtain the multi-view feature matrix of the original data set

[0008] (2) For a given multi-view data X v , according to the low-rank self-representation subspace clustering method, the representation coefficient matrix Z of each visual data v Low-rank constraints and rank structure consistency constraints are performed respectively, and a multi-view subspace clustering model based on low-rank matrix decomposition and rank structure consistency constraints is constructed;

[0009] (3) For the constructed multi-view subspace clustering model, the alternating direction multiplier method is used to solve the optimization problem and obtain a multi-view self-representation coefficient matrix with consistent rank structure;

[0010] (4) The multi-view self-representation coefficient matrices are fused to obtain the correlation matrix W, and spectral clustering is performed on the correlation matrix W to obtain the final clustering result.

[0011] According to the low-rank self-representation subspace clustering method, the representation coefficient matrix Z of each visual data is v Low-rank constraints and rank structure consistency constraints are performed respectively, and a multi-view subspace clustering model based on low-rank matrix decomposition and rank structure consistency constraints is constructed. The designed model is solved based on the alternating direction multiplier method ADMM. Therefore, it can make full use of the complementary information between multi-view data and ensure the clustering structure consistency of the self-representation matrix, which greatly improves the clustering performance.

[0012] A multi-view subspace clustering device is also provided, comprising:

[0013] Acquisition module, which obtains the multi-view feature matrix of the original dataset

[0014] Building a module that, for a given multi-view data X v , according to the low-rank self-representation subspace clustering method, the representation coefficient matrix Z of each visual data v Low-rank constraints and rank structure consistency constraints are performed respectively, and a multi-view subspace clustering model based on low-rank matrix decomposition and rank structure consistency constraints is constructed;

[0015] A solution module, which uses the alternating direction multiplier method to solve the optimization problem of the constructed multi-view subspace clustering model and obtains a multi-view self-representation coefficient matrix with consistent rank structure;

[0016] The fusion module fuses the multi-view self-representation coefficient matrices to obtain the correlation matrix W, and performs spectral clustering on the correlation matrix W to obtain the final clustering result. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The figure shows the overall flow chart of the multi-view subspace clustering method according to the present invention. DETAILED DESCRIPTION

[0018] like Figure 1 As shown in Figure 2, this multi-view subspace clustering method includes the following steps:

[0019] (1) Obtain the multi-view feature matrix of the original data set

[0020] (2) For a given multi-view data X v , according to the low-rank self-representation subspace clustering method, the representation coefficient matrix Z of each visual data v Low-rank constraints and rank structure consistency constraints are performed respectively, and a multi-view subspace clustering model based on low-rank matrix decomposition and rank structure consistency constraints is constructed;

[0021] (3) For the constructed multi-view subspace clustering model, the alternating direction multiplier method is used to solve the optimization problem and obtain a multi-view self-representation coefficient matrix with consistent rank structure;

[0022] (4) The multi-view self-representation coefficient matrices are fused to obtain the correlation matrix W, and spectral clustering is performed on the correlation matrix W to obtain the final clustering result.

[0023] According to the low-rank self-representation subspace clustering method, the representation coefficient matrix Z of each visual data is v Low-rank constraints and rank structure consistency constraints are performed respectively, and a multi-view subspace clustering model based on low-rank matrix decomposition and rank structure consistency constraints is constructed. The designed model is solved based on the alternating direction multiplier method ADMM. Therefore, it can make full use of the complementary information between multi-view data and ensure the clustering structure consistency of the self-representation matrix, which greatly improves the clustering performance.

[0024] Preferably, in step (2), matrix decomposition is performed on the self-representation coefficient matrix of each view, and

[0025] Preferably, in step (1), for each sample in the original data set, different feature extractors are used to extract different types of data features, thereby forming a multi-view data feature matrix Where n is the number of samples in the dataset, V is the number of multi-views, and d v is the dimension of the v-th view data feature.

[0026] Preferably, in step (2), the multi-view subspace clustering model based on rank structure consistency is:

[0027]

[0028] xT v =X v Z v +E v ,rank(Z1)=…=rank(Z V )≤k,

[0029] where Z v ∈R n×n is the low-rank representation coefficient matrix of the v-th view, Fitting the noise of the v-th visual data feature, λ is the regularization parameter to balance the importance of low-rank terms and noise sparse terms, rank(Z1)=…=rank(Z V )≤k is the rank consistency structural constraint, which ensures the structural consistency of the multi-view self-representation coefficient matrix, k is the rank upper bound of the low-rank representation coefficient matrix, k<<n.

[0030] Preferably, in the matrix decomposition of step (2), ||Z v || * =||C|| * , rank(Z v )=rank(C) For any view v, the model that is easy to optimize and solve is:

[0031]

[0032] Among them, the view-specific representation coefficient matrix is ​​decomposed into the multiplication of three matrices, and the left and right matrices are orthogonal, and a common kernel matrix C is shared in the middle, which ensures the structural consistency of the self-representation coefficient matrix and can maximize the mining of complementary information to improve the clustering effect.

[0033] Preferably, in step (3), the alternating direction multiplier method is used to solve formula (2) to obtain a multi-view self-representation coefficient matrix with consistent rank structure. The augmented Lagrangian function of formula (2) is:

[0034]

[0035] in <A,B> represents the inner product of matrices A and B, μ is the penalty coefficient, is the Lagrange multiplier; all variables in the above augmented Lagrangian function are optimized alternately until convergence.

[0036] Preferably, in step (3), the step of alternately optimizing all variables in the augmented Lagrangian function until convergence is:

[0037] Fix other variables and update L v :

[0038]

[0039] Fix other variables and update R v :

[0040]

[0041] Fix other variables and update C:

[0042]

[0043] Fix other variables and update Z v :

[0044]

[0045] Fix other variables and update E v :

[0046]

[0047] Fix other variables and update the Lagrange multiplier and penalty coefficient:

[0048]

[0049] in

[0050] Repeat the above variable update steps until the convergence condition is reached: Get the view-specific representation coefficient matrix Z for the final fusion clustering v .

[0051] Preferably, in step (4), for the specific representation coefficient matrix Z v Perform fusion to obtain the final correlation matrix W and perform spectral clustering to obtain the clustering results:

[0052]

[0053] Then, the clustering result indicators, including ACC, NMI, and F-score, were calculated and analyzed.

[0054] A multi-view subspace clustering device is also provided, comprising:

[0055] Acquisition module, which obtains the multi-view feature matrix of the original dataset

[0056] Building a module that, for a given multi-view data X v , according to the low-rank self-representation subspace clustering method, the representation coefficient matrix Z of each visual data v Low-rank constraints and rank structure consistency constraints are performed respectively, and a multi-view subspace clustering model based on low-rank matrix decomposition and rank structure consistency constraints is constructed;

[0057] A solution module, which uses the alternating direction multiplier method to solve the optimization problem of the constructed multi-view subspace clustering model and obtains a multi-view self-representation coefficient matrix with consistent rank structure;

[0058] The fusion module fuses the multi-view self-representation coefficient matrices to obtain the correlation matrix W, and performs spectral clustering on the correlation matrix W to obtain the final clustering result.

[0059] Preferably, in the building module, matrix decomposition is performed on the self-representation coefficient matrix of each view, and

[0060] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of protection of the technical solution of the present invention.

Claims

1. A multi-view subspace clustering method, characterized by: This method is applied in the field of image processing to cluster multi-view data, where multi-view data refers to data features from multiple different perspectives describing the same sample. The method includes the following steps: (1) Obtain the multi-view feature matrix of the original data set (2) For a given multi-view data X v , according to the low-rank self-representation subspace clustering method, the representation coefficient matrix Z of each visual data v Low-rank constraints and rank structure consistency constraints are performed respectively, and a multi-view subspace clustering model based on low-rank matrix decomposition and rank structure consistency constraints is constructed; (3) For the constructed multi-view subspace clustering model, the alternating direction multiplier method is used to solve the optimization problem and obtain a multi-view self-representation coefficient matrix with consistent rank structure; (4) The multi-view self-representation coefficient matrix is ​​fused to obtain the correlation matrix W, and spectral clustering is performed on the correlation matrix W to obtain the final clustering result; In the step (2), matrix decomposition is performed on the self-representation coefficient matrix of each view. and In step (1), for each sample in the original data set, different feature extractors are used to extract different types of data features, thereby forming a multi-view data feature matrix v∈{1, 2, ..., V}, where n is the number of samples in the dataset, V is the number of multi-views, and d v is the dimension of the v-th view data feature; In step (2), the multi-view subspace clustering model based on rank structure consistency is: s.t.X v =X v Z v +E v ,rank(Z1)=…=rank(Z V )≤k, where Z v ∈R n×n is the low-rank representation coefficient matrix of the v-th view, Fitting the noise of the v-th visual data feature, λ is the regularization parameter to balance the importance of low-rank terms and noise sparse terms, rank(Z1)=…=rank(Z V )≤k is a rank consistency structural constraint to ensure the structural consistency of the multi-view self-representation coefficient matrix, k is the rank upper bound of the low-rank representation coefficient matrix, k<<n; in the matrix decomposition of step (2), ||Z v || * =||C|| * , rank(Z v )=rank(C) For any view v, the model that is easy to optimize and solve is: xT v =X v Z v +E v , Among them, the specific representation coefficient matrix is ​​decomposed into three matrix multiplications, and the left and right matrices are orthogonal, and a common core matrix C is shared in the middle, which ensures the structural consistency of the self-representation coefficient matrix and can maximize the mining of complementary information to improve the clustering effect; In step (3), the alternating direction multiplier method is used to solve formula (2) to obtain a multi-view self-representation coefficient matrix with consistent rank structure. The augmented Lagrangian function of formula (2) is: in <A,B> represents the inner product of matrices A and B, μ is the penalty coefficient, is the Lagrange multiplier; alternately optimize all variables in the above augmented Lagrangian function until convergence; In step (3), the steps of alternately optimizing all variables in the augmented Lagrangian function until convergence are: Fix other variables and update L v : Fix other variables and update R v : Fix other variables and update C: Fix other variables and update Z v : Fix other variables and update E v : Fix other variables and update the Lagrange multiplier and penalty coefficient: in Repeat the above variable update steps until the convergence condition is reached: Get the view-specific representation coefficient matrix Z for the final fusion clustering v ; In the step (4), for the specific representation coefficient matrix Z v Perform fusion to obtain the final correlation matrix W and perform spectral clustering to obtain the clustering results:

2. A multi-view subspace clustering device, configured to execute the method according to claim 1.

Images