Low-rank multi-view subspace clustering method based on sparse regularization

Abstract

The invention discloses a low-rank multi-view subspace clustering method based on sparse regularization. The low-rank multi-view subspace clustering method comprises the following steps: receiving a multi-view data set to be clustered; cleaning the data, checking whether missing values exist in the data or not, and then selecting to fill or delete the missing values according to conditions to obtain a standard data set; performing sparse subspace representation on the data; performing low-rank representation on the subspaces; constructing a public potential subinterval, and mapping data of different views to the public potential subinterval; constructing a similar matrix and performing spectral clustering on the similar matrix; according to the method, a common subspace is found, and potential structures of data in all view angles are optimally represented, so that the quality and the accuracy of a clustering result are improved.

Description

Technical Field

[0001] The invention relates to the technical field of machine learning and artificial intelligence, and in particular to a low-rank multi-view subspace clustering method based on sparse regularization. Background Art

[0002] Clustering plays an important role in machine learning and artificial intelligence for several reasons, supporting its use as a foundational technique for many processes and applications in these fields.

[0003] Multi-view subspace clustering (MVSC) is an advanced clustering technique that is particularly well suited for processing data that naturally arises from multiple sources or "views". This approach is based on the principle that different views of the data can provide complementary information that should be integrated when performing clustering. The goal of MVSC is to find a common subspace that best represents the underlying structure of the data across all views, thereby improving the quality and accuracy of the clustering results. By leveraging multiple views of the data, MVSC can achieve higher clustering accuracy than single-view clustering methods, especially when these views are complementary. Research has shown that integrating multiple views can make the clustering process more robust to noise and redundancy in individual views, because clean and informative parts of each view can be exploited.

[0004] Therefore, multi-view subspace clustering has been applied in various fields, such as image and video analysis, bioinformatics, and social network analysis. Among them, low-rank prior is a key technique to capture the global structure of data across multiple views while ensuring that the representation is compact and meaningful. The low-rank prior is based on the assumption that the data of all views are located in or close to a low-dimensional subspace, and this inherent structure can be exploited to improve clustering performance. Key contributions in the field of low-rank subspace methods include some notable works such as Latent Multi-view Subspace Clustering (LMSC), Multimodal Sparse and Low-rank Subspace Clustering (MLRSSC), Flexible Multi-view Subspace Clustering Representation Learning (FMR), and Dual Shared Specific Multi-view Subspace Clustering (DSS-MSC). These methods are based on the mature framework of low-rank representation and have demonstrated competitive clustering performance in empirical studies.

[0005] In particular, recent studies have introduced an efficient and effective method, known as promoted low-rank multi-view subspace clustering (FLMSC). They factorized the view-specific representation matrix into two small factor matrices, namely, the orthogonal dictionary and the latent representation, which can fully explore the latent subspace structure of multiple views. However, there is still a problem with this approach. They focused on using the F-norm to measure the error between the observed data and the reconstructed data. However, it is well known that the F-norm is sensitive to outliers because it sums the values ​​after squaring them, which may increase the impact proportionally on larger differences.

[0006] “A study of graph-based system for multi-view clustering” proposes a general system for multi-view clustering, called graph-based system (GBS). The impact of different graph metrics on GBS is discussed, and a new graph construction method based on manifold learning and sparse representation is introduced. In order to address the limitations of existing graph-based multi-view clustering methods, a novel multi-view clustering method based on GBS is proposed. The method can automatically weight the graph constructed for each view to learn a unified graph, which gives the final clustering without any additional clustering steps. Extensive experiments on toy data and real-world data demonstrate the superior performance of the method.

[0007] “Auto-weighted multi-view clustering via kernelized graph learning” proposes a novel multi-view learning model that simultaneously performs the multi-view clustering task and learns similarity relations in kernel space. If there are c clusters, the optimal graph obtained can be directly partitioned into exactly c connected components. In addition, the proposed model can automatically assign the best weights to each view without adding extra parameters as in previous methods. Since the performance is usually sensitive to the input kernel matrix, the model is further extended with multi-kernel learning capabilities. With this joint model, the three subtasks of building the most accurate similarity graph, automatically assigning ideal weights to each view, and finding the clustering indicator matrix can be accomplished simultaneously. Through this joint learning, each subtask can enhance each other.

[0008] Diversity-based multi-view low-rank sparse subspace clustering algorithm: This method introduces sparse low-rank constraints into the multi-view subspace clustering model, and uses the diversity regularization term to capture the inherent differences in each view, thereby obtaining a comprehensive and discriminative feature space. The final cluster assignment is implemented using the spectral clustering algorithm. The algorithm is committed to integrating information from different views and describing the data more comprehensively by learning subspaces from multiple views. Each view provides information about different aspects of the data, and the consideration of diversity helps to understand the complex structure of the data more comprehensively. By using a low-rank matrix to represent the data of each view, the algorithm attempts to capture the potential structure in the data. Low-rank representation helps to reduce the dimensionality of the data and improve the algorithm's understanding of the overall structure of the data. Sparsity is introduced so that in the data representation of each view, only a few elements are critical to describing the important structure of the data. This helps to improve the algorithm's robustness to noise and reduce the interference of redundant information. The algorithm fuses low-rank sparse representations from different views to form a more comprehensive data representation. This fusion integrates the unique information provided by different views by considering diversity, which helps to better reflect the overall structure of the data. By learning the subspace of the integrated representation, the algorithm is able to model the data in a lower dimensional space. This helps improve clustering performance, especially when dealing with high-dimensional data. The algorithm is usually accompanied by an optimization process, whose goal is to minimize the reconstruction error of the data in the subspace while satisfying the constraints of low rank and sparsity. Compared with other methods, this diversity-based multi-view low-rank sparse subspace clustering algorithm improves robustness to noise and redundancy through low rank and sparsity constraints while comprehensively utilizing information from multiple views.

[0009] Diversity-based Multi-view Low-rank Sparse Subspace Clustering Algorithm: A clustering method for processing multi-view data. Multi-view data refers to multiple data sets from different sources or different feature extraction methods. In the real world, we often face situations where each view provides information about different aspects of the data. This algorithm aims to overcome the shortcomings of single-view clustering and integrate information from multiple views to improve clustering performance. The algorithm can effectively integrate information from different views and use the unique perspective provided by each view to more comprehensively describe the data. Representing the data of each view through a low-rank matrix helps capture the potential structure in the data. Low-rank representation helps reduce dimensionality, reduce redundant information, and improve the robustness of the algorithm. Sparsity is introduced so that in the data representation of each view, only a few elements are critical to describing the important structure of the data. This helps improve the robustness of the algorithm to noise. Sparsity is introduced so that in the data representation of each view, only a few elements are critical to describing the important structure of the data. This helps improve the robustness of the algorithm to noise. The algorithm takes into account the diversity between different views and improves the ability to understand the overall structure of the data by fusing information from each view. By integrating multi-view information and adopting low-rank sparse subspace learning, the algorithm can generally improve the clustering performance, especially when dealing with complex, high-dimensional datasets.

[0010] Self-weighted multi-view subspace clustering based on low-rank sparse constraints: is a clustering method for processing multi-view data. Multi-view data refers to data from multiple views or feature sets, each of which provides information about different aspects of the data. The algorithm aims to perform self-weighted multi-view subspace clustering through low-rank and sparse constraints to improve the clustering performance of multi-view data. The algorithm can effectively model multi-view data and integrate the information provided by each view. The data of each view can be regarded as a subspace, and the goal of the algorithm is to learn a global low-rank and sparse representation to better reveal the overall structure of the data. By introducing low-rank constraints, the algorithm attempts to find a low-rank representation to reflect the potential structure in the data. This helps to reduce the dimensionality of the data and better capture the essential characteristics of the data. Sparse constraints are introduced so that only information that is critical to clustering is retained in the representation. By making most of the elements in the representation zero, sparsity helps to reduce the impact of noise and improve the robustness of the algorithm. The algorithm usually learns the self-weight of each view, that is, determines the weight of each view's contribution to the overall clustering. This helps to adapt to the importance differences of different views in the data. By learning a low-rank and sparse self-weighted multi-view representation, the algorithm projects the data into a subspace and performs clustering in the subspace to divide the data into different categories. Due to the constraints of low rank and sparseness, the algorithm has a certain resistance to noise and redundant information, which improves the robustness of clustering.

[0011] Clustering is a foundational technique in machine learning and artificial intelligence, and is essential for understanding and analyzing data from multiple sources or "views". Multi-view subspace clustering (MVSC) has emerged as an advanced method that aims to integrate these different views to discover common subspaces, thereby improving the accuracy and robustness of clustering results. Most existing methods still have two problems. First, they mainly focus on exploiting the low-rank consistency between multiple views, but often ignore the low-rank structure within each view. Second, they often encounter expensive time overheads, usually due to the cost of matrix inversion and singular value decomposition (SVD) at each iteration. The main disadvantages of traditional techniques are as follows:

[0012] (1) Differences in view representation capabilities: Since each view has different capabilities in representing data, it is more reasonable to assign different weights to each view. Most existing multi-view clustering methods do not fully utilize the diversity of information views, making the affinity matrix unclear and inaccurate, and unable to accurately describe the potential structure of multi-view data, resulting in poor clustering results.

[0013] (2) High-dimensional data processing: For high-dimensional data, feature distribution is usually sparser, and the traditional Euclidean distance method for similarity measurement is not applicable at all. To solve this problem, some studies have used low-rank constraint methods to learn low-dimensional subspaces from high-dimensional data, which not only reduces the computational complexity but also improves the robustness to noisy data.

[0014] (3) Extraction of high-order information: When exploring high-order representations of data, there is a lack of constraints on the local geometric structure of the data, a lack of consideration of high-order relationships in the encoded similarity tensor, and a lack of extraction of high-order relationships between point pairs. Summary of the invention

[0015] To solve the problems existing in the prior art, the purpose of the present invention is to provide a low-rank multi-view subspace clustering method based on sparse regularization, which aims to find a common subspace that best represents the potential structure of the data at all perspectives, thereby improving the quality and accuracy of the clustering results.

[0016] To achieve the above object, the technical solution adopted by the present invention is: a low-rank multi-view subspace clustering method based on sparse regularization, comprising the following steps:

[0017] Step 1: Receive a multi-view dataset to be clustered: collect views and construct a multi-view dataset based on sample data points in each view;

[0018] Step 2: Clean the data, check whether there are missing values ​​in the data, and then choose to fill or delete the missing values ​​according to the situation to obtain a standard data set. Collect view sample data bases to construct a sample data set X for each view, satisfying a set of multi-view data samples X = [X (1)T ,...,X (V)T ] T ,in represents the vth view;

[0019] Step 3: Perform a sparse subspace representation on the data: only consider the low-rank structure within each view. The self-expression form of multi-view data is as follows:

[0020]

[0021] Among them, Z (v) represents the subspace of the vth view, E (v) represents the error matrix of the vth view, A (v) is the given dictionary;

[0022] Step 4: Perform low-rank representation on the subspace: Extend the low-rank representation to multiple views to obtain:

[0023]

[0024] Step 5: construct a common potential subinterval and map the data of different views to the common potential subinterval;

[0025] Step 6: Construct a similarity matrix and perform spectral clustering on the similarity matrix.

[0026] As a further improvement of the present invention, the step 5 is specifically as follows:

[0027] Step 5.1: Introduce the following consistency items:

[0028]

[0029] Step 5.2: The formulation of the low-rank multi-view subspace clustering FLMSC promoted by the consistency term is as follows:

[0030]

[0031]

[0032] Among them, λ 1 and λ 2 are two positive equilibrium parameters;

[0033] Step 5.3, replace the F norm in the above formula with the L1 norm to obtain low-rank multi-view subspace clustering based on sparse regularization:

[0034]

[0035]

[0036] Step 5.4: Based on the alternating direction multiplier method, use the optimization algorithm to solve the minimization problem and construct the corresponding augmented Lagrangian function as follows:

[0037]

[0038]

[0039] Step 5.5: Update variables

[0040] When other variables are fixed, solve for variable Z (v) The following minimization subproblem:

[0041]

[0042] The equivalent rewrite is:

[0043]

[0044] By introducing the auxiliary variable H and omitting the superscript for simplification, we get the constrained optimization problem:

[0045]

[0046] Step 5.6: Update variables

[0047] When other variables are fixed, solve the problem about variable L (v) The following minimization subproblem:

[0048]

[0049] The constrained optimization problem is further simplified to the following form:

[0050]

[0051] in

[0052] Step 5.7. Update variables

[0053] When other variables are fixed, solve the problem about variable C. (v) The following minimization subproblem:

[0054]

[0055] Further simplified to the following form:

[0056]

[0057] in:

[0058]

[0059] As a further improvement of the present invention, a semi-quadratic splitting algorithm is used to solve the constrained optimization problem in step 5.5:

[0060]

[0061] where η is a penalty parameter that forces X-XZ and H to be close to the same fixed point; subsequently, H and Z are updated according to the following two subproblems.

[0062] Sub-question 1:

[0063]

[0064] Sub-question 2:

[0065]

[0066] For subproblem 1, we introduce the concept of contraction operator. For real number x and threshold τ, contraction operator Defined as: It means that when the absolute value of x is less than τ, then will become 0, otherwise x will be subtracted by τ; extending this to the matrix and applying it to each element, the solution to subproblem 1 is given by:

[0067]

[0068] For sub-problem 2, the following problem is obtained by scaling:

[0069]

[0070] The solution is obtained by setting the derivative of subproblem 2 to zero:

[0071]

[0072] Then:

[0073]

[0074] make get:

[0075]

[0076] As a further improvement of the present invention, the method for solving the constrained optimization problem in step 5.6 is as follows:

[0077] For any matrix G∈R m×n , assuming that the singular value decomposition (SVD) of the matrix G is UΛV T , consider the following constraints:

[0078]

[0079] Its closed-form solution is as follows:

[0080] Y=UV T

[0081] Since G=UΛV T , U T U=I and V T V=I, then:

[0082] Tr(Y T G)

[0083] =Tr(Y T UΛV T )

[0084] =Tr(ΛV T Y T U)

[0085] ≤Tr(ΛI)

[0086] where δ i is the i-th singular value of the matrix G; obviously, when ΛP=ΛI, that is, V T Y T When U = I, the above equation holds; therefore, the closed-form solution is Y = UV T ;

[0087] By (v) Perform SVD decomposition, that is The solution to the problem is as follows:

[0088]

[0089] As a further improvement of the present invention, in step 5.7, the variable C is solved by the singular value threshold method. (v) The minimization subproblem is:

[0090] For a given matrix F and a positive parameter τ>0, the problem The optimal solution is given by given;

[0091] in is the singular value decomposition of the matrix F; at the same time, Θτ (·) represents the singular value threshold operator; the singular value threshold operator is defined as follows:

[0092] Θ τ (∑ F )=max(0,∑ F -τ)+min(0,∑ F +τ)

[0093] By setting Variable C (v) The closed-form solution of is as follows:

[0094]

[0095] express The singular value decomposition of .

[0096] As a further improvement of the present invention, in step 6, the final affinity matrix S is obtained: and

[0097] The present invention is a multi-view clustering method for processing data from multiple sources or "views". By integrating the complementary information provided by different viewpoints, the method aims to find a common subspace that best represents the latent structure of the data at all views, thereby improving the quality and accuracy of the clustering results. The method exploits data from multiple views and can achieve higher clustering accuracy than single-view clustering methods, especially when the viewpoints are complementary. The method can also make the clustering process more robust to noise and redundancy because it can exploit the clean and informative parts of each viewpoint.

[0098] The beneficial effects of the present invention are:

[0099] Multi-view subspace clustering aims to fuse information from different data views to improve clustering accuracy. Low-rank representation is usually used to capture the potential structure of the data in order to better adapt to the differences and correlations between different views. However, since it relies on the Frobenius norm for error measurement, the algorithm is sensitive to outliers and noise in the data. To address this problem, this paper proposes a low-rank multi-view subspace clustering (LMSC-DL) method based on sparse regularization, which is optimized by sparse regularization and alternating direction multiplier (ADMM) algorithm. Sparse regularization has two advantages: (i) Sparse regularization can promote the sparsity of the solution. This means that they encourage the model to use fewer features or parameters by driving the coefficients of less important features to zero. This is particularly useful in cases where feature selection and model interpretability are important because it highlights which features are most relevant to prediction. (ii) Sparse regularization is very effective in preventing overfitting, especially in high-dimensional settings where the number of features greatly exceeds the number of observations. By encouraging the model to focus on fewer variables, it reduces the complexity of the model and enhances its ability to fit noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 A flowchart of a subspace clustering method in an embodiment of the present invention;

[0101] Figure 2 This is the overall process of the embodiment of the present invention;

[0102] Figure 3 Schematic diagram of AR values ​​under different noise levels in an embodiment of the present invention;

[0103] Figure 4 Schematic diagram of F-score values ​​under different noise levels in an embodiment of the present invention;

[0104] Figure 5 Schematic diagram of MINI values ​​under different noise levels in an embodiment of the present invention

[0105] Figure 6 Schematic diagram of ACC values ​​under different noise levels in an embodiment of the present invention;

[0106] Figure 7 Schematic diagram of Reuter data set analysis in an embodiment of the present invention.

[0107] Figure 8 Schematic diagram of VIS / NIRBBC data set analysis in an embodiment of the present invention. DETAILED DESCRIPTION

[0108] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0109] Example

[0110] like Figure 1 and Figure 2 As shown, a low-rank multi-view subspace clustering method based on sparse regularization includes: receiving a multi-view data set to be clustered; performing low-rank and sparse self-representation on data features to obtain subspaces of each view; combining each view to construct a potential complete subspace; constructing an affinity matrix through the potential complete subspace; performing spectral clustering on the affinity matrix; specifically including:

[0111] Receiving a multi-view dataset to be clustered: collecting views, and constructing a multi-view dataset according to sample data points in each view.

[0112] Clean the data, check if there are any missing values ​​in the data, and then choose to fill or delete these missing values ​​as appropriate. Find and delete duplicate records in the data set to avoid unnecessary impact on the analysis. Identify and handle outliers, which may have an adverse effect on the model. Make sure the data is on the same scale so that the model can better understand them. Convert the data to the correct data type, such as converting text data to numerical data. If some columns are meaningless for analysis, consider deleting them. Check if there are any unreasonable characters or symbols in the data and handle them appropriately. Obtain a standard data set and collect view sample data. The base station can construct a sample data set X for each view, satisfying a set of multi-view data samples x = [x (1)T , …, X (V)T ] T ,in represents the vth view;

[0113] The data is represented in a sparse subspace. Further extending LRR to multiple views and considering the consistency of all views, the subspace representations of different views should be shared consistent subspaces. If only the low-rank structure within each view is considered, the self-expression of multi-view data can be written as:

[0114]

[0115] Among them, Z (v) ∈R n×n represents the subspace of the vth view, E (v) ∈R d×n represents the error matrix of the vth view, A (v) ∈R d×n A dictionary representing the v-th view.

[0116] Low-rank representation of subspace: Low-rank representation can reduce high-dimensional data to a subspace of lower dimensionality. This helps to reduce the complexity of the data, remove redundant information, and improve the efficiency of the clustering algorithm. In addition, low-rank representation can also reduce the impact of noise, making the clustering results more accurate. Low-rank representation can help us capture the potential structure of the data. By mapping the data to a low-rank subspace, we can better understand the relationship between the data, thereby better adapting to the differences and complementarities between different views. In multi-view clustering, we usually hope that there is a certain consistency between different views. Low-rank representation can help us find potential features shared between different views, thereby enhancing consistency across views. Further extending LRR to multi-view, we obtain the following model:

[0117]

[0118]

[0119] And get the following equivalent form:

[0120]

[0121] Multi-view clustering aims to reveal the low-rank structure inside each view. However, the existing multi-view low-rank representation (LRR) form has a very high computational cost in each iteration, which is the cubic power of the data samples. In order to improve its scalability in solving large-scale practical problems, the idea of ​​low-rank matrix decomposition is introduced.

[0122] Constructing common latent subintervals: When exploring multi-view clustering, it is crucial to construct common latent subintervals. These subintervals can capture the shared information between different views, thereby improving the accuracy of clustering. By mapping data from different views to these common subintervals, we can better understand the latent structure of the data and better adapt to the differences and complementarities between different views.

[0123] For any matrix X∈R m×n , where rank(X) = r ≤ d, can be decomposed into two smaller matrices U∈R m×d and V∈R d×n , so that X = UV. If the columns of the matrix U are orthogonal, that is, U T U=I, then the following properties apply:

[0124] ||X|| * =||V|| *

[0125] Proof: Since X = UV mesh U T U=I, then Decompose the view subspace representation into two small matrices as follows:

[0126]

[0127] Among them, L (v) ∈R n×γ represents the orthogonal dictionary (γ<<n) in the υth view, whose columns are orthogonal. In addition, represents the potential representation in the v-th view, which can effectively capture the (v) The new multi-view LRR formula can be expressed as:

[0128]

[0129] satisfy

[0130] The above formula can effectively improve the clustering performance, but it still cannot find the consistent subspace in different views. To this end, let X∈R m×n and Y∈R m×n are two matrices with the same dimensions, which can be further decomposed into smaller matrices: X = UV and Y = WB, where U, W∈R m×d , V, B∈R d×n . If U, W are column orthogonal matrices, and ||V|| * =||B|| * If it holds, then it has the following properties:

[0131] ||X|| * =||Y|| *

[0132] The above formula reveals that the structural consistency between different views can be measured by the potential representation in each view. Inspired by this, the following consistency term is introduced:

[0133]

[0134] The above consistency term effectively measures the average consistency of different latent representations. The formulation of the promoted low-rank multi-view subspace clustering FLMSC can be realized using the consistency term as follows:

[0135]

[0136]

[0137] Here, λ 1 and λ 2 are two positive equilibrium parameters.

[0138] Replacing the F norm in the above formula with the L1 norm, we get the low-rank multi-view subspace clustering based on sparse regularization (LMSC-DL):

[0139]

[0140]

[0141] Based on the framework of alternating direction multiplier method ADMM, an effective optimization algorithm is proposed to solve the above minimization problem. First, the corresponding augmented Lagrangian function is constructed as follows:

[0142]

[0143]

[0144] in Represents Lagrange multipliers, and μ represents penalty parameters. It is not easy to optimize all variables at the same time. Therefore, this embodiment adopts an iterative optimization scheme to update variables one by one. Alternating Direction Method of Multipliers (ADMM) is an iterative algorithm for solving optimization problems. Its basic principle is to decompose the original optimization problem into multiple sub-problems and gradually approach the optimal solution by alternatingly updating variables and Lagrange multipliers. In each iteration, ADMM divides the problem into two parts: a sub-problem solved according to the current variables and Lagrange multipliers, and a step of updating variables and Lagrange multipliers. This division makes the solution of each sub-problem simpler and more efficient. By alternating these two steps, ADMM finds a balanced solution so that both the objective function and the constraints are satisfied. The ADMM algorithm is widely used in various optimization problems, especially in problems with structured or distributed characteristics, because it can decompose the complexity of the problem into more easily tractable sub-problems, thereby improving the solution efficiency and convergence.

[0145] Update variables

[0146] When other variables are fixed, we can solve for the variable Z (v) The following minimization subproblem:

[0147]

[0148] This can be equivalently rewritten as:

[0149]

[0150] By introducing the auxiliary variable H and omitting the superscript for simplification, we have:

[0151]

[0152] This is a constrained optimization problem. The Half Quadratic Splitting (HQS) algorithm is used because of its simplicity and fast convergence. It is then solved by minimizing:

[0153]

[0154] where η is a penalty parameter that forces X-XZ and H to be close to the same fixed point. Subsequently, H and Z can be updated according to the following two subproblems.

[0155] Sub-question 1:

[0156]

[0157] Sub-question 2:

[0158]

[0159] In order to solve the first sub-problem, we need to introduce the concept of shrinkage operator. For a real number x and a threshold τ, the shrinkage operator Defined as: It means that when the absolute value of x is less than τ, then will become 0, otherwise x will be subtracted by τ. Extending this to matrices, apply it to each element. It is easy to prove that the solution to the above subproblem can be given by:

[0160]

[0161] For the second sub-problem, by scaling we can get the following problem:

[0162]

[0163] The solution can be obtained by setting the derivatives of the above subproblems to zero:

[0164]

[0165] Then:

[0166]

[0167] make get:

[0168]

[0169] Update variables

[0170] When other variables are fixed, we can solve the problem of variable L (v) The following minimization subproblem:

[0171]

[0172] sL (v)T L (v) =I

[0173] This constraint problem can be further simplified to the following form:

[0174]

[0175] in

[0176] For any matrix G∈R m×n , assuming that the singular value decomposition (SVD) of the matrix G is UΛV T , consider the following constraints:

[0177]

[0178] Its closed-form solution is as follows:

[0179] Y=UV T

[0180] Proof: Since G=UΛV T , U T U=I and V T V=I, then:

[0181] Notice, where δ i is the i-th singular value of matrix A. Obviously, when ΛP=ΛI, that is, V T Y T When U = I, the above equation holds. Therefore, the closed-form solution is Y = UV T .

[0182] By (v) Perform SVD decomposition, that is The solution to the problem can be obtained as follows:

[0183]

[0184] Update variables

[0185] When other variables are fixed, (v) The following minimization subproblem of can be written as:

[0186]

[0187] This can be further simplified to the following form:

[0188]

[0189] in:

[0190]

[0191] The above problems can be effectively solved by the singular value threshold (SVT) method. The details of the SVT method are as follows:

[0192] For a given matrix F and a positive parameter τ>0, the optimal solution to the following problem is:

[0193]

[0194] It is given by the following formula:

[0195]

[0196] in is the SVD decomposition of the matrix F. At the same time, Θ τ (·) indicates the SVT operator. The SVT operator is defined as follows:

[0197] Θ τ (∑ F )=max(0,∑ F -τ)+min(0,∑ F +τ)

[0198] Based on this, by setting Variable C (v) The closed-form solution of is as follows:

[0199]

[0200] In particular, express SVD decomposition of .

[0201] S106. Constructing a similarity matrix can help measure the similarity between different views. By calculating the similarity between data samples, potential features shared between different views can be found. This helps to better understand the structure of the data. In this optimization algorithm, the above steps are updated in an iterative manner until the stopping criterion is met. Subsequently, the final affinity matrix S can be obtained as follows:

[0202]

[0203] and

[0204]

[0205] Finally, spectral clustering is performed on the similarity matrix.

[0206] The proposed algorithm in this embodiment is compared with four state-of-the-art clustering algorithms, namely, convenient low-rank multi-view subspace clustering (FLMSC), scalable multi-view subspace clustering (SMVSC) with unified anchors, large-scale multi-view subspace clustering (LMVSC), and graph-based multi-view clustering (GMC). The performance is verified using the BBC dataset, the Reuter dataset, and the VISNIR dataset. The BBC has a total of 2225 documents covering 5 annotation topics. A sampled subset of the original BBC is used in the experiment, containing 685 documents and four different views, each of which contains 4659, 4633, 4665, and 4684 documents, respectively. The Reuter dataset contains documents in five different languages. A subset of 1200 documents is randomly selected in the experiment, with 200 documents in each category. The VISNIR dataset is a collection of face images generated from two different sources (views), each of which corresponds to a specific lighting condition. In the experiment, a subset of 1056 images was constructed as clustering data. The following evaluation indicators were used to evaluate the clustering performance:

[0207] F-score: takes into account precision and recall and is used to calculate the overall performance of clustering.

[0208] Normalized Mutual Information (NMI): It is used to measure the similarity between two clusters in a data set. It is a measure of the interdependence between two variables and is applicable to cluster assignments obtained by different algorithms or methods.

[0209] Accuracy (ACC): A commonly used evaluation metric in the context of clustering, especially when true labels are available. It measures the proportion of data points that are correctly assigned to their true cluster.

[0210] Adjusted Rand Index (AR): Evaluates the similarity between two clusters by considering all pairs of samples and counting the number of pairs that are assigned to the same or different clusters in the predicted and true clustering. AR adjusts the original Rand Index to account for the expected similarity between clusters due to chance.

[0211] Comparison with the State-of-the-Art In the first experiment, noise was added to 1% of the elements in the BBC dataset, and the noise level varied from 0 to 0.5. Then, we compared the AR, F-score, NMI, and ACC of various algorithms. The results are shown in Figure 3-Figure 6 shown.

[0212] LMSC-DL shows the best F-score at all noise levels. The performance of FLMSC decreases with the increase of noise, but the decrease is relatively gentle, indicating that it has a certain robustness in dealing with noise. The F-scores of SMVSC, LMVSC and GMC decrease significantly with the increase of noise, especially GMC, whose performance decreases significantly at high noise levels. For NMI, LMSC-DL also maintains the lead at all noise levels. NMI is based on mutual information to evaluate the similarity between the clustering results and the true distribution. The higher NMI value shows that LMSC-DL can better restore the true structure of the data under different noise conditions. At the same time, the NMI values ​​of other algorithms (especially GMC) decrease rapidly under high noise conditions. For the ACC indicator, LMSC-DL and FLMSC show higher accuracy and are relatively more stable, and can still maintain high accuracy even under high noise levels. The accuracy of SMVSC, LMVSC and GMC decreases with the increase of noise. The Rand Index (AR) evaluates the consistency of two data distributions and takes into account randomness. The AR values ​​of the LMSC-DL algorithm are relatively high at all noise levels, indicating that its clustering results are more consistent with the true class label distribution. GMC shows the largest performance degradation with increasing noise, while the other algorithms show some degree of noise resistance.

[0213] The overall trend shows that the LMSC-DL algorithm exhibits stability and high performance on all evaluation metrics, especially in high noise environments. The FLMSC algorithm is usually in the second place, showing good performance, but is not as stable as LMSC-DL under high noise levels. SMVSC, LMVSC, and GMC are affected by noise to varying degrees.

[0214] In the second experiment, 1%, 5%, 10%, 20%, and 50% noise elements were added to the BBC dataset, and the noise level was 0.1. Then, the F-score, NMI, ACC, and AR of various algorithms were compared. The following conclusions were drawn based on the experiment:

[0215] 1) The F-scores of all algorithms decrease with increasing sparsity, with LMSC-DL and FLMSC being least affected. The performance of GMC decreases significantly and remains low at all sparsity levels.

[0216] 2) For NMI, again, all algorithms show a degradation when the sparsity increases, with LMSC-DL having the least impact. GMC performs poorly at higher sparsity levels.

[0217] 3) For ACC, we see that all methods experience a sharp drop as sparsity increases, among which LMSC-DL is the most robust but still affected.

[0218] 4) For AC, all algorithms show a degradation when the sparsity increases. LMSC-DL and FLMSC seem to have better robustness than other algorithms.

[0219] In general, the performance of all algorithms decreases as sparsity increases, which is expected since sparser data tends to have less information for the algorithms to exploit during clustering. LMSC-DL appears to be the most robust across all the metrics evaluated, maintaining higher values ​​than the other algorithms as sparsity increases.

[0220] In the third experiment, the performance of the algorithm was compared on the Reuter and VISNIR datasets, adding 1% noise to the datasets and the noise level was 0.1. The results are shown in Figure 2. Figure 7 (Reuter dataset) and Figure 8 (VIS / NIRBBC).

[0221] Combining the two graphs ( Figure 7 and Figure 8 ) The performance of different methods on the Reuters and VISNIR datasets is compared and analyzed. The following is an overall comprehensive analysis: LMSC-DL is the most stable and powerful method on both datasets, with excellent classification accuracy and clustering effect. GMC performs very well on the VISNIR dataset, but has a clear disadvantage on the Reuters dataset.

[0222] LMVSC performs well on the VISNIR dataset, especially in the clustering-related NMI and AR indicators, but performs moderately on the Reuters dataset. FLMSC and SMVSC perform generally well on both datasets, especially SMVSC, which performs significantly worse on the VISNIR dataset. Overall, LMSC-DL can be considered the most versatile method, while GMC is more suitable for tasks similar to the VISNIR dataset.

[0223] Multi-view clustering aims to fuse information from different data views to improve clustering accuracy. Existing algorithms usually use low-rank expressions to capture the potential structure of the data to better adapt to the differences and complementarities between different views. However, since it relies on the Frobenius norm metric for error measurement, the algorithm is more sensitive to outliers and noisy data in the data. To solve this problem, this embodiment proposes a low-rank multi-view clustering method (LMSC-DL) based on sparse regularization. This method enhances the robustness of the model to outliers by introducing sparse regularization, so that the model retains the potential structure of the data while reducing the interference of noise and outliers.

[0224] The above-mentioned embodiments only express the specific implementation of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A low-rank multi-view subspace clustering method based on sparse regularization, characterized in that: The following steps are involved: Step 1: Receive a multi-view dataset to be clustered: collect views and construct a multi-view dataset based on sample data points in each view; Step 2: Clean the data, check whether there are missing values ​​in the data, and then choose to fill or delete the missing values ​​according to the situation to obtain a standard data set. Collect view sample data bases to construct sample data sets X for each view, satisfying a set of multi-view data samples X = [ X (1)T ,...,X VT ] T ,in represents the vth view; Step 3: Perform a sparse subspace representation on the data: only consider the low-rank structure within each view. The self-expression form of multi-view data is as follows: Among them, Z (v) represents the subspace of the vth view, E (v) represents the error matrix of the vth view, A (v is the given dictionary; Step 4: Perform low-rank representation on the subspace: Extend the low-rank representation to multiple views to obtain: Step 5: construct a common potential subinterval and map the data of different views to the common potential subinterval; Step 6: Construct a similarity matrix and perform spectral clustering on the similarity matrix.

2. The low-rank multi-view subspace clustering method based on sparse regularization according to claim 1, characterized in that: The step 5 is specifically as follows: Step 5.1, introduce the following consistency items: Step 5.2: The formulation of the low-rank multi-view subspace clustering FLMSC promoted by the consistency term is as follows: Among them, λ1 and λ2 are two positive equilibrium parameters; Step 5.3, replace the F norm in the above formula with the L1 norm to obtain low-rank multi-view subspace clustering based on sparse regularization: Step 5.4: Based on the alternating direction multiplier method, use the optimization algorithm to solve the minimization problem and construct the corresponding augmented Lagrangian function as follows: Step 5.5: Update variables When other variables are fixed, solve for variable Z (v) The following minimization subproblem: The equivalent rewrite is: By introducing the auxiliary variable H and omitting the superscript for simplification, we get the constrained optimization problem: Step 5.6: Update variables When other variables are fixed, solve the problem about variable L (v) The following minimization subproblem: s.t.L( v)T L (v) =I The constrained optimization problem is further simplified to the following form: in Step 5.

7. Update variables When other variables are fixed, solve the problem about variable C (v) The following minimization subproblem: Further simplified to the following form: in:

3. The low-rank multi-view subspace clustering method based on sparse regularization according to claim 2, characterized in that: The semi-quadratic splitting algorithm is used to solve the constrained optimization problem in step 5.5: where η is a penalty parameter that forces X-AZ and H to be close to the same fixed point; subsequently, H and Z are updated according to the following two subproblems. Sub-question 1: Sub-question 2: For subproblem 1, we introduce the concept of contraction operator. For real number x and threshold τ, contraction operator Defined as: It means that when the absolute value of x is less than τ, then will become 0, otherwise x will be subtracted by τ; extending this to the matrix and applying it to each element, the solution to subproblem 1 is given by: For sub-problem 2, the following problem is obtained by scaling: The solution is obtained by setting the derivative of subproblem 2 to zero: Then: make get:

4. The low-rank multi-view subspace clustering method based on sparse regularization according to claim 3, characterized in that: The solution to the constrained optimization problem in step 5.6 is as follows: For any matrix G∈R m×n , assuming that the singular value decomposition (SVD) of the matrix G is UΛV T , consider the following constraints: Its closed-form solution is as follows: Y=UV T Since G=UΛV T ,U T U=I and V T V=I, then: Tr ( Y T G) =Tr ( AND T ULV T =Tr ( LV T Y T U ≤Tr ( ΛI) where δ i is the i-th singular value of the matrix G; obviously, when ΛP=ΛI, that is, V T Y T When U = I, the above equation holds; therefore, the closed-form solution is Y = UV T ; By (v) Perform SVD decomposition, that is The solution to the problem is as follows:

5. The low-rank multi-view subspace clustering method based on sparse regularization according to claim 4, characterized in that: In step 5.7, the variable C is solved by the singular value threshold method (v) The minimization subproblem is: For a given matrix F and a positive parameter τ>0, the problem The optimal solution is given by given; in is the singular value decomposition of the matrix F; at the same time, Θ τ (·) represents the singular value threshold operator; the singular value threshold operator is defined as follows: I τ (S F )=max(0,Σ F -τ)+min(0,Σ F +t By setting Variable C (v) The closed-form solution of is as follows: express The singular value decomposition of .

6. The low-rank multi-view subspace clustering method based on sparse regularization according to claim 5, characterized in that: In step 6, the final affinity matrix S is obtained: and

Images