Robust convex biclustering method based on metric learning
By introducing a robust convex dual clustering method that dynamically adjusts weights, the problem of convex dual clustering is solved, efficient and accurate clustering is achieved under complex data, and the application scope of clustering algorithms is expanded.
Patent Information
- Application Number
- CN202510612038.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-13
- Publication Date
- 2025-08-26
AI Technical Summary
The existing convex double clustering method cannot dynamically adjust the weights of different features when facing data sets containing outliers, resulting in inaccurate clustering results and easy to fall into local optimal solutions. It also uses a standard Euclidean distance to be sensitive to noise, resulting in distortion of the result.
The metric learning mechanism is introduced, by constructing a robust convex bicluster model based on metric learning, using k-nearest neighbor reduction regularization and alternating direction multipliers method (ADMM) to optimize the model, dynamically adjust the weights between data points, reduce the impact of outliers, and ensure the global optimal solution through the iterative optimization process.
It improves the robustness and stability of the clustering method, and can maintain efficient and accurate clustering results in complex data environments, avoid local optimal traps, and adapt to variable data distribution.
Smart Images
Figure CN120541552A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of pattern recognition, and in particular to a robust convex biclustering method based on metric learning. Background Art
[0002] Biclustering is an advanced data analysis technique that can simultaneously cluster samples and features, aiming to reveal more complex patterns and structures in the data. However, a significant challenge is that many biclustering models face the problem of non-convexity when solving their inherent optimization problems, making them prone to falling into local minima. This directly affects the global optimality and reliability of the clustering results.
[0003] To overcome this challenge, the convex biclustering method was proposed and has received widespread attention. Its core advantage lies in its ability to fundamentally ensure that the algorithm converges to the global optimal solution by constructing a convex optimization model, thus avoiding the pitfalls of local optimality and significantly improving the stability and performance of clustering. However, this method typically uses a fixed Euclidean distance as a similarity metric, making it extremely sensitive to outliers in the data. The presence of these outliers can amplify noise interference, leading to distortion of the final results.
[0004] Therefore, to further improve the practicality and accuracy of clustering algorithms, developing more robust clustering methods has become a key research focus. Such methods should be robust enough to withstand the interference of outliers in the dataset. Such methods will be better able to adapt to complex and changing data environments, providing more solid and reliable support for data analysis and decision-making.
[0005] While existing convex biclustering methods have attracted considerable attention in cluster analysis due to their global optimality and independence from initialization, their performance significantly degrades when dealing with contaminated data containing outlier features. Specifically, the objective function of convex biclustering methods is typically designed to be convex to ensure the existence and solvability of a global optimal solution, but this design does not account for the presence of outliers in the data. The presence of outliers can significantly affect the accuracy of clustering results, causing them to deviate from the true data structure. The specific technical issues that need to be addressed are as follows:
[0006] Non-robustness: When faced with a dataset containing outliers, existing convex biclustering methods cannot dynamically adjust the weights of different features to reduce the impact of outliers on the clustering process, which may lead to inaccurate clustering results.
[0007] Trapped in local optimality: Although the convex biclustering method avoids the local optimal trap common in traditional non-convex clustering methods, when dealing with contaminated or complex data sets, the clustering results may fall into suboptimal solutions due to the presence of outliers.
[0008] Objective function design limitations: Although the objective function of the original convex biclustering method can be solved accurately and scientifically, it uses the standard Euclidean metric to measure the distance between data points and their corresponding cluster centers. Therefore, its performance will degrade significantly when there are feature outliers. Summary of the Invention
[0009] In view of the deficiencies in the prior art, the present invention provides a robust convex biclustering method based on metric learning, which solves the problems in the background technology.
[0010] The object of the present invention is achieved as follows: a robust convex biclustering method based on metric learning, comprising the following steps:
[0011] Step 1) Initialize the variables, data representation and symbol interpretation required in the model;
[0012] Step 2) Construct a robust convex biclustering model based on metric learning, introduce metric learning into the model, and use k-nearest neighbors to reduce the number of non-zero terms in the regularized sum;
[0013] Step 3) Optimize the model using the alternating direction multiplier method ADMM;
[0014] Step 4) Through continuous iteration, until the objective function converges;
[0015] Step 5) Calculate the final clustering results based on the variables and output the results.
[0016] Furthermore, the data representation and symbol interpretation in step 1) are specifically as follows: define the data set as the matrix X∈R p×n , where p represents the feature dimension and n represents the number of samples; each sample is represented by a p-dimensional feature vector and stored in the rows of the matrix; x i represents the feature vector of the i-th sample, x j Represents the value of all samples on the jth feature, x i,j Represents the data in the i-th row and j-th column of the matrix X; define a cluster center matrix U∈R with the same size as the original data matrix X p×n , this matrix is used to store the cluster center representation in the clustering process and is initialized to an all-zero matrix; in addition, the metric matrix B∈R is defined p×p , a data-specific weighted distance is introduced and initialized to the identity matrix; the regularization parameter γ is defined.
[0017] Furthermore, the specific process of step 2) constructing a robust convex biclustering model based on metric learning includes:
[0018] 2-1) The original convex biclustering formula is as follows:
[0019]
[0020] 2-2) Based on the original convex biclustering model, the metric matrix B is introduced, and the matrix B is required to be positive definite to ensure the quadratic form (x i -u i ) T B(x i -u i ) is non-negative, the robust convex biclustering objective function based on metric learning is obtained as follows:
[0021]
[0022] Where γ is the regularization parameter, which is used to balance the contribution of row-column fusion terms and sparse anomaly terms; w i,j =w j，i and are the weights between row and column pairs, respectively.
[0023] Furthermore, the weight calculation method is as follows: the weight between the i-th sample point and the j-th sample point is set to: Among them, if the jth sample is the k-nearest neighbor of the i-th sample, =1, otherwise it is 0; the constant φ is non-negative; the value of φ = 0 corresponds to uniform weight; positive weights are restricted to the nearest neighbors.
[0024] Furthermore, the specific method of using the alternating direction multiplier method ADMM to optimize the model in step 3) is: using the quadratic operator To expand U's updates;
[0025] U=argmin f(U)+H(UU k )
[0026] Among them U k is U obtained in the previous iteration, and α is selected to ensure that H(U) is positive definite, with the value α=2r(σ(X))+1, where r is a pre-set value and σ(X) is the largest singular value of the matrix X;
[0027] Let f(U)+H(UU k ) with respect to U is 0, and the update of U for each iteration is:
[0028]
[0029] Furthermore, step 5) is specifically as follows: merge the labels corresponding to the rows and columns into one label; the label corresponding to the row is: [r1, r2, ..., r n ], the labels corresponding to the columns are: [c1,c2,…,c n ], for each row label rj and each column label c j , generate combined labels (r i , c j ), and then assign a new unique label to each combined label; arrange them from top to bottom, and if the combined labels are the same, the new labels are also the same. After all are assigned, the final clustering result is output.
[0030] Compared with the prior art, the present invention has the following beneficial effects:
[0031] This paper introduces a dynamic distance adjustment mechanism based on metric learning into the objective function of convex biclustering, effectively reducing the impact of outliers on clustering results. Furthermore, the adaptive nature of metric learning dynamically optimizes the distance metric based on the data distribution, further improving the adaptability of clustering to complex data distributions. This design ensures that clustering remains stable and efficient even when dealing with data containing noise and outliers.
[0032] By introducing metric learning, this method not only retains the advantages of convex biclustering but also inherits the initialization independence advantage of convex biclustering. Therefore, it avoids the local optimum trap common in traditional non-convex biclustering methods and can stably converge to the optimal solution. This ensures the accuracy and reliability of the clustering results.
[0033] This paper proposes a new approach for combining traditional clustering algorithms with new technologies such as deep learning. The introduction of metric learning also opens up the possibility of diversifying clustering methods. This is particularly true in heterogeneous data analysis scenarios, where specialized distance metrics can be designed for different data types, significantly expanding the application scope of clustering algorithms. This cross-disciplinary integration is expected to further advance research in clustering methods and related fields.
[0034] In summary, a robust convex biclustering method based on metric learning provides a more effective and reliable solution for the field of cluster analysis by improving robustness, optimizing stability, and expanding applicability. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0036] Figure 1 It is a flow chart of the present invention.
[0037] Figure 2This is a flow chart of the ADMM optimization model used in the present invention.
[0038] Figure 3 This is a flow chart of calculating clustering results based on variables in the present invention. DETAILED DESCRIPTION
[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0040] The overall process of the present invention is as follows Figure 1 As shown, the submodule processes 3 and 5 included are respectively as follows Figure 2 、 Figure 3 As shown, the following is a detailed description with reference to each flow chart.
[0041] This method uses a dataset with outliers for analysis. It first initializes the variables needed in the model, then introduces metric learning into the model. Weights are used to ensure that all data points contribute to the calculation, thereby improving the model's generalization ability. K-nearest neighbors are used to reduce the number of nonzero terms in the regularized sum, simplifying the calculation. Variable updates are then performed according to the ADMM rule, with continuous iteration until the objective function converges. After convergence, the corresponding labels are obtained using V1 and V2, and then merged into a single label. Ultimately, this method can effectively address the sensitivity to outliers in the biclustering process.
[0042] The present invention will be further described below with reference to specific examples.
[0043] like Figure 1-3 A robust convex biclustering method based on metric learning is shown, comprising the following steps:
[0044] Step 1: Data representation and symbol interpretation
[0045] First, define the data set as a matrix X∈R p×n , where p represents the feature dimension and n represents the number of samples. Each sample is represented by a p-dimensional feature vector and stored in the rows of the matrix. Specifically, x i represents the eigenvector of the i-th sample (i.e., the i-th row of the matrix), x j represents the value of all samples on the jth feature (i.e. the jth column of the matrix), x i,j Represents the data in the i-th row and j-th column of the matrix X. Define a cluster center matrix U∈R with the same size as the original data matrix X p×n, this matrix is used to store the cluster center representation in the clustering process and is initialized to a full zero matrix. In addition, the metric matrix B∈R is defined p×p , a data-specific weighted distance is introduced and initialized to the identity matrix. The regularization parameter γ is defined.
[0046] Step 2: Construct robust convex biclustering based on metric learning
[0047] The metric matrix B is introduced into the convex biclustering model to weight and metrically transform the distances between data points, thereby enhancing the flexibility and robustness of the optimization problem. The structure of the matrix B indicates which features of the data are more consistent with cluster assignment. In particular, when B is a diagonal matrix, larger diagonal values of B correspond to features with higher correlation or lower noise corruption. The specific construction process of the robust convex biclustering model based on metric learning is as follows:
[0048] (1) The original convex biclustering formula is as follows:
[0049]
[0050] (2) Based on the original convex biclustering model, after introducing the metric matrix B, the matrix B is required to be positive definite, ensuring that the quadratic form (x i -u i ) T B(x i -u i ) is non-negative, that is, the metric is meaningful. Thus, the robust convex biclustering objective function based on metric learning obtained by the present invention is as follows:
[0051]
[0052] Where γ is the regularization parameter, which is used to balance the contribution of row-column fusion terms and sparse anomaly terms. i,j =w j,i and are the weights between row and column pairs, respectively.
[0053] We set the weight between the i-th sample point and the j-th sample point as: Among them, if the jth sample is the k-nearest neighbor of the i-th sample, It is equal to 1, otherwise it is 0. The term slows down the merging of distant points. The nonnegative constant φ controls the strength of the similarity-driven fusion process; a value of φ = 0 corresponds to uniform weighting. Restricting positive weights to nearest neighbors improves computational efficiency and clustering quality. While these two factors in defining weights act similarly, their combination increases the sensitivity of the clustering path to the local density of the data. Weights between features are calculated similarly.
[0054] Step 3: Use the alternating direction multiplier method ADMM to optimize the model
[0055] For the convenience of calculation, the matrix form will be used in the following. For a given undirected graph G = (V, E), where V is a set of n vertices and E is a defined edge set. For a point pair (i, j), the present invention uses l(i, j) in the enumeration order, and |E| is the number of edges in the graph G. The present invention gives two incidence matrices and They all belong to the matrix R p×|E| The kth element in the l(i,j)th column of . If k=i, then Otherwise it is 0. Similarly, if k=j, then Otherwise it is 0. That is [(u i -u j )] (i,j)∈E =UQ, [(u i -u j )] (i,j)∈E =Q T U, the present invention defines matrices D1 and D2, and the difference matrix of the coefficient matrix can be written as UD1=UQ, D2U=Q T U. Two auxiliary variables V1 and V2 are introduced, where V1 and V2 respectively save the differences between samples and features in the center matrix, thereby reformulating the robust biclustering problem as the following equivalent constraint problem:
[0056]
[0057] Among them, V 1.l Represents the l(i,j)th column of matrix V1. Similarly, V 2.l Represents the l(i,j)th column of matrix V2. The augmented Lagrangian function corresponding to this problem is as follows:
[0058]
[0059] Among them, Y and W are Lagrangian dual variables, and λ and ρ are penalty parameters.
[0060] ADMM minimizes the augmented Lagrangian function for only one block of the variables at a time before updating the dual variables. Therefore, the above problem can be broken down into six different subproblems for further solution. The specific process is as follows:
[0061] (1) Fixing B and updating U is equivalent to finding the following minimum U:
[0062]
[0063] make The above formula can be written as:
[0064]
[0065] In order to achieve efficient U update, the present invention uses the quadratic operator To expand U's updates.
[0066] U=argmin f(U)+H(UU k )
[0067] Among them U k =U obtained in the previous iteration, α must be selected to ensure that H(U) is positive definite, so that the subproblem maintains convexity. Here, the present invention takes the value α = 2r(σ(X)) + 1, where r is a value that needs to be set in advance and σ(X) is the largest singular value of the matrix X.
[0068] Let f(U)+H(UU k ) With respect to the gradient of U being 0, the present invention can easily obtain the update of U in each iteration as:
[0069]
[0070] Where I is the identity matrix.
[0071] (2) Fixing U and updating B is equivalent to finding the following minimum B:
[0072] min Tr|(XU) T B(XU)]
[0073] stlog det(B)≥0.
[0074] To find the optimal solution for B, let A = (XU)(XU) T The Lagrange formula is:
[0075] L(B,u)=Tr|(XU) T B(XU)]-ulogdet(B)
[0076] =Tr(AB))-ulog det(B)
[0077] Its Karush-Kuhn-Tucker condition yields:
[0078]
[0079] The complementarity condition is: ulogdet(B)=0.
[0080] A is full rank, that is, no feature is completely redundant, then the closed solution of B above is:
[0081] B=det(A)A -1
[0082] (3) To update V1, the present invention needs to minimize the following function:
[0083]
[0084] V1 is determined by the proximal mapping, and the V1 update formula is:
[0085]
[0086] Among them, σ col =[(γ||v 1.1 ||2),...,(γ / ||V1. |E| ||2)], for the matrix M∈R p×q ,definition [M] col，2 =[||M .1 ||2, ..., ||M .q ||2]. It is a proximal operation for each column.
[0087] (4) To update V2, the present invention needs to minimize the following function:
[0088]
[0089] The update of V2 is also determined by the proximal mapping. The update formula of V2 is:
[0090]
[0091] Among them, σ row =[(γ / ||V 21. ||2),...,(γ / ||V 2|E|. ||2)] T , define the matrix M∈R p×n , [M] row，2 =[||M 1. ||2, ..., ||M p. ||2] T .
[0092] It performs proximal operations on each row.
[0093] (5) Fix the others and update the Lagrangian dual variable Y. The update formula of Y is:
[0094] Y=Y+λ(UD1-V1)
[0095] (6) Fix the others and update the Lagrangian dual variable W. The update formula of W is:
[0096] W=W+ρ(D2U-V2)
[0097] Step 4: Through continuous iteration, until the objective function converges
[0098] Iterate from the initial value and, following the ADMM rule, alternately update the model's variables: U, B, V1, V2, and the Lagrange dual variables Y and W. When the preset maximum number of iterations is reached, the iteration stops and the objective function value is calculated under the current state to check whether the change in the objective function value meets the convergence criteria. If not, continue iterating to update the variables until convergence is achieved and the final V1 and V2 are saved.
[0099] Step 5: Calculate the final clustering results based on variables V1 and V2.
[0100] (1) Once V1 and V2 are determined, we only need to use breadth-first search to identify the connected components of the graph induced by V1 and V2 respectively. Specifically, we calculate the 2-norm of each vector in V1 and V2. For example, if the number of elements in the vector is n, the corresponding 2-norm calculation formula is as follows:
[0101]
[0102] If the difference is less than or equal to the specified error threshold, the node pairs corresponding to these indices are considered connected. The corresponding node indices i and j are extracted, and then an edge is established between them through G = graph(i, j). After traversing all node pairs, the resulting graphs G1 and G2 describe the connectivity of all node pairs that meet this condition.
[0103] (2) G1 and G2 are searched using a breadth-first search, starting from each unvisited node, to find all connected nodes and assign them the same label. This continues until all nodes are visited. If there are no edges in the graph, each node is considered a separate cluster. Finally, the corresponding row and column labels are output.
[0104] (3) The present invention combines the labels corresponding to the rows and columns into one label. Specifically, the labels corresponding to the rows are: [r1, r2, ..., r n ], the labels corresponding to the columns are: [c1,c2,…,c n ], for each row label r j and each column label c j , generate combined labels (r i , c j), and then assign a new unique label to each combined label. Arrange them from top to bottom, and if their combined labels are the same, the new labels will also be the same. After all the assignments are made, the final clustering result is output.
[0105] The clustering method proposed in this invention is generally described as follows:
[0106] Input: original data X∈R p×n , hyperparameter γ.
[0107] Initialization: cluster center matrix U, metric matrix B.
[0108] Output: metric matrix B, cluster centers U, difference matrices V1 and V2, Lagrange multipliers Y and W
[0109]
[0110] The above embodiments are only intended to help understand the method and core concept of the present invention. It should be noted that, without departing from the principles of the present invention, a number of improvements and modifications may be made to the present invention by those skilled in the art, and such improvements and modifications also fall within the scope of protection of the claims of the present invention.
Claims
1. A robust convex biclustering method based on metric learning, characterized by: The following steps are involved: Step 1) Initialize the variables, data representation and symbol interpretation required in the model; Step 2) Build a robust convex biclustering model based on metric learning, introduce metric learning into the model, and use k-nearest neighbors to reduce the number of non-zero terms in the regularized sum; Step 3) Optimize the model using the alternating direction multiplier method ADMM; Step 4) Through continuous iteration, until the objective function converges; Step 5) Calculate the final clustering results based on the variables and output the results.
2. A robust convex biclustering method based on metric learning according to claim 1, characterized in that: The data representation and symbol interpretation in step 1) are as follows: define the data set as the matrix X∈R p×n , where p represents the feature dimension and n represents the number of samples; each sample is represented by a p-dimensional feature vector and stored in the rows of the matrix; x i represents the feature vector of the i-th sample, x j Represents the value of all samples on the jth feature, x i,j Represents the data in the i-th row and j-th column of the matrix X; define a cluster center matrix U∈R with the same size as the original data matrix X p×n , this matrix is used to store the cluster center representation in the clustering process and is initialized to an all-zero matrix; in addition, the metric matrix B∈R is defined p×p , a data-specific weighted distance is introduced and initialized to the identity matrix; the regularization parameter γ is defined.
3. A robust convex biclustering method based on metric learning according to claim 2, characterized in that: Step 2) The specific process of building a robust convex biclustering model based on metric learning includes: 2-1) The original convex biclustering formula is as follows: ； 2-2) Based on the original convex biclustering model, the metric matrix B is introduced, and the matrix B is required to be positive definite to ensure the quadratic form is non-negative, and the robust convex biclustering objective function based on metric learning is obtained as follows: ； Where γ is the regularization parameter, which is used to balance the contribution of row-column fusion terms and sparse anomaly terms; w i,j = w j，i and are the weights between row and column pairs, respectively.
4. A robust convex biclustering method based on metric learning according to claim 3, characterized in that: The specific method for calculating the weight is: set the weight between the i-th sample point and the j-th sample point to: , where if the jth sample is the k-nearest neighbor of the ith sample, =1, otherwise it is 0; the constant ϕ is non-negative; the value of ϕ=0 corresponds to uniform weight; positive weights are restricted to the nearest neighbors.
5. A robust convex biclustering method based on metric learning according to claim 4, characterized in that: Step 3) The specific method of using the alternating direction multiplier method ADMM to optimize the model is: use the quadratic operator H(U) = To expand U's updates; ； Among them U k For U obtained in the previous iteration, select α to ensure that H(U) is positive definite, and take the value α=2r(σ(X))+1, where r needs to be set in advance and σ(X) is the largest singular value of the matrix X; Let f(U)+H(UU k ) with respect to U is 0, and the update of U for each iteration is:
6. A robust convex biclustering method based on metric learning according to claim 5, characterized in that: Step 5) Specifically: merge the labels corresponding to the rows and columns into one label; the label corresponding to the row is: [r1, r2, ..., r n ], the labels corresponding to the columns are: [c1,c2,…,c n ], for each row label r j and each column label c j , generate combined labels (r i , c j ), and then assign a new unique label to each combined label; arrange them from top to bottom, and if the combined labels are the same, the new labels are also the same. After all are assigned, the final clustering result is output.