An integrated clustering method based on discrete labels
By assigning weights to each clustering result and applying regularization terms in ensemble clustering, and replacing the spectral embedding matrix with a binary membership indicator matrix, the problems of information loss and unreliable clustering results in traditional ensemble spectral clustering are solved, achieving higher quality clustering and relation extraction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGDONG YUNZHAO MEDICAL TECH CO LTD
- Filing Date
- 2023-10-17
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies suffer from low clustering accuracy, insufficient stability and robustness in open relation extraction. The two-step pipeline of traditional ensemble spectral clustering algorithms leads to information loss and unreliable clustering results.
An ensemble clustering method based on discrete labels is adopted. By assigning weights to each clustering result and applying a regularization term, high-quality pseudo-labels are constructed to guide the clustering process. A binary membership indicator matrix is used instead of the spectral embedding matrix to prevent information loss and balance the weights of each clustering result.
It improves the quality of the clustering process and the accuracy of pseudo-labels, obtains more reasonable hypergraphs and sample categories, and enhances the effect of open relation extraction.
Smart Images
Figure CN117390188B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an ensemble clustering method, and more particularly to an ensemble clustering method based on discrete labels, which applies a regularization term to the ensemble clustering to balance the weights between each clustering result, thereby obtaining a more reasonable hypergraph and the category of each sample. Background Technology
[0002] In data mining applications, clustering is a classic machine learning problem that divides data into several sets based on similarity, with promising applications in interest recommendation, image segmentation, and anomaly detection. Most open relation extraction methods extract relational patterns between named entities using unsupervised methods, then cluster semantically equivalent patterns into relation clusters. However, due to the lack of supervised information and low clustering accuracy, they suffer from low stability and robustness. Therefore, open relation extraction methods need to leverage unsupervised ensemble learning to create high-quality pseudo-labels, which can then be used as supervised information to improve the learning of relation features. To address this issue, clustering ensemble methods have emerged. Ensemble clustering combines multiple weak clustering results to obtain a more robust and stable clustering result. Although ensemble clustering has achieved good results, traditional ensemble clustering algorithms often use weaker clustering results, which can easily lead to unreliable clustering results. Traditional ensemble spectral clustering mainly includes two steps: 1) constructing a low-dimensional embedding representation of the data based on the feature vectors of the graph Laplacian; 2) applying K-means to the constructed low-dimensional data to obtain the clustering result. Therefore, for ensemble spectral clustering algorithms, such a two-step pipeline can lead to suboptimal result quality due to information loss during the two steps. Furthermore, existing algorithms often treat all clustering results equally, making the final consistent clustering result susceptible to the negative impact of unreliable clustering results. Thus, existing methods for extracting relational facts from open-domain corpora suffer from low clustering accuracy, affecting the final relation extraction performance. Summary of the Invention
[0003] To address the aforementioned problems in existing technologies, this invention proposes an integrated clustering method based on discrete labels.
[0004] By applying a regularization term to the ensemble clustering method to balance the weights between each clustering result, the clustering process is guided to create high-quality pseudo-labels. Finally, through multiple iterations of clustering, the relationship types in the text are discovered more accurately, resulting in a more reasonable hypergraph and the category of each sample.
[0005] The technical solution adopted in this invention is as follows:
[0006] An ensemble clustering method based on discrete labels is proposed to create high-quality pseudo-labels to guide the clustering process, and finally, through multiple iterations of clustering, accurately discover the relation types in the text; the process includes:
[0007] Step S1: Construct the distance matrix for each clustering result;
[0008] For a having Dataset of one instance The dataset is obtained through clustering methods. Clustering results Through integration Learn a consensus clustering result from the clustering results;
[0009] The distance matrix for each clustering result is as follows: ;
[0010] in, It is the first The label indicator matrix of each clustering result. It is the first The number of clusters in each clustering result It is an identity matrix, therefore we can obtain It is the first The distance relationships between samples in a clustering result; m, n, v are natural numbers greater than 1; T is the transpose of the matrix.
[0011] Step S2: Assign a weight to the distance matrix of each clustering result. , thus obtaining a weighted consensus distance graph matrix;
[0012] ;
[0013] in, Is every Each clustering result is assigned a weight, and the sum of the weights is 1.
[0014] Since base clustering methods can be weak, potentially leading to unreliable base clusters, the resulting hypergraph may also be imperfect. Therefore, re-weighting the constructed hypergraph based on the quality of the base clusters can significantly improve the final clustering quality. This patent assigns a weight to the distance matrix of each base cluster, thus obtaining a more robust consensus distance graph matrix.
[0015] Step S3: Obtain the discrete label matrix;
[0016] To address clustering instability and spectral embedding matrix The element may have negative values; therefore, a continuously relaxed spectrum is embedded in the matrix. Transform into a binary membership indicator matrix Therefore, the consensus distance matrix is changed to:
[0017] ;
[0018] Step S4: Obtain weight regularization;
[0019] To prevent the weights from becoming overly polarized, a regularization term is added to the weights, along with a weight constraint term for each clustering result from the original method. The objective function for ensemble clustering based on discrete labels is derived as follows:
[0020]
[0021] Where λ is the applied regularization term, used to balance the weights of each clustering result and prevent overfitting.
[0022] Instead of treating the results in each cluster equally, we design weights for each base cluster. The main motivation for weighting clusters is to assign lower weights to unstable clusters. This ensemble clustering fully utilizes multi-level information such as sample features and category structure.
[0023] In the discrete label-based ensemble clustering method, step S1 involves constructing the distance matrix for each clustering result as follows:
[0024] For each label vector Construct a binary membership indicator matrix :
[0025]
[0026] Each cluster (now represented as a hyperedge) has a column if the binary membership indicator matrix One row in the table corresponds to one object with a known label. Representative at the In the base classes, the first The sample belongs to the first Each category / cluster;
[0027] Based on the coarse-resolution perspective, if two objects belong to the same cluster, their similarity is 1; otherwise, their similarity is 0. A cluster is created for each object. Binary similarity matrix; therefore , The element represents the degree of kinship between two objects that are in the same cluster; let Then the matrix In the middle element Representing the The first sample and the first Distance relationships between samples.
[0028] In the discrete label-based ensemble clustering method, step S4 involves solving the objective function using an iterative optimization method. The solution process is as follows:
[0029] Step S41, when the matrix When fixed, for weights Solving this problem, the objective function can be transformed into:
[0030]
[0031] in Since the sum of the constraint weights is 1, a Lagrange function needs to be constructed:
[0032]
[0033] Lagrange function pairs Differentiating and setting it to 0, we get:
[0034]
[0035] Through mathematical simplification, we can obtain
[0036] ;
[0037] Step S42, when the weight When fixed, for the binary membership indicator matrix Solving this problem, the objective function can be transformed into:
[0038]
[0039] in, , A consensus distance graph matrix is obtained by weighting each clustering result.
[0040] When the objective function does not meet the convergence condition, steps S41 and S42 are iterated sequentially, and finally the binary membership indicator matrix is output. This allows us to obtain a more reasonable hypergraph and the category of each sample.
[0041] The aforementioned discrete label-based ensemble clustering method employs the standard coordinate descent algorithm to optimize the objective function described in step S42; to solve... The first line fixes the other lines to constants; let To remove the first row of the matrix , To remove the first row and first column of the matrix The objective function then becomes:
[0042]
[0043] in , ; express The indicator matrix before the update;
[0044] The update rules for the other rows can be derived similarly;
[0045] Finally, through the following formula
[0046]
[0047] get The optimal solution.
[0048] Beneficial effects of the invention:
[0049] Traditional ensemble spectral clustering mainly involves two steps: 1) constructing a low-dimensional embedding representation of the data based on the feature vectors of the Laplacian graph; 2) applying K-means to the constructed low-dimensional data to obtain the clustering results. Therefore, for ensemble spectral clustering algorithms, such a two-step pipeline can lead to suboptimal result quality due to information loss during the two steps.
[0050] Compared with traditional ensemble spectral clustering methods, this invention eliminates the need to form a continuous relaxed spectral embedding matrix before performing K-means (for ensemble spectral clustering algorithms, such a pipeline leads to suboptimal results due to significant information loss / loss in the two steps, resulting in reduced clustering performance, and K-means is also unstable). Instead, it uses a binary membership indicator matrix to replace the spectral embedding matrix, preventing information loss caused by this two-step pipeline. Furthermore, by applying a regularization term to the ensemble clustering, each clustering result is assigned a weight to balance the weights among them, rather than treating each clustering result equally. Therefore, this method can obtain a more reasonable hypergraph and the category of each sample.
[0051] 1. This invention presents an ensemble clustering method based on discrete labels. By improving the ensemble clustering algorithm module, it enhances the quality of pseudo-labels in relation extraction tasks, thereby improving entity pair representation and clustering quality. The quality of the clustering process is significantly improved by weighting each clustering result and replacing the spectral embedding matrix with a binary membership indicator matrix. This invention designs weights for each clustering result, rather than treating all clustering results equally. The main motivation for weighting clusters is to assign lower weights to unstable clusters, aiming to fully utilize multi-level information such as sample features and category structure for ensemble clustering.
[0052] 2. This invention presents an ensemble clustering method based on discrete labels. By introducing the ensemble clustering algorithm into the unsupervised ensemble clustering framework, it allows the open relation extraction task to retain the advantages of unsupervised learning while possessing the strong feature discrimination ability of supervised learning. This invention eliminates the need to form a continuous relaxed spectral embedding matrix before performing K-means, as these two steps result in significant information loss and reduced clustering performance. Furthermore, this invention applies a regularization term to the ensemble clustering method, assigning a weight to each clustering result to balance the weights among them, rather than treating each clustering result equally. Therefore, this invention's ensemble clustering method can obtain a more reasonable hypergraph and category for each sample.
[0053] 3. Existing technologies merely reference ensemble clustering modules, but the pseudo-labels generated by these modules are of low quality and cannot effectively guide the clustering process. This invention, based on a discrete label-based ensemble clustering method, constructs a new ensemble clustering module. Instead of treating the quality of each clustering result equally, it significantly improves the quality of pseudo-labels, thereby better performing open relation extraction tasks and more accurately discovering relational models in the text. Attached Figure Description
[0054] Figure 1 Figures a and b show the clustering results of two ensemble datasets obtained using traditional clustering methods;
[0055] Figure 2 The figure shows the clustering results obtained using the discrete label-based ensemble clustering method of this invention. Detailed Implementation
[0056] To make the technical concept and advantages of the invention clearer, the technical solution of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the following embodiments are only used to explain and illustrate preferred embodiments of the present invention and should not be construed as limiting the scope of patent protection claimed by the present invention.
[0057] Example 1
[0058] This invention provides an ensemble clustering method for discrete tags to create high-quality pseudo-tags, thereby guiding the clustering process. Finally, through multiple iterations of clustering, the method accurately discovers the relationship types in the text. The clustering process is as follows:
[0059] enter: Each clustering result has a parameter λ.
[0060] 1. Construct the distance matrix for each clustering result:
[0061] 2. Obtain the weighted consensus distance graph matrix:
[0062] 3. Discrete label-based ensemble clustering: When the objective function does not meet the convergence condition, the following steps are iterated in sequence:
[0063] (1) Based on solution process 1, and update ;
[0064] (2) Based on solution process 2, and update ;
[0065] Output: Binary membership indicator matrix .
[0066] Example 2
[0067] The discrete label-based ensemble clustering method in this embodiment differs from that in Embodiment 1 in that the specific process is as follows:
[0068] Step S1: Construct the distance matrix for each clustering result;
[0069] For a having Dataset of one instance The dataset is obtained through clustering methods. Clustering results Through integration Learn a consensus clustering result from the clustering results;
[0070] The distance matrix for each clustering result is as follows: ;
[0071] in, It is the first The label indicator matrix of each clustering result. It is the first The number of clusters in each clustering result It is an identity matrix, therefore we can obtain It is the first The distance relationships between samples in a clustering result; m, n, v are natural numbers greater than 1; T is the transpose of the matrix.
[0072] Step S2: Assign a weight to the distance matrix of each clustering result. , thus obtaining a weighted consensus distance graph matrix;
[0073] ;
[0074] in, Is every Each clustering result is assigned a weight, and the sum of the weights is 1.
[0075] Since base clustering methods can be weak, potentially leading to unreliable base clusters, the resulting hypergraph may also be imperfect. Therefore, re-weighting the constructed hypergraph based on the quality of the base clusters can significantly improve the final clustering quality. This patent assigns a weight to the distance matrix of each base cluster, thus obtaining a more robust consensus distance graph matrix.
[0076] Step S3: Obtain the discrete label matrix;
[0077] To address clustering instability and spectral embedding matrix The element may have negative values; therefore, a continuously relaxed spectrum is embedded in the matrix. Transform into a binary membership indicator matrix Therefore, the method is changed to:
[0078] ;
[0079] Step S4: Obtain weight regularization;
[0080] To prevent the weights from becoming overly polarized, a regularization term is added to the weights, along with a weight constraint term for each clustering result from the original method. The objective function for ensemble clustering based on discrete labels is derived as follows:
[0081]
[0082] Where λ is the applied regularization term, used to balance the weights of each clustering result and prevent overfitting.
[0083] Instead of treating all clusters equally, weights are designed for each base cluster. The main motivation for weighting clusters is to assign lower weights to unstable clusters. Ensemble clustering leverages multi-level information such as sample features and category structure.
[0084] In step S4, based on discrete label ensemble clustering, the objective function is solved using an iterative optimization method. The solution process includes:
[0085] Step S41, when the matrix When fixed, for weights Solving this problem, the objective function can be transformed into:
[0086]
[0087] in Since the sum of the constraint weights is 1, a Lagrange function needs to be constructed:
[0088]
[0089] Lagrange function pairs Differentiating and setting it to 0, we get:
[0090]
[0091] Through mathematical simplification, we can obtain
[0092] ;
[0093] Step S42, when the weight When fixed, for the binary membership indicator matrix Solving this problem, the objective function can be transformed into:
[0094]
[0095] in, , A consensus distance graph matrix is obtained by weighting each clustering result.
[0096] When the objective function does not meet the convergence condition, steps S41 and S42 are iterated sequentially, and finally the binary membership indicator matrix is output. This allows us to obtain a more reasonable hypergraph and the category of each sample.
[0097] Example 3
[0098] The difference between the discrete label-based ensemble clustering method in this embodiment and that in embodiment 2 is that in step S4, the standard coordinate descent algorithm is used to optimize the objective function described in step S42.
[0099] To solve The first line fixes the other lines to constants; let To remove the first row of the matrix , To remove the first row and first column of the matrix With these symbols, the problem / objective function of S42 becomes:
[0100]
[0101] in , ; express The indicator matrix before the update;
[0102] The update rules for the other rows can be derived similarly;
[0103] Finally, through the following formula
[0104]
[0105] get The optimal solution.
[0106] Example 4
[0107] This invention utilizes an ensemble clustering method based on discrete labels to create high-quality pseudo-labels, thereby guiding the clustering process. Finally, through multiple iterative clustering iterations, it accurately discovers the relation types in the text. Its specific implementation process includes:
[0108] 1) Construct the distance matrix for each clustering result.
[0109] Given a Dataset of one instance The dataset can be obtained through various clustering methods. Clustering results Ensemble clustering aims to integrate... We use clustering results to learn a consensus clustering result.
[0110] For each label vector This paper constructs a binary membership indicator matrix. (Transformation of label vectors into label matrices):
[0111]
[0112] in, It is the first The label vector of each clustering result. It is the first A label matrix for each clustering result. Each cluster (now represented as a hyperedge) has one column. If the binary membership indicator matrix... One row in the table corresponds to one object with a known label. Representative at the In the base classes, the first The sample belongs to the first Each category is a cluster.
[0113] From a coarse-resolution perspective, if two objects belong to the same cluster, their similarity is 1; otherwise, it is 0. It is easy to create clusters for each object. Binary similarity matrix. Therefore... , The element represents the degree of kinship between two objects that are in the same cluster.
[0114] make Then the matrix In the middle element Representing the The first sample and the first Distance relationships between samples.
[0115] 2) Obtain the weighted consensus distance graph matrix
[0116] Because the clustering method may be weak, potentially leading to unreliable clustering results, the hypergraph constructed from these unreliable results may also be imperfect. Therefore, re-weighting the hypergraph based on the quality of the clustering results can significantly improve the final clustering quality. This patent requires assigning a weight to the distance matrix of each clustering result to obtain a more robust consensus distance graph matrix.
[0117]
[0118] 3) Obtain the discrete label matrix
[0119] Most spectral clustering algorithms use a spectral embedding matrix. Then, a discretization program such as k-means is run to obtain the final clustering result. However, k-means is not suitable for... The requirements are high, which makes clustering performance unstable. This paper addresses the instability of clustering and the spectral embedding matrix... The element may have negative values; therefore, a continuously relaxed spectrum is embedded in the matrix. To transform into a binary membership indicator matrix Therefore, the model is changed to:
[0120]
[0121] 4) Obtain weight regularization
[0122] To enhance the regularization of the weights, a weight constraint term is added for each clustering result. The objective function for ensemble clustering based on discrete labels proposed in this patent is:
[0123]
[0124] Here, λ is the applied regularization term, used to balance the weights of each clustering result and prevent overfitting. This patent uses an iterative optimization method to solve the objective function. The solution process is as follows:
[0125] 1. When the matrix When fixed, for weights Solve the problem. The objective function can then be transformed into:
[0126]
[0127] in Since the sum of the constraint weights is 1, a Lagrange function needs to be constructed:
[0128]
[0129] Lagrange function pairs Differentiating and setting it to 0, we get:
[0130]
[0131] Through simple mathematical simplification, we can obtain
[0132]
[0133] 2. When the weight When fixed, for the binary membership indicator matrix Solve the problem. The objective function can then be transformed into:
[0134]
[0135] in, , The consensus distance graph matrix is obtained by weighting each clustering result. The standard coordinate descent algorithm is used to optimize the above problem. For example, to solve... In the first line, we fix the other lines to constants. Let To remove the first row of the matrix , To remove the first row and first column of the matrix With these symbols, the problem becomes:
[0136]
[0137] in , . express The indicator matrix before the update. The update rules for the other rows can be derived similarly. Therefore, The optimal solution can be obtained through:
[0138]
[0139] Although the problem above is very simple, it is very time-consuming when the number of clusters constructed is large.
[0140] Therefore, the complexity of this technique is linearly related to the number of data clusters and is not suitable for data with too large a number of clusters.
[0141] Compared with traditional ensemble spectral clustering methods, this invention eliminates the need to form a continuous relaxed spectral embedding matrix before performing K-means. Instead, it uses a binary membership indicator matrix to replace the spectral embedding matrix, preventing information loss caused by the two-step pipeline of traditional ensemble spectral clustering methods. Furthermore, by applying a regularization term to the ensemble clustering, each clustering result is assigned a weight to balance the weights among the clustering results (rather than treating each clustering result equally). Therefore, the discrete label-based ensemble clustering method of this invention can obtain a more reasonable hypergraph and the category of each sample.
[0142] Experiments have shown that:
[0143] Figure 1 a, Figure 1 Figure b shows the clustering results of two ensemble datasets using traditional clustering methods. The ensemble datasets were obtained, and the clustering results of the two ensemble datasets are shown below. Figure 1 As shown, their results were extremely poor.
[0144] Figure 2 The image shows the clustering results obtained using the ensemble clustering method described in Embodiment 4 of this invention. By utilizing the structured information of the ensemble data through the discrete label-based ensemble clustering method of this invention, better clustering results can be obtained, such as… Figure 2 As shown. (Through) Figure 1 and Figure 2 The comparison shows that the ensemble clustering method based on discrete labels can obtain a more reasonable hypergraph and a category for each sample.
Claims
1. An ensemble clustering method based on discrete labels to create high-quality pseudo-labels, thereby guiding the clustering process, and finally accurately discovering the relation types in the text through multiple iterative clustering; characterized in that, The process includes: Step S1: Construct the distance matrix for each clustering result; For a having The dataset X = [X1, X2, …, X] contains instances. n The dataset was obtained through clustering methods. Clustering results Through integration Learn a consensus clustering result from the clustering results; The distance matrix for each clustering result is as follows: ; in, It is the first The label indicator matrix of each clustering result. It is the first The number of clusters in each clustering result It is an identity matrix, therefore we can obtain It is the first The distance relationships between samples in each clustering result; m, n, v are natural numbers greater than 1; T is the transpose of the matrix; Step S2: Assign a weight to the distance matrix of each clustering result. , thus obtaining a weighted consensus distance graph matrix; ； in, Is every Each clustering result is assigned a weight, and the sum of the weights is 1. Step S3: Obtain the discrete label matrix; To address clustering instability and spectral embedding matrix The element may have negative values; therefore, a continuously relaxed spectrum is embedded in the matrix. Transform into a binary membership indicator matrix Therefore, the method is changed to: ； Step S4: Obtain weight regularization; To prevent the weights from becoming overly polarized, a regularization term is added to the weights, along with the original weight constraint term for each clustering result. The objective function for ensemble clustering based on discrete labels is derived as follows: in, The applied regularization term is used to balance the weights of each clustering result and prevent overfitting.
2. The ensemble clustering method based on discrete labels according to claim 1, characterized in that: In step S1, the process of constructing the distance matrix for each clustering result is as follows: For each label vector Construct a binary membership indicator matrix ; Each cluster has a column if the binary membership indicator matrix... One row in the table corresponds to one object with a known label. Representative at the In the base classes, the first The sample belongs to the first Each category / cluster; Based on the coarse-resolution perspective, if two objects belong to the same cluster, their similarity is 1; otherwise, their similarity is 0. A cluster is created for each object. Binary similarity matrix; therefore , The element represents the degree of kinship between two objects that are in the same cluster; let Then the matrix In the middle element Representing the The first sample and the first Distance relationships between samples.
3. The ensemble clustering method based on discrete labels according to claim 1 or 2, characterized in that: In step S4, based on discrete label ensemble clustering, an iterative optimization method is used to solve the objective function. The solution process is as follows: Step S41, when the matrix When fixed, for weights Solving this problem, the objective function can be transformed into: in Since the sum of the constraint weights is 1, a Lagrange function needs to be constructed: Lagrange function pairs Differentiating and setting it to 0, we get: Through mathematical simplification, we can obtain ； Step S42, when the weight When fixed, for the binary membership indicator matrix Solving this problem, the objective function can be transformed into: in, , A consensus distance graph matrix is obtained by weighting each clustering result. When the objective function does not meet the convergence condition, steps S41 and S42 are iterated sequentially, and finally the binary membership indicator matrix is output. This allows us to obtain a more reasonable hypergraph and the category of each sample.
4. The ensemble clustering method based on discrete labels according to claim 3, characterized in that: The objective function described in step S42 is optimized using the standard coordinate descent algorithm; to solve... The first line fixes the other lines to constants; let To remove the first row of the matrix , To remove the first row and first column of the matrix Then the objective function becomes: in , ; express The indicator matrix before the update; The update rules for the other rows can be derived similarly; Finally, through the following formula get The optimal solution.