An overlapping region judgment and overlapping cluster division method and system
By constructing a hypergraph and obtaining a minimal cover set, combined with a perfect matching model, the rigor and overlap control issues of overlapping clustering in complex node systems in existing technologies are solved, achieving accurate and efficient data segmentation and improving the recognition accuracy and adaptability of overlapping clustering.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOUTHWEST JIAOTONG UNIV
- Filing Date
- 2026-03-18
- Publication Date
- 2026-06-19
AI Technical Summary
Existing technologies struggle to accurately reflect the true structure of complex node systems with multiple membership relationships in data analysis and machine learning. Furthermore, existing overlapping clustering methods lack a solid mathematical theory in terms of partitioning rigor and overlap control, resulting in partitioning results containing redundancy and overlap and poor interpretability.
By constructing a hypergraph, generating hyperedges based on similarity, and obtaining a set of minimal covers, and combining a perfect matching model to determine overlapping regions, a greedy algorithm and minimal cover theory are used to achieve efficient and rigorous overlapping clustering.
It achieves accurate discovery and division of complex overlapping structures, improves the theoretical rigor and interpretability of clustering results, automatically identifies overlapping regions without the need for manual threshold setting, and adapts to complex overlapping patterns.
Smart Images
Figure CN122241294A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data mining, and in particular to a method and system for determining overlapping regions and dividing overlapping clusters. Background Technology
[0002] Currently, in data analysis and machine learning, traditional clustering methods such as K-means, hierarchical clustering, and spectral clustering are based on mutual exclusion assumptions, making it difficult to accurately reflect the true structure of complex node systems with multiple membership relationships. Based on this, the proposed overlapping clustering method allows data points to belong to multiple clusters simultaneously, but existing methods are mainly based on simple graph models, which cannot fully express the higher-order relationships between nodes. Furthermore, they lack a solid mathematical theory in terms of the rigor of partitioning and the control of overlap, resulting in partitioning results that may contain a large amount of redundant overlap, poor interpretability, and difficulty in meeting the application scenarios with clear constraints on the scale of overlap.
[0003] Among them, existing representative methods, such as the network clustering framework for high-order motif clustering proposed by Benson et al., do not directly address the control of minimizing overlap, nor do they provide strict theoretical criteria to determine the inevitability of overlap; the overlapping community detection framework for integrated subspaces proposed by Changyang et al. does not make full use of the high-order relations described by the hypergraph, and its extended modularity optimization is difficult to guarantee the minimum overlap; the matrix factorization method proposed by He et al. is mainly for signed networks and lacks a theoretical framework for system overlap partitioning and discrimination applicable to general node systems.
[0004] Therefore, how to develop a clustering method that can uniformly model high-order relationships in node systems and achieve minimum overlap partitioning based on rigorous mathematical theory is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of this, the present invention provides a method and system for determining overlapping regions and dividing overlapping clusters, which overcomes the above-mentioned defects.
[0006] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for determining overlapping regions and classifying overlapping clusters, the specific steps of which are as follows: Calculate the similarity between data points in a given dataset, generate hyperedges based on multiple data points using the similarity, and construct a hypergraph with the data points as vertices and the hyperedges as connections. With the goal of covering all vertices, obtain the minimum coverage set based on the hyperedge set; The minimum coverage set is compared with a predefined perfect matching model. If the minimum coverage set matches the perfect matching model, it is determined to be a non-overlapping region; if the minimum coverage set does not match the perfect matching model, it is determined to be an overlapping region. Based on the judgment results, each data point is mapped to one or more corresponding hyperedges to obtain overlapping clustering results.
[0007] Optionally, the step of obtaining the hyperedge is as follows: Calculate the similarity between each data point in the given dataset, and construct the hyperedge based on multiple data points whose similarity exceeds a preset similarity threshold. Optionally, the steps for obtaining the minimal cover set are as follows: Step 21: Initialize the covered vertex set and the uncovered vertex set; Step 22: If the set of uncovered vertices is not empty, proceed to steps 23-24; if the set of uncovered vertices is empty, proceed to step 25. Step 23: Obtain the number of vertices that are not covered within the coverage area of each hyperedge. ; Step 24, The largest superedge is added to the covered set, and the vertices covered by the superedge are removed from the uncovered vertex set. Then, proceed to step 22. Step 25: Output the covering set as the minimal covering set.
[0008] Optionally, the perfect matching model is defined as: a set of superedges that covers all vertices and whose intersection with any two superedges is empty.
[0009] Optionally, if overlapping regions exist, an overlap measure is also included, specifically: the degree of overlap of the hypergraph is quantified by calculating the intersection of any two hyperedges in the minimal cover set.
[0010] Secondly, this application provides a system for determining overlapping regions and classifying overlapping clusters, including: The hypergraph construction module is used to calculate the similarity between data points in a given dataset, generate hyperedges based on multiple data points using the similarity, and construct a hypergraph with the data points as vertices and the hyperedges as connections. The hypergraph optimization module is used to obtain the minimum coverage set based on the hyperedge set with the goal of covering all vertices. The overlapping region determination module is used to compare the minimum coverage set with a predefined perfect matching model. If the minimum coverage set is consistent with the perfect matching model, it is determined that there is no overlapping region; if the minimum coverage set is inconsistent with the perfect matching model, it is determined that there is an overlapping region. The overlapping clustering partitioning module is used to map each data point to one or more corresponding hyperedges based on the judgment result, so as to obtain the overlapping clustering result.
[0011] Optionally, the hypergraph construction module includes: The similarity calculation unit is used to calculate the similarity between data points in a given dataset; The hyperedge generation unit is used to set a similarity threshold and group data points with similarity higher than the similarity threshold into the same set, and generate hyperedges based on the data point set; The hypergraph construction unit is used to construct a hypergraph with the data points as vertices and the hyperedges as connections.
[0012] Optionally, the overlapping region determination module further includes an overlapping region quantization unit, which is used to quantify the degree of overlap of the hypergraph by calculating the intersection of any two hyperedges in the minimal cover set.
[0013] According to the specific embodiments provided in this application, this application has the following technical effects: 1. This application introduces hypergraph covering theory and minimum covering solution into the overlapping clustering problem for the first time. It utilizes the natural characterization of high-order similarity relationships between data by hypergraphs and the minimum covering theory to automatically and efficiently determine the cluster structure. This fundamentally overcomes the core defects of existing technologies that rely on preset cluster numbers and cannot guarantee the minimization of overlap from a global perspective. It achieves more accurate and essential discovery and division of complex overlapping structures.
[0014] 2. This application establishes a minimum coverage objective of "covering all samples with the fewest hyperedges" and employs a greedy algorithm for efficient solution. A deterministic optimization strategy ensures the global sparse optimality of the obtained clustering scheme in terms of coverage. This deterministic optimization strategy ensures that each cluster corresponds to a hyperedge with a clear semantic meaning, enhancing the theoretical rigor and structural interpretability of the clustering results.
[0015] 3. This application introduces a perfect match comparison mechanism, which automatically identifies essentially overlapping regions by theoretically comparing the minimal coverage results with an ideal perfect match model, without relying on manually setting an overlap threshold. This mechanism establishes strict overlap judgment criteria at the mathematical level, improving adaptability and recognition accuracy for complex overlap patterns. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a schematic diagram of the method flow in one embodiment of this application; Figure 2The diagram below illustrates an overlapping clustering process according to an embodiment of this application; (a) shows the original data points; (b) shows a diagram of neighborhood construction; (c) shows a diagram of a hypergraph; (d) shows a diagram of minimal cover set construction; and (e) shows the overlapping clustering result. Detailed Implementation
[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0019] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0020] This embodiment discloses a method for determining overlapping regions and classifying overlapping clusters, such as... Figure 1 As shown, the specific steps are as follows: Step 1: Calculate the similarity between data points in the given dataset, generate hyperedges based on multiple data points using the similarity, and construct a hypergraph with data points as vertices and hyperedges as connections. Step 2: With the goal of covering all vertices, obtain the minimum coverage set based on the hyperedge set; Step 3: Compare the minimum coverage set with the predefined perfect matching model. If the minimum coverage set matches the perfect matching model, it is determined to be a non-overlapping region; if the minimum coverage set does not match the perfect matching model, it is determined to be an overlapping region. Step 4: Based on the judgment results, map each data point to one or more corresponding hyperedges to obtain the overlapping clustering results.
[0021] Furthermore, the goal of this stage is to solve the problem of overlapping data points using hypergraph covering and minimal covering theories, and to establish an optimization framework based on hypergraph structure to achieve efficient solutions for minimizing data overlap. The process is as follows: Figure 2As shown, the core idea of this scheme is to construct hyperedges to form a hypergraph using the similarity between data points, and remove redundant edges using the minimum cover theory. Furthermore, it combines the comparison of perfect matching and cover models to identify overlapping regions of data points. If the hypergraph cover is equivalent to a perfect match, then the data points do not overlap; if not, the covered region is the overlapping region of the data points, and the minimum overlap problem is solved by calculating the minimum cover. To achieve this goal, it is necessary to design hypergraph construction and similarity measurement methods. First, the similarity of data points is calculated, and hyperedges are established by setting a similarity threshold. Then, an optimization strategy based on a greedy algorithm is designed to select the fewest hyperedges from the given hypergraph to cover all vertices, thereby achieving efficient hypergraph coverage. Based on this, the construction mechanism based on hypergraph overlap partitioning is revealed through theoretical analysis and experimental verification, and a training algorithm for the model is designed. Finally, the effectiveness and rationality of the model are verified using standard datasets of various application categories to ensure the feasibility and practicality of the scheme.
[0022] In one embodiment, the step of obtaining the hyperedge is as follows: Calculate the similarity between data points in a given dataset, and construct hyperedges based on multiple data points whose similarity exceeds a preset similarity threshold.
[0023] Furthermore, assume that there exists a dataset ,in, Indicates the first Data points, ; The dimension of the data points; Let the set be the set of real numbers. For each data point... Perform similarity calculations, and can use Nearest neighbor algorithms, density algorithms, and other methods are used to determine its adjacency structure, denoted as... ;in, For Centered on, size is ; This represents a subset relationship. Specifically, selection... Individual and The most similar data points.
[0024] (1); In the formula, Points for neighbors; This indicates that among all data points, the selection should be the one that matches the previous one. Most similar There are 10 data points. Neighbor relationships are obtained through nearest neighbor or density algorithms, and a similarity metric is used. To measure the similarity between them, a similarity threshold is designed. This determines which data points will form superedges. This is a function used to calculate similarity. Specifically, if... Then data points and There exist hyperedges between them, and they are placed in the same hyperedge. Formally, the set of hyperedges is defined as follows: ,in, For the first A super edge, Specific hypergraphs The construction can be represented as: (2); Among them, the hyperedge of formula (2) It is a point set. .
[0025] In one embodiment, the steps for obtaining the minimal cover set are as follows: Step 21: Initialize the covered vertex set and the uncovered vertex set; Step 22: If the set of uncovered vertices is not empty, proceed to steps 23-24; if the set of uncovered vertices is empty, proceed to step 25. Step 23: Obtain the number of vertices that are not covered within the coverage area of each hyperedge. ; Step 24, Add the largest superedge to the covered set, and remove the vertices covered by the superedge from the uncovered vertex set. Then proceed to step 22. Step 25: Output the covering set as the minimal covering set.
[0026] Furthermore, the objective of the hypergraph covering problem is to select the minimum number of hyperedges to ensure that each vertex is covered by at least one hyperedge. The objective function is expressed as: (3); In the formula, It is a binary decision variable, representing whether to choose a hyperedge. The constraints ensure that all vertices All vertices are covered by at least one superedge. Therefore, a greedy algorithm is designed to select the superedge that covers the most uncovered vertices at each step. The decision variables are Make: (4); The steps of the greedy algorithm are as follows: Step 1: Initialization: Setting all vertices The state is uncovered, i.e., the initial covered set. .
[0027] Step 2: In each step, select a super edge. Cover as many uncovered vertices as possible until all vertices are covered.
[0028] Step 3: For each hyperedge We define the number of uncovered vertices in a covered area. .
[0029] Step 4: Select the superedge that covers the most vertices. Add it to the current overlay set And update the set of vertices covered.
[0030] Step 5: Repeat the iteration until all vertices are covered.
[0031] Once the set of covered superedges is selected using a greedy algorithm... To remove redundant edges and optimize the hypergraph structure, the minimum cover theory can be further applied to ensure that no redundant hyperedges participate in the cover. The final overlapping clustering result can be obtained through the mapping from hyperedges to clusters, for example, a vertex. Belongs to multiple hyperedges , ,but It can be considered to belong to these two clusters, indicating that it overlaps between the two clusters.
[0032] In one embodiment, the perfect matching model is defined as: a set of hyperedges that covers all vertices and whose intersection with any two hyperedges is empty.
[0033] In one embodiment, if there are overlapping regions, an overlap measure is also included, specifically: the degree of overlap of the hypergraph is quantified by calculating the intersection of any two hyperedges in the minimal cover set.
[0034] Furthermore, a perfect matching is defined as the existence of a set of hyperedges in a hypergraph that covers all vertices, and the intersection of any two hyperedges is empty; that is, there exists a set of hyperedges. ,satisfy ;in, This is a union operation; This is an intersection operation; For the perfect match One super-edge; For the perfect match One super-edge; It is an empty set; `universal` is a quantifier indicating any, all (each). Design a perfect matching model for a hypergraph and compare the minimum cover result obtained by the greedy algorithm with the perfect matching model.
[0035] If the coverage result is consistent with the perfect match, then each vertex belongs to only one hyperedge, meaning that the data points do not overlap. Otherwise, if the coverage result is inconsistent with the perfect match, then there is an overlapping region, that is, at least one data point belongs to multiple hyperedges, as formally expressed in formula (5).
[0036] (5); In the formula, "Existent" is a quantifier indicating the existence of (at least one). In this case, the data point... The overlapping regions, representing the areas of overlapping data points, are the intersections of the hyperedges. Therefore, the degree of overlap of the entire hypergraph can be measured by calculating the size of the intersection of each pair of hyperedges.
[0037] (6); In the formula, Represents two super edges and The size of the intersection of hypergraphs, i.e., the number of vertices they commonly contain. This metric helps us understand the distribution of data points across different clusters or hyperedges. Efficient partitioning based on hypergraph-based overlapping clustering is shown in the example. Figure 2 As shown: (a) is the original data points, (b) is the hyperedge constructed based on the local structural relationship of the data points, (c) is the hypergraph constructed based on the local structural relationship, (d) is the search for the minimum cover of the hypergraph, and (e) is the minimum cover of this hypergraph, while completing the overlapping clustering.
[0038] Furthermore, the efficient hypergraph-based overlapping clustering partitioning algorithm is loosely or tightly coupled with the previously proposed graph neural network-based and variational autoencoder-based overlapping representation learning to form a deep overlapping clustering model. The designed algorithm is then validated using various standard datasets and performance testing standards.
[0039] This embodiment also discloses an overlapping region judgment and overlapping clustering system, including: The hypergraph construction module is used to calculate the similarity between data points in a given dataset, generate hyperedges based on multiple data points using the similarity, and construct a hypergraph with data points as vertices and hyperedges as connections. The hypergraph optimization module is used to obtain the minimum coverage set based on the hyperedge set with the goal of covering all vertices. The overlapping region judgment module is used to compare the minimal coverage set with the predefined perfect matching model. If the minimal coverage set is consistent with the perfect matching model, it is determined that there is no overlapping region; if the minimal coverage set is inconsistent with the perfect matching model, it is determined that there is an overlapping region. The overlapping clustering partitioning module is used to map each data point to one or more corresponding hyperedges based on the judgment results, so as to obtain the overlapping clustering results.
[0040] Optionally, the hypergraph building module includes: The similarity calculation unit is used to calculate the similarity between data points in a given dataset; The hyperedge generation unit is used to set a similarity threshold and group data points with similarity higher than the similarity threshold into the same set, and generate hyperedges based on the data point set; Hypergraph building unit is used to construct a hypergraph with data points as vertices and hyperedges as connections.
[0041] Optionally, the overlapping region determination module also includes an overlapping region quantization unit, which is used to quantify the degree of overlap of the hypergraph by calculating the intersection of any two hyperedges in the minimal cover set.
[0042] The technical solution disclosed in this embodiment can be directly applied to data scenarios with complex relational structures, such as social network analysis, multi-label classification, and bioinformatics.
[0043] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0044] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for determining overlapping regions and dividing overlapping clusters, characterized in that, The specific steps are as follows: Calculate the similarity between data points in a given dataset, generate hyperedges based on multiple data points using the similarity, and construct a hypergraph with the data points as vertices and the hyperedges as connections. With the goal of covering all vertices, obtain the minimum coverage set based on the hyperedge set; The minimum coverage set is compared with a predefined perfect matching model. If the minimum coverage set matches the perfect matching model, it is determined to be a non-overlapping region; if the minimum coverage set does not match the perfect matching model, it is determined to be an overlapping region. Based on the judgment results, each data point is mapped to one or more corresponding hyperedges to obtain overlapping clustering results.
2. The overlapping region judgment and overlapping clustering method according to claim 1, characterized in that, The steps for obtaining the hyperedge are as follows: Calculate the similarity between each data point in the given dataset, and construct the hyperedge based on multiple data points whose similarity exceeds a preset similarity threshold.
3. The method for determining overlapping regions and clustering overlapping regions according to claim 1, characterized in that, The steps for obtaining the minimal cover set are as follows: Step 21: Initialize the covered vertex set and the uncovered vertex set; Step 22: If the set of uncovered vertices is not empty, proceed to steps 23-24; if the set of uncovered vertices is empty, proceed to step 25. Step 23: Obtain the number of vertices that are not covered within the coverage area of each hyperedge. ; Step 24, The largest superedge is added to the covered set, and the vertices covered by the superedge are removed from the uncovered vertex set. Then, proceed to step 22. Step 25: Output the covering set as the minimal covering set.
4. The method for determining and clustering overlapping regions according to claim 1, characterized in that, The perfect matching model is defined as: a set of superedges that covers all vertices and whose intersection is empty for any two superedges.
5. The method for determining overlapping regions and clustering overlapping regions according to claim 1, characterized in that, If overlapping regions exist, an overlap metric is also included, specifically: the degree of overlap of the hypergraph is quantified by calculating the intersection of any two hyperedges in the minimal cover set.
6. A system for determining overlapping regions and classifying overlapping clusters, characterized in that, include: The hypergraph construction module is used to calculate the similarity between data points in a given dataset, generate hyperedges based on multiple data points using the similarity, and construct a hypergraph with the data points as vertices and the hyperedges as connections. The hypergraph optimization module is used to obtain the minimum coverage set based on the hyperedge set with the goal of covering all vertices. The overlapping region determination module is used to compare the minimum coverage set with a predefined perfect matching model. If the minimum coverage set is consistent with the perfect matching model, it is determined that there is no overlapping region; if the minimum coverage set is inconsistent with the perfect matching model, it is determined that there is an overlapping region. The overlapping clustering partitioning module is used to map each data point to one or more corresponding hyperedges based on the judgment result, so as to obtain the overlapping clustering result.
7. The overlapping region judgment and overlapping clustering system according to claim 6, characterized in that, The hypergraph construction module includes: The similarity calculation unit is used to calculate the similarity between data points in a given dataset; The hyperedge generation unit is used to set a similarity threshold and group data points with similarity higher than the similarity threshold into the same set, and generate hyperedges based on the data point set; The hypergraph construction unit is used to construct a hypergraph with the data points as vertices and the hyperedges as connections.
8. The overlapping region judgment and overlapping clustering system according to claim 6, characterized in that, The overlapping region determination module also includes an overlapping region quantization unit, which is used to quantify the degree of overlap of the hypergraph by calculating the intersection of any two hyperedges in the minimal cover set.