A robust hierarchical clustering method

Abstract

The application discloses a robust hierarchical clustering method, which performs cleaning, filling and standardization processing on an input data set; constructs a restricted hierarchical separation tree and a corresponding tree metric space according to the processed data set; selects a clustering center on the constructed hierarchical separation tree with probability, performs hierarchical clustering according to the clustering center, and outputs a clustering result; and measures the change amplitude of the algorithm under the condition of random deletion of data points by introducing an average sensitivity quantitative index, and evaluates the robustness and consistency of the algorithm to data disturbance. The method solves the problem that the output result of hierarchical clustering may change dramatically under slight data modification when the data distribution remains unchanged, so that the clustering result can remain stable even under normal data disturbance, thereby improving the practical applicability and explanation ability of the algorithm.

Description

Technical Field

[0001] This invention relates to the field of hierarchical clustering algorithm technology, and in particular to a robust hierarchical clustering method. Background Technology

[0002] Hierarchical clustering is one of the oldest and most widely used data clustering methods. Its goal is not merely to discover single partitions of data, but to extract the hierarchical structure of the data by constructing a rooted tree (often called a dendrogram). This hierarchical structure reflects the relationships between data at multiple granular levels, providing powerful interpretability for data analysis. The leaves of the tree represent individual data points, while the internal nodes represent the clustering results. Common implementations of hierarchical clustering include single-link, full-link, and average-link. The single-link method merges the most similar points from two clusters; the full-link method considers the least similar points; and the average-link method calculates the similarity by averaging the similarity of all points. These methods measure the similarity between clusters in different ways, directly affecting the structural characteristics and interpretability of the clustering results.

[0003] Current research on hierarchical clustering largely focuses on improving clustering quality, such as optimizing the hierarchy and consistency of the trees. However, the crucial issue of cluster stability is often overlooked. Stability can be defined as the ability of an algorithm to maintain consistent output when dealing with data perturbations (such as random deletion of data points) while keeping the data distribution constant. This stability is usually quantified and measured using average sensitivity, which is particularly important for practical applications because the output is often used as the basis for understanding the data structure. Some existing research attempts to improve stability by identifying and eliminating outliers, but outlier detection may not be applicable to all situations, and even without outliers, small changes in data points can still lead to significant changes in clustering. Summary of the Invention

[0004] The purpose of this invention is to provide a robust hierarchical clustering method that solves the problem that the output results of hierarchical clustering may change drastically with small data modifications when the data distribution remains unchanged. This method enables the clustering results to remain stable even under normal data perturbations, thereby improving the practical applicability and interpretability of the algorithm.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A robust hierarchical clustering method, the method comprising:

[0007] Step 1: Clean, fill, and standardize the input dataset;

[0008] Step 2: Based on the dataset processed in Step 1, construct a restricted hierarchical splitting tree and its corresponding tree metric space;

[0009] Step 3: Select cluster centers on the constructed hierarchical splitting tree with probability, perform hierarchical clustering based on the cluster centers, and output the clustering results;

[0010] Step 4: By introducing the quantitative metric of average sensitivity, we measure the magnitude of change of the algorithm when data points are randomly deleted, and evaluate its robustness and consistency to data perturbation.

[0011] As can be seen from the technical solution provided by the present invention, the above method solves the problem that hierarchical clustering output results may change drastically with minor data modifications when the data distribution remains unchanged, so that the clustering results can remain stable even under normal data perturbation, thereby improving the practical applicability and interpretability of the algorithm. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of a robust hierarchical clustering method provided in an embodiment of the present invention;

[0014] Figure 2 This is a schematic diagram illustrating the dimensional partitioning process of a two-dimensional dataset with a side length of 4, as described in an embodiment of the present invention.

[0015] Figure 3 This is a schematic diagram of the one-dimensional discrete dataset to be processed according to an embodiment of the present invention;

[0016] Figure 4 This is a schematic diagram of the hierarchical separation tree obtained after partitioning according to the method of the embodiments of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, and do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0018] like Figure 1The diagram illustrates a robust hierarchical clustering method provided in an embodiment of the present invention. The method includes:

[0019] Step 1: Clean, fill, and standardize the input dataset;

[0020] In this step, during the cleaning of the input dataset, missing values ​​need to be handled first by imputing them using the mean, interpolation, or predicted values. If some points in the dataset have missing values ​​exceeding half of the dimension, they are directly deleted.

[0021] Then, the dataset is deduplicated to remove duplicate samples or records, ensuring the independence and uniqueness of the dataset. Subsequently, outliers need to be identified and handled, as outliers may significantly interfere with distance calculations or clustering results. Specifically, outliers are identified through statistical methods (such as z-score or interquartile range) and removed or corrected according to specific needs.

[0022] Finally, the cleaned dataset is standardized or normalized. Specifically:

[0023] Normalization scales data to the [0, 1] interval, which is suitable for situations where the range of feature values ​​varies greatly; standardization transforms data into a distribution with a mean of 0 and a standard deviation of 1, which is suitable for feature data with different units or scales.

[0024] By performing the above preprocessing operations, the scale differences between features can be effectively eliminated, so that the contribution of each feature to the distance or similarity calculation during the clustering process is balanced, thereby improving the robustness and interpretability of the clustering results.

[0025] Step 2: Based on the dataset processed in Step 1, construct a restricted hierarchical splitting tree and its corresponding tree metric space;

[0026] In this step, P = {p1, ..., p} is used. n Let} denote the preprocessed dataset, with a size of n. The dataset is a set of points in discrete Euclidean space. Then, the maximum and minimum distances between any two points in the dataset are recorded, with the minimum distance considered as 1. The dataset is then resized, and Δ represents the maximum distance in the processed dataset, which is the ratio of the minimum distance to the maximum distance in the original dataset. This results in a new discrete set of points, still denoted as P. d represents the dimension of the dataset;

[0027] For two points p and q in Euclidean space, the Euclidean distance between them is defined as Dist(p, q) = ||pq||2. For a point p and a set of centers C, the distance between them is defined as Dist(p, C) = min c∈CDist(p, c), where if we want to consider the discrete point set in Euclidean space... Hierarchical clustering is performed, and its clustering cost function Cost(P, C) is defined as follows:

[0028]

[0029] Before dividing the data by side length Δ, the points in the point set are randomly offset by the same amount, with the offset sampled uniformly across the range [0, ..., Δ]. d After random offsetting, the probability of points in the dataset falling on the dividing boundary is extremely small, and the distribution of the original dataset remains unchanged.

[0030] Next, we need to partition the dataset dimensionally. Dimensional partitioning is a prerequisite for constructing a restricted hierarchical splitting tree. The specific dimensional partitioning process is as follows:

[0031] A restricted hierarchical split tree is an embedding of a dataset where all edge weights are positive, all leaf nodes are at the same level, edges at each level have the same weight, and the weight of each level gradually decreases along the path from the root to the leaf.

[0032] Constructing a restricted hierarchical splitting tree (2-RHST) involves a quadtree structure, starting with a d-dimensional cube with side length Δ, and recursively subdividing the space into 2... d There are three equal sub-cubes, each non-empty sub-cube having its side length halved at each level, connected to its parent node with an edge weight of . Where i is the level of the hierarchical split tree; this recursive subdivision continues until the side length of the sub-cube is reduced to 1, at which point each sub-cube contains exactly one point, thus obtaining a hierarchical split tree. The hierarchical split tree has all leaf nodes on the same level, and all non-leaf nodes on the tree are empty. Non-leaf nodes are used later for hierarchical cluster center labeling.

[0033] For example, such as Figure 2 The diagram shown illustrates the process of partitioning a two-dimensional dataset with a side length of 4, according to an embodiment of the present invention. Figure 2 This paper presents a preprocessed two-dimensional dataset with an initial side length of 4 and containing 7 data points. The dataset is then recursively partitioned spatially, as follows:

[0034] First partitioning: Divide the original rectangle into four equal parts along its center point, generating four sub-rectangles with a side length of 2. It's worth noting that although the lower left quadrant contains only one isolated data point, it still needs further partitioning according to the hierarchical splitting tree rules.

[0035] Second partitioning: Each sub-rectangle is further divided into four equal parts, at which point the side length of the sub-rectangle is reduced to 1. After two iterations of partitioning, all data points are precisely assigned to independent subspaces, strictly satisfying the termination condition that each unit-length sub-rectangle contains only one data point.

[0036] It should be noted that the distribution characteristics of this dataset are not naturally formed, but are achieved through steps 1 and 2. This standardization and step-by-step operation ensures that the spatial partitioning algorithm can converge to the ideal state in a finite number of iterations, thus building a regular data structure foundation for subsequent data analysis tasks.

[0037] like Figure 3 The diagram shown is a schematic representation of the one-dimensional discrete dataset to be processed according to an embodiment of the present invention. Figure 4 The diagram shown is a schematic representation of the hierarchical splitting tree obtained after partitioning according to the method of this embodiment of the invention. It is assumed here that there are no duplicate points in the dataset, which can be achieved by... Figure 3 This is achieved by preprocessing a one-dimensional discrete dataset.

[0038] After processing with a hierarchical splitting tree, the metric space is transformed from Euclidean space to a tree metric space. In the tree metric space, the distance Dist between two points p and q is... T (p, q) is defined as the shortest path length from p to q on a hierarchical split tree; similarly, for a point p and a center set C, the distance between them is defined as Dist. T (p, C) = min c∈ C Dist T (p, c), for a point set P in the tree metric space, perform hierarchical clustering, with the hierarchical clustering cost function Cost. T (P, C) is defined as:

[0039]

[0040] Therefore, based on the dataset processed in step 1, a restricted hierarchical splitting tree and its corresponding tree metric space are obtained.

[0041] Step 3: Select cluster centers on the constructed hierarchical splitting tree with probability, perform hierarchical clustering based on the cluster centers, and output the clustering results;

[0042] The specific process in this step is as follows:

[0043] Each time, a point c is selected from the point set P with probability. t As the center of round t, the probability distribution is determined by the following formula:

[0044] c t∝exp(-λCost T (P,x∪S t-1 ))

[0045] Cost T (P,x∪S t-1 This indicates adding point x to the set S of centers selected in the previous t-1 rounds. t-1 The cost in the tree metric space; λ is a parameter controlling the selection distribution; when t-1 = 0, let... Specifically, λ is a uniformly random sample from:

[0046]

[0047] x here opt represents the optimal solution in each round, i.e., the point that minimizes the cost in each round; n represents the size of the point set P; ∈ is a parameter that controls the precision; x opt The value is dynamically changed in each round and is obtained by recording the minimum cost of adding a center in each round before selecting a center.

[0048] In the iterative operation of selecting the center in a hierarchical split tree, after selecting the center point x (which exists as a data instance in the leaf node of the tree) at time t, the following normalization operation is performed:

[0049] 1) Ancestor chain backtracking mark: Traverse backwards along the path from the leaf node to the root node to locate the ancestor node v that is unmarked and farthest from the current leaf node (i.e., the highest level unmarked internal node in the tree structure).

[0050] 2) Node state update: Set the identifier of the ancestor node v to the current center point x to complete the binding of internal nodes and data instances (Note: In the initial state, the identifier fields of all internal nodes are empty, and only leaf nodes carry the original data point information).

[0051] 3) Iterative update of the center set: The newly labeled center point x is dynamically injected into the center set S of the previous time step. t-1 The set of cluster centers S at the current moment is formed. t ;

[0052] 4) Obtain the central set S at time t. t Then, based on the markings on the tree, i.e. the number of markings of the central nodes, all points are assigned to the nearest center to form clusters;

[0053] Since clustering is performed after constructing the hierarchical splitting tree, this ensures that adding a new center at time t in the tree metric space will split one of the clusters at time t-1 into two, satisfying the property of hierarchical clustering.

[0054] Step 4: By introducing the quantitative metric of average sensitivity, we measure the magnitude of change of the algorithm when data points are randomly deleted, and evaluate its robustness and consistency to data perturbation.

[0055] In this step, for the hierarchical clustering algorithm, the input point set P = {p1, p2, ..., p...} n} represents n points, and the goal is to measure the average sensitivity of the algorithm's output after uniformly and randomly deleting a point;

[0056] For example: P (i) ={p1, ..., p i-1 p i+1 , ..., p n} represents the dataset after the i-th point is deleted. For a deterministic k-median algorithm, ... The output clustering result of point set P is represented by... Represents the point set P (i) The output of the algorithm is denoted by A, and the average sensitivity on the point set P is defined as:

[0057]

[0058] Here, π is used to denote a bijection from {0, ..., k} to itself;

[0059] For randomized algorithms, average sensitivity is measured by the distance between their output distributions; specifically, the average sensitivity (relative to the total variance distance) α(A, P) of randomized algorithm A to dataset P is defined as:

[0060]

[0061] Where d TV (A(x), A(x) (i) )) represents A(x) and A(x) (i) The total variation distance between distributions quantifies the average change in the algorithm's output distribution when a single data point is removed; n represents the size of the dataset.

[0062] If we use the function f(A, P, P) {(i)} ) indicates the transformation from the output of algorithm A(P) to the output of algorithm A(P). (i) The cost of ) is the symmetric difference between the clustering results output by the two algorithms. Therefore, the average sensitivity β(A, P) is defined as:

[0063]

[0064] For the k-median hierarchical clustering problem, given a dataset P and the number of clusters k, for any precision control parameter ∈, it will output a set of cluster centers S and the clustering results, with the expected cost satisfying:

[0065] Cost T (P, S)≤O(d·logΔ)·(1+∈) k ·OPT(P,k)

[0066] At least The probability is true, and the average sensitivity of the algorithm is Here, OPT(P, k) represents the optimal value of dataset P given the number of clusters k.

[0067] The above method selects the center point probabilistically by weighing the costs, avoiding deterministic selection based solely on the minimum cost, thereby enhancing the robustness and versatility of the algorithm.

[0068] It is worth noting that the contents not described in detail in the embodiments of the present invention belong to the prior art known to those skilled in the art.

[0069] In summary, the method described in the embodiments of the present invention has the following advantages:

[0070] 1. This invention introduces the quantitative index of "average sensitivity" to systematically evaluate the stability of hierarchical clustering algorithms to data perturbation. Compared with existing methods, under the premise that the data distribution remains unchanged, it can effectively reduce the significant changes in clustering results caused by missing data, noise or outliers, thereby greatly improving the robustness of the algorithm.

[0071] 2. The algorithm designed in this invention is not only applicable to datasets with outliers or missing values, but also applicable to all data points, ensuring that the clustering results remain consistent even under normal data perturbation. This makes the algorithm adaptable to more practical scenarios, such as medical data analysis, market research, or high-dimensional data clustering, which are application areas with high stability requirements.

[0072] 3. This invention can generate more consistent clustering results, making the hierarchical structure of hierarchical clustering more reliable. Because the results are more stable, users can more easily understand the internal structure of the data from multiple granular levels, thereby improving the explanatory power and decision-making reference value of the clustering results in practical applications.

[0073] 4. Compared with traditional methods, this invention does not require the design of complex outlier detection or classification mechanisms. Instead, it optimizes the algorithm directly for all data points through a more natural robustness standard. This not only simplifies the implementation of the algorithm but also improves its versatility and practicality.

[0074] 5. This invention introduces average sensitivity analysis from a completely new perspective, quantifies the impact of data perturbation on clustering results and incorporates it into the algorithm design. This is a direction that has not been studied in existing hierarchical clustering algorithms. This method expands the theoretical framework of clustering algorithms and has high academic value and practical significance.

[0075] Furthermore, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware, and the corresponding program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0076] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A robust hierarchical clustering method, characterized in that, The method includes: Step 1: Clean, fill, and standardize the input dataset; Step 2: Based on the dataset processed in Step 1, construct a restricted hierarchical splitting tree and its corresponding tree metric space; Step 3: Select cluster centers on the constructed hierarchical splitting tree with probability, perform hierarchical clustering based on the cluster centers, and output the clustering results; Step 4: By introducing the quantitative metric of average sensitivity, we measure the magnitude of change of the algorithm when data points are randomly deleted, and evaluate its robustness and consistency to data perturbation.

2. The robust hierarchical clustering method according to claim 1, characterized in that, In step 1, during the cleaning process of the input dataset, missing values ​​need to be handled first. They are filled using the mean, interpolation, or predicted values. If some points in the dataset have missing values ​​exceeding half of the dimension, they are deleted directly. Then, the dataset is deduplicated to remove duplicate samples or records, ensuring the independence and uniqueness of the dataset. The next step is to identify and process outliers, specifically by using statistical methods to identify outliers and then choosing to remove or correct them based on specific needs. Finally, the cleaned dataset is standardized or normalized. Specifically: Normalization scales data to the [0, 1] range, which is suitable for situations where the range of feature values ​​varies greatly; standardization transforms data into a distribution with a mean of 0 and a standard deviation of 1, which is suitable for feature data with different units or scales.

3. The robust hierarchical clustering method according to claim 1, characterized in that, In step 2, P = {p1, ..., p} is used. n Let} represent the preprocessed dataset, where the size of the dataset is n, and the dataset is a set of points in discrete Euclidean space; Then, record the maximum and minimum distances between any two points in the point set. Treat the minimum distance as 1 and resize the dataset. Use Δ to represent the maximum distance in the processed dataset, which is the ratio of the minimum distance to the maximum distance in the original dataset. The processed dataset is a new discrete point set, still denoted as P. d represents the dimension of the dataset; For two points p and q in Euclidean space, the Euclidean distance between them is defined as Dist(p, q) = ||pq||2. For a point p and a set of centers C, the distance between them is defined as Dist(p, C) = min c∈C Dist(p, c), where if we want to consider the discrete point set in Euclidean space... Hierarchical clustering is performed, and its clustering cost function Cost(P, C) is defined as follows: Before dividing the data by side length Δ, the points in the point set are randomly offset by the same amount, with the offset sampled uniformly across the range [0, ..., Δ]. d After random offsetting, the probability of points in the dataset falling on the dividing boundary is extremely small, and the distribution of the original dataset remains unchanged. Next, we need to partition the dataset dimensionally. Dimensional partitioning is a prerequisite for constructing a restricted hierarchical splitting tree. The specific dimensional partitioning process is as follows: A restricted hierarchical split tree is an embedding of a dataset where all edge weights are positive. All leaf nodes of the tree are at the same level, and the edges at each level have the same weight. The weight of each level gradually decreases along the path from the root to the leaf. Constructing a restricted hierarchical splitting tree involves a quadtree structure, starting with a d-dimensional cube with side length Δ, and recursively subdividing the space into 2... d There are three equal sub-cubes, each non-empty sub-cube having its side length halved at each level, connected to its parent node with an edge weight of . Where i is the level of the hierarchical split tree; this recursive subdivision continues until the side length of the sub-cube is reduced to 1, at which point each sub-cube contains exactly one point, thus obtaining a hierarchical split tree. The hierarchical split tree has all leaf nodes on the same level, and all non-leaf nodes on the tree are empty. Non-leaf nodes are used later for hierarchical cluster center labeling. After processing with a hierarchical splitting tree, the metric space is transformed from Euclidean space to a tree metric space. In the tree metric space, the distance Dist between two points p and q is... T (p, q) is defined as the shortest path length from p to q on a hierarchical split tree; similarly, for a point p and a center set C, the distance between them is defined as Dist. T (p, C) = min c∈C Dist T (p, c), for a point set P in the tree metric space, perform hierarchical clustering, with the hierarchical clustering cost function Cost. T (P, C) is defined as: Therefore, based on the dataset processed in step 1, a restricted hierarchical splitting tree and its corresponding tree metric space are obtained.

4. The robust hierarchical clustering method according to claim 1, characterized in that, The process of step 3 is as follows: Each time, a point c is selected from the point set P with probability. t As the center of round t, the probability distribution is determined by the following formula: c t ∝exp(-λCost T (P,x∪S t-1 )) Cost T (P,x∪S t-1 This indicates adding point x to the set S of centers selected in the previous t-1 rounds. t-1 The cost in the tree metric space; λ is a parameter controlling the selection distribution; when t-1 = 0, let... Specifically, λ is a uniformly random sample from: x here opt This represents the optimal solution in each round, i.e., the point that minimizes the cost in each round; n represents the size of the point set P; ∈ is a parameter that controls the precision; x opt The value is dynamically changed in each round and is obtained by recording the minimum cost of adding a center in each round before selecting a center. In the iterative operation of selecting the center in a hierarchical split tree, after selecting the center point x at time t, the following normalization operation is performed: 1) Ancestor chain backtracking mark: Traverse backwards along the path from the leaf node to the root node, locate the ancestor node v that is unmarked and farthest from the current leaf node on the path, which is the highest level unmarked internal node in the tree structure. 2) Node state update: Set the identifier of the ancestor node v to the current center point x to complete the binding of the internal node to the data instance; 3) Iterative update of the center set: The newly labeled center point x is dynamically injected into the center set S of the previous time step. t-1 The set of cluster centers S at the current moment is formed. t ; 4) Obtain the central set S at time t. t Then, based on the markings on the tree, i.e. the number of markings of the central nodes, all points are assigned to the nearest center to form clusters; Since clustering is performed after constructing the hierarchical splitting tree, this ensures that adding a new center at time t in the tree metric space will split one of the clusters at time t-1 into two, satisfying the property of hierarchical clustering.

5. The robust hierarchical clustering method according to claim 4, characterized in that, In step 4, for the hierarchical clustering algorithm, the input point set P = {p1, p2, ..., p...} n } represents n points, and the goal is to measure the average sensitivity of the algorithm's output after uniformly and randomly deleting a point; Use P (i) ={p1, ..., p i-1 p i+1 , ..., p n } represents the dataset after the i-th point is deleted. For a deterministic k-median algorithm, we use... The output clustering result of point set P is represented by... Represents the point set P (i) The output of the algorithm is denoted by A, and the average sensitivity on the point set P is defined as: Here, π is used to denote a bijection from {0, ..., k} to itself; For randomized algorithms, average sensitivity is measured by the distance between their output distributions; specifically, the average sensitivity α(A, P) of randomized algorithm A to dataset P is defined as: Where d TV (A(x), A(x) (i) )) represents A(X) and A(x) (i) The total variation distance between distributions quantifies the average change in the algorithm's output distribution when a single data point is removed; n represents the size of the dataset. If we use the function f(A, P, P) {(i)} ) indicates the transformation from the output of algorithm A(P) to the output of algorithm A(P). (i) The cost of ) is the symmetric difference between the clustering results output by the two algorithms. Therefore, the average sensitivity β(A, P) is defined as: For the k-median hierarchical clustering problem, given a dataset P and the number of clusters k, for any precision control parameter ∈, it will output a set of cluster centers S and the clustering results, with the expected cost satisfying: Cost T (P,S)≤O(d·logΔ)·(1+∈) k ·0PT(P,k) At least The probability is true, and the average sensitivity of the algorithm is Here, OPT(P, k) represents the optimal value of dataset P given the number of clusters k.

Images