Entropy-Terminated Weighted Hierarchical Clustering Method and System

Abstract

A weighted hierarchical clustering method and system based on entropy termination is proposed, relating to the field of cluster analysis. It addresses the significant shortcomings of existing traditional hierarchical clustering algorithms in adaptability to different types of data. The method includes: Step 1, constructing the shortest connected path, connecting all sample points in the shortest connected path, building a minimum spanning tree based on the sample points, and assigning a corresponding weight to each edge; Step 2, in the splitting phase, deleting the edge with the largest weight from Step 1, calculating the weights based on the ratio of the weight to the entropy before and after each split, and plotting a line graph to quickly determine the final number of clusters; Step 3, testing the number of clusters in Steps 1 and 2 using a test dataset to obtain the test results of the entropy-termination-based weighted hierarchical clustering, thus realizing the entropy-termination-based weighted hierarchical clustering method. It is also applicable to pattern recognition, image recognition, and bioinformatics technologies.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of clustering analysis, and particularly relates to a weighted hierarchical clustering method and system based on entropy termination. BACKGROUND

[0002] As an important technique in data mining and machine learning, clustering analysis aims to divide data points into different clusters, so that data points in the same cluster have high similarity, while data points between different clusters have great differences. In many fields such as computer vision, data analysis, medical images, clustering analysis plays a key role. With the increasing scale and complexity of data, the demand for efficient and accurate clustering methods is increasingly urgent.

[0003] According to its internal logic and operation mode, clustering methods can be mainly divided into partition clustering, hierarchical clustering and spectral clustering.

[0004] Further subdivision, according to the standard used, partition clustering can be divided into two categories based on distance and density. Among them, the density-based clustering method, such as DBSCAN and DPC, has shown unique advantages. They analyze the density of data points and divide high-density areas into clusters. The outstanding feature of this method is that it can accurately find clusters of any shape and has good ability in dealing with noise and outliers. In the distance-based clustering method, K-means as one of the commonly used algorithms has distinct characteristics. It divides the data set into a fixed number of clusters, each cluster is represented by a cluster center (centroid), and the data points in the cluster are as close to the center as possible, so as to realize the clustering of data.

[0005] Hierarchical clustering algorithm can be divided into two types according to its operation direction, which are agglomerative and divisive. Both types construct a clustering tree by iteratively merging or splitting data points. In the agglomerative algorithm, single-linkage. The single-linkage algorithm defines the distance between clusters as the distance between two sample points in two clusters, and selects the nearest two points to merge when iteratively merging two clusters. This way is relatively intuitive. In the divisive algorithm, DIANA is a typical representative.

[0006] Spectral clustering is a graph-based clustering method that treats data points as nodes in a graph and the similarity or distance between nodes as edges in the graph. The core idea of spectral clustering is to construct a similarity matrix and perform eigenvalue decomposition, then use traditional clustering algorithms (such as K-means) to cluster the eigenvectors. This method can identify samples of arbitrary shape, converge to global optimal solution, and has strong adaptability to data distribution, with excellent clustering results. Among them, FBMC is a method that can directly solve the discrete exponential matrix without any processing. This method uses bipartite graphs to speed up the construction and optimization of affinity graphs. And two optimization schemes FBMC1 and FBMC2 are proposed. These two methods add balance factors in the model to reduce the biased clustering results.

[0007] Although density-based clustering algorithms such as DBSCAN and DPC have significant advantages in handling complex data sets and identifying clusters of arbitrary shape, they also face some challenges in practical use. DBSCAN requires users to specify the neighborhood radius and the minimum number of points within the neighborhood, which requires users to have a deep understanding of the distribution characteristics of the data in order to set the parameters reasonably. The DPC algorithm requires the determination of the cutoff distance, which not only requires users to have a deep understanding of the algorithm principle, but often needs to be tested and adjusted several times to obtain the best result. Especially for users who are not familiar with the DPC clustering algorithm, it is a difficult process to determine the appropriate parameters. In contrast, K-means and traditional hierarchical clustering algorithms are relatively simple to operate, usually only need to specify the number of clusters, which makes them widely used in some scenarios where the clustering effect is not particularly high and the result needs to be obtained quickly. However, traditional hierarchical clustering algorithms are not perfect, and their adaptability is poor when dealing with diverse data sets, and they are usually more suitable for specific types of data sets, which limits their application range to some extent. Therefore, in-depth research and improvement of clustering algorithms to balance the complexity and adaptability of the algorithm have become an important topic in the field of clustering research.

[0008] Traditional hierarchical clustering algorithms often show insufficient adaptability when faced with diverse data. This adaptability problem mainly reflects the sensitivity of the algorithm to data distribution characteristics. Different types of data distribution may lead to biased clustering results. Moreover, noise has a greater impact on it. These algorithms usually rely heavily on user experience, such as determining clustering parameters and selecting appropriate splitting or merging strategies, which requires a lot of prior knowledge and repeated trials. This not only increases the cost of use, but also makes non-professional users face great challenges when applying these algorithms. SUMMARY

[0009] The present application aims to solve the problem that the hierarchical clustering algorithm in the prior art often shows insufficient adaptability when facing diverse data, and the adaptability problem mainly lies in the sensitivity of the data distribution characteristics, different types of data distribution may cause deviation of the clustering result, and noise has a greater impact. It is highly dependent on the user's experience, such as determining the clustering parameters and selecting the appropriate splitting or merging strategy, which requires a lot of prior knowledge and repeated attempts. This not only increases the use cost, but also makes non-professional users face great challenges when applying these algorithms.

[0010] To solve the above technical problems, the present application is realized by the following technical solutions:

[0011] Scheme one, the present application provides a weighted hierarchical clustering method based on entropy termination, the method comprises:

[0012] Step 1, constructing the shortest connected path, connecting all sample points in the shortest connected path, constructing the minimum spanning tree based on the sample points, and assigning a corresponding weight to each edge;

[0013] Step 2, in the splitting stage, deleting the edge with the largest weight in step 1, calculating the weight based on the ratio of the weight and the entropy before and after each splitting, and drawing a broken line graph, which is used to quickly determine the number of final clusters;

[0014] Step 3, testing the number of clusters in step 1 and step 2 by using test data set, obtaining the test result of the weighted hierarchical clustering based on entropy termination, and realizing the weighted hierarchical clustering method based on entropy termination.

[0015] Further, a preferred embodiment is provided, and the shortest connected path in step 1 is realized by Dijkstra algorithm.

[0016] Further, a preferred embodiment is provided, and the method for calculating the weight in step 2 comprises:

[0017] Calculate the degree of each point, select the point with a degree of 1, assign a value to the level of the connected edge, and delete the edge; repeat the operation until the level calculation of all edges is completed, after the level calculation is completed, the product of the length and the level of each edge is taken as the maximum weight of the edge, and the edge with the maximum weight is selected at each splitting.

[0018] Further, a preferred embodiment is provided, and step 2 further comprises the step of dividing the number of clusters.

[0019] Further, a preferred embodiment is provided, and the test data set in step 3 is all unlabeled data set, and step 3 further comprises the step of identifying the number of clusters from the unlabeled data set.

[0020] Furthermore, in a preferred embodiment, step 3 further includes a step of dimensionality reduction of the data in the test dataset.

[0021] Furthermore, a preferred implementation method is provided in which the t-SNE method is used to achieve dimensionality reduction in step 3.

[0022] Option 2: A weighted hierarchical clustering system based on entropy termination, wherein the system includes:

[0023] The shortest connected path construction module is used to construct the shortest connected path, connect all sample points in the shortest connected path, construct the minimum spanning tree based on the sample points, and assign a corresponding weight to each edge;

[0024] The weight calculation module is used to delete the edge with the largest weight in the shortest connected path construction module during the splitting phase. It calculates the weight based on the ratio of the weight to the entropy before and after each split and draws a line graph to quickly determine the number of final clusters.

[0025] The testing module is used to test the number of clusters in the shortest connectivity path construction module and the weight calculation module using a test dataset, and obtain the test results of the entropy-terminated weighted hierarchical clustering, thus realizing the entropy-terminated weighted hierarchical clustering method.

[0026] Option 3: A computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method described in any one of Options 1.

[0027] Option 4: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in any one of Options 1.

[0028] The advantages of this invention are:

[0029] The weighted hierarchical clustering method and system based on entropy termination described in this invention first constructs a minimum spanning tree and assigns a corresponding weight to each edge. During the splitting phase, the edge with the largest weight among the current edges is deleted first. Furthermore, this application improves the algorithm by using a line graph composed of the ratio of weight to entropy before and after each split to quickly determine the final number of clusters. This algorithm is tested on commonly used datasets, and the results show that, compared with traditional hierarchical clustering algorithms, the splitting hierarchical clustering algorithm exhibits significant advantages, significantly improving adaptability to diverse datasets while lowering the barrier to entry, enabling more users to easily apply and obtain high-quality clustering results.

[0030] This invention is also applicable to the fields of pattern recognition, image recognition, and bioinformatics. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the single-linkage method described in Implementation Method Eleven.

[0032] Figure 2 This is a schematic diagram of a polyline plot drawn based on the ratio of the weight of the selected edge to the entropy during splitting, as described in Implementation Method 1.

[0033] Figure 3 This is a clustering diagram of the five clusters determined by the maximum descent point as described in Implementation Method Eleven.

[0034] Figure 4 This is a comparative diagram of the various clustering algorithms described in Implementation Method Eleven.

[0035] Figure 5 This is a schematic diagram illustrating the results of different clustering methods for the dataset described in Implementation Method Eleven.

[0036] Figure 6 This is a schematic diagram illustrating the optimization of data with poor results as described in Implementation Method Eleven.

[0037] Figure 7 This is a schematic diagram of the result after dimensionality reduction of high-dimensional data as described in Implementation Method Eleven.

[0038] Figure 8 This is a comparative illustration of applying the split hierarchical clustering algorithm described in Implementation Method Eleven to the Olivetti face dataset. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0040] Implementation Method 1: This implementation method proposes a weighted hierarchical clustering method based on entropy termination, the method comprising:

[0041] Step 1: Construct the shortest connected path, connect all sample points in the shortest connected path, construct the minimum spanning tree based on the sample points, and assign a corresponding weight to each edge;

[0042] Step 2: In the splitting phase, delete the edge with the largest weight in Step 1. Calculate the weights based on the ratio of weight to entropy before and after each split and draw a line graph to quickly determine the final number of clusters.

[0043] Step 3: Use the test dataset to test the number of clusters in Step 1 and Step 2, obtain the test results of the entropy-terminated weighted hierarchical clustering, and implement the entropy-terminated weighted hierarchical clustering method.

[0044] Implementation Method 2: This implementation method is a further limitation of the weighted hierarchical clustering method based on entropy termination described in Implementation Method 1. In step 1, the shortest connected path is constructed using the Dijkstra algorithm.

[0045] Implementation Method 3: This implementation method further defines the entropy-terminated weighted hierarchical clustering method described in Implementation Method 1. The method for calculating the weights in step 2 includes:

[0046] Calculate the degree of each point, select the point with a degree of 1, assign a degree to the edge connected to it, and delete the edge; repeat this operation until the degree of all edges is calculated. After the degree calculation is completed, the product of the length of each edge and its degree is used as the final weight of the edge, and the edge with the largest weight is selected at each split.

[0047] Implementation Method 4: This implementation method further defines the weighted hierarchical clustering method based on entropy termination described in Implementation Method 1. Step 2 also includes the step of dividing the number of clusters.

[0048] Implementation Method 5: This implementation method further defines the weighted hierarchical clustering method based on entropy termination described in Implementation Method 1. The test datasets mentioned in step 3 are all unlabeled datasets, and step 3 also includes a step of identifying the number of clusters from the unlabeled datasets.

[0049] Implementation Method Six: This implementation method further defines the weighted hierarchical clustering method based on entropy termination described in Implementation Method One. Step 3 also includes a step of dimensionality reduction of the data in the test dataset.

[0050] Implementation Method Seven: This implementation method is a further limitation of the weighted hierarchical clustering method based on entropy termination described in Implementation Method Six. In step 3, the t-SNE method is used to achieve dimensionality reduction.

[0051] Implementation Method 8: This implementation method proposes a weighted hierarchical clustering system based on entropy termination, the system comprising:

[0052] The shortest connected path construction module is used to construct the shortest connected path, connect all sample points in the shortest connected path, construct the minimum spanning tree based on the sample points, and assign a corresponding weight to each edge;

[0053] The weight calculation module is used to delete the edge with the largest weight in the shortest connected path construction module during the splitting phase. It calculates the weight based on the ratio of the weight to the entropy before and after each split and draws a line graph to quickly determine the number of final clusters.

[0054] The testing module is used to test the number of clusters in the shortest connectivity path construction module and the weight calculation module using a test dataset, and obtain the test results of the entropy-terminated weighted hierarchical clustering, thus realizing the entropy-terminated weighted hierarchical clustering method.

[0055] Implementation Method Nine: This implementation method provides a computer device, including a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the method described in any one of Implementation Methods One to Seven.

[0056] Implementation Method 10: This implementation method provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in any one of Implementation Methods 1 to 7.

[0057] Implementation Method Eleven: This implementation method provides an example, which is used to explain the above-described implementation methods one through eight. The specific example is as follows:

[0058] See Figures 1 to 7 This embodiment describes an innovative entropy-terminated weighted hierarchical clustering algorithm to overcome the limitations of traditional hierarchical clustering algorithms. The core objective of this algorithm is to significantly improve its adaptability to diverse datasets while simultaneously lowering the barrier to entry, enabling more users to easily apply it and obtain high-quality clustering results.

[0059] The algorithm design described in this embodiment provides an effective approach to progressively refine clustering from the global to the local. Specifically, we first connect all sample points by constructing the shortest connected paths, initially using Dijkstra's algorithm with a priority queue to construct these paths. The advantage of this method is that it keeps the time complexity of the construction process low, achieving O(n log n) time complexity. 2This approach, which calculates edge weights based on log n (log n), is more efficient for handling large-scale data. Instead of relying on a single factor, we propose a novel, comprehensive approach to weighting edges. While splitting based solely on edge length might work for certain types of data, such as those with large inter-cluster distances, as it tends to select the longest edge (similar to a reversed single-linkage algorithm), this simple edge-length-based method becomes problematic when dealing with data that has small inter-cluster distances but large intra-cluster edge distances. In such cases, simply selecting the longest edge can cause the algorithm to get stuck in regions with low point density, as the longest edge might be an edge point of a low-density cluster, and such data often contains a large number of such edge points, ultimately leading to poor clustering results. See also... Figure 1 As shown, an experiment was conducted using test3 data as an example. Based on the above problems, this application proposes a novel weight calculation method that cleverly combines distance and point location factors. By introducing a weighting approach, we can effectively reduce the weight of edges between edge points within a cluster while increasing the weight of edges between clusters, thus ensuring that inter-cluster connections are prioritized during the splitting process. The specific steps of the algorithm include: In an undirected connected graph, first, accurately calculate the degree of each point; then, selectively select points with a degree of 1, assign a level to the edges connected to them, and then delete the edges. Repeat this series of operations until the level calculation of all edges is completed. After the level calculation is completed, the product of the length and level of each edge is taken as the final weight of that edge, and the edge with the largest weight is selected during each split. The detailed weight calculation steps are as follows:

[0060]

[0061]

[0062] In addition, to further optimize the algorithm, we introduce the concept of entropy to help determine the number of clusters. In traditional clustering algorithms, determining the appropriate number of clusters is often a tedious and difficult process, requiring users to experiment with numerous parameters, yet still failing to obtain the optimal clustering results. Our proposed entropy-based method provides an effective way to solve this problem. The core idea of ​​this method is to utilize the changing pattern of entropy in each round of edge weights. Specifically, during normal splitting, the entropy of edge weights in each round does not suddenly increase, but when the selected edge is an inter-cluster connection, the weight of the selected edge decreases significantly between the first and last rounds. By recording the ratio of the weight of the selected edge to the entropy at each split and plotting the corresponding line graph, we can clearly observe this trend. Figure 2 Users can identify the clusters with the largest decrease in size based on the line chart, and use this as the final splitting result. For example, ... Figure 3The clustering results of the five clusters determined by the maximum descent point in the line graph are presented, visually demonstrating the effectiveness of the method. This entropy-based cluster selection method greatly reduces the burden on users and improves the automation and accuracy of the clustering process.

[0063] Furthermore, to comprehensively test the effectiveness of our proposed algorithm, we carefully selected 12 sets of artificial data and 3 sets of real data, and compared them with DBSCAN, DPC, K-means, single-linkage, and FBMC2 clustering algorithms. Since the clustering results of DBSCAN, DPC, and single-linkage are largely influenced by parameters, we calculated the optimal parameters for DBSCAN, DPC, and single-linkage algorithms iteratively. The optimal parameters for K-means and FBMC2 are affected by the number of iterations; therefore, we set the maximum number of iterations to 30. We then conducted a fair comparison using the results under these optimal parameter conditions (the maximum number of clusters for single-connected hierarchical clustering was set to 500). The optimal parameters are detailed in Table 1.

[0064] Table 1. Optimal parameters for various methods

[0065]

[0066]

[0067]

[0068] First, we selected four datasets, each containing 5000 points (S1, S2, S3, and S4), for our experiments. Considering that none of these datasets were pre-labeled, we used three internal evaluation metrics—DBI (Davies-Bouldin Index), SC (Silhouette Coefficient), and CH (Calinski-Harabasz Index)—to comprehensively evaluate the feasibility of our method. The corresponding results are presented in [link to results]. Figure 4 And in Table 2.

[0069] from Figure 4The results clearly show that when processing datasets S1 and S2, due to the small difference between the inter-cluster distance and intra-cluster distance, the single-linkage method tends to misclassify some points on the periphery of clusters as new clusters. Although this approach can form more clusters and identify the core parts of the clusters to some extent, the effect is poor from the perspective of clustering quality. For S3 and S4, with even smaller inter-cluster distances, the single-linkage method cannot even complete the clustering task correctly. For the other five clustering methods, they all obtained relatively suitable cluster partitions on these four datasets. However, for the DBSCAN algorithm, from S1 to S4, as the inter-cluster distance gradually decreases, a smaller neighborhood radius and more points are needed to form core points. This leads to many points on the periphery of clusters being incorrectly identified as interference points, thus affecting the quality of the clustering results. In contrast, DPC, K-means, FBMC2, and our proposed ETWHC algorithm, due to their noise-ignoring characteristics, obtained better clustering results on these datasets.

[0070] Table 3. Evaluation results of clustering results from different clustering methods.

[0071]

[0072]

[0073] Further analysis of the data in Table 2 reveals that DPC, K-means, FBMC2, and ETWHC algorithms all demonstrate their advantages. These algorithms are designed to ensure the similarity of data points within clusters, a characteristic that enables them to obtain good data clustering results when processing these datasets

[34] . Specifically, DPC and K-means algorithms effectively divide data points into appropriate clusters through their respective specific mechanisms, maintaining the compactness within clusters. The FBMC2 algorithm, based on its principle of minimizing the connection weights between clusters, makes the clustering results perform well in terms of intra-cluster similarity. The ETWHC algorithm we propose, through a new weight calculation method and entropy termination condition, also ensures good intra-cluster similarity. For the DBSCAN algorithm, since its algorithm principle requires ensuring the density of core points, it inevitably treats some edge points as interference points when processing this type of dataset, which to some extent reduces the accuracy of the clustering results. For the single-linkage algorithm, in datasets S1 and S2, since the distinction between inter-cluster distance and intra-cluster edge distance is not very obvious, a large number of intra-cluster edge points are incorrectly classified into new clusters. To obtain relatively good clustering results, a larger number of clusters needs to be specified in the initial settings; for example, the number of clusters in S1 is set to 96, and the initial number of clusters in S2 is set to 290. However, for S3 and S4, due to the severe overlap of many cluster edges, the inter-cluster spacing and intra-cluster spacing are almost indistinguishable. Even with a maximum cluster size of 500, a satisfactory clustering result cannot be obtained. These experimental results fully demonstrate the effectiveness and superiority of the ETWHC algorithm proposed in this application in handling such challenging datasets.

[0074] Secondly, we selected six representative datasets (2-cluster, 2d-3c-no123, D31, Flame, Jain, and Spiral) to conduct a comprehensive qualitative and quantitative comparison of DBSCAN, DPC, K-means, single-linkage, FBMC2, and ETWHC. Given that all six datasets contain labels, we specifically selected three widely accepted external evaluation metrics: NMI (Normalized Mutual Information), ARI (Adjusted Rand Index), and FMI (Fowlkes-Mallows Index) to accurately evaluate the effectiveness of our proposed method. The corresponding results are presented in [Table showing results]. Figure 5 And in Table 3.

[0075] from Figure 5It can be clearly observed that the clustering results of ETWHC all follow the distribution of the original data, with only a small number of points with excessively close inter-cluster distances failing to be correctly distinguished. In contrast, the results of other clustering methods deviate from the original distribution to some extent. Specifically, the DBSCAN algorithm, due to its strict requirements on the density around the core points, causes some discrete points on the periphery of the clusters to be misclassified into appropriate clusters, a phenomenon particularly evident in the D31 dataset. Furthermore, when the dataset contains clusters with excessively close inter-cluster distances or excessively large intra-cluster distances, such as the 2d-3c-no123 and Jain datasets, the DBSCAN algorithm may encounter the problem of some data not being clustered according to the original distribution. For the DPC algorithm, because it clusters around a few core points, this mechanism leads to some cluster partitioning failing to follow the original data distribution in some datasets, such as 2-cluster and Jain. Both K-means and FBMC2 algorithms partition based on the similarity between objects, which makes them perform poorly when dealing with chained datasets (such as 2-cluster and spiral datasets) because the nature of this data structure makes similarity-based partitioning prone to misclassification. The single-linkage algorithm tends to merge clusters that are close together, a characteristic that means that when processing datasets like 2d-3c-no123, some data on the periphery of clusters cannot be effectively merged.

[0076] Further analysis of the data in Table 3 reveals that ETWHC achieved excellent scores for clustering 2-cluster, 2d-3c-no123, D31, and flame datasets. The DBSCAN algorithm received a slightly lower score due to uneven inter-cluster density and small inter-cluster distance in the 2d-3c-no123 dataset, but still achieved relatively good scores on other datasets. The single-linkage algorithm received a poor external score because the inter-cluster distance and intra-cluster distance were not clearly distinguished in the 2d-3c-no123 and Jain datasets, but performed reasonably well on other datasets. The DPC, K-means, and FBMC2 algorithms, primarily focused on achieving high intra-cluster similarity, did not achieve ideal external scores when processing the 2-cluster and Jain datasets. Furthermore, the K-means algorithm, by ignoring the location information of the data, also received poor scores when processing the flame and spiral datasets.

[0077] Table 3. Evaluation results of clustering results from different clustering methods

[0078]

[0079]

[0080] Because the ETWHC method only calculates distance information and does not consider density information, it results in poor performance for clusters with connected data points. For example, aggregation and cluto-t4-8k suffer from poor ETWHC clustering results because data points directly connect different clusters. Figure 6 As shown in the first and second rows, we implemented a specific optimization method. We used the sum of the densities of the two points connected by the line segment as the denominator of the weights, adding density information to the weights to increase the influence of positional information. The results are as follows. Figure 6 The third and fourth rows show the results. The results demonstrate that better recognition results were achieved after targeted optimization.

[0081] Table 4. Optimization strategies for data with poor results

[0082]

[0083]

[0084] In addition, we selected three classic real datasets (i.e., Iris dataset, seeds dataset, wdbc Wisconsin breast cancer dataset, isolet dataset, and Yales face dataset) to conduct a more in-depth qualitative and quantitative comparison of DBSCAN, DPC, K-means, single-linkage, FBMC2, and ETWHC. For the high-dimensional data isolet and Yales, we chose to use PCA for dimensionality reduction

[35] . Since these five datasets all contain labels and are all high-dimensional data, we selected three external evaluation metrics, NMI, ARI, and FMI, to measure the effectiveness of the algorithm. For ease of analysis and visualization, we used the t-SNE (t-distributedStochastic Neighbor Embedding) algorithm to reduce the dimensionality of these three datasets

[37] , converting them into a two-dimensional representation. The corresponding results are presented in the figure below. Figure 6 As shown in Table 4.

[0085] Table 4. Evaluation results of clustering results from different clustering methods

[0086]

[0087] As shown in the figure, on the Iris dataset, DPC, K-means, single-linkage, FBMC2, and ETWHC perform well in clustering, effectively capturing the inherent structure of the data. However, the DBSCAN algorithm fails to accurately cluster according to the original distribution. On the Seeds dataset, DBSCAN, DPC, K-means, FBMC2, and ETWHC also demonstrate good clustering performance, further proving their effectiveness in handling real-world data. The single-linkage algorithm, however, still performs poorly. On the wdbc dataset, its unique data structure—having only two clusters with completely overlapping edges—makes it difficult for a single clustering method to achieve the same high results as the previous datasets. Nevertheless, among these six clustering algorithms, DPC, K-means, FBMC2, and ETWHC still achieve relatively good clustering results. In contrast, DBSCAN and single-linkage algorithms fail to effectively separate the two clusters, resulting in clustering results that do not accurately reflect the distribution characteristics of the original data. For high-dimensional data such as isolet and Yaleb, FBMC2 and ETWHC methods showed good adaptability, while DPC and K-means performed poorly, and DBSCAN and single-linkage were difficult to adapt to high-dimensional data.

[0088] In-depth analysis of these data reveals that the DPC, K-means, FBMC2, and ETWHC algorithms have relatively low requirements for cluster shape and distribution, focusing more on the similarity measurement between data points. Therefore, they can cluster data into several highly similar clusters. This characteristic allows them to achieve satisfactory clustering results even when cluster overlap exists in the original data. In contrast, the DBSCAN algorithm's design philosophy emphasizes the distinction between cluster cores and edges; that is, core points need to have a high density, while edge points have a lower density. Therefore, even with overlapping clusters, although the edge point density of a single cluster is low, the overlapping edges of multiple clusters mean that the density of edge points in the original distribution is not significantly lower than the density of core points. This leads to lower evaluations of DBSCAN's clustering results on the wdbc and Yaleb datasets. The single-linkage algorithm, due to its excessive focus on clear boundaries between clusters, performs poorly when dealing with complex data structures such as the seed, wdbc, isolet, and Yaleb datasets.

[0089] Finally, see Figure 8As shown, this application applies ETWHC to the Olivetti face dataset for testing. The Olivetti dataset, provided by AT&T Cambridge Labs, contains a set of facial images from 40 different subjects, with 10 different 112*92 pixel images per subject, forming a 400x10304 data matrix. In this experiment, we selected the first 10 datasets, totaling 100 images. To reduce storage and computational complexity, we first compressed the original images from 112*92 to 15*15. Then, we performed Principal Component Analysis (PCA) to filter out attributes with a cumulative contribution rate greater than 90%, thereby reducing data dimensionality while retaining key information. Finally, we performed clustering using ETWHC, and the results are shown below. Figure 8 As shown in the figure, in the clustering results, the first, fourth, and eighth images of the first group were assigned to the second group, and the second image of the tenth group was assigned to the eighth group, with a calculated accuracy of 96%. This result fully demonstrates that the algorithm has good adaptability and robustness even for complex face data. When experimented on all 40 groups of data, the accuracy was 0.81%, which is significantly better than the 76% obtained by C. Moraru's recently proposed SNN-DPC algorithm. This series of experimental results strongly demonstrates that the ETWHC algorithm not only performs excellently on traditional datasets, but also exhibits broad applicability and superior performance in complex real-world application scenarios (such as face image data clustering), providing strong evidence for its application in more fields.

[0090] This implementation proposes a novel clustering method—the split hierarchical clustering algorithm ETWHC. Compared to traditional hierarchical clustering algorithms, ETWHC effectively handles situations where inter-cluster similarity is not significant and intra-cluster edge point density is low. Simultaneously, it avoids the limitations of algorithms like DPC, K-means, and FBMC, which may neglect data distribution when pursuing intra-cluster similarity. Compared to DBSCAN, ETWHC performs better in handling imbalanced inter-cluster distributions. Overall, ETWHC exhibits good adaptability to most data types, considering not only intra-cluster similarity but also adapting to different data distribution characteristics, achieving results similar to or better than other traditional clustering algorithms at a relatively low cost.

[0091] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0092] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A weighted hierarchical clustering method based on entropy termination, characterized in that, The method comprises: Step 1, constructing a shortest connected path, connecting all sample points in the shortest connected path, constructing a minimum spanning tree based on the sample points, and assigning a corresponding weight to each edge; Step 2, in the splitting stage, deleting the edge with the largest weight in step 1, and drawing a corresponding broken line graph based on the ratio of the weight of the selected edge to the entropy at each splitting, which is used to quickly determine the number of final clusters; Step 3, testing the number of clusters in steps 1 and 2 using a test data set to obtain a test result of the entropy-terminated weighted hierarchical clustering, and implementing the entropy-terminated weighted hierarchical clustering method; the test data set includes a face image data set; The method for calculating the weight of the edge in step 2 comprises: Calculating the degree of each point, selecting a point with a degree of 1, assigning a level to the connected edge, and deleting the edge; repeat the operation until the level calculation of all edges is completed, and after the level calculation is completed, the product of the length and the level of each edge is taken as the maximum weight of the edge, and the edge with the largest weight is selected at each splitting.

2. The entropy termination based weighted hierarchical clustering method of claim 1, wherein, The construction of the shortest connected path in step 1 is realized by Dijkstra algorithm.

3. The entropy termination based weighted hierarchical clustering method of claim 1, wherein, Step 2 further comprises a step of dividing the number of clusters.

4. The entropy termination based weighted hierarchical clustering method of claim 1, wherein, The test data set in step 3 is a no-label data set, and step 3 further comprises a step of identifying the number of clusters from the no-label data set.

5. The entropy termination based weighted hierarchical clustering method of claim 1, wherein, Step 3 further comprises a step of dimension reduction on the data in the test data set.

6. The entropy termination based weighted hierarchical clustering method of claim 5, wherein, The dimension reduction in step 3 is realized by t-SNE method.

7. A system for entropy termination based weighted hierarchical clustering, characterized in that, The system comprises: A shortest connected path construction module for constructing a shortest connected path, connecting all sample points in the shortest connected path, constructing a minimum spanning tree based on the sample points, and assigning a corresponding weight to each edge; A weight calculation module for deleting the edge with the largest weight in the shortest connected path construction module in the splitting stage, drawing a corresponding broken line graph based on the ratio of the weight of the selected edge to the entropy at each splitting, and quickly determining the number of final clusters; A test module for testing the number of clusters in the shortest connected path construction module and the weight calculation module using a test data set to obtain a test result of the entropy-terminated weighted hierarchical clustering, and implementing the entropy-terminated weighted hierarchical clustering method; the test data set includes a face image data set; The method for calculating the weight of the edge in the weight calculation module comprises: Calculating the degree of each point, selecting a point with a degree of 1, assigning a level to the connected edge, and deleting the edge; repeat the operation until the level calculation of all edges is completed, and after the level calculation is completed, the product of the length and the level of each edge is taken as the maximum weight of the edge, and the edge with the largest weight is selected at each splitting.

8. Computer device comprising a memory and a processor, characterized in that The memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method of any one of claims 1-6.

9. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the processor executes the computer program, the steps of the method of any one of claims 1-6 are realized.

Images