Weighted hierarchical clustering method and system based on entropy termination

Abstract

The invention discloses a weighted hierarchical clustering method and system based on entropy termination, and relates to the field of clustering analysis. The problem that an existing traditional hierarchical clustering algorithm is obviously insufficient in adaptability when coping with different types of data is solved. The method comprises the following steps: step 1, constructing a shortest communication path, connecting all sample points in the shortest communication path, constructing a minimum spanning tree based on the sample points, and endowing each edge with a corresponding weight; 2, in the splitting stage, deleting the edge with the maximum weight in the step 1, calculating the weight based on the ratio of the weight to the entropy before and after each splitting, and drawing a broken line graph for quickly determining the number of final clusters; and step 3, testing the number of the clusters in the step 1 and the step 2 by using the test data set to obtain a test result of the weighted hierarchical clustering based on entropy termination, thereby realizing the weighted hierarchical clustering method based on entropy termination. The method is also suitable for the technical fields of pattern recognition, image recognition and bioinformatics.

Description

Technical Field

[0001] The present invention relates to the technical field of cluster analysis, and in particular to a weighted hierarchical clustering method and system based on entropy termination. Background Art

[0002] Cluster analysis, as an important technology in data mining and machine learning, aims to divide data points into different clusters so that data points within the same cluster have high similarity, while data points between different clusters have large differences. Cluster analysis plays a key role in many fields, such as computer vision, data analysis, and medical imaging. With the continuous increase in data size and complexity, the demand for efficient and accurate clustering methods is becoming increasingly urgent.

[0003] Clustering methods can be mainly divided into partition clustering, hierarchical clustering and spectral clustering according to their internal logic and operation methods.

[0004] Looking further into the details, partition clustering can be divided into two categories, distance-based and density-based, according to the standards adopted. Among them, density-based clustering methods, such as DBSCAN and DPC, show unique advantages. They divide high-density areas into clusters by analyzing the density of data points. The outstanding feature of this method is that it can accurately discover clusters of any shape and has good ability in dealing with noise and outliers. Among the distance-based clustering methods, K-means, as one of the commonly used algorithms, has distinct characteristics. It achieves data clustering by dividing the data set into a fixed number of clusters, each cluster is represented by the cluster center (centroid), and the data points in the cluster are as close to the center as possible.

[0005] Hierarchical clustering algorithms can be divided into two types according to their operation direction: agglomerative and divisive. Both types build clustering trees by iteratively merging or splitting data points. In agglomerative algorithms, single-linkage. The single-linkage algorithm defines the distance between clusters as the distance between two sample points in two clusters. When iteratively merging two clusters, the two closest points are selected for merging. This method is relatively intuitive. In divisive algorithms, DIANA is a typical representative.

[0006] Spectral clustering is a clustering method based on graph theory. It regards data points as nodes in the graph and the similarity or distance between nodes as edges in the graph. The core idea of ​​spectral clustering is to construct a similarity matrix and perform eigendecomposition on it, and then cluster the eigenvectors using traditional clustering algorithms (such as K-means). This method can identify sample spaces of arbitrary shapes and converge to the global optimal solution. It has strong adaptability to data distribution and excellent clustering effect. Among them, FBMC is a method that can directly solve discrete exponential matrices without any processing. This method uses bipartite graphs to accelerate the construction and optimization process of affinity graphs. Two optimization schemes FBMC1 and FBMC2 are proposed. These two methods specifically add balance factors to the model to alleviate skewed clustering results.

[0007] Although density-based clustering algorithms such as DBSCAN and DPC have significant advantages in processing complex data sets and identifying clusters of arbitrary shapes, they also face some challenges in actual use. DBSCAN requires users to specify the neighborhood radius and the minimum number of points in the neighborhood, which requires users to have a deep understanding of the distribution characteristics of the data in order to set the parameters reasonably. The DPC algorithm needs to determine the cutoff distance, which not only requires users to have a deep understanding of the algorithm principles, but also often requires multiple trials and adjustments to obtain the best results. Especially for users who are not familiar with the DPC clustering algorithm, determining the appropriate parameters is a more difficult process. In contrast, K-means and traditional hierarchical clustering algorithms are relatively simple in operation, usually only requiring the number of clusters to be specified, which makes them widely used in some scenarios where the clustering effect is not particularly high and the results need to be obtained quickly. However, traditional hierarchical clustering algorithms are not perfect. When dealing with diverse data sets, their adaptability is poor and they are usually more suitable for specific types of data sets, which limits their application scope to a certain extent. Therefore, in-depth research and improvement of clustering algorithms to balance the complexity and adaptability of the algorithms has become an important topic in the current clustering research field.

[0008] Traditional hierarchical clustering algorithms often show the defect of insufficient adaptability when facing diverse data. This adaptability problem is mainly reflected in its sensitivity to data distribution characteristics. Different types of data distribution may lead to deviations in clustering results. Moreover, noise has a greater impact on them. These algorithms are usually highly dependent on the user's experience. For example, when determining clustering parameters and choosing appropriate splitting or merging strategies, a lot of prior knowledge and repeated attempts are required. This not only increases the cost of use, but also makes it a huge challenge for non-professional users to apply these algorithms. Summary of the invention

[0009] The present invention aims to solve the defect that the hierarchical clustering algorithm in the prior art often exhibits insufficient adaptability when facing diverse data. The adaptability problem is mainly reflected in its sensitivity to the data distribution characteristics. Different types of data distribution may lead to deviations in clustering results, and noise also has a greater impact on it. It is usually highly dependent on the user's experience. For example, when determining clustering parameters and selecting appropriate splitting or merging strategies, a lot of prior knowledge and repeated attempts are required. This not only increases the cost of use, but also makes non-professional users face huge challenges when applying these algorithms.

[0010] To solve the above technical problems, the present invention is achieved through the following technical solutions:

[0011] Solution 1: The present invention proposes a weighted hierarchical clustering method based on entropy termination, the method comprising:

[0012] Step 1: construct the shortest connected path, connect all sample points in the shortest connected path, construct a minimum spanning tree based on the sample points, and assign corresponding weights to each edge;

[0013] Step 2: In the splitting phase, delete the edge with the largest weight in step 1, calculate the weight based on the ratio of the weight to the entropy before and after each split, and draw a line graph to quickly determine the number of final clusters;

[0014] Step 3: Use the test data set to test the number of clusters in step 1 and step 2, obtain the test results of the weighted hierarchical clustering based on entropy termination, and implement the weighted hierarchical clustering method based on entropy termination.

[0015] Furthermore, a preferred implementation is provided, in which the construction of the shortest connection path in step 1 is implemented by the Dijkstra algorithm.

[0016] Further, a preferred implementation is provided, in which the method for calculating the weight in step 2 includes:

[0017] Calculate the degree of each point, select the point with a degree of 1, assign the level of the edge connected to it, and delete the edge; repeat this operation until the level calculation of all edges is completed. After the level calculation is completed, the product of the length of each edge and the level is used as the final weight of the edge, and the edge with the largest weight is selected at each split.

[0018] Furthermore, a preferred implementation is provided, wherein step 2 also includes a step of dividing the number of clusters.

[0019] Furthermore, a preferred implementation manner is provided, wherein the test data sets described in step 3 are all unlabeled data sets, and step 3 also includes a step of identifying the number of clusters from the unlabeled data sets.

[0020] Furthermore, a preferred implementation is provided, wherein step 3 also includes a step of reducing the dimension of the data in the test data set.

[0021] Furthermore, a preferred implementation is provided, in which the t-SNE method is used in step 3 to achieve dimensionality reduction.

[0022] Solution 2: A weighted hierarchical clustering system based on entropy termination, the system comprising:

[0023] The shortest connected path construction module is used to construct the shortest connected path, connect all sample points in the shortest connected path, build a minimum spanning tree based on the sample points, and assign corresponding weights to each edge;

[0024] The weight calculation module is used to delete the edge with the largest weight in the shortest connected path construction module during the splitting phase, calculate the weight based on the ratio of the weight to the entropy before and after each splitting, and draw a line graph to quickly determine the number of final clusters;

[0025] The test module is used to test the number of clusters in the shortest connected path construction module and the weight calculation module using a test data set, obtain the test results of the weighted hierarchical clustering based on entropy termination, and implement the weighted hierarchical clustering method based on entropy termination.

[0026] Solution three: A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes any one of the methods described in Solution one.

[0027] Solution 4: A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of Solution 1.

[0028] The present invention is beneficial in that:

[0029] The weighted hierarchical clustering method and system based on entropy termination described in the present invention first constructs a minimum spanning tree and assigns corresponding weights to each edge. In the splitting stage, the edge with the largest weight among the current edges is deleted first. In addition, the present application has also improved the algorithm, using a line graph composed of the ratio of the weight to the entropy before and after each split to quickly determine the number of final clusters. The algorithm was applied to commonly used data sets for testing, and the results showed that compared with the traditional hierarchical clustering algorithm, the divisive hierarchical clustering algorithm of the present application showed significant advantages, significantly improved the adaptability to diverse data sets, and at the same time lowered the threshold for using the algorithm, allowing more users to easily apply and obtain high-quality clustering results.

[0030] The present invention is also applicable to the technical fields of pattern recognition, image recognition and bioinformatics. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of the single-linkage method described in Implementation Example 11.

[0032] Figure 2 This is a schematic diagram of a broken line drawn based on the ratio of the weight of the edge selected during splitting to the entropy described in Implementation Method 1.

[0033] Figure 3 A clustering diagram of the five clusters determined by the maximum drop point described in Implementation Method 11.

[0034] Figure 4 This is a schematic diagram for comparing various clustering algorithms described in Implementation Example 11.

[0035] Figure 5 This is a schematic diagram of the results of different clustering methods for a data set as described in Implementation Example 11.

[0036] Figure 6 This is a schematic diagram of optimizing data with poor results as described in Implementation Example 11.

[0037] Figure 7 This is a schematic diagram of the result after reducing the dimensionality of high-dimensional data as described in Implementation Method 11.

[0038] Figure 8 This is a comparative schematic diagram of applying the divisive hierarchical clustering algorithm described in Implementation Example 11 to the Olivetti face dataset. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the implementation methods of the present application clearer, the technical solutions in the implementation methods of the present application will be clearly and completely described below in conjunction with the drawings in the implementation methods of the present application. Obviously, the described implementation methods are only part of the implementation methods of the present application, not all of the implementation methods.

[0040] Embodiment 1: This embodiment proposes a weighted hierarchical clustering method based on entropy termination, the method comprising:

[0041] Step 1: construct the shortest connected path, connect all sample points in the shortest connected path, construct a minimum spanning tree based on the sample points, and assign corresponding weights to each edge;

[0042] Step 2: In the splitting phase, delete the edge with the largest weight in step 1, calculate the weight based on the ratio of the weight to the entropy before and after each split, and draw a line graph to quickly determine the number of final clusters;

[0043] Step 3: Use the test data set to test the number of clusters in step 1 and step 2, obtain the test results of the weighted hierarchical clustering based on entropy termination, and implement the weighted hierarchical clustering method based on entropy termination.

[0044] Implementation method 2: This implementation method further limits the weighted hierarchical clustering method based on entropy termination described in implementation method 1. In step 1, the shortest connection path is constructed by Dijkstra algorithm.

[0045] Implementation method 3: This implementation method further limits the weighted hierarchical clustering method based on entropy termination described in implementation method 1. The method for calculating the weight in step 2 includes:

[0046] Calculate the degree of each point, select the point with a degree of 1, assign the level of the edge connected to it, and delete the edge; repeat this operation until the level calculation of all edges is completed. After the level calculation is completed, the product of the length of each edge and the level is used as the final weight of the edge, and the edge with the largest weight is selected at each split.

[0047] Implementation mode 4: This implementation mode further limits the weighted hierarchical clustering method based on entropy termination described in implementation mode 1, and step 2 also includes a step of dividing the number of clusters.

[0048] Implementation method five: This implementation method further limits the weighted hierarchical clustering method based on entropy termination described in implementation method one. The test data sets described in step 3 are all unlabeled data sets, and step 3 also includes the step of identifying the number of clusters from the unlabeled data set.

[0049] Implementation method 6: This implementation method further limits the weighted hierarchical clustering method based on entropy termination described in implementation method 1, and step 3 also includes a step of reducing the dimension of the data in the test data set.

[0050] Implementation method seven: This implementation method is a further limitation of the weighted hierarchical clustering method based on entropy termination described in implementation method six. In step 3, the t-SNE method is used to achieve dimensionality reduction.

[0051] Embodiment 8: This embodiment proposes a weighted hierarchical clustering system based on entropy termination, the system comprising:

[0052] The shortest connected path construction module is used to construct the shortest connected path, connect all sample points in the shortest connected path, build a minimum spanning tree based on the sample points, and assign corresponding weights to each edge;

[0053] The weight calculation module is used to delete the edge with the largest weight in the shortest connected path construction module during the splitting phase, calculate the weight based on the ratio of the weight to the entropy before and after each splitting, and draw a line graph to quickly determine the number of final clusters;

[0054] The test module is used to test the number of clusters in the shortest connected path construction module and the weight calculation module using a test data set, obtain the test results of the weighted hierarchical clustering based on entropy termination, and implement the weighted hierarchical clustering method based on entropy termination.

[0055] Embodiment 9. This embodiment proposes a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method described in any one of embodiments 1 to 7.

[0056] Embodiment 10: This embodiment proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method described in any one of Embodiments 1 to 7 are implemented.

[0057] Implementation 11: This implementation proposes an example, which is used to explain the above-mentioned implementations 1 to 8. The specific example is as follows:

[0058] See also Figures 1 to 7 In order to overcome the limitations of traditional hierarchical clustering algorithms, we propose an innovative weighted hierarchical clustering algorithm based on entropy termination. The core goal of this algorithm is to significantly improve its adaptability to diverse data sets and at the same time lower the threshold for using the algorithm, so that more users can easily apply it and obtain high-quality clustering results.

[0059] The algorithm design described in this embodiment provides an effective way to gradually refine the clustering from the overall to the local. Specifically, we first connect all sample points by constructing the shortest connected path, and first use the Dijkstra algorithm of the priority queue to achieve the construction of the shortest connected path. The advantage of this method is that the time complexity of the construction process can be controlled at a low level, reaching O(n 2logn), which makes it more efficient when processing large-scale data. In calculating the weight of the edge, we do not simply rely on a single factor, but propose a new method that takes it into comprehensive consideration. If the splitting operation is performed only according to the length of the edge, this method may perform well for certain types of data, such as data with large inter-cluster distances, because it tends to select the edge with the largest distance, similar to the inverse single-linkage algorithm. However, when faced with data with small inter-cluster distances and large distances between edge points within a cluster, this simple method based on edge length will have serious problems. In this case, simply selecting the longest edge may cause the algorithm to fall into areas with low point density, because the longest edge may be an edge point of a cluster with a smaller density, and there are often a large number of such edge points in such data, which ultimately produces poor clustering results. See Figure 1 As shown, an experimental example is given using test3 data as an example. Based on the above problems, this application proposes a novel weight calculation method, which cleverly combines the distance and point location factors. By introducing the weight method, we can effectively reduce the weight of the edges between edge points within the cluster, while increasing the weight of the edges between clusters, thereby ensuring that the connections between clusters are given priority during the splitting process. The specific steps of the algorithm include: in an undirected connected graph, first accurately calculate the degree of each point, then selectively select points with a degree of 1, assign the level of the edges connected to them, and then delete the edge. Repeat this series of operations until the level calculation of all edges is completed. After the level calculation is completed, the product of the length of each edge and the level is used as the final weight of the edge, and the edge with the largest weight is selected at each split. The detailed weight calculation steps are as follows:

[0060]

[0061]

[0062] In addition, in order to further optimize the algorithm, we introduced the concept of entropy to assist in determining the number of clusters. In traditional clustering algorithms, determining the appropriate number of clusters is often a tedious and difficult process, requiring users to try a large number of parameters, but may still not be able to obtain the best clustering results. The entropy-based method we proposed provides an effective way to solve this problem. The core idea of ​​this method is to use the changing law of the entropy of edge weights in each round. Specifically, in the normal splitting process, the entropy of edge weights in each round will not suddenly increase. When the selected edges are inter-cluster connections, the weights of the edges selected in each round will drop significantly between the previous and next rounds. By recording the ratio of the weight of the selected edge to the entropy at each split and drawing the corresponding line graph, we can clearly observe this trend of change. Figure 2 The user can find the number of clusters with the largest drop according to the line graph and use it as the final split result. Figure 3The clustering results of the five clusters determined by the maximum drop point in the line graph are shown, which intuitively demonstrates the effectiveness of the method. This entropy-based cluster number selection method greatly reduces the burden on users and improves the automation and accuracy of the clustering process.

[0063] In addition, to fully test the effectiveness of our proposed algorithm, we carefully selected 12 sets of artificial data and 3 sets of real data, and compared them with DBSCAN, DPC, K-means, single-linkage, and FBMC2 clustering algorithms. Since the clustering results of DBSCAN, DPC, and single-linkage are largely affected by parameters, we calculated the optimal parameters of DBSCAN, DPC, and single-linkage algorithms through iterations, while K-means and FBMC2 algorithms are affected by the number of iterations, and the maximum number of iterations is set to 30. The results under this optimal parameter state are fairly compared (the maximum number of clusters for single-connected hierarchical clustering is set to 500), and the optimal parameters are detailed in Table 1:

[0064] Table 1. Optimal parameters for various methods

[0065]

[0066]

[0067]

[0068] First, we selected four datasets (i.e., S1, S2, S3, and S4) each containing 5,000 points to conduct experiments. Considering that none of these four datasets have pre-labeled labels, we selected three internal evaluation criteria, namely, DBI (Davies-Bouldin Index), SC (Silhouette Coefficient), and CH (Calinski-Harabasz Index), to comprehensively evaluate the feasibility of our method. The corresponding results are shown in Figure 4 and Table 2.

[0069] from Figure 4It can be clearly seen from the results that when processing data sets S1 and S2, due to the small difference between the cluster distance and the distance within the cluster, the single-linkage method tends to misjudge some of the points outside the cluster as new clusters. Although this method can form more clusters to a certain extent and identify the core part of the cluster, the effect is poor from the perspective of clustering quality. For S3 and S4 with smaller cluster distances, the single-linkage method cannot even complete the clustering task correctly. For the other five clustering methods, they all obtained relatively suitable cluster divisions on these four data sets. However, for the DBSCAN algorithm, from S1 to S4, due to the gradual decrease in cluster distance, a smaller neighborhood radius and more points are required to form core points, which leads to many points outside the cluster being mistakenly identified as interference points, thus affecting the quality of the clustering results. In contrast, DPC, K-means, FBMC2 and the ETWHC algorithm proposed by us have the characteristic of ignoring noise and obtain better clustering results on these data sets.

[0070] Table 3. Index evaluation results of clustering results of different clustering methods.

[0071]

[0072]

[0073] Further analysis of the data in Table 2 shows that the DPC, K-means, FBMC2, and ETWHC algorithms all demonstrate their advantages. These algorithms are designed to ensure the similarity of data points within a cluster as much as possible. This feature enables them to obtain good data clustering results when processing these data sets

[34] . Specifically, the DPC and K-means algorithms effectively divide data points into appropriate clusters through their own specific mechanisms, maintaining the compactness within the clusters. The FBMC2 algorithm, based on its principle of minimizing the connection weights between clusters, makes the clustering results perform well in terms of intra-cluster similarity. The ETWHC algorithm proposed by us also ensures good intra-cluster similarity through a new weight calculation method and entropy termination condition. For the DBSCAN algorithm, since its algorithm principle requires ensuring the density of core points, when processing such data sets, it is inevitable that some edge points will be regarded as interference points, which reduces the accuracy of the clustering results to a certain extent. For the single-linkage algorithm, in data sets S1 and S2, since the distinction between the inter-cluster distance and the intra-cluster edge distance is not very obvious, a large number of edge points within the cluster are mistakenly divided into new clusters. In order to obtain relatively good clustering results, a larger number of clusters needs to be specified in the initial settings, for example, the number of clusters of S1 is set to 96, and the initial number of clusters of S2 is set to 290. However, for S3 and S4, since the edges of many clusters overlap seriously, there is almost no difference between the distance between clusters and the distance within clusters. Even if the maximum number of clusters is set to 500, a satisfactory clustering result cannot be obtained. This series of experimental results fully demonstrates the effectiveness and superiority of the ETWHC algorithm proposed in this application in processing such challenging data sets.

[0074] Secondly, we selected six representative data (2-cluster, 2d-3c-no123, D31, Flame, Jain, spiral) to conduct a comprehensive qualitative and quantitative comparison of DBSCAN, DPC, K-means, single-linkage, FBMC2, and ETWHC. Given that these six data contain labels, we deliberately selected three widely recognized external evaluation indicators, namely NMI (Normalized Mutual Information), ARI (Adjusted Rand Index), and FMI (Fowlkes-Mallows Index), to accurately evaluate the effectiveness of our proposed method. The corresponding results are shown in Figure 5 and Table 3.

[0075] from Figure 5It can be clearly observed that the clustering results of ETWHC all follow the distribution of the original data, and only a small number of points with too close distances between clusters cannot be correctly distinguished. In contrast, the results of other clustering methods deviate from the original distribution to a certain extent. Specifically, the DBSCAN algorithm has strict requirements on the density around the core points, resulting in some discrete points on the periphery of the cluster not being divided into appropriate clusters. For example, this phenomenon is particularly obvious in the D31 dataset. In addition, when there are situations in which the cluster spacing is too tight or the intra-cluster spacing is too large in the dataset, such as the 2d-3c-no123 and Jain datasets, the DBSCAN algorithm will have the problem that some data cannot be clustered according to the original distribution. For the DPC algorithm, since it clusters with several core points as the center, this mechanism leads to the fact that in some datasets, such as 2-cluster and jain, the division of some clusters fails to follow the original data distribution. For K-means and FBMC2 algorithms, they divide objects based on similarity, which makes them perform poorly when processing chain data sets (such as 2-cluster, spiral), because the special nature of this data structure makes it easy to misjudge based on similarity. For the single-linkage algorithm, it tends to select clusters that are close to each other for merging. This feature makes it impossible to effectively merge some data outside the cluster when processing data sets such as 2d-3c-no123.

[0076] Further analysis of the data in Table 3 shows that the clustering results of 2-cluster, 2d-3c-no123, D31 and flame using ETWHC all obtained relatively excellent scores. The DBSCAN algorithm had a slightly lower evaluation result due to the uneven density between clusters and small cluster distances in the 2d-3c-no123 dataset, but still obtained relatively good scores on the rest of the data. The single-linkage algorithm obtained a poor external score due to the unclear distinction between cluster distances and intra-cluster distances in the 2d-3c-no123 and Jain datasets, but performed well on the rest of the data. The DPC, K-means and FBMC2 algorithms mainly focus on obtaining higher intra-cluster similarities, which makes them not obtain ideal external scores when processing the 2-cluster and jain datasets. In addition, the K-means algorithm also obtained poor scores when processing the flame and spiral datasets because it ignores the location information of the data.

[0077] Table 3. Index evaluation results of clustering results of different clustering methods

[0078]

[0079]

[0080] Since the ETWHC method only calculates the distance information without considering the density information, the effect is poor for data with connected data points. For example, aggregation and cluto-t4-8k have data points between different clusters that directly connect them, which leads to poor clustering results of ETWHC. Figure 6 As shown in the first and second rows, we have carried out a special optimization method for this purpose. We use the sum of the densities of the two points connected by the line segment as the denominator of the weight and add density information to the weight to increase the influence of the position information. The results are as follows Figure 6 The third and fourth rows are shown. From the results, it can be seen that after targeted optimization, better recognition results are obtained.

[0081] Table 4. Optimization for data with poor results

[0082]

[0083]

[0084] In addition, we also selected three classic real-world datasets (i.e., the Iris dataset, the seeds dataset, the wdbc dataset, the Breast Cancer Wisconsin (Diagnostic) dataset, the isolet sound dataset, and the Yales face dataset) to conduct a more in-depth qualitative and quantitative comparison of DBSCAN, DPC, K-means, single-linkage, FBMC2, and ETWHC. For data with higher dimensions, isolet and Yaleb chose to use PCA for dimensionality reduction

[35] . Since these five datasets all contain labels and are high-dimensional data, we selected three external evaluation indicators, NMI, ARI, and FMI, to measure the effectiveness of the algorithms. For ease of analysis and visualization, we used the t-SNE (t-distributed Stochastic Neighbor Embedding) algorithm to reduce the dimensionality of these three data

[37] and convert them into a two-dimensional space representation. The corresponding results are shown in Figure 6 and Table 4 shows.

[0085] Table 4. Index evaluation results of clustering results of different clustering methods

[0086]

[0087] As can be seen from the figure, on the Iris dataset, the clustering results of DPC, K-means, single-linkage, FBMC2 and ETWHC performed well and were able to effectively capture the intrinsic structure of the data, while the DBSCAN algorithm failed to accurately cluster according to the original distribution. On the seeds dataset, DBSCAN, DPC, K-means, FBMC2 and ETWHC also showed good clustering performance, which once again proved the effectiveness of these algorithms in processing actual data, while the performance of the single-linkage algorithm was still unsatisfactory. On the wdbc dataset, since there are only two clusters and the edges of the two clusters completely overlap, this special data structure makes it difficult for a single clustering method to obtain the same good results as the previous datasets. However, among the six clustering algorithms, DPC, K-means, FBMC2 and ETWHC still achieved relatively good clustering results. In contrast, DBSCAN and single-linkage algorithms failed to effectively separate the two clusters, resulting in clustering results that cannot accurately reflect the distribution characteristics of the original data. For data with higher dimensions, isolet and Yaleb, FBMC2 and ETWHC methods show good adaptability, while DPC and K-means have poorer effects, and DBSCAN and single-linkage are difficult to adapt to high-dimensional data.

[0088] Through in-depth analysis of these data, it can be seen that the four algorithms, DPC, K-means, FBMC2 and ETWHC, have relatively low requirements on the shape and distribution of clusters. They focus more on the similarity measurement between data points, so they can cluster data into several clusters with high similarity. This feature enables them to obtain satisfactory clustering results when there is cluster overlap in the original data. Relatively speaking, the design concept of the DBSCAN algorithm pays more attention to the difference between cluster core and cluster edge, that is, the core points need to have a higher density, while the density of edge points is lower. Therefore, in the case of cluster overlap, although the density of edge points of a single cluster is low, due to the overlap of the edges of multiple clusters, the density of edge points of the cluster in the original distribution is not significantly lower than the density of core points, which leads to the low evaluation of DBSCAN clustering results on wdbc and Yaleb datasets. The single-linkage algorithm, due to excessive attention to the obvious boundaries between clusters, has poor evaluation results when processing data with complex structures such as seeds, wdbc, isolet and Yaleb datasets.

[0089] Finally, see Figure 8As shown, this application applies ETWHC to the Olivetti face dataset for testing. The Olivetti dataset is provided by AT&T Cambridge Laboratory. It contains a set of facial images from 40 different subjects. Each subject has 10 different 112*92 pixel images, forming a 400x10304 data matrix. In this experiment, we selected the first 10 groups of data, totaling 100 images. In order to reduce the complexity of storage and calculation, we first compressed the original image from 112*92 to 15*15. Then, principal component analysis PCA is performed to filter out attributes with a cumulative contribution rate greater than 90%, so as to reduce the data dimension while retaining the main information. Finally, clustering is performed through ETWHC, and the results are shown as follows. Figure 8 As shown in the figure. As can be seen from the figure, in the clustering results, the first, fourth and eighth images of the first group are assigned to the second group, and the second image of the tenth group is assigned to the eighth group. The calculated accuracy is 96%. This result fully proves that the algorithm has good adaptability and robustness for face data with higher complexity. When all 40 groups of data were experimented, the accuracy was 0.81%, which is significantly better than the 76% obtained by the SNN-DPC algorithm recently proposed by C.Moraru. This series of experimental results strongly show that the ETWHC algorithm not only performs well on traditional data sets, but also shows wide applicability and superior performance in complex practical application scenarios (such as face image data clustering), providing a strong basis for its application in more fields.

[0090] This embodiment proposes a new clustering method - the divisive hierarchical clustering algorithm ETWHC. Compared with the traditional hierarchical clustering algorithm, the divisive hierarchical clustering algorithm ETWHC can effectively deal with the situation where the similarity between clusters is not obvious and the density of edge points within the cluster is low. At the same time, it avoids the limitations of algorithms such as DPC, K-means and FBMC that may ignore the distribution of data when pursuing similarity within clusters. Compared with DBSCAN, the divisive hierarchical clustering algorithm ETWHC performs better in dealing with the problem of unbalanced distribution between clusters. Overall, the divisive hierarchical clustering algorithm ETWHC has good adaptability to most data. It not only considers the similarity within the cluster, but also can adapt to the characteristics of different data distributions. At a lower cost of use, it achieves similar or better results than other traditional clustering algorithms.

[0091] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0092] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A weighted hierarchical clustering method based on entropy termination, characterized in that: The method comprises: Step 1: construct the shortest connected path, connect all sample points in the shortest connected path, construct a minimum spanning tree based on the sample points, and assign corresponding weights to each edge; Step 2: In the splitting phase, delete the edge with the largest weight in step 1, calculate the weight based on the ratio of the weight to the entropy before and after each split, and draw a line graph to quickly determine the number of final clusters; Step 3: Use the test data set to test the number of clusters in step 1 and step 2, obtain the test results of the weighted hierarchical clustering based on entropy termination, and implement the weighted hierarchical clustering method based on entropy termination.

2. The weighted hierarchical clustering method based on entropy termination according to claim 1, characterized in that: The shortest connection path constructed in step 1 is implemented by Dijkstra's algorithm.

3. The weighted hierarchical clustering method based on entropy termination according to claim 1, characterized in that: The method for calculating the weight in step 2 includes: Calculate the degree of each point, select the point with a degree of 1, assign the level of the edge connected to it, and delete the edge; repeat this operation until the level calculation of all edges is completed. After the level calculation is completed, the product of the length of each edge and the level is used as the final weight of the edge, and the edge with the largest weight is selected at each split.

4. The weighted hierarchical clustering method based on entropy termination according to claim 1, characterized in that: Step 2 also includes the step of dividing the number of clusters.

5. The weighted hierarchical clustering method based on entropy termination according to claim 1, characterized in that: The test data sets described in step 3 are all unlabeled data sets, and step 3 also includes a step of identifying the number of clusters from the unlabeled data sets.

6. The weighted hierarchical clustering method based on entropy termination according to claim 1, characterized in that: Step 3 also includes the step of reducing the dimension of the data in the test data set.

7. The weighted hierarchical clustering method based on entropy termination according to claim 6, characterized in that: In step 3, t-SNE method is used to achieve dimensionality reduction.

8. A weighted hierarchical clustering system based on entropy termination, characterized in that The system comprises: The shortest connected path construction module is used to construct the shortest connected path, connect all sample points in the shortest connected path, build a minimum spanning tree based on the sample points, and assign corresponding weights to each edge; The weight calculation module is used to delete the edge with the largest weight in the shortest connected path construction module during the splitting phase, calculate the weight based on the ratio of the weight to the entropy before and after each splitting, and draw a line graph to quickly determine the number of final clusters; The test module is used to test the number of clusters in the shortest connected path construction module and the weight calculation module using a test data set, obtain the test results of the weighted hierarchical clustering based on entropy termination, and implement the weighted hierarchical clustering method based on entropy termination.

9. A computer device comprising a memory and a processor, characterized in that A computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images