Hierarchical clustering method integrating distance features, shape features and time sequence features

Abstract

The invention belongs to the related technical field of data processing, and discloses a hierarchical clustering method integrating distance features, shape features and time sequence features, which comprises the following steps of: in an initial stage, regarding each time sequence process as an independent cluster, and calculating distances of different features among the time sequence processes; according to the contribution degree of each distance, constructing a composite similarity index DST integrating distance, shape and time sequence characteristics; calculating the distance between different clusters according to the constructed composite similarity index and the determined inter-cluster connection mode, and continuously iterating and merging the nearest clusters; when all clusters are combined into a large cluster, clustering is completed, and the optimal clustering number is determined according to the contour coefficient after each time of combination. According to the method, the composite similarity index DST is constructed, the weight is automatically calculated according to the contribution degree of different features to the current time sequence process, the multi-dimensional difference between the time sequence processes can be more comprehensively reflected, and the clustering result is more in line with the actual dynamic evolution rule of the time sequence process.

Description

Technical Field

[0001] The present invention belongs to the technical field related to data processing, and more specifically, relates to a hierarchical clustering method that integrates distance features, shape features and time series features. Background Art

[0002] Hierarchical clustering is a clustering method that constructs a multi-level tree structure by gradually merging or splitting data points. Its core principle is to iteratively form a hierarchical relationship of clusters from bottom to top or top to bottom based on the similarity measurement between data points, and finally display the multi-scale clustering structure of the data through a tree diagram. Similarity indices are often used in cluster analysis to measure the similarity or difference between data points. Their core function is to determine the similarity between data points, thereby guiding the algorithm to classify samples with inherent similarity into the same category. Therefore, the selection of similarity indices will directly affect the clustering algorithm's ability to identify and divide the data's inherent structure. In the existing technology, the standard hierarchical clustering method uses Euclidean distance as the similarity indicator. Euclidean distance mainly focuses on the absolute spatial distance between data points and is difficult to capture the inherent dynamic changes of time series data. Therefore, in order to achieve more precise and effective clustering of time series data, given the characteristics of water level processes that exhibit multi-scale fluctuations while taking into account both overall trends and local details, it is urgent to propose a hierarchical clustering method that integrates distance features, shape features, and time series features. This method sets a composite similarity indicator that integrates distance features, shape features, and time series features. The weights of different features are automatically calculated based on their contribution to the current time series process, which is used to reveal the inherent structural characteristics and dynamic evolution of the water level process. Summary of the Invention

[0003] In response to the above defects or improvement needs of the existing technology, the present invention provides a hierarchical clustering method that integrates distance features, shape features and time series features. Its purpose is to fully capture the similarities between different time series processes in the three dimensions of amplitude, morphology and time series, thereby overcoming the defect that the standard hierarchical clustering method cannot capture the complex dynamic characteristics of the time series process based solely on Euclidean distance, thereby solving the technical problems of low clustering accuracy of time series processes, neglect of structural information, and large intra-class heterogeneity.

[0004] To achieve the above object, according to one aspect of the present invention, a hierarchical clustering method based on integrated distance features, shape features, and time series features is provided, comprising the following steps: S1: In the initial stage, each temporal process is regarded as an independent cluster, and the distances between different features of each temporal process are calculated; S2: Based on the calculated different feature distances, a composite similarity index is constructed based on the contribution of each distance, which integrates distance, shape, and time series features. S3: Clarify the connection mode between clusters and determine the distance calculation method between different clusters; S4: Calculate the distances between different clusters based on the constructed composite similarity index and the determined inter-cluster connection method and iteratively merge the closest clusters; S5: Clustering is completed when all clusters are merged into one large cluster, and the optimal number of clusters is determined based on the silhouette coefficient after each merger.

[0005] Preferably, S1 comprises the following steps: S11: Obtain the initial time series process data set, and regard each process in the data set as an independent cluster as the starting point for clustering; S12: Calculate the distances of different features between each temporal process in sequence, including complexity invariant distance, first-order difference Euclidean distance, and dynamic time warping distance; Preferably, the calculation formula of the complexity invariant distance is as follows: in, and Represents two different sequences, Representation sequence and The Euclidean distance between represents the complexity estimate, express and The complexity-invariant distance between Preferably, the calculation formula of the first-order difference Euclidean distance is as follows: in, represents the sequence length, and Represents the first-order derivative of different sequences at a certain moment, which can be regarded as the basic approximation of the local slope. express and The first-order difference Euclidean distance between them; Preferably, the calculation formula of the dynamic time warping distance is as follows: in, and There are two time series and The two points in express and The dynamic time warping distance between them is , the final result is ; Preferably, S2 comprises the following steps: S21: Normalize the complexity invariant distance, first-order difference Euclidean distance, and dynamic time warping distance calculated in S1. The calculation formula is as follows: in, represents the normalized distance, represents the original distance, and Represent the maximum and minimum values ​​of the original distance respectively; S22: The normalized complexity invariant distance in step S21 is the distance feature , the normalized first-order difference Euclidean distance is the shape feature , the normalized dynamic time warping distance is the temporal feature The weight of each characteristic index is calculated according to its contribution to the similarity between different time series processes. The calculation formula is as follows: in: Represents the weights of different feature indicators, Indicates the relative ranking under different feature indicators, is the value of the sample, is the sample mean, is the standard deviation of the sample; If the Z score is less than 0, the weight of the distance will be greater than 1, and the contribution of the distance metric will be strengthened; if the Z score is 0, the feature index will remain unchanged; otherwise the weight of the feature will be less than 1, and the contribution of the distance metric will be reduced; S23: Construct a composite similarity index DST based on the weights of the different feature indices calculated in step S22. The calculation formula is as follows: in: 、 、 Respectively represent the distance feature distance, shape feature distance and time series feature distance between different time series processes, 、 、 Respectively represent the weights of distance features, shape features, and time series features in constructing comprehensive distance metrics; Preferably, S3 includes the following steps: S31: Average connectivity is selected as the connection method between different clusters to reveal the inherent hierarchical structure and similarity of the data. The formula for calculating the inter-cluster distance is as follows: in: and represent the number of samples in the cluster, represents the distance between different sequences, express and The inter-cluster distance; S32: In the initial stage of clustering, each time series process is regarded as an independent cluster, and the composite similarity index DST between each time series process is the cluster distance between two clusters; Preferably, S4 includes the following steps: S41: First iteration merges the two clusters with the smallest DST distance in the distance matrix D and , forming a new cluster ; S42: The distance between the new cluster and other clusters is calculated based on the constructed composite similarity index DST and the selected average connection method, and the new cluster is deleted. and Corresponding rows and columns, add new clusters The distance to all other clusters, update the distance matrix D; S43: In each iteration, the two closest clusters are merged according to the new distance matrix, and S42 is repeated continuously. After each iteration, the distance matrix is ​​updated and the next merge is performed. After each iteration, the silhouette coefficient of the current clustering is calculated. .

[0006] Preferably, the silhouette coefficient The calculation formula is as follows: in, Representation sample The average distance to all other samples in the same cluster, Representation sample The average distance to all samples in the nearest cluster.

[0007] In general, compared with the prior art, the above technical solution conceived by the present invention provides a hierarchical clustering method that integrates distance features, shape features, and time series features, which has the following beneficial effects: 1. Significantly improved accuracy: A composite similarity metric (DST) was constructed that integrates distance, shape, and time series features. Weights are automatically calculated based on the contribution of different features to the current time series process. This metric can more comprehensively reflect the multidimensional differences between time series processes, resulting in better clustering results than a single metric, significantly improving clustering accuracy.

[0008] 2. More complete preservation of structural information: Traditional distance metrics such as Euclidean distance emphasize amplitude, while dynamic time warping (DST) tends to ignore morphological and temporal features when rearranging time structures. The composite similarity metric, DST, focuses on amplitude while also taking into account change trends and shapes, making the clustering results more consistent with the actual dynamic evolution of the temporal process.

[0009] 3. Greater flexibility: The combination of composite similarity metrics (DST) does not use fixed or subjectively set weights. Instead, it automatically calculates weights based on the contribution of different features to the current time series process. This data-driven adaptive weighting mechanism avoids human bias and is more adaptable to time series data of different types and characteristics, significantly improving clustering quality and generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flow chart of clustering using the hierarchical clustering method combined with the composite similarity index DST; Figure 2 It is the changing trend of silhouette coefficient of 200 groups of water level processes under different similarity indices and different cluster numbers; Figure 3 This is the clustering result diagram of 200 groups of water level processes using Euclidean distance as the similarity index; Figure 4 This is the clustering result diagram of 200 groups of water level processes using dynamic time warping as the similarity indicator; Figure 5 This is the clustering result diagram of 200 groups of water level processes using Chebyshev distance as the similarity index; Figure 6 This is the clustering result diagram of 200 groups of water level processes using the composite similarity index DST as the distance metric. DETAILED DESCRIPTION

[0011] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0012] See also Figure 1 ,A hierarchical clustering method integrating distance features, shape features and time series features, comprising the following steps: S1: In the initial stage, each temporal process is regarded as an independent cluster, and the distances between different features of each temporal process are calculated; S2: Based on the calculated different feature distances, a composite similarity index DST is constructed based on the contribution of each distance, which integrates distance, shape, and time series features. S3: Clarify the connection mode between clusters and determine the distance calculation method between different clusters; S4: Calculate the distances between different clusters based on the constructed composite similarity index and the determined inter-cluster connection method and iteratively merge the closest clusters; S5: Clustering is completed when all clusters are merged into one large cluster, and the optimal number of clusters is determined based on the silhouette coefficient after each merger.

[0013] S1 includes the following steps: S11: Obtain the initial time series process data set, and regard each process in the data set as an independent cluster as the starting point for clustering; S12: Calculate the distances of different features between each temporal process in sequence, including complexity invariant distance, first-order difference Euclidean distance, and dynamic time warping distance; The complexity invariant distance calculation formula is as follows: in, and Represents two different sequences, Representation sequence and The Euclidean distance between represents the complexity estimate, express and The complexity-invariant distance between The calculation formula of the first-order difference Euclidean distance is as follows: in, represents the sequence length, and Represents the first-order derivative of different sequences at a certain moment, which can be regarded as the basic approximation of the local slope. express and The first-order difference Euclidean distance between them; The calculation formula of dynamic time warping distance is as follows: in, and There are two time series and The two points in express and The dynamic time warping distance between them is , the final result is ; Preferably, S2 includes the following steps: S21: Normalize the complexity invariant distance, first-order difference Euclidean distance, and dynamic time warping distance calculated in S1. The calculation formula is as follows: in, represents the normalized distance, represents the original distance, and Represent the maximum and minimum values ​​of the original distance respectively; S22: The normalized complexity invariant distance in step S21 is the distance feature , the normalized first-order difference Euclidean distance is the shape feature , the normalized dynamic time warping distance is the temporal feature The weight of each characteristic index is calculated according to its contribution to the similarity between different time series processes. The calculation formula is as follows: in: Represents the weights of different feature indicators, Indicates the relative ranking under different feature indicators, is the value of the sample, is the sample mean, is the standard deviation of the sample; If the Z score is less than 0, the weight of the distance will be greater than 1, and the contribution of the distance metric will be strengthened; if the Z score is 0, the feature index will remain unchanged; otherwise the weight of the feature will be less than 1, and the contribution of the distance metric will be reduced; S23: Construct a composite similarity index DST based on the weights of the different feature indices calculated in step S22. The calculation formula is as follows: in: 、 、 Respectively represent the distance feature distance, shape feature distance and time series feature distance between different time series processes, 、 、 Respectively represent the weights of distance features, shape features, and time series features in constructing comprehensive distance metrics; S3 includes the following steps: S31: Average connectivity is selected as the connection method between different clusters to reveal the inherent hierarchical structure and similarity of the data. The formula for calculating the inter-cluster distance is as follows: in: and represent the number of samples in the cluster, represents the distance between different sequences, express and The inter-cluster distance; S32: In the initial stage of clustering, each time series process is regarded as an independent cluster, and the composite similarity index DST between each time series process is the cluster distance between two clusters; S4 includes the following steps: S41: First iteration merges the two clusters with the smallest DST distance in the distance matrix D and , forming a new cluster ; S42: The distance between the new cluster and other clusters is calculated based on the constructed composite similarity index DST and the selected average connection method, and the new cluster is deleted. and Corresponding rows and columns, add new clusters The distance to all other clusters, update the distance matrix D; S43: In each iteration, the two closest clusters are merged according to the new distance matrix, and S42 is repeated continuously. After each iteration, the distance matrix is ​​updated and the next merge is performed. After each iteration, the silhouette coefficient of the current cluster is calculated. , the calculation formula is as follows: in, Representation sample The average distance to all other samples in the same cluster, Representation sample The average distance to all samples in the nearest cluster.

[0014] This example uses 200 sets of water level processes corresponding to 200 sets of Pareto solutions obtained from the multi-objective scheduling of the Baihetan Hydropower Station in September 2022 as the original data. It aims to deeply explore the inherent spatiotemporal correlations of different water level processes through efficient clustering methods, more finely capture the differences in shape and trend of water level processes, and perform more scientific and reasonable clustering, so as to conduct a more in-depth analysis of typical water level processes.

[0015] This embodiment uses a hierarchical clustering method combined with a composite similarity index DST that integrates distance features, shape features, and time series features to cluster 200 groups of water level processes. In order to verify the advantage of the composite similarity index DST over the traditional distance metric in capturing subtle differences in water level curves, three commonly used similarity indices, namely Euclidean distance (EU), dynamic time warping (DTW), and Chebyshev distance (Cheb), are selected and combined with the hierarchical clustering method as controls, and the silhouette coefficient, CH index, and DB index are selected as clustering effect evaluation indicators.

[0016] Since the optimal number of clusters in the hierarchical clustering method is obtained by calculating the silhouette coefficient under different cluster numbers rather than being set manually, in order to avoid the clustering results being too dispersed due to too large a k value or too rough due to too small a k value, this embodiment limits the cluster number k to between 2 and 10 for research. The changing trend of the silhouette coefficient under different similarity indices with the cluster number k is shown in the figure below. Figure 2 As shown in the figure, it can be seen that traditional single similarity indicators tend to reach the maximum silhouette coefficient at low cluster numbers and then begin to decline, indicating that these indicators are prone to underfitting or oversimplification, while the silhouette coefficient change trend corresponding to the composite similarity indicator shows a more stable and significant cluster separation.

[0017] The clustering evaluation index results under the optimal number of clusters are shown in Table 1. It can be seen from the table that under the optimal number of clusters, the use of DST as the similarity index shows significant advantages in the clustering results of water level processes compared with the traditional single similarity index. Among them, the silhouette coefficient, CH index and DB index obtained when DST is used to cluster water level processes are improved by 9.9%, 15.7% and 6.6% respectively compared with the optimal values ​​obtained by other indicators, indicating that DST can ensure the intra-class compactness while enhancing the inter-class separability, significantly improving the overall quality and stability of water level clustering.

[0018] Figure 3 This is the clustering result for the optimal number of clusters using the standard hierarchical clustering method, using Euclidean distance (EU) as the similarity metric. The clustering results show that using Euclidean distance as the similarity metric only roughly divides the 200 water level processes into three categories, making it difficult to distinguish the subtle differences between them. This is primarily because Euclidean distance focuses solely on the overall distance between processes in numerical space, without delving deeply into the morphological characteristics of the time series curve, such as the location of inflection points, the amplitude and timing of peaks and troughs, and so on. Therefore, when different water level processes have similar numerical ranges but different shapes and trends, Euclidean distance often cannot distinguish them, resulting in overly simplistic clustering results that are unable to meet the needs of detailed analysis of water level time series patterns.

[0019] Figure 4The clustering results for the optimal number of clusters using Dynamic Time Warping (DTW) as the similarity metric are shown. The clustering results show that the algorithm can effectively align phase differences and local time shifts across water level processes using DTW. However, since DTW only seeks to minimize the cumulative alignment error during optimization and does not impose constraints on the global morphology or local geometric features of the sequence, it is still difficult to distinguish differences in microscopic details, which is insufficient to meet the requirements of high-precision time series clustering.

[0020] Figure 5 This is the clustering result under the optimal number of clusters when Chebyshev distance (Cheb) is used as the similarity indicator. From the clustering results, it can be seen that when using Chebyshev distance as the similarity indicator, the algorithm can quickly identify the water level process that has significantly deviated at a certain moment by virtue of its sensitivity to the maximum coordinate difference at any moment in the sequence. However, since the Chebyshev distance only measures the overall similarity based on the maximum deviation of a single point, it completely ignores the cumulative differences and overall morphological trends of the two curves at other time points. Therefore, when the water level process shows medium-amplitude fluctuations in multiple time periods, this measurement is difficult to reflect these scattered changes, and thus cannot meet the needs of fine clustering of the overall evolution and local microstructure of the water level time series.

[0021] Figure 6 The clustering results for the optimal number of clusters using the constructed composite similarity metric, DST, as the distance metric are presented. The clustering results show that using DST as a similarity metric for water level process clustering not only balances distance differences, curve shape, and temporal evolution, but also evenly divides 200 groups of water level processes into several categories. Water level processes within each category exhibit high clustering, while the central curves between categories exhibit significant evolutionary differences. This demonstrates that, driven by the data-driven similarity contribution, DST continuously adjusts the weights of each feature to ensure that each iteration aligns global trends while sensitively capturing microstructures such as local peaks, troughs, and slope changes. This avoids the problems of overly coarse clustering using Euclidean distance and oversensitivity to single-point deviations using Chebyshev distance. Furthermore, compared to DTW, which focuses solely on minimizing the alignment path, DST improves its ability to identify sudden jumps and inflection point distributions through a weighted fusion of shape and temporal features, thereby achieving higher discrimination and classification accuracy across diverse water level patterns.

[0022] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A hierarchical clustering method that integrates distance features, shape features, and time series features, characterized by: The following steps are involved: S1: In the initial stage, each temporal process is regarded as an independent cluster, and the distances between different features of each temporal process are calculated; S2: Based on the calculated different feature distances, a composite similarity index is constructed based on the contribution of each distance, which integrates distance, shape, and time series features. S3: Clarify the connection mode between clusters and determine the distance calculation method between different clusters; S4: Calculate the distances between different clusters based on the constructed composite similarity index and the determined inter-cluster connection method and iteratively merge the closest clusters; S5: Clustering is completed when all clusters are merged into one large cluster, and the optimal number of clusters is determined based on the silhouette coefficient after each merger.

2. The hierarchical clustering method according to claim 1, wherein: S1 includes the following steps: S11: Obtain the initial time series process data set, and regard each process in the data set as an independent cluster as the starting point for clustering; S12: Calculate the distances of different features between each time series process in turn, including complexity invariant distance, first-order difference Euclidean distance, and dynamic time warping distance.

3. The hierarchical clustering method according to claim 2, which integrates distance features, shape features, and time series features, is characterized by: The calculation formula of the complexity invariant distance is as follows: in, and Represents two different sequences, Representation sequence and The Euclidean distance between represents the complexity estimate, express and The complexity-invariant distance between them.

4. The hierarchical clustering method according to claim 3, wherein: The calculation formula of the first-order difference Euclidean distance is as follows: in, represents the sequence length, and Represents the first-order derivative of different sequences at a certain moment, which can be regarded as the basic approximation of the local slope. express and The first-order difference Euclidean distance between them.

5. The hierarchical clustering method according to claim 4, which integrates distance features, shape features, and time series features, is characterized in that: The calculation formula of the dynamic time warping distance is as follows: in, and There are two time series and The two points in express and The dynamic time warping distance between them is , the final result is .

6. The hierarchical clustering method according to claim 5, wherein: S2 includes the following steps: S21: Normalize the complexity invariant distance, first-order difference Euclidean distance, and dynamic time warping distance calculated in S1. The calculation formula is as follows: in, represents the normalized distance, represents the original distance, and Represent the maximum and minimum values ​​of the original distance respectively; S22: The normalized complexity invariant distance in step S21 is the distance feature , the normalized first-order difference Euclidean distance is the shape feature , the normalized dynamic time warping distance is the temporal feature The weight of each characteristic index is calculated according to its contribution to the similarity between different time series processes. The calculation formula is as follows: in: Represents the weights of different feature indicators, Indicates the relative ranking under different feature indicators, is the value of the sample, is the sample mean, is the standard deviation of the sample; S23: Construct a composite similarity index DST based on the weights of the different feature indices calculated in step S22. The calculation formula is as follows: in: 、 、 Respectively represent the distance feature distance, shape feature distance and time series feature distance between different time series processes, 、 、 They respectively represent the weights of distance features, shape features and time series features in constructing comprehensive distance measurement indicators.

7. The hierarchical clustering method according to claim 6, wherein: S3 includes the following steps: S31: Average connectivity is selected as the connection method between different clusters to reveal the inherent hierarchical structure and similarity of the data. The formula for calculating the inter-cluster distance is as follows: in: and represent the number of samples in the cluster, represents the distance between different sequences, express and The inter-cluster distance; S32: In the initial stage of clustering, each time series process is regarded as an independent cluster, and the composite similarity index DST between each time series process is the cluster distance between two clusters.

8. The hierarchical clustering method according to claim 7, wherein: S4 includes the following steps: S41: First iteration merges the two clusters with the smallest DST distance in the distance matrix D and , forming a new cluster ; S42: The distance between the new cluster and other clusters is calculated based on the constructed composite similarity index DST and the selected average connection method, and the new cluster is deleted. and Corresponding rows and columns, add new clusters The distance to all other clusters, update the distance matrix D; S43: In each iteration, the two closest clusters are merged according to the new distance matrix, and S42 is repeated continuously. After each iteration, the distance matrix is ​​updated and the next merge is performed. After each iteration, the silhouette coefficient of the current clustering is calculated. .

9. The hierarchical clustering method according to claim 8, characterized by: The silhouette coefficient The calculation formula is as follows: in, Representation sample The average distance to all other samples in the same cluster, Representation sample The average distance to all samples in the nearest cluster.

Images