Flood classification method and system based on dynamic time warping and multi-cluster coupling

Abstract

The invention discloses a flood classification method and system based on dynamic time warping and multi-cluster coupling. The method comprises the following steps: firstly, collecting and processing flood flow time sequence data to obtain a data set composed of standardized floods; secondly, calculating the similarity between every two of the K floods by adopting a dynamic time warping method to form a similar matrix; and flood classification is carried out based on DTW coupling hierarchical clustering and K-medoids multi-clustering. The system comprises a data acquisition module, a data preprocessing module, a comparison module and a classification output module. According to the method, hierarchical clustering is optimized again, and a multi-clustering method is combined, so that the reasonability of flood classification is improved, and the accuracy of flood classification is improved. The method can be applied to the fields of flood early warning, flood control dispatching and the like, and has important practical value and social significance.

Description

Technical Field

[0001] The present invention belongs to the technical field of data analysis and relates to a flood classification system that combines a dynamic time warping (DTW) algorithm with a multi-clustering method to improve the calculation accuracy of flood process similarity and classification reliability. Background Art

[0002] The study of surface hydrological processes forms the theoretical foundation of flood forecasting and exhibits complex characteristics such as randomness, fuzziness, nonlinearity, and nonstationarity (time-varying properties). Flood similarity research aims to explore the characteristics and patterns inherent in flood time series data. Traditional flood classification methods suffer from the following problems: ① Most methods rely on the characteristic values of flood discharge time series data for classification, while few methods directly use flood discharge time series data for process similarity classification. ② In existing methods for directly calculating process similarity using flood discharge time series data, the different contributions of high- and low-discharge time series components of the same flood to the similarity calculation are not considered. ③ Flood discharge time series data are typically standardized on a case-by-case basis, which is significantly influenced by the timing of the flood's start and end points. This can lead to unrealistic similarities between high- and low-discharge flood peaks and low-discharge flood peaks after standardization.

[0003] The Dynamic Time Warping (DTW) algorithm effectively handles unequal-length sequences and phase shifts when calculating time series similarity. This allows similarity calculations to be performed directly on the shape of flood processes between two flood events of varying lengths, allowing for the measurement of similarity between any two flood events. Traditional DTW uses uniform weights for peak and non-peak areas. However, peak areas are of particular interest, serving as a crucial factor in flood warnings and forecasts, and a key component of flood management in the humid southern regions. Therefore, increasing the weighting of high-flow time series in similarity calculations is crucial.

[0004] In the era of big data, machine learning and data mining have become two closely related and important technologies for learning knowledge and discovering patterns. Clustering is a crucial method in data mining research and applications. Cluster analysis encompasses time series data representation, feature extraction, similarity metrics, and clustering algorithms. Clustering algorithms often have their own strengths and weaknesses. K-medoids is one of the most common clustering algorithms, but it suffers from uncertain number of clusters, sensitivity to initial centers, and a tendency to fall into local optima during clustering calculations. Therefore, the synergy of multiple clustering methods can effectively overcome the limitations of a single algorithm. Summary of the Invention

[0005] One objective of this invention is to provide a flood classification method based on dynamic time warping coupled with multi-clustering to improve the accuracy of flood similarity calculation and the precision of flood classification. This method addresses the issues of inaccurate similarity measures and unstable clustering results in flood classification by improving the contribution allocation strategy of DTW and combining hierarchical clustering with K-medoids.

[0006] The method of the present invention is specifically as follows:

[0007] Step (1) collects and processes flood flow time series data, and obtains a dataset consisting of K standardized flood fields after processing.

[0008] Step (2) The dynamic time warping method is used to calculate the time warping distance between the two standardized floods in the K fields, and the similarity matrix of the K floods is obtained;

[0009] (2-1) Select any two flood flow time series data and calculate the boundary value of the high and low flow time series data of the flood flow time series data that comes first based on the maximum flow value and the minimum flow value of the flood;

[0010] (2-2) Set the distance coefficients of high and low flow time series data, and calculate the distance between any two points on the two flood flow time series data. The distance coefficient of high flow time series data is smaller than the distance coefficient of low flow time series data.

[0011] (2-3) Using the dynamic time warping distance algorithm, the time warping distance between the two floods is calculated;

[0012] (2-4) According to the time sequence traversal, the time warped distance between the two time series data of the K-field standardized flood flow is calculated, and finally the similarity matrix of the K-field flood is obtained.

[0013] Step (3) Similar flood classification, flood classification based on DTW coupled hierarchical clustering and K-medoids multi-clustering;

[0014] (3-1) Using the DTW coupled hierarchical clustering method, the flood clustering process is calculated, isolated points are removed, the number of flood classifications R and the corresponding classifications are determined, and finally the silhouette coefficient S1 of the classification is calculated;

[0015] (3-2) The number of categories R determined by hierarchical classification is selected as the number of categories of K-medoids‌, and any of the two most similar sets of standardized secondary flood flow time series data in each category after hierarchical classification is selected as the initial center of the K-medoids‌ cluster; the two most similar sets of standardized secondary flood flow time series data in a category are the two floods with the smallest dynamic time warping distance;

[0016] (3-3) The K-medoids‌ clustering method is used to iteratively calculate the flood classification until the termination condition is met. The silhouette coefficient of the classification is recorded as S2;

[0017] (3-4) Select the classification with the higher silhouette coefficient score between the silhouette coefficients S1 and S2 as the final flood classification number R', and obtain the corresponding classification.

[0018] Another object of the present invention is to provide a flood classification system based on dynamic time warping coupled with multi-clustering, for implementing the aforementioned flood classification method. The flood classification system includes a data acquisition module, a data preprocessing module, a comparison module, and a classification output module.

[0019] The data acquisition module is used to collect flow time series data within the target basin and target time range;

[0020] The data preprocessing module is used to clean, process the flood flow time series data at the whole time, interpolate and standardize them;

[0021] The comparison module is used to obtain pairwise similarities between multiple floods;

[0022] The classification output module is used to obtain and output the determined flood type.

[0023] The beneficial effects of the present invention include: in the processing of flood flow time series data, the present invention optimizes the original individual standardization of each flood to the standardization of flood flow time series data in units of years, with the advantages of ① compensating for the possible errors in the selection of flood start and end times on flood standardization and flood similarity judgment; ② compensating for the inconsistency of the mean and mean difference in the standardization process of a single flood, which leads to similarity calculation errors. For example, two floods with high flow due to high water level rise and low flow due to low water level rise are consistent after standardization.

[0024] Dynamic Time Warping (DTW) gives equal weight to each matching distance step. In practice, we believe that the similarity of floods in high-flow areas should contribute more than the similarity of floods in low-flow areas. This paper improves the accuracy of flood similarity calculations by optimizing the DTW algorithm and assigning different coefficients to high-flow and low-flow time series.

[0025] Hierarchical clustering can demonstrate the characteristics of the clustering process and conduct preliminary classification of flood flow time series data. The objectives are: 1. Identify isolated flood points; 2. Visually determine the degree of flood similarity, thereby assisting in flood classification; and 3. Provide initial values for the application of other classification methods. K-medoids classification is a further optimization of hierarchical clustering, combined with a multi-clustering algorithm, to improve the rationality of flood classification and contribute to the improvement of flood classification accuracy. The method of the present invention can be applied to fields such as flood warning and flood control scheduling, and has important practical value and social significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0027] like Figure 1 As shown in the figure, the flood classification method based on dynamic time warping and multi-clustering coupling is as follows:

[0028] Step (1) Collect and process flood flow time series data:

[0029] (1-1) Based on the hydrological yearbook data, collect the flow time series data of each flood in the target basin and target time range.

[0030] (1-2) Flow data cleaning: Clean the flow time series data and remove abnormal data that exceeds historical extremes and does not conform to hydrological laws.

[0031] (1-3) Hourly processing of traffic data: If the extracted traffic data is hourly hourly data, it will be retained; if the extracted traffic data is non-hourly hourly data, it will be compared with the adjacent hourly data according to the rounding principle. If there is no data at the adjacent hourly time, the non-hourly hourly data will be stored as the adjacent hourly data; if there is data at the adjacent hourly time, the data will be compared with the adjacent hourly data. If it is greater than the adjacent hourly data, the non-hourly hourly data will replace the original adjacent hourly data and be stored. Otherwise, the non-hourly hourly data will be discarded and the original hourly data will be retained unchanged.

[0032] (1-4) Interpolation of flow time series data: If the interval between adjacent flow time series data is greater than or equal to 2 hours, the intermediate process flow time series data is linearly interpolated to supplement the hourly flow time series data during the period, and finally the hourly flow time series data for the entire period (q, t) is obtained, t = {1, 2, 3, …, T}, q is the flow, t is the time, and T represents the total time;

[0033] (1-5) Standardization of flow data time series data: The hourly flow time series data q is standardized according to the year, and q' is the standardized hourly flow time series data. , μ is the mean of annual flow time series data, σ is the standard deviation of annual flow time series data;

[0034] (1-6) The start and end time of each flood are verified by combining the original data, and a data set Q consisting of K standardized floods is obtained according to the start time, where Q = {Q1, Q2, Q3, …, QK}.

[0035] Step (2) uses the dynamic time warping (DTW) method to calculate the pairwise similarities between the K floods and form a similarity matrix.

[0036] (2-1) Determine the boundary between high and low flow: Select any two flood flow time series data, the flood flow time series data with the earlier time series , flood flow time series data with the later time series , M and N are and Length; Considering that the similarity of high flow time series data should be more important than the similarity of low flow time series data in the DTW similarity calculation of two floods, the dividing line value of high and low flow time series data of the flood flow time series data QA with the earlier time series is calculated. , is the maximum flow value of the flood, is the minimum flow value of the flood, is the proportionality coefficient;

[0037] (2-2) Determine the DTW distance coefficient: the distance between any two points on two traffic time series data , m=1,2,3,…,M, n=1,2,3,…,N; when calculating the distance between any two points on two data, assign a distance coefficient α to high-flow time series data and a distance coefficient β to low-flow time series data. Considering that the greater the distance between any two points, the lower the similarity, α<β;

[0038] (2-3) Calculate the similarity between two floods:

[0039] Perform similarity measurement on two flood flow time series data QA and QB with lengths m and n respectively, calculate the distance between each pair of points in the two series, and obtain the distance matrix based on QA and QB;

[0040] The cumulative distance between the first i elements of the flood flow time series data QA and the first j elements of the flood flow time series data QB ; Represents the distance between the i-th element of the flood flow time series data QA and the j-th element of the flood flow time series data QB;

[0041] According to the principle of dynamic programming, we backtrack from the end point (m,n) to the starting point (1,1) to find the optimal path r(m,n). The path selection after each data point matching follows continuity and monotonicity, and the boundaries are consistent.

[0042] Boundary conditions: The starting point and end point of the twisted path must correspond to the two end elements on the diagonal of the distance matrix, namely (1,1) and (i,j).

[0043] Monotonicity: ensure that the warp path must be monotonically increasing on the time axis, that is, the i and j indexes of a given path can only increase;

[0044] Continuity: can only match adjacent points, cannot match across a point;

[0045] In order to eliminate the influence of different warped path lengths, the total distance is normalized by the path length L to obtain the time warped distance , .

[0046] (2-4) Calculate the similarity matrix of all floods. Sort the floods in order of flood number, and calculate the time warp distance between the time series data of K standardized secondary flood flows from small to large, and finally obtain the similarity matrix K×K of K floods. .

[0047] Step (3) Similar flood classification, flood classification based on DTW coupled hierarchical clustering and K-medoids multi-clustering.

[0048] (3-1) The DTW coupled hierarchical clustering method is used to calculate the flood clustering process, remove isolated points, determine the flood classification number R and the corresponding classification, and finally calculate the silhouette coefficient S1 of its classification.

[0049] (3-1-1) During the first classification, each flood is considered as Class 1, and the classification set is recorded as Q (0) , let Q (0) =Q, the time warped distance between the time series data of the normalized secondary flood flow in the known K fields, recorded as the initial distance matrix ;

[0050] (3-1-2) Select the distance matrix D (0) The two floods corresponding to the minimum value in the , are merged into one category, recorded as new category C, , Represents the smallest number in the K×K matrix, that is, the minimum distance; after clustering is completed, the number of categories will be reduced by one, and the new category set will be recorded as

[0051] (3-1-3) Calculate the new class C and the original Q(0) The dynamic time warping distance of each category, after clustering, the distance matrix between each category is updated to ;

[0052] (3-1-4) Repeat (3-1-2) and (3-1-3) until they are clustered into one category;

[0053] (3-1-5) Draw a cluster tree based on the system clustering results and remove isolated points; Isolation point judgment conditions: Assume that each type of flood event must meet ≥Y events, and those that do not meet the conditions are considered isolated points, 1≤Y≤3; Determine a reasonable number of flood classifications R, 2≤R≤5. If there are multiple options that meet the above conditions at the same time, calculate the silhouette coefficient of each option separately, and select the category with the largest silhouette coefficient S1 and the corresponding flood classification number R as the preferred option.

[0054] (3-2) Select the number of categories R determined by hierarchical classification as the number of K-medoids‌ categories, and select any of the two most similar sets of standardized secondary flood flow time series data in each category after hierarchical classification as the initial centers of the K-medoids‌ cluster. The two most similar sets of standardized secondary flood flow time series data in a category are the two floods with the smallest dynamic time warping distance.

[0055] (3-3) The K-medoids‌ clustering method is used to iteratively calculate the flood classification until the termination condition is met. The silhouette coefficient of the classification is recorded as S2.

[0056] (3-3-1) DTW-based classification and recognition. Calculate the distance between all data points except the initial center medoids and each K-medoids, and assign them to the cluster Ci containing the closest medoids. The distance metric uses the Dynamic Time Warping algorithm.

[0057] (3-3-2) Calculate the new center of each of the R categories. Traverse all categories, assume that all data points Qi∈Ci are the new center medoids for each category, and calculate the total distance Z of all other data points to the new center medoids, and select the center medoids corresponding to the minimum distance as the new center; the new center , Qi traverses all points in Ci, DTW(Qi,Qj) is the dynamic time warping distance function, the new center It is the point that minimizes the total distance inside Ci.

[0058] (3-3-3) Repeat (3-3-1) and (3-3-2) until the termination condition is met; the termination condition is that the set number of iterations is reached or the K-medoids in all categories no longer change.

[0059] (3-3-4) Calculate the classification silhouette coefficient S2 of DTW coupled hierarchical clustering and K-medoids.

[0060] (3-4) Select the classification with the higher silhouette coefficient score between the silhouette coefficients S1 and S2 as the final flood classification number R', and obtain the corresponding classification.

[0061] The coupling of multiple clustering methods is not limited to the combination of the above two clustering methods. Other clustering methods can be introduced according to data characteristics, such as density clustering, graph clustering, fuzzy C clustering and other classification methods, in order to improve the rationality and accuracy of classification.

[0062] The flood classification system for executing the aforementioned flood classification method includes a data acquisition module, a data preprocessing module, a comparison module, and a classification output module.

[0063] The data acquisition module is used to collect standardized flow time series data of floods within the target basin and target time range.

[0064] The data preprocessing module is used to clean, process the standardized flow time series data of floods at the hourly rate, interpolate and standardize them.

[0065] The comparison module is used to obtain pairwise similarities between multiple floods.

[0066] The classification output module is used to obtain and output the determined flood type.

Claims

1. A flood classification method based on dynamic time warping and multi-clustering coupling, characterized by: Step (1) collecting and processing flood flow time series data to obtain a dataset consisting of K standardized flood fields; Step (2) The dynamic time warping method is used to calculate the time warping distance between the two standardized floods in the K fields, and the similarity matrix of the K floods is obtained; Step (3) Similar flood classification, flood classification based on DTW coupled hierarchical clustering and K-medoids multi-clustering; (3-1) Using the DTW coupled hierarchical clustering method, the flood clustering process is calculated, isolated points are removed, the number of flood classifications R and the corresponding classifications are determined, and finally the silhouette coefficient S1 of the classification is calculated; (3-2) The number of categories R determined by hierarchical classification is selected as the number of categories of K-medoids‌, and any of the two most similar sets of standardized secondary flood flow time series data in each category after hierarchical classification is selected as the initial center of the K-medoids‌ cluster; the two most similar sets of standardized secondary flood flow time series data in a category are the two floods with the smallest dynamic time warping distance; (3-3) The K-medoids‌ clustering method is used to iteratively calculate the flood classification until the termination condition is met. The silhouette coefficient of the classification is recorded as S2; (3-4) Select the classification with the higher silhouette coefficient score between the silhouette coefficients S1 and S2 as the final flood classification number R', and obtain the corresponding classification.

2. The flood classification method based on dynamic time warping and multi-clustering coupling according to claim 1 is characterized in that: Step (1) is as follows: (1-1) Collect flow time series data within the target basin and target time range based on hydrological yearbook data; (1-2) Clean the flow time series data and remove abnormal data that does not conform to the hydrological laws; (1-3) If the extracted traffic data is hourly data, it shall be retained; if the extracted traffic data is non-hourly data, it shall be compared with the adjacent hourly data according to the rounding principle. If there is no data at the adjacent hourly data, the non-hourly data shall be stored as the adjacent hourly data; if there is data at the adjacent hourly data, the data shall be compared with the adjacent hourly data. If it is greater than the adjacent hourly data, the non-hourly data shall replace the original adjacent hourly data and be stored. Otherwise, the non-hourly data shall be discarded and the original hourly data shall be retained unchanged. (1-4) If the interval between adjacent flow time series data is greater than or equal to 2 hours, the intermediate process flow time series data is supplemented with the hourly flow time series data during the period by linear interpolation, and finally the hourly flow time series data for the entire period is obtained; (1-5) Based on the mean value and standard deviation of annual flow time series data, the hourly flow time series data are standardized according to the year; (1-6) The start and end time of each flood are verified by combining the original data, and a data set consisting of K standardized floods is obtained according to the start time.

3. The flood classification method based on dynamic time warping and multi-clustering coupling according to claim 1 is characterized in that: Step (2) is as follows: (2-1) Select any two flood flow time series data and calculate the boundary value of the high and low flow time series data of the flood flow time series data that comes first based on the maximum flow value and the minimum flow value of the flood; (2-2) Set the distance coefficients of high and low flow time series data, and calculate the distance between any two points on the two flood flow time series data. The distance coefficient of high flow time series data is smaller than the distance coefficient of low flow time series data. (2-3) Using the dynamic time warping distance algorithm, the time warping distance between the two floods is calculated; (2-4) According to the time sequence traversal, the time warped distance between the two time series data of the K-field standardized flood flow is calculated, and finally the similarity matrix of the K-field flood is obtained.

4. The flood classification method based on dynamic time warping and multi-clustering coupling according to claim 3 is characterized in that: Steps (2-3) calculate the time warp distance between the two floods as follows: Perform similarity measurement on two flood flow time series data QA and QB with lengths m and n respectively, calculate the distance between each pair of points in the two series, and obtain the distance matrix based on QA and QB; The cumulative distance between the first i elements of the flood flow time series data QA and the first j elements of the flood flow time series data QB ; Represents the distance between the i-th element of the flood flow time series data QA and the j-th element of the flood flow time series data QB; According to the principle of dynamic programming, we backtrack from the end point (m,n) to the starting point (1,1) to find the optimal path r(m,n). The path selection after each data point matching follows continuity and monotonicity, and the boundaries are consistent. Normalize the total distance by the path length L to get the time warp distance , .

5. The flood classification method based on dynamic time warping and multi-clustering coupling according to claim 1 is characterized in that: Step (3-1) is as follows: (3-1-1) During the first classification, each flood is considered as Class 1, and the classification set is recorded as Q (0) , let Q (0) =Q, the time warped distance between the time series data of the normalized secondary flood flow in the known K fields, recorded as the initial distance matrix ; (3-1-2) Select the distance matrix D (0) The two floods corresponding to the minimum value in the , are merged into one category, recorded as new category C, , Represents the smallest number in the K×K matrix, that is, the minimum distance; after clustering is completed, the number of categories will be reduced by one, and the new category set will be recorded as ; (3-1-3) Calculate the new class C and the original Q (0) The dynamic time warping distance of each category, after clustering, the distance matrix between each category is updated to ; (3-1-4) Repeat (3-1-2) and (3-1-3) until they are clustered into one category; (3-1-5) Draw a cluster tree based on the system clustering results and remove isolated points; Isolation point judgment conditions: Assume that each type of flood event must meet ≥Y events, and those that do not meet the conditions are considered isolated points, 1≤Y≤3; Determine a reasonable number of flood classifications R, 2≤R≤5. If there are multiple options that meet the above conditions at the same time, calculate the silhouette coefficient of each option separately, and select the category with the largest silhouette coefficient S1 and the corresponding flood classification number R as the preferred option.

6. The flood classification method based on dynamic time warping and multi-clustering coupling according to claim 1, characterized in that: Step (3-3) is as follows: (3-3-1) DTW-based classification and recognition; calculate the distance between all data points except the initial center medoids and each K-medoids, and assign them to the cluster Ci where the closest medoids are located. The distance metric uses the dynamic time warping algorithm; (3-3-2) Calculate the new center of each of the R categories; traverse all categories, assume that all data points Qi∈Ci are new center medoids for each category, and calculate the total distance Z of all other data points to the new center medoids, and select the center medoids corresponding to the minimum distance as the new center; the new center , Qi traverses all points in Ci, DTW(Qi,Qj) is the dynamic time warping distance function, the new center It is the point that minimizes the total distance inside Ci; (3-3-3) Repeat (3-3-1) and (3-3-2) until the termination condition is met; the termination condition is that the set number of iterations is reached or the K-medoids in all categories no longer change; (3-3-4) Calculate the classification silhouette coefficient S2 of DTW coupled hierarchical clustering and K-medoids.

7. A system for implementing the flood classification method based on dynamic time warping and multi-clustering coupling as described in any one of claims 1 to 6, characterized in that: Including data acquisition module, data preprocessing module, comparison module, classification output module; The data acquisition module is used to collect flow time series data within the target basin and target time range; The data preprocessing module is used to clean, process the flood flow time series data at the whole time, interpolate and standardize them; The comparison module is used to obtain pairwise similarities between multiple floods; The classification output module is used to obtain and output the determined flood type.

Images