A change point detection method for multidimensional hydrological time series data
Through the combination of principal component analysis algorithm and feature vectors, the variable points in multi-dimensional hydrological time series data are identified, which solves the instability and computational complexity problems of existing methods in high-dimensional data processing, and achieves more accurate variable points Improve detection and model generalization capabilities.
Patent Information
- Application Number
- CN202411034326.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-07-31
AI Technical Summary
Existing hydrological data variable point detection methods perform poorly when processing high-dimensional and multivariate time series data, and are susceptible to noise and data complexity, resulting in unstable detection results, high computational complexity, and difficult to meet the needs of real-time monitoring and analysis. , and it is easy to overfit.
A method is adopted to collect multi-dimensional hydrological time series data, use principal component analysis algorithm to calculate the projection matrix and zero eigenvector, calculate the L2 norm and the minimum eigenvalue ratio factor number, divide the data subset, and process it recursively to identify variable points. Improve detection accuracy and robustness.
This method can more accurately detect variable points in multi-dimensional hydrological time series data, improve the generalization ability and prediction accuracy of the hydrological model. It is suitable for reservoir scheduling, flood warning and water resource management scenarios, and has high robustness and Adaptability.
Smart Images

Figure CN118568642B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of hydrological data analysis, and in particular to a change point detection method applied to multidimensional hydrological time series data. Background Art
[0002] Change point detection in hydrological data is of great significance in water resources management and research. It can identify structural changes in time series data and help understand the impact of climate change and human activities on hydrological systems. It is widely used in flood risk management, reservoir scheduling optimization, groundwater resource assessment, water quality monitoring, and watershed management. Especially in long-term hydrological simulation, the input-output relationship may change over time, which in turn affects the generalization ability of the model. Therefore, in the change point detection of hydrological data, we should not only focus on the change points of a single data, but also the mutation of the mapping relationship between multidimensional data. In practical applications, it is necessary to detect change points in multidimensional hydrological time series data so that changes in the input-output mapping relationship can be considered in the training of hydrological models, thereby improving the generalization ability of the model.
[0003] Change point detection methods in hydrological applications mainly include statistical methods, Bayesian methods, nonparametric methods and machine learning methods. Statistical methods such as the sliding average method and cumulative sum test are common change point detection techniques. The sliding average method reduces fluctuations by smoothing time series data to identify change points. However, this method has limited effect on identifying mutation points, especially when the data fluctuates greatly or there is a lot of noise. The cumulative sum test detects change points by the mutation of the cumulative sum curve, which is suitable for detecting small changes. However, it is not effective when dealing with nonlinear changes and noisy data. The Bayesian method handles data with large noise by calculating the posterior probability and dynamically adjusting the model parameters. This method can better deal with uncertainty and changes in the data, but the calculation is complex and costly. In addition, the Bayesian method has great difficulty in model construction and parameter selection, requiring researchers to have high statistical and programming skills. Nonparametric methods do not rely on specific data distribution assumptions, have high flexibility, and are often used for change point detection. However, these methods have high computational complexity when dealing with high-dimensional data and a large number of samples, and are easily affected by data noise. For example, kernel density estimation and K-nearest neighbor methods do not need to assume the distribution form of data when detecting change points, but they consume a lot of computing time and resources, making them difficult to use in real-time applications. Machine learning methods such as support vector machines identify change points by building classification models, which are suitable for linear and nonlinear change point detection. However, support vector machines are sensitive to parameter selection and have high computational costs. Deep learning models such as convolutional neural networks and recurrent neural networks can automatically learn complex patterns in time series, but they rely on a large amount of labeled data, consume a lot of computing resources during the training process, and the model's interpretability is poor. Ensemble learning methods such as random forests and gradient boosting trees improve detection accuracy and robustness by combining multiple basic models. Their models are highly complex, take a long time to train, and require high hyperparameter tuning.
[0004] In summary, existing change point detection methods still have some shortcomings in processing complex data, real-time performance, parameter sensitivity, evaluation criteria, and overfitting risks. First, existing change point detection methods perform poorly when processing high-dimensional and multivariate time series data, and are easily affected by noise and data complexity, resulting in unstable detection results. Secondly, the existing change point detection methods have high computational complexity and are difficult to meet the needs of real-time monitoring and analysis. Finally, when processing noisy data, existing change point detection methods are prone to overfitting, resulting in too many detected change points or inaccurate locations. In the application of hydrological research, the above problems are particularly prominent. For example, in scenarios such as reservoir scheduling, flood warning, and water resources management, the accuracy and real-time performance of change point detection are crucial to the scientificity and timeliness of decision-making. Summary of the invention
[0005] In a first aspect, an embodiment of the present invention provides a change point detection method applied to multidimensional hydrological time series data, the method comprising:
[0006] Collect multidimensional hydrological time series data to obtain a multidimensional hydrological time series data set;
[0007] According to the minimum training window of the hydrological model to be trained using the multidimensional hydrological time series dataset, the left boundary data and the right boundary data of the multidimensional hydrological time series dataset are determined. For the left and right boundary data and the data between the left and right boundary data, starting from the left boundary data, the data on the left and right sides of the left boundary data are divided into a left sub-dataset and a right sub-dataset, and the projection matrix and the zero eigenvector matrix corresponding to the left and right sub-datasets are calculated. According to the projection matrix and the zero eigenvector matrix corresponding to the left sub-dataset, the L2 norm corresponding to the left sub-dataset is calculated. According to the projection matrix and the zero eigenvector matrix corresponding to the right sub-dataset, the L2 norm corresponding to the right sub-dataset is calculated. norm, add the L2 norms corresponding to the left and right sub-datasets to obtain the sum of the L2 norms, and determine whether the sum of the L2 norms is less than the reference threshold. If not, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset, and the above processing is performed. If less than, the reference threshold is assigned to the sum of the L2 norms, and the minimum eigenvalue ratio factor number corresponding to the left and right sub-datasets is calculated to determine whether the minimum eigenvalue ratio factor number corresponding to the left and right sub-datasets is consistent. If consistent, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset, and the above processing is performed. If inconsistent, the corresponding data is determined as a change point;
[0008] Recursively perform the above process on the left sub-dataset and the right sub-dataset corresponding to the change point.
[0009] In some implementations of the first aspect, the multidimensional hydrological time series data includes hydrometeorological characteristics, hydrogeological characteristics, hydrological river basin characteristics, and water resource characteristics.
[0010] In some possible implementations of the first aspect, collecting multidimensional hydrological time series data to obtain a multidimensional hydrological time series data set includes:
[0011] Collect n×p multidimensional hydrological time series data, determine the input and output of the hydrological model, and standardize the multidimensional hydrological time series data to obtain a multidimensional hydrological time series data set; where n is the number of data, and p is the sum of the number of features and the number of outputs of the hydrological model.
[0012] In some implementations of the first aspect, determining left boundary data and right boundary data of the multidimensional hydrological time series dataset according to a minimum training window of a hydrological model to be trained using the multidimensional hydrological time series dataset includes:
[0013] The data sequence number range of the multidimensional hydrological time series data set is recorded as 1 to n, and the minimum training window is recorded as min_size, so as to determine that the left boundary sequence number and the right boundary sequence number of the multidimensional hydrological time series data set are min_size and n-min_size respectively, and then determine that the data corresponding to min_size is the left boundary data, and the data corresponding to n-min_size is the right boundary data.
[0014] In some implementations of the first aspect, calculating a projection matrix and a zero eigenvector matrix corresponding to the left and right sub-data sets includes:
[0015] The left and right sub-datasets are processed using the principal component analysis algorithm to obtain the projection matrix corresponding to the left and right sub-datasets determined by the number of factors;
[0016] Calculate the covariance matrix corresponding to the left and right sub-datasets, and use it to calculate the zero eigenvector matrix corresponding to the left and right sub-datasets.
[0017] In some implementations of the first aspect, the number of factors is obtained by estimating a data cumulative eigenvalue contribution threshold of not less than 95% for a multidimensional hydrological time series dataset.
[0018] In some implementations of the first aspect, calculating the covariance matrix corresponding to the left and right sub-data sets includes:
[0019] The left and right sub-datasets are standardized, and the corresponding covariance matrix is calculated based on the standardized left and right sub-datasets.
[0020] In a second aspect, an embodiment of the present invention provides a change point detection device applied to multidimensional hydrological time series data, the device comprising:
[0021] A collection module, used to collect multi-dimensional hydrological time series data to obtain a multi-dimensional hydrological time series data set;
[0022] The detection module is used to determine the left boundary data and the right boundary data of the multidimensional hydrological time series data set according to the minimum training window of the hydrological model to be trained using the multidimensional hydrological time series data set, and for the left and right boundary data and the data between the left and right boundary data, starting from the left boundary data, the data on the left and right sides of the left boundary data are divided into a left sub-data set and a right sub-data set, and the projection matrix and the zero eigenvector matrix corresponding to the left and right sub-data sets are calculated. According to the projection matrix and the zero eigenvector matrix corresponding to the left sub-data set, the L2 norm corresponding to the left sub-data set is calculated, and according to the projection matrix and the zero eigenvector matrix corresponding to the right sub-data set, the L2 norm corresponding to the right sub-data set is calculated. The corresponding L2 norm is calculated, and the L2 norms corresponding to the left and right sub-datasets are added to obtain the sum of the L2 norms. It is determined whether the sum of the L2 norms is less than the reference threshold value. If not, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset, and the above processing is performed. If it is less than, the reference threshold is assigned to the sum of the L2 norms, and the number of minimum eigenvalue ratio factors corresponding to the left and right sub-datasets is calculated to determine whether the number of minimum eigenvalue ratio factors corresponding to the left and right sub-datasets is consistent. If they are consistent, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset, and the above processing is performed. If they are inconsistent, the corresponding data is determined as a change point;
[0023] The detection module is also used to recursively perform the above processing on the left sub-dataset and the right sub-dataset corresponding to the change point.
[0024] In a third aspect, an embodiment of the present invention provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described above.
[0025] In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to cause a computer to execute the method described above.
[0026] In the embodiments of the present invention, the change points in multidimensional hydrological time series data can be effectively detected, and can be widely used in hydrological scenarios such as reservoir scheduling, flood warning and water resources management, to serve the machine learning modeling using multidimensional hydrological time series data, and improve the generalization ability and prediction accuracy of the hydrological model.
[0027] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are provided for a better understanding of the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:
[0029] Figure 1 A flow chart of a change point detection method applied to multi-dimensional hydrological time series data provided by an embodiment of the present invention is shown;
[0030] Figure 2 A model training result provided by an embodiment of the present invention is shown.
[0031] Figure 3 A structural diagram of a change point detection device for multi-dimensional hydrological time series data provided by an embodiment of the present invention is shown;
[0032] Figure 4 A block diagram of an exemplary electronic device capable of implementing an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0034] In addition, the term "and / or" in the present invention is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects before and after are in an "or" relationship.
[0035] In order to solve the technical problems in the background technology, an embodiment of the present invention provides a change point detection method applied to multidimensional hydrological time series data. In conjunction with the accompanying drawings, a change point detection method applied to multidimensional hydrological time series data provided by an embodiment of the present invention is described in detail through a specific embodiment.
[0036] Figure 1 FIG. 4 shows a flow chart of a change point detection method for multidimensional hydrological time series data provided by an embodiment of the present invention. Figure 1 As shown, the change point detection method 100 may include the following steps:
[0037] S110, collecting multidimensional hydrological time series data to obtain a multidimensional hydrological time series data set.
[0038] Among them, the multidimensional hydrological time series data may include hydro-meteorological characteristics, hydro-geological characteristics, hydrological river basin characteristics, and water resource characteristics, etc., which are not limited here.
[0039] In some embodiments, n×p multidimensional hydrological time series data can be collected to determine the input and output of the hydrological model to be trained using the multidimensional hydrological time series data set, and the multidimensional hydrological time series data can be standardized to obtain the multidimensional hydrological time series data set; wherein n is the number of data, and the multidimensional hydrological time series data is required to be a continuous time series data, and p is the sum of the number of features and the number of outputs of the hydrological model.
[0040] S120, according to the minimum training window of the hydrological model to be trained using the multidimensional hydrological time series dataset, determine the left boundary data and the right boundary data of the multidimensional hydrological time series dataset, for the left and right boundary data and the data between the left and right boundary data, starting from the left boundary data, divide the data on the left and right sides of the left boundary data into a left sub-dataset and a right sub-dataset, calculate the projection matrix and the zero eigenvector matrix corresponding to the left and right sub-datasets, calculate the L2 norm corresponding to the left sub-dataset according to the projection matrix and the zero eigenvector matrix corresponding to the left sub-dataset, calculate the L2 norm corresponding to the right sub-dataset according to the projection matrix and the zero eigenvector matrix corresponding to the right sub-dataset , add the L2 norms corresponding to the left and right sub-datasets to obtain the sum of the L2 norms, and determine whether the sum of the L2 norms is less than the reference threshold. If not, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset, and the above processing is performed. If less than, the reference threshold is assigned to the sum of the L2 norms, and the number of minimum eigenvalue ratio factors corresponding to the left and right sub-datasets is calculated to determine whether the number of minimum eigenvalue ratio factors corresponding to the left and right sub-datasets is consistent. If they are consistent, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset, and the above processing is performed. If they are inconsistent, the corresponding data is determined as a change point.
[0041] In some embodiments, the data sequence number range of the multidimensional hydrological time series data set can be recorded as 1 to n, and the minimum training window can be recorded as min_size, so as to determine that the left boundary sequence number and the right boundary sequence number of the multidimensional hydrological time series data set are min_size and n-min_size respectively, and then determine that the data corresponding to min_size is the left boundary data, and the data corresponding to n-min_size is the right boundary data.
[0042] Starting from the left boundary data, the data on the left and right sides of the left boundary data are divided into a left sub-dataset and a right sub-dataset respectively.
[0043] The left and right sub-datasets are processed using the principal component analysis (PCA) algorithm to obtain projection matrices corresponding to the left and right sub-datasets that are determined by the number of factors. For example, the above processing can be represented by the following formula:
[0044] (1)
[0045] (2)
[0046] in, Represents the projection matrix corresponding to the left sub-dataset; Represents the projection matrix corresponding to the right sub-dataset; Represents the left sub-dataset; Represents the right sub-dataset; It indicates the number of factors obtained by estimating the cumulative eigenvalue contribution threshold of the multidimensional hydrological time series dataset to be no less than 95%.
[0047] Calculate the covariance matrix corresponding to the left and right sub-datasets, for example, standardize the left and right sub-datasets, and calculate the corresponding covariance matrix based on the standardized left and right sub-datasets. Exemplarily, the above calculation process can be expressed by the following formula:
[0048] (3)
[0049] (4)
[0050] (5)
[0051] (6)
[0052] in, Represents the standardized left sub-dataset; Represents the normalized right sub-dataset; Represents the left sub-dataset; Represents the right sub-dataset; Represents the covariance matrix corresponding to the left sub-dataset; Represents the covariance matrix corresponding to the right sub-dataset.
[0053] According to the covariance matrix corresponding to the left and right sub-datasets, the zero eigenvector matrix corresponding to the left and right sub-datasets is calculated. For example, the covariance matrix corresponding to the left and right sub-datasets is , Perform eigenvalue decomposition and set {λ 1 1 ,λ 12 ,...,λ 1 e} and {λ 2 1 ,λ 2 2 ,...,λ 2 f} respectively and The eigenvalues in ascending order have corresponding eigenvectors. {v 1 1 ,v 1 2 ,...,v 1 e}and ={v 2 1 ,v 2 2 ,...,v 2 f}, in order to find the null space, the eigenvectors corresponding to the minimum eK eigenvalues are selected as the estimates of the zero eigenvector matrix corresponding to the left sub-dataset, and the eigenvectors corresponding to the fK eigenvalues are selected as the estimates of the zero eigenvector matrix corresponding to the right sub-dataset. Exemplarily, the above calculation process can be expressed by the following formula:
[0054] (7)
[0055] (8)
[0056] (9)
[0057] (10)
[0058] in, Represents the covariance matrix corresponding to the left sub-dataset; Represents the covariance matrix corresponding to the right sub-dataset; express Eigenvectors corresponding to eigenvalues in ascending order; express Eigenvectors corresponding to eigenvalues in ascending order; Represents the zero eigenvector matrix corresponding to the left sub-dataset; Represents the zero eigenvector matrix corresponding to the right sub-dataset.
[0059] According to the projection matrix and zero eigenvector matrix corresponding to the left sub-dataset, the L2 norm corresponding to the left sub-dataset is calculated, and according to the projection matrix and zero eigenvector matrix corresponding to the right sub-dataset, the L2 norm corresponding to the right sub-dataset is calculated, and the L2 norms corresponding to the left and right sub-datasets are added to obtain the sum of the L2 norms. Exemplarily, the above calculation process can be represented by the following formula:
[0060] + (11)
[0061] in, represents the sum of L2 norms; Represents the zero eigenvector matrix corresponding to the left sub-dataset; Represents the zero eigenvector matrix corresponding to the right sub-dataset; Represents the projection matrix corresponding to the left sub-dataset; Represents the projection matrix corresponding to the right sub-dataset.
[0062] Determine whether the sum of the L2 norms is less than the reference threshold. If not, divide the data on the left and right sides of the next data into left sub-datasets and right sub-datasets respectively, and perform the above processing. If less than, assign the reference threshold to the sum of the L2 norms, and calculate the minimum eigenvalue ratio factor number corresponding to the left and right sub-datasets. For example, based on and The eigenvalues are arranged in ascending order, and the number of factors with the minimum eigenvalue ratio corresponding to the left and right sub-datasets is estimated by finding the index value with the minimum eigenvalue ratio. Here, since the index value starts from 0, 1 needs to be added to match the estimated number of factors. Exemplarily, the above calculation process can be expressed by the following formula:
[0063] (12)
[0064] in, Indicates the minimum eigenvalue ratio factor number corresponding to the left sub-dataset; Indicates the number of minimum eigenvalue ratio factors corresponding to the right sub-dataset.
[0065] Determine whether the number of minimum eigenvalue ratio factors corresponding to the left and right sub-datasets is consistent. If they are consistent, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset respectively, and the above processing is performed. If they are inconsistent, the corresponding data is determined as a change point.
[0066] S130, recursively perform the above processing on the left sub-dataset and the right sub-dataset corresponding to the change point.
[0067] Specifically, the left sub-dataset and the right sub-dataset corresponding to the change point are recursively processed respectively, and the above process is repeated until each sub-dataset is smaller than the minimum training window, and finally all the change points are output.
[0068] According to the embodiments of the present invention, at least the following technical effects are achieved:
[0069] (1) By combining features and outputs, this method can more accurately detect change points in the data. Features can help understand the internal changes between features and inputs, while outputs show the overall changes in the system. By more accurately identifying the changes in the input-output relationship of systems such as rivers and reservoirs at different time points, we can better understand and manage water resources.
[0070] (2) It can process complex multidimensional hydrological time series data and is not only suitable for the analysis of a single variable, but also performs well in multivariate joint analysis, making it more suitable for scenarios that require comprehensive consideration of multiple hydrological factors.
[0071] (3) Through the combination of eigenvector orthogonality and covariance matrix, this method has high robustness and can work stably in noisy data environments. At the same time, the dynamic adjustment and model space mapping of this method improve its adaptability in different hydrological scenarios and is suitable for a variety of hydrological management needs.
[0072] In summary, this method effectively improves the accuracy, robustness and adaptability of change point detection through comprehensive features and outputs, model space mapping and dynamic adjustment, and has significant advantages in applications such as reservoir scheduling, flood warning and water resources management.
[0073] For further understanding, the application scenario of the power transformation detection method provided by the embodiment of the present invention is described below in combination with a specific embodiment:
[0074] (1) Data collection:
[0075] Taking the YZ reservoir in the X basin as an example, the daily-scale multidimensional hydrological time series data from 2002 to 2009 were analyzed. The data include outflow differences (as the output of the hydrological model), inflow, basin CN value, monthly electricity consumption of provinces, representative monthly crop water requirements, meteorological characteristics and time characteristics. First, the characteristic variables were preliminarily screened by using the Spearman correlation coefficient and variance threshold of 0.1; then, according to the needs of the hydrological task simulation scenario, the inflow characteristics were introduced to form a multidimensional hydrological time series data set. Finally, the advantages of the Spearman correlation coefficient were confirmed by the ADF test and the Shapiro-Wilk normality test, and the significant correlation test was passed.
[0076] (2) Import change point detection method:
[0077] As mentioned above, the maximum eigenvalue cumulative contribution rate is used to estimate the number of factors for the multidimensional hydrological time series data set processed in step (1). This number of factors is usually less than the sum of the output and the number of features of the data set. After setting an appropriate minimum training window, the data set is imported into the change point algorithm for detection. The algorithm converges quickly and detects the change point as 1576. The data are arranged in chronological order to ensure its continuity.
[0078] (3) Real records and model verification:
[0079] As mentioned above, the change point detection method used in step (2) was verified, and it was found that the change point corresponded to April 25, 2006 in the time dimension. During this period, the YZ Reservoir experienced high-intensity rainfall. In August of the same year, the YZ Reservoir opened the gate to release floodwater, verifying the accuracy of the change point detection method. In the construction of the hydrological model, the original multidimensional hydrological time series dataset was divided into two left and right sub-datasets based on the change point, randomly divided at a ratio of 3:1, and created time lag features. The hydrological model was trained by machine learning majority voting method (random forest, gradient boosting tree, SVR, KNN, MLP), and monthly and seasonal features were added to help capture time-related periodic changes. The model evaluation indicators are NSE and RMSE, and the unit of RMSE is cubic meters per second. The hydrological model was trained with all the data before the change point segmentation. The NSE and RMSE of the total model for the change point left and change point right test sets were 0.72 and 46.41 m, respectively. 3 / s and 0.91, 37.69m 3 / s. The NSE and RMSE of the test set of model 1 and model 2 constructed by the left and right sub-datasets are 0.77 and 42.74 m respectively. 3 / s and 0.93,33.85m 3 / s (see Figure 2 ). It can be seen that the change point detection method can effectively improve the performance of the model. Among them, the random forest model performs best. The feature importance analysis shows that the left sub-dataset is insensitive to the water requirements of various crops, while the right sub-dataset depends on the water requirements of crops, which provides an objective understanding of the mechanism changes in the YZ reservoir scheduling decision.
[0080] Obviously, the change point detection method effectively improves the accuracy, robustness and adaptability of change point detection in helping to detect complex hydrological tasks driven by complex multi-dimensional hydrological time series data, and has significant advantages in applications such as reservoir scheduling, flood warning and water resources management.
[0081] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present invention.
[0082] The above is an introduction to a method embodiment. The following is a further explanation of the solution of the present invention through an apparatus embodiment. Figure 3 FIG. 4 shows a structural diagram of a change point detection device for multi-dimensional hydrological time series data provided by an embodiment of the present invention. Figure 3 As shown, the change point detection device 300 may include:
[0083] The acquisition module 310 is used to acquire multi-dimensional hydrological time series data to obtain a multi-dimensional hydrological time series data set.
[0084] The detection module 320 is used to determine the left boundary data and the right boundary data of the multidimensional hydrological time series data set according to the minimum training window of the hydrological model to be trained using the multidimensional hydrological time series data set, and for the left and right boundary data and the data between the left and right boundary data, starting from the left boundary data, the data on the left and right sides of the left boundary data are divided into a left sub-data set and a right sub-data set, respectively, and the projection matrix and the zero eigenvector matrix corresponding to the left and right sub-data sets are calculated, and the L2 norm corresponding to the left sub-data set is calculated according to the projection matrix and the zero eigenvector matrix corresponding to the left sub-data set, and the L2 norm corresponding to the right sub-data set is calculated according to the projection matrix and the zero eigenvector matrix corresponding to the right sub-data set. Corresponding L2 norm, add the L2 norms corresponding to the left and right sub-datasets to obtain the sum of the L2 norms, and judge whether the sum of the L2 norms is less than the reference threshold. If not, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset respectively, and the above processing is performed. If it is less than, the reference threshold is assigned to the sum of the L2 norms, and the number of minimum eigenvalue ratio factors corresponding to the left and right sub-datasets is calculated to judge whether the number of minimum eigenvalue ratio factors corresponding to the left and right sub-datasets is consistent. If they are consistent, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset respectively, and the above processing is performed. If they are inconsistent, the corresponding data is determined as a change point.
[0085] The detection module 320 is further used to recursively perform the above processing on the left sub-dataset and the right sub-dataset corresponding to the change point.
[0086] Understandably, Figure 3 Each module / unit in the change point detection device 300 shown has the function of realizing Figure 1The functions of the various steps in the change point detection method 100 shown can achieve their corresponding technical effects, and for the sake of brevity, they will not be repeated here.
[0087] Figure 4 A block diagram of an exemplary electronic device capable of implementing an embodiment of the present invention is shown. Electronic device 400 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 400 may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown in the present invention, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required in the present invention.
[0088] like Figure 4 As shown, the electronic device 400 may include a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.
[0089] Multiple components in the electronic device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0090] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as method 100. For example, in some embodiments, the method 100 may be implemented as a computer program product, including a computer program, which is tangibly contained in a computer-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the method 100 described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the method 100 in any other appropriate manner (e.g., by means of firmware).
[0091] The various embodiments described above in the present invention can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0092] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0093] In the context of the present invention, a computer-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0094] It should be noted that the present invention also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute method 100 and achieve the corresponding technical effect achieved by the embodiments of the present invention executing the method. For the sake of concise description, they are not repeated here.
[0095] In addition, the present invention also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method 100 is implemented.
[0096] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and the present invention is not limited here.
[0097] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A change point detection method for multidimensional hydrological time series data, characterized in that: The method comprises: Collect multidimensional hydrological time series data to obtain a multidimensional hydrological time series data set; According to the minimum training window of the hydrological model to be trained using the multidimensional hydrological time series dataset, the left boundary data and the right boundary data of the multidimensional hydrological time series dataset are determined. For the left and right boundary data and the data between the left and right boundary data, starting from the left boundary data, the data on the left and right sides of the left boundary data are divided into a left sub-dataset and a right sub-dataset, and the projection matrix and the zero eigenvector matrix corresponding to the left and right sub-datasets are calculated. According to the projection matrix and the zero eigenvector matrix corresponding to the left sub-dataset, the L2 norm corresponding to the left sub-dataset is calculated. According to the projection matrix and the zero eigenvector matrix corresponding to the right sub-dataset, the L2 norm corresponding to the right sub-dataset is calculated. norm, add the L2 norms corresponding to the left and right sub-datasets to obtain the sum of the L2 norms, and determine whether the sum of the L2 norms is less than the reference threshold. If not, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset, and the above processing is performed. If less than, the reference threshold is assigned to the sum of the L2 norms, and the minimum eigenvalue ratio factor number corresponding to the left and right sub-datasets is calculated to determine whether the minimum eigenvalue ratio factor number corresponding to the left and right sub-datasets is consistent. If consistent, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset, and the above processing is performed. If inconsistent, the corresponding data is determined as a change point; Recursively perform the above process on the left sub-dataset and the right sub-dataset corresponding to the change point; The step of determining the left boundary data and the right boundary data of the multidimensional hydrological time series dataset according to the minimum training window of the hydrological model to be trained using the multidimensional hydrological time series dataset comprises: The data serial number range of the multidimensional hydrological time series data set is recorded as 1 to n, and the minimum training window is recorded as min_size, so as to determine that the left boundary serial number and the right boundary serial number of the multidimensional hydrological time series data set are min_size and n-min_size respectively, and then determine that the data corresponding to min_size is the left boundary data, and the data corresponding to n-min_size is the right boundary data; The method of calculating the projection matrix and the zero eigenvector matrix corresponding to the left and right sub-datasets, calculating the L2 norm corresponding to the left sub-dataset according to the projection matrix and the zero eigenvector matrix corresponding to the left sub-dataset, calculating the L2 norm corresponding to the right sub-dataset according to the projection matrix and the zero eigenvector matrix corresponding to the right sub-dataset, and adding the L2 norms corresponding to the left and right sub-datasets to obtain the sum of the L2 norms includes: The principal component analysis algorithm is used to process the left and right sub-datasets to obtain the projection matrix corresponding to the left and right sub-datasets, which is determined by the number of factors. The processing process is expressed by the following formula: M1=PCA(X1,K) T <h2 style=";text-align:left;direction:ltr">M2 = PCA(X2,K)<h2 style=";text-align:left;direction:ltr"> T Among them, M1 represents the projection matrix corresponding to the left sub-dataset; M2 represents the projection matrix corresponding to the right sub-dataset; X1 represents the left sub-dataset; X2 represents the right sub-dataset; K represents the number of factors obtained by estimating the data cumulative eigenvalue contribution threshold of not less than 95% for the multidimensional hydrological time series dataset; Calculate the covariance matrix corresponding to the left and right sub-datasets; specifically, standardize the left and right sub-datasets, and calculate the corresponding covariance matrix based on the standardized left and right sub-datasets. The calculation process is represented by the following formula: Z1=standardize(X1) Z2=standardize(X2) C1=COV(Z1) C2=COV(Z2) Among them, Z1 represents the standardized left sub-dataset; Z2 represents the standardized right sub-dataset; X1 represents the left sub-dataset; X2 represents the right sub-dataset; C1 represents the covariance matrix corresponding to the left sub-dataset; C2 represents the covariance matrix corresponding to the right sub-dataset; According to the covariance matrix corresponding to the left and right sub-datasets, the zero eigenvector matrix corresponding to the left and right sub-datasets is calculated; specifically, the covariance matrices C1 and C2 corresponding to the left and right sub-datasets are subjected to eigenvalue decomposition, and {λ 1 1,λ 1 2,...,λ 1 e } and {λ 2 1,λ 2 2,...,λ 2 f } are the eigenvalues of C1 and C2 in ascending order, and their corresponding eigenvectors are V 1 = {v 1 1,v 1 2,...,v 1 e } and V 2 = {v 2 1,v 2 2,...,v 2 f }, in order to find the null space, the eigenvectors corresponding to the smallest eK eigenvalues are selected as the estimates of the zero eigenvector matrix corresponding to the left sub-dataset, and the eigenvectors corresponding to the fK eigenvalues are selected as the estimates of the zero eigenvector matrix corresponding to the right sub-dataset. The calculation process is expressed by the following formula: Where C1 represents the covariance matrix corresponding to the left sub-dataset; C2 represents the covariance matrix corresponding to the right sub-dataset; V 1 represents the eigenvector corresponding to the eigenvalues of C1 in ascending order; V 2 represents the eigenvector corresponding to the eigenvalues of C2 in ascending order; B1 represents the zero eigenvector matrix corresponding to the left sub-dataset; B2 represents the zero eigenvector matrix corresponding to the right sub-dataset; According to the projection matrix and zero eigenvector matrix corresponding to the left sub-dataset, the L2 norm corresponding to the left sub-dataset is calculated. According to the projection matrix and zero eigenvector matrix corresponding to the right sub-dataset, the L2 norm corresponding to the right sub-dataset is calculated. The L2 norms corresponding to the left and right sub-datasets are added to obtain the sum of the L2 norms. The calculation process is represented by the following formula: Among them, G represents the sum of L2 norms; B1 represents the zero eigenvector matrix corresponding to the left sub-dataset; B2 represents the zero eigenvector matrix corresponding to the right sub-dataset; M1 represents the projection matrix corresponding to the left sub-dataset; M2 represents the projection matrix corresponding to the right sub-dataset; The calculating of the minimum eigenvalue ratio factor number corresponding to the left and right sub-data sets includes: Based on the eigenvalues of the covariance matrix C1 corresponding to the left sub-dataset and the covariance matrix C2 corresponding to the right sub-dataset in ascending order, the minimum eigenvalue ratio factor number corresponding to the left and right sub-datasets is estimated by finding the index value with the minimum eigenvalue ratio; The recursive processing is performed on the left sub-dataset and the right sub-dataset corresponding to the change point, and the above processing is repeated, including: Recursively perform the left sub-dataset and the right sub-dataset corresponding to the change point, and repeat the above process until each sub-dataset is smaller than the minimum training window, and finally output all the change points.
2. The method according to claim 1, characterized in that: The multi-dimensional hydrological time series data includes hydro-meteorological characteristics, hydro-geological characteristics, hydro-basin characteristics and water resources characteristics.
3. The method according to claim 1, characterized in that The multidimensional hydrological time series data is collected to obtain a multidimensional hydrological time series data set, including: Collect n×p multidimensional hydrological time series data, determine the input and output of the hydrological model, and standardize the multidimensional hydrological time series data to obtain a multidimensional hydrological time series data set; where n is the number of data, and p is the sum of the number of features and the number of outputs of the hydrological model.
4. A change point detection device for multidimensional hydrological time series data, characterized in that: The device comprises: A collection module, used to collect multi-dimensional hydrological time series data to obtain a multi-dimensional hydrological time series data set; The detection module is used to determine the left boundary data and the right boundary data of the multidimensional hydrological time series data set according to the minimum training window of the hydrological model to be trained using the multidimensional hydrological time series data set, and for the left and right boundary data and the data between the left and right boundary data, starting from the left boundary data, the data on the left and right sides of the left boundary data are divided into a left sub-data set and a right sub-data set, and the projection matrix and the zero eigenvector matrix corresponding to the left and right sub-data sets are calculated. According to the projection matrix and the zero eigenvector matrix corresponding to the left sub-data set, the L2 norm corresponding to the left sub-data set is calculated, and according to the projection matrix and the zero eigenvector matrix corresponding to the right sub-data set, the L2 norm corresponding to the right sub-data set is calculated. The corresponding L2 norm is calculated, and the L2 norms corresponding to the left and right sub-datasets are added to obtain the sum of the L2 norms. It is determined whether the sum of the L2 norms is less than the reference threshold value. If not, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset, and the above processing is performed. If it is less than, the reference threshold is assigned to the sum of the L2 norms, and the number of minimum eigenvalue ratio factors corresponding to the left and right sub-datasets is calculated to determine whether the number of minimum eigenvalue ratio factors corresponding to the left and right sub-datasets is consistent. If they are consistent, the data on the left and right sides of the next data are divided into a left sub-dataset and a right sub-dataset, and the above processing is performed. If they are inconsistent, the corresponding data is determined as a change point; The detection module is further used to recursively perform the above processing on the left sub-dataset and the right sub-dataset corresponding to the change point respectively; The detection module is specifically used for: The data serial number range of the multidimensional hydrological time series data set is recorded as 1 to n, and the minimum training window is recorded as min_size, so as to determine that the left boundary serial number and the right boundary serial number of the multidimensional hydrological time series data set are min_size and n-min_size respectively, and then determine that the data corresponding to min_size is the left boundary data, and the data corresponding to n-min_size is the right boundary data; The detection module is specifically used for: The principal component analysis algorithm is used to process the left and right sub-datasets to obtain the projection matrix corresponding to the left and right sub-datasets, which is determined by the number of factors. The processing process is expressed by the following formula: M1=PCA(X1,K) T <h2 style=";text-align:left;direction:ltr">M2 = PCA(X2,K)<h2 style=";text-align:left;direction:ltr"> T Among them, M1 represents the projection matrix corresponding to the left sub-dataset; M2 represents the projection matrix corresponding to the right sub-dataset; X1 represents the left sub-dataset; X2 represents the right sub-dataset; K represents the number of factors obtained by estimating the data cumulative eigenvalue contribution threshold of not less than 95% for the multidimensional hydrological time series dataset; Calculate the covariance matrix corresponding to the left and right sub-datasets; specifically, standardize the left and right sub-datasets, and calculate the corresponding covariance matrix based on the standardized left and right sub-datasets. The calculation process is represented by the following formula: Z1=standardize(X1) Z2=standardize(X2) C1=COV(Z1) C2=COV(Z2) Among them, Z1 represents the standardized left sub-dataset; Z2 represents the standardized right sub-dataset; X1 represents the left sub-dataset; X2 represents the right sub-dataset; C1 represents the covariance matrix corresponding to the left sub-dataset; C2 represents the covariance matrix corresponding to the right sub-dataset; According to the covariance matrix corresponding to the left and right sub-datasets, the zero eigenvector matrix corresponding to the left and right sub-datasets is calculated; specifically, the covariance matrices C1 and C2 corresponding to the left and right sub-datasets are subjected to eigenvalue decomposition, and {λ 1 1,λ 1 2,...,λ 1 e } and {λ 2 1,λ 2 2,...,λ 2 f } are the eigenvalues of C1 and C2 in ascending order, and their corresponding eigenvectors are V 1 = {v 1 1,v 1 2,...,v 1 e } and V 2 = {v 2 1,v 2 2,...,v 2 f }, in order to find the null space, the eigenvectors corresponding to the smallest eK eigenvalues are selected as the estimates of the zero eigenvector matrix corresponding to the left sub-dataset, and the eigenvectors corresponding to the fK eigenvalues are selected as the estimates of the zero eigenvector matrix corresponding to the right sub-dataset. The calculation process is expressed by the following formula: Where C1 represents the covariance matrix corresponding to the left sub-dataset; C2 represents the covariance matrix corresponding to the right sub-dataset; V 1 represents the eigenvector corresponding to the eigenvalues of C1 in ascending order; V 2 represents the eigenvector corresponding to the eigenvalues of C2 in ascending order; B1 represents the zero eigenvector matrix corresponding to the left sub-dataset; B2 represents the zero eigenvector matrix corresponding to the right sub-dataset; According to the projection matrix and zero eigenvector matrix corresponding to the left sub-dataset, the L2 norm corresponding to the left sub-dataset is calculated. According to the projection matrix and zero eigenvector matrix corresponding to the right sub-dataset, the L2 norm corresponding to the right sub-dataset is calculated. The L2 norms corresponding to the left and right sub-datasets are added to obtain the sum of the L2 norms. The calculation process is represented by the following formula: Among them, G represents the sum of L2 norms; B1 represents the zero eigenvector matrix corresponding to the left sub-dataset; B2 represents the zero eigenvector matrix corresponding to the right sub-dataset; M1 represents the projection matrix corresponding to the left sub-dataset; M2 represents the projection matrix corresponding to the right sub-dataset; The detection module is specifically used for: Based on the eigenvalues of the covariance matrix C1 corresponding to the left sub-dataset and the covariance matrix C2 corresponding to the right sub-dataset in ascending order, the minimum eigenvalue ratio factor number corresponding to the left and right sub-datasets is estimated by finding the index value with the minimum eigenvalue ratio; The detection module is specifically used for: Recursively perform the left sub-dataset and the right sub-dataset corresponding to the change point, and repeat the above process until each sub-dataset is smaller than the minimum training window, and finally output all the change points.
5. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 3.
6. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 3.
