Multi-measuring-point space coupling dam data management method adaptive to water conservancy long sequence

Abstract

The invention provides a multi-measuring-point space coupling dam data management method adaptive to a water conservancy long sequence, and the method comprises the steps: carrying out the clustering of preprocessed dam time sequence monitoring data through employing an improved K-means clustering method, dividing dam measuring points into a plurality of measuring point clusters according to the clustering result, and determining the double-coupling features of the measuring points in each measuring point cluster; based on the double-coupling characteristics of all the measuring points in the cluster to which each measuring point belongs, an effective neighbor set corresponding to each measuring point is determined for each measuring point, and normal measuring points and abnormal measuring points are obtained through division by taking the effective neighbor sets of the measuring points as calculation references; comparing and evaluating synchronism deviation values of the abnormal measuring points and the corresponding effective neighbors based on time sequence similar characteristics of the abnormal measuring points and the effective neighbors of the abnormal measuring points so as to divide measuring point types; and carrying out data restoration on the measuring point data of which the type is single measuring point abnormity. According to the invention, based on the improved K-means clustering method and the LOF method, a whole link of abnormal data identification and restoration coupled with a time-space relationship is formed, and manual connection is not needed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data acquisition and monitoring, and particularly relates to a multi-measuring-point spatial coupling dam data management method suitable for long water conservancy sequences. BACKGROUND

[0002] As the core infrastructure of water conservancy projects, it is crucial to ensure the safe operation of dams, and dam safety monitoring data is the core basis for evaluating the real-time serviceability of the dam body. Only under the premise of ensuring the authenticity and integrity of the monitoring data, can the dam safety risk be accurately warned.

[0003] However, the dam monitor is in a complex outdoor environment for a long time, and will face natural phenomena such as temperature fluctuations, water level changes, geological subsidence, and sensor aging, which will cause the monitoring data to be easily contaminated by noise, and abnormal values such as "short jump" and "numerical offset" will occur.

[0004] Since the above abnormal values are not caused by the safety risk of the dam itself, it is necessary to identify and clean the abnormal values in the monitoring data before predicting the safety risk of the dam using the monitoring data, so as to eliminate the noise in the monitoring data, and then repair the missing part to restore the real dam safety monitoring data.

[0005] Existing dam monitoring abnormal value data focuses on isolated time series analysis of single measuring point data, that is, identifying abnormal values in the monitoring data of a single sensor itself. Such a method is prone to systematic bias, which will affect the accuracy of the data repair result. SUMMARY

[0006] The present application provides a multi-measuring-point spatial coupling dam data management method suitable for long water conservancy sequences, which solves the defect that existing dam monitoring abnormal value data focuses on isolated time series analysis of single measuring point data, and realizes an abnormal monitoring data identification and repair method using multi-measuring-point spatial coupling.

[0007] The present application provides a multi-measuring-point spatial coupling dam data management method suitable for long water conservancy sequences, comprising: An improved K-means clustering method is used to cluster the preprocessed dam time series monitoring data, so as to divide the dam measuring points into multiple measuring point clusters according to the clustering results, and determine the double coupling characteristics of the measuring points in each measuring point cluster, the double coupling characteristics including static neighbor characteristics representing the static value closeness of the measuring point to the remaining measuring points in the cluster, and a time series similarity feature matrix representing the time series data change synchronization of the measuring point to the remaining measuring points in the cluster; Based on the double coupling characteristics of all the measuring points in the cluster to which each measuring point belongs, the effective neighbor set corresponding to each measuring point is determined, and the local outlier factor method is used to perform traversal calculation on all the measuring points based on the effective neighbor set of the measuring point as the calculation reference, so that the normal measuring points and abnormal measuring points are divided; Based on the time sequence similarity feature comparison of the abnormal measuring point and its effective neighbor, the synchronization deviation value of the abnormal measuring point and its corresponding effective neighbor is evaluated, and the abnormal measuring point type whose synchronization deviation value is greater than a preset synchronization threshold is determined as a multi-measuring point cooperative anomaly, otherwise, it is determined as a single-measuring point anomaly; Data repair is performed on the measuring point data of the type of single-measuring point anomaly According to the multi-measuring point space coupling dam data management method suitable for water conservancy long sequence provided by the application, the improved K-means clustering method is used for clustering the preprocessed dam time sequence monitoring data, and the dam measuring points are divided into a plurality of measuring point clusters according to the clustering result, which specifically includes: Based on the preprocessed dam time sequence monitoring data, a weighted undirected graph is constructed, wherein each measuring point is taken as a graph node, the preprocessed measuring point data of each measuring point is taken as the feature of the graph node, a connecting line is added between two graph nodes with spatial proximity or time sequence similarity as the edge between the graph nodes, and the edge weight value is determined; The weighted degree centrality of each node is calculated, and based on the weighted degree centrality and the spatial distribution uniformity, a preset number of candidate centroids with high weighted degree centrality and uniform spatial position distribution are screened out; Based on the composite distance of the fusion feature distance, the graph topological distance and the time sequence distance, each measuring point is distributed to the cluster to which the initial centroid belongs, and the centroid is updated through iteration, so that the preset number of measuring point clusters is divided based on the centroid obtained through iteration.

[0008] According to the multi-measuring point space coupling dam data management method suitable for water conservancy long sequence provided by the application, the step of constructing a weighted undirected graph based on the preprocessed dam time sequence monitoring data specifically includes: Each measuring point is defined as a graph node, and the preprocessed measuring point data of each measuring point is taken as the feature of the graph node; If the Euclidean distance between any two measuring points is less than a first preset distance threshold, it is determined that the two measuring points have spatial proximity; If the Pearson correlation coefficient of the time sequence monitoring data between any two measuring points is greater than a preset correlation threshold, it is determined that the two measuring points have time sequence similarity; A connecting line is added between two graph nodes with spatial proximity or time sequence similarity as the edge between the graph nodes, and the edge weight value between the two measuring points is determined based on the Euclidean distance and the Pearson correlation coefficient, so that the greater the edge weight value, the stronger the spatiotemporal coupling between the two measuring points.

[0009] According to the application, a multi-measuring point space coupling dam data management method suitable for water conservancy long sequences is provided. The weighted degree centrality is arranged in descending order to generate a candidate node list; The node with the highest weighted degree centrality is fixed as the first candidate centroid, and the nodes in the candidate node list are sequentially traversed to calculate the graph shortest path of each selected candidate centroid, and if the graph shortest distance between the node and each selected candidate centroid is not less than the second preset distance threshold, the node is fixed as a new candidate centroid; Otherwise, the node is adjusted and the next node in the candidate node list is checked until the preset number of candidate centroids is selected.

[0010] According to the application, a multi-measuring point space coupling dam data management method suitable for water conservancy long sequences is provided. In the cluster to which each measuring point belongs, the measuring points with static neighbor characteristics less than a preset neighbor threshold and time sequence similarity characteristics less than a preset experience threshold are determined as effective neighbors to construct an effective neighbor set of each measuring point. The preset neighbor threshold is half of the mean value of all static neighbor characteristics in the measuring point cluster to which the current measuring point belongs.

[0011] According to the application, a multi-measuring point space coupling dam data management method suitable for water conservancy long sequences is provided. C The following formula is used for calculation: ; In the formula, represents the effective neighbor set, and the time sequence similarity characteristic between the measuring point i and the effective neighbor j , is the time sequence similarity characteristic mean value of the measuring point i and all effective neighbors thereof.

[0012] According to the application, a multi-measuring point space coupling dam data management method suitable for water conservancy long sequences is provided. The measuring point data of the type of single measuring point anomaly is removed from the preprocessed dam time sequence monitoring data, and then input into a data repair model to obtain the repaired dam time sequence monitoring data output by the data repair model. The data repair model is obtained based on a Bi-LSTM model and based on dam historical time sequence monitoring data.

[0013] The application further provides a dam data management system for long sequence multi-measuring point spatial coupling of water conservancy, comprising: The clustering module is configured to cluster the preprocessed dam time sequence monitoring data using an improved K-means clustering method, divide dam measuring points into multiple measuring point clusters according to a clustering result, and determine double coupling features of measuring points in each measuring point cluster, wherein the double coupling features include a static neighbor feature representing static value closeness of a measuring point to other measuring points in a cluster and a time sequence similarity feature matrix representing time sequence data change synchronization of the measuring point to the other measuring points in the cluster. The dividing module is configured to determine an effective neighbor set of each measuring point based on double coupling features of all measuring points in a cluster to which the measuring point belongs, and perform traversal calculation on all measuring points using a local outlier factor method with the effective neighbor set of the measuring point as a calculation reference, so as to divide normal measuring points and abnormal measuring points. The determining module is configured to compare and evaluate synchronization deviation values of an abnormal measuring point and its effective neighbors based on time sequence similarity features of the abnormal measuring point and the effective neighbors, determine a type of the abnormal measuring point as a multi-measuring point cooperative abnormality if the synchronization deviation value is greater than a preset synchronization threshold, and otherwise, determine the type as a single-measuring point abnormality. The repair module is configured to perform data repair on measuring point data of a type of single-measuring point abnormality.

[0014] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for dam data management of long sequence multi-measuring point spatial coupling of water conservancy when executing the program.

[0015] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the method for dam data management of long sequence multi-measuring point spatial coupling of water conservancy.

[0016] The application further provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the method for dam data management of long sequence multi-measuring point spatial coupling of water conservancy.

[0017] The application provides a dam data management method for adapting to long sequence multi-point spatial coupling of water conservancy, which forms an abnormal data identification and repair full link coupling with time and space relationship without manual connection through an improved K-means clustering method and a LOF method, specifically fuses static neighbors and time sequence similar characteristics through the improved K-means clustering method, and accurately mines point correlation clusters, so that sensor failure is avoided from being misjudged as a dam body problem in subsequent abnormal identification, and engineering decision risk is further reduced. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is one of the flowcharts of the dam data management method for adapting to long sequence multi-point spatial coupling of water conservancy provided by the application; Figure 2 is the second flowchart of the dam data management method for adapting to long sequence multi-point spatial coupling of water conservancy provided by the application; Figure 3 is a classification result schematic diagram of a point cluster in an example of the dam data management method for adapting to long sequence multi-point spatial coupling of water conservancy provided by the application; Figure 4 is a monitoring data schematic diagram of a point in point cluster 0 in an example of the dam data management method for adapting to long sequence multi-point spatial coupling of water conservancy provided by the application; Figure 5 is a monitoring data schematic diagram of a point in point cluster 1 in an example of the dam data management method for adapting to long sequence multi-point spatial coupling of water conservancy provided by the application; Figure 6 is a monitoring data schematic diagram of a point in point cluster 2 in an example of the dam data management method for adapting to long sequence multi-point spatial coupling of water conservancy provided by the application; Figure 7 is a double-coupling feature heat map of point cluster 0 in an example of the dam data management method for adapting to long sequence multi-point spatial coupling of water conservancy provided by the application; Figure 8 is a double-coupling feature heat map of point cluster 1 in an example of the dam data management method for adapting to long sequence multi-point spatial coupling of water conservancy provided by the application; Figure 9 is a double-coupling feature heat map of point cluster 2 in an example of the dam data management method for adapting to long sequence multi-point spatial coupling of water conservancy provided by the application; Figure 10 is an abnormal monitoring data schematic diagram of an abnormal measuring point in an example of a dam data management method for adapting to long sequence multi-measuring point spatial coupling of water conservancy provided by the application; Figure 11 is a data comparison schematic diagram of repaired abnormal monitoring data of an abnormal measuring point in an example of a dam data management method for adapting to long sequence multi-measuring point spatial coupling of water conservancy provided by the application; Figure 12 is a structural schematic diagram of a dam data management system for adapting to long sequence multi-measuring point spatial coupling of water conservancy provided by the application; Figure 13 is a structural schematic diagram of an electronic device provided by the application. DETAILED DESCRIPTION

[0020] To make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0021] The present application will be described below with reference to the drawings. Figures 1 to 11 The present application introduces a dam data management method for adapting to long sequence multi-measuring point spatial coupling of water conservancy, as shown in Figure 1 , which comprises the following steps. Step 101, using an improved K-means clustering method to cluster the preprocessed dam time series monitoring data, so as to divide the dam measuring points into multiple measuring point clusters according to the clustering results, and determine the double coupling characteristics of the measuring points in each measuring point cluster, wherein the double coupling characteristics include a static neighbor characteristic representing the static value closeness of the measuring point to the remaining measuring points in the cluster and a time series similarity characteristic matrix representing the time series data change synchronism of the measuring point to the remaining measuring points in the cluster. Considering that there is significant spatial coupling between monitoring measuring points of different sections and different positions of the dam, for example, the monitoring sequence change trend of the seepage pressure measuring points of the same section has synchronism due to the same hydrogeological condition, in the present application, the synchronism is used to determine whether the monitoring data abnormality is caused by dam structure problems.

[0022] Specifically, first, the time series monitoring data of each measuring point of the dam is obtained, and the time series seepage pressure monitoring data is obtained in this embodiment for illustration.

[0023] In order to eliminate the influence of the magnitude difference of monitoring values of different monitoring points on subsequent clustering and anomaly calculation, and ensure the consistency of the calculation data format, first, the original time series monitoring data of each monitoring point is collected to obtain the original dam time series monitoring data, the data dimension is "monitoring point number-time-original seepage pressure value", then the original time series monitoring data of each monitoring point is standardized, and the following formula is selected for standardization: ; In the formula, is the standardized monitoring value of the monitoring point i at time t , is the original monitoring value of the monitoring point i at time t , is the mean value of the original data of the monitoring point i , is the standard deviation of the original data of the monitoring point i .

[0024] After standardization, the mean value of the monitoring data of each monitoring point is unified to 0, and the standard deviation is unified to 1, so that the order of magnitude interference caused by the installation height and range difference of different monitoring points is effectively avoided.

[0025] The preprocessed dam time series monitoring data includes the preprocessed time series monitoring data of each monitoring point of the dam, which is used as the input of the improved K-means clustering module to obtain the division result of the monitoring point cluster.

[0026] Specifically, compared with the conventional K-means clustering method, the improved K-means clustering method in the embodiment can accurately identify the space-time relationship between monitoring points by constructing a graph structure of the monitoring points of the dam, and consider the spatial coupling relationship and time sequence synchronization between monitoring points when obtaining the initial centroid, so that the monitoring data of the monitoring points corresponding to the finally selected initial centroid has similar time sequence change and uniform spatial distribution.

[0027] In the application, the improved K-means clustering method is used to divide the monitoring points of the dam into multiple monitoring point clusters, and the double coupling features of each monitoring point are output simultaneously, including static neighbor feature and time sequence similarity feature.

[0028] Specifically, for each monitoring point, the overall numerical closeness of its monitoring sequence to other monitoring points in the cluster is calculated, for example, the Euclidean distance between the monitoring point and other monitoring points in the cluster is calculated as its static neighbor feature: ; In the formula, is the static neighbor feature value of the monitoring point i and other monitoring points j in the cluster, which represents thei In the t Real-time monitoring data, n It represents the time point; the smaller the value, the closer the static numerical distribution of the measurement points is.

[0029] Optionally, the Pearson correlation coefficient can be used to quantify the synchronicity of temporal changes between measurement points in order to obtain temporal similarity characteristics: ; In the formula, Let be the temporal similarity feature value between measurement point i and other measurement points j within the same cluster. Let i be the standardized sequence mean of measurement point i. Let j be the standardized sequence mean. The closer the result is to 1, the stronger the synchronization of the time series trends between the two measurement points.

[0030] In summary, during the clustering stage, this invention outputs the classification results of the measurement point clusters and the dual coupling features of each measurement point for subsequent anomaly identification. The static nearest neighbor features provide the range of associated measurement points for anomaly identification, while the temporal similarity features provide the coupling basis for anomaly judgment, thereby realizing the reuse of features and module collaboration within the framework.

[0031] Step 102: Based on the dual coupling characteristics of all measurement points in the cluster to which each measurement point belongs, determine the effective nearest neighbor set corresponding to each measurement point, and use the effective nearest neighbor set of the measurement point as the calculation benchmark to perform traversal calculation on all measurement points using the local outlier factor method, thereby dividing normal measurement points and abnormal measurement points. This invention also designs a dual-feature T-LOF anomaly identification module, which is used to determine the type of a measuring point based on the classification results of the measuring point cluster and the dual coupling features of each measuring point. The dual-feature T-LOF anomaly identification module improves upon the traditional Local Anomaly Factor (LOF) algorithm to address the characteristics of dam monitoring data, which are highly time-series and prone to sudden changes.

[0032] Specifically, considering that dam seepage pressure data is affected by rainfall and water level changes, exhibiting a "weekly fluctuation" characteristic, a sliding window with a size of 14 days and a step size of 1 day was first set before selecting effective nearest neighbors. This process divided the complete, standardized time-series data of all measuring points—that is, the pre-processed dam time-series monitoring data—into several consecutive event windows, denoted as... By using windowing, we can ensure the ability to capture short-term mutations and anomalies, while avoiding misjudgments caused by judging from a single time point.

[0033] Based on this, for each window Target measurement points inside i Based on its dual coupling feature, effective nearest neighbors are selected to obtain each target measurement point. i The corresponding effective nearest neighbor setN i .

[0034] The effective neighbor needs to meet the static neighbor condition and the timing similarity condition at the same time, ensuring that the neighbor measuring point and the target measuring point have strong correlation, thereby avoiding the judgment deviation caused by the random selection of neighbors in the traditional LOF algorithm.

[0035] On this basis, the effective neighbor set of the target measuring point currently calculated is taken as the calculation reference, and the local outlier factor method introducing the timing trend feature is used to calculate the local outlier value (T-LOF value) of the target measuring point.

[0036] In a specific embodiment, in order to fuse the influence of numerical difference and timing trend on abnormal judgment, the target measuring point i and the effective neighbor j The weighted distance in the window : ; In the formula, is the weighted distance of the measuring point i and j in the window , time is the number of time points in the window, =10 -6 (avoiding the small value of the denominator being 0), is used to standardize the numerical difference at different time points in the window, ensuring that the distance calculation is not disturbed by extreme values.

[0037] Then the local reachable density (LRD) is calculated, which is used to measure the isolation degree of the target measuring point in the window. The smaller the value, the more isolated the measuring point is, and the more likely it is an abnormal value. The calculation formula is as follows: ; In the formula, is the local reachable density of the measuring point i in the window , is the weighted distance of the effective neighbor j to its own k effective neighbor ( ), and the operation in the formula can avoid the density misjudgment caused by the too small distance of a single point, ensuring the stability of the calculation result.

[0038] Finally, the T-LOF value of the measuring point is calculated based on the LRD. In order to amplify the timing mutation characteristics (such as seepage pressure jump) of the dam data, the adjacent window density difference is introduced on the basis of the local density ratio of the traditional LOF algorithm, forming the T-LOF value, and the calculation formula is as follows: ; wherein, is the measuring point i The time series local anomaly factor in the window W_t, the larger the value, the higher the abnormality possibility; is the near neighbor j The local density ratio of the target i reflects the isolation degree of the relative near neighbor; the parameter i =0.5 is the time series trend sensitive coefficient (based on instance verification setting, balancing the influence of numerical anomaly and time series mutation); α is the measuring point The local reachable density in the previous window, i is the adjacent window density difference, which can effectively capture the mutation of the time series trend. After calculating the local outliers (T-LOF values) of all measuring points in the above manner, 3

[0039] The principle is used to determine the abnormality judgment threshold: manually select the normal measuring points in the time period without abnormal records in each measuring point cluster in the historical data, calculate the T-LOF values of these normal measuring points in 100 normal windows, and then determine the abnormality judgment threshold by taking the mean value and the standard deviation of the T-LOF values of the normal measuring points. .

[0040] If the T-LOF value of the measuring point is greater than TH , the measuring point is determined as an abnormal measuring point, otherwise, it is determined as a normal measuring point.

[0041] Step 103, based on the time series similarity feature comparison of the abnormal measuring point and its effective near neighbor, the synchronism deviation value of the abnormal measuring point and its corresponding effective near neighbor is evaluated, and the type of the abnormal measuring point whose synchronism deviation value is greater than the preset synchronization threshold is determined as a multi-measuring point cooperative anomaly, otherwise, it is determined as a single-measuring point anomaly; Further, for the obtained abnormal measuring point, the consistency of the time series similarity feature of the abnormal measuring point and its effective near neighbor is evaluated to determine whether the behavior of the abnormal measuring point deviates from the behavior of its effective near neighbor.

[0042] The synchronism deviation value represents the deviation degree of the time series similarity feature of the abnormal measuring point and the time series similarity feature of its effective near neighbor, the larger the synchronism deviation value, the more likely the current abnormal measuring point deviates from its effective near neighbor set, and the more likely the anomaly of this measuring point is caused by the anomaly of its effective near neighbor set, that is, the local leakage of the dam body and other structural problems lead to multi-measuring point anomaly.

[0043] The smaller the synchronization deviation value is, the higher the synchronization between the effective neighbors of the current abnormal measuring point is, and the more the abnormality of the current abnormal measuring point is caused by individual abnormality such as sensor failure, that is, the abnormal value of the current abnormal measuring point is caused by noise rather than structural problem of the dam body.

[0044] The preset synchronization threshold is determined according to historical data experience, the type of the abnormal measuring point with the synchronization deviation value greater than the preset synchronization threshold is marked as a multi-measuring-point cooperative abnormality, which represents that the dam structure needs to be checked, otherwise, it is marked as a single-measuring-point abnormality, and the data of the single-measuring-point abnormality needs to be repaired to obtain dam monitoring data with higher accuracy.

[0045] In step 104, the data of the measuring point of the type of single-measuring-point abnormality is repaired.

[0046] In a feasible implementation, first, the data of the measuring point marked as a single-measuring-point abnormality is deleted from the preprocessed dam time series monitoring data, and then the remaining data is input into a data repair model.

[0047] Optionally, the data repair model can be an existing machine learning model constructed based on a long short-term memory network, a gated recurrent unit, a convolutional neural network, and a Transformer, which predicts monitoring data in a future period from monitoring data in a historical period to realize data repair.

[0048] The improved K-means clustering method and the LOF method are used to form an abnormal data recognition and repair whole link coupled with space-time relationship, without manual connection, and the improved K-means clustering method is used to fuse static neighbors and time series similarity features to accurately mine measuring point correlation clusters, so that sensor failure is avoided to be misjudged as a dam problem in subsequent abnormality identification, and the risk of engineering decision is further reduced.

[0049] The improved K-means clustering method is used to cluster the preprocessed dam time series monitoring data, and the dam measuring points are divided into a plurality of measuring point clusters according to the clustering results, and the method is suitable for water conservancy long sequence multi-measuring-point space coupled dam data management method. A weighted undirected graph is constructed based on the preprocessed dam time series monitoring data, wherein each measuring point is used as a graph node, the preprocessed measuring point data of each measuring point is used as the feature of the graph node, a connection line is added between two graph nodes with spatial proximity or time series similarity as the edge between the graph nodes, and the edge weight value is determined. In view of the spatial coupling characteristics and time sequence synchronization of the dam multiple measuring points, the K-means clustering method in the embodiment optimizes the initial centroid selection, cluster assignment and centroid updating rule of the traditional K-means method three times to overcome the instability of the traditional algorithm caused by random selection of the centroid, so that the correlation characteristics of the measuring points cannot be fully utilized, and thus the classification of the abnormal measuring points cannot be realized through accurate and effective neighbors.

[0050] To this end, first, the graph topology structure of the measuring points is constructed.

[0051] For a graph structure data with n nodes and m edges, denoted as G= (V, E, W), wherein V is the initial node set, E is the initial edge set, and W is the edge weight set; A represents the adjacency matrix of G, and D represents the degree information diagonal matrix of G; each node in G has corresponding features, which are denoted as X (i), wherein i represents the node index, and X (i) represents the feature of the i-th node. is the feature dimension of the node in G.

[0052] On this basis, for the spatial attributes and time sequence monitoring data of the dam measuring points, a weighted undirected graph can be constructed in the above manner, each measuring point is taken as an independent measuring point to form a node in the graph, and the preprocessed time sequence monitoring data of each measuring point is taken as the feature of each node. If two nodes satisfy spatial proximity or time sequence similarity, it is considered that there is an edge between the two nodes.

[0053] Optionally, the edge weight value is determined according to the spatial proximity value and / or the time sequence similarity value, so that the edge weight value between two nodes with closer spatial distance / larger time sequence similarity is larger.

[0054] The weighted degree centrality of each node is calculated, and based on the weighted degree centrality and the spatial distribution uniformity, a preset number of candidate centroids with high weighted degree centrality and uniform spatial position distribution are screened out. Further, the initial centroid selection guided by the graph topology structure is realized. First, the weighted degree centrality of each node is calculated. In the formula, is the adjacent node of the node i , and the correlation strength of the node is quantified by calculating the weighted degree centrality.

[0055] On this basis, among the multiple nodes with high weighted degree centrality, multiple nodes with uniform spatial position distribution are selected as candidate centroids according to the spatial position distribution of the nodes.

[0056] ​​In the embodiment, based on engineering experience and actual verification, the preset number is determined as 3, that is, three candidate centroids are selected, the measuring points are divided into three clusters, and a more optimal classification result can be achieved, so that the correlation of the measuring points in each cluster is the strongest.

[0057] Based on the composite distance of the fusion feature distance, the graph topology distance and the time sequence distance, each measuring point is distributed to the cluster to which the initial centroid with the closest composite distance belongs, and the centroid is updated through iteration, so that the preset number of measuring point clusters is obtained based on the centroid obtained through iteration.

[0058] After the selection of the candidate centroid is completed, each measuring point is classified into the cluster to which the initial centroid with the closest distance belongs.

[0059] In the embodiment, the composite distance of the fusion feature distance, the graph topology distance and the time sequence distance is defined, and is used for classifying the measuring points: ; In the formula, and are weight values for balancing each distance factor. is the standardized feature Euclidean distance between the measuring point i and the candidate centroid . In the embodiment, based on engineering experience, the value of is set as 0.4, and the value of is set as 0.3.

[0060] ; In the formula, represents the time sequence distance, n is the time sequence length.

[0061] is the graph weighted shortest path distance between the measuring point i and the candidate centroid . The value can be the reciprocal of the edge weight value.

[0062] is the time sequence Euclidean distance between the measuring point i and the candidate centroid . First, the standard time sequence Euclidean distance is calculated, in which, T is the time sequence length, is the feature value of the centroid at the time t, and then the normalized processing is performed to obtain the final time sequence Euclidean distance: ; In the formula, is the maximum value of the original time sequence Euclidean distance between all measuring points and each centroid.

[0063] Repeat the cluster assignment-centroid update process based on the defined composite distance until the change of all centroids is less than a preset threshold: Or the number of iterations reaches 50 (the maximum number of iterations), stop calculation, finally get 3 associated measuring point clusters (recorded as label0, label1, label2).

[0064] In the dam data management method suitable for water conservancy long sequence multi-measuring point space coupling of the application, the step of constructing a weighted undirected graph based on the preprocessed dam time series monitoring data specifically comprises: Define each measuring point as a graph node, and the preprocessed measuring point data of each measuring point as the features of the graph node; If the Euclidean distance between any two measuring points is less than a first preset distance threshold, it is determined that the two measuring points are spatially adjacent; In this embodiment, the Euclidean distance between two measuring points is used as the determination condition of spatial proximity, and if the Euclidean distance between two measuring points is less than a first preset distance threshold, it is considered that the two measuring points are spatially adjacent, and a connection edge needs to be constructed between the two measuring points.

[0065] If the Pearson correlation coefficient of the time series monitoring data between any two measuring points is greater than a preset correlation threshold, it is determined that the two measuring points are time series similar; The Pearson correlation coefficient of the time series data between two measuring points is used as the determination condition of time series similarity, and if the Pearson correlation coefficient between the feature data of two measuring points is greater than 0.5, it is considered that the two measuring points are time series similar, and a connection edge needs to be constructed between the two measuring points.

[0066] A connection line is added between the two graph nodes that are spatially adjacent or time series similar as the edge between the graph nodes, and the edge weight value between the two measuring points is determined based on the Euclidean distance and the Pearson correlation coefficient, so that the greater the edge weight value, the stronger the spatio-temporal coupling between the two measuring points.

[0067] Further, in this embodiment, the edge weight value couples the Euclidean distance and the Pearson correlation coefficient of the two measuring points: ; In the formula, is a balance coefficient set based on engineering experience, is the spatial Euclidean distance between the two measuring points, is the spatial feature standard deviation, is the Pearson correlation coefficient of the time series data of the two measuring points. In general, the greater the edge weight, the stronger the spatio-temporal coupling between the two measuring points The application is adapted to a water conservancy long sequence multi-measuring point space coupling dam data management method, and a step of screening a preset number of candidate centroids with high weighted degree centrality and uniform spatial position distribution based on weighted degree centrality and spatial distribution uniformity, specifically comprising: arranging the weighted degree centrality in descending order to generate a candidate node list; fixing the node with the highest weighted degree centrality as the first candidate centroid, sequentially traversing the candidate node list, calculating the graph shortest path of the node and each selected candidate centroid, and fixing the node as a new candidate centroid in the case that the graph shortest distance between the node and each selected candidate centroid is not less than a second preset distance threshold; Otherwise, the node is adjusted and the next node in the candidate node list is continuously checked until a preset number of candidate centroids are screened.

[0068] The embodiment provides a candidate centroid screening method to screen a preset number of candidate centroids with high weighted degree centrality and uniform spatial distribution from the nodes with high weighted degree centrality.

[0069] The second preset distance threshold is half of the average graph distance in the cluster.

[0070] It can be understood that when the graph shortest distance between two nodes is less than the second preset distance threshold, the distance between the two nodes is relatively close, and the spatial distribution is not uniform.

[0071] On this basis, in order to screen candidate nodes with high weighted degree centrality and uniform spatial distribution, the weighted degree centrality is first arranged in descending order to generate a candidate node list, then the node with the highest weighted degree centrality is fixed as the first candidate centroid, and subsequent candidate centroids are sequentially screened according to the list order: For the node in the candidate node list, the graph shortest distance between the node and the fixed candidate centroid is calculated, and the node is fixed as a new candidate centroid in the case that the graph shortest distance between the node and each selected candidate centroid is not less than the second preset distance threshold.

[0072] The new candidate centroid is sequentially determined in the above manner until the number of selected candidate centroids reaches the preset number.

[0073] The application is adapted to a water conservancy long sequence multi-measuring point space coupling dam data management method, and a step of determining the effective neighbor set corresponding to each measuring point based on the double coupling features of all measuring points in the cluster to which each measuring point belongs, specifically comprising: In the cluster to which each measuring point belongs, the measuring point with a static neighbor feature less than a preset neighbor threshold and a time series similarity feature less than a preset experience threshold is determined as an effective neighbor of the measuring point, and an effective neighbor set of each measuring point is constructed; The preset proximity threshold is half of the mean value of the static proximity features of all the measuring points in the cluster to which the current measuring point belongs.

[0074] In the embodiment, the effective proximity of the measuring point is screened through the static proximity feature and the time sequence similarity feature of the measuring point.

[0075] First, the preset proximity threshold is determined In the embodiment, the preset proximity threshold is half of the mean value of the static proximity features of all the measuring points in the cluster to which the current measuring point belongs, and a plurality of measuring points in the cluster are found, the static proximity features of which are less than the preset proximity threshold (i.e. ).

[0076] On this basis, the second screening is performed according to the time sequence similarity feature, and the measuring points with the time sequence similarity feature greater than the preset empirical threshold are retained as the effective proximity of the current measuring point, wherein the preset empirical threshold makes the number of the effective proximity finally screened to be 1 / 3 of the total number of the measuring points in the cluster to which the current measuring point belongs, and in the embodiment, the preset empirical threshold is 0.6, that is, the second screening retains the measuring points with the time sequence similarity feature greater than 0.6.

[0077] The effective proximity screened is used to construct an effective proximity set N i .

[0078] In the dam data management method adapted to the water conservancy long sequence multi-measuring point space coupling, the synchronization deviation value C is calculated by the following formula: ; In the formula, represents the time sequence similarity feature between the measuring point and the effective proximity, i and the time sequence similarity feature mean value of the measuring point and all the effective proximities thereof. j i

[0079] The synchronization deviation value calculation method in the embodiment is shown in the above formula, based on the above calculation method and the actually collected historical monitoring data, the preset synchronization threshold is set to 0.15, and under this threshold, when C>0.15, it is indicated that more than 50% of the effective proximities in the measuring point cluster synchronously appear abnormal, and therefore the abnormal category of the corresponding measuring point i is determined as multi-measuring point cooperative abnormality, that is, the actual abnormal monitoring value may not be the measuring point i , but a structural problem such as local leakage of the dam body causes the cooperative abnormality of the multiple measuring points associated with i .

[0080] If C≤0.15, it is indicated that only the corresponding measuring point i is abnormal, and the measuring point​​​​i Multiple effective near-neighbor behaviors of the monitoring point i The anomaly of the monitoring point i The monitoring data of the monitoring point

[0081] In the above manner, the double-feature T-LOF anomaly identification method provided by the application outputs the abnormal value position (monitoring point-time) and the abnormal type (single monitoring point / multi-monitoring point cooperation), which are transmitted to the subsequent data repair module to provide the repair range and background information for the missing value repair.

[0082] The application is suitable for a dam data management method with long sequence and multi-monitoring point space coupling, and the step of repairing the monitoring point data of the type of single monitoring point anomaly, specifically comprises: The monitoring point data of the type of single monitoring point anomaly is removed from the preprocessed dam time series monitoring data, and then input into a data repair model to obtain the repaired dam time series monitoring data output by the data repair model; The data repair model is obtained based on the Bi-LSTM model and the dam historical time series monitoring data.

[0083] In this embodiment, the data repair model is constructed based on Bi-LSTM, which is used to repair the missing data, and it can be understood that the missing data is the dam time series monitoring data after removing the monitoring point data of the type of single monitoring point anomaly.

[0084] In view of the long time series and nonlinear characteristics of dam monitoring data, the missing data formed after the abnormal value is removed by the abnormal identification module is repaired to ensure the integrity and physical authenticity of the output data. The data repair model increases the backward LSTM layer based on the traditional LSTM to form a bidirectional learning structure (Bi-LSTM) to capture the historical trend and future trend of the data at the same time, which is more suitable for the time series continuity of dam seepage pressure data. The training process is as follows: Collect the dam historical monitoring data, and select the abnormal monitoring point according to the above steps, remove the values marked as abnormal to form the seepage pressure time series data containing missing values, then divide the data into training set and test set according to the ratio of 8:2, then use the sliding window to construct the model input sample, that is, use the seepage pressure values of the past s time points as the input to predict the seepage pressure value of the current time point, and s is the sequence length.

[0085] In view of the sequence characteristics of the long time series seepage pressure data of the dam, the model parameters of Bi-LSTM are optimized to balance the repair accuracy and calculation efficiency, and the core parameters are as follows: the number of hidden layer units is 128 (to avoid overfitting and underfitting), the sequence length is 30 (to cover short-term fluctuations and medium-term trends), the batch size is 32 (to improve training efficiency), and the learning rate is 2e -4 (To avoid gradient explosion or slow convergence), Dropout probability 0.3 (to prevent overfitting), training epochs 50 (to ensure the model converges sufficiently).

[0086] The Bi-LSTM model consists of three parts: a forward LSTM layer, a backward LSTM layer, and an output layer. The forward LSTM layer processes data from the beginning of the sequence (past) to the end (future), and outputs the time step. t The forward hidden state of is given by the following iteration formula: ; In the formula, for t Always in a forward-hidden state. Forward LSTM unit, for t- At time 1, the forward-hidden state is active. θ These are the parameters for the forward LSTM unit. The backward LSTM layer processes data from the end of the sequence (future) to the beginning (past), outputting the time step. t Backward hidden state: ; In the formula, for t Backward hidden state at any moment For backward LSTM units, for t +1 time backward hidden state, These are the parameters for the backward LSTM unit. Finally, the output layer concatenates the forward and backward hidden states and outputs the repaired value through a fully connected layer: ; In the formula, for t The value of the repair at any moment. For splicing operations, This is the output layer weight matrix. b This is the output layer bias term. During training, this module reuses the associated measurement point cluster data output by the clustering module. This means that the time-series trends of measurement points within the same cluster are synchronized, which helps the model learn the true changing patterns of dam seepage pressure data and improves repair accuracy. The module's output is seepage pressure time-series data without missing or anomalies, which is then passed to the data output module.

[0087] Furthermore, since the data repair model is used to repair the standardized monitoring data, the repaired data output by the data repair model also needs to be destandardized to obtain the final monitoring data, which will be convenient for subsequent dam safety assessment.

[0088] In this embodiment, the inverse normalization formula corresponds to the used normalization formula, which is: .

[0089] Subsequently, the data is arranged in the format of measurement point number-time-osmotic pressure value after cleaning, and each measurement point corresponds to a complete time series data.

[0090] On this basis, a complete collection, identification and repair process is as shown in Figure 2 .

[0091] In a specific example, for a large reservoir with a total storage capacity of 1.86 billion m 3 , the normal storage level is 79 m, the dead water level is 51 m, and the check flood level is 80.29 m.

[0092] There are 11 earth-rock dams in the reservoir area, and the monitoring items include deformation and seepage monitoring. Taking 25 seepage monitoring points of 1# dam as an example, the monitoring data from 2022 to 2025 is analyzed. In this period, there are abnormal values such as jump and burst phenomena in different measurement points.

[0093] Among them, the measurement point arrangement is shown in Table 1: Table 1

[0094] On this basis, the improved K-means clustering method provided by the application is used to cluster the above measurement points into three clusters, and the obtained measurement point clusters are shown in Table 2: Table 2

[0095] The visualization of the classification results of the measurement point clusters is shown in Figure 3 , it can be seen that the improved K-means clustering method provided by the application can better group the measurement points, and the measurement points in the group have obvious aggregation effect on the image.

[0096] Further, the normalized monitoring data sequences of the measurement points in the same cluster are plotted in the same graph to compare and verify the consistency of the change trend of the related measurement point groups and the measurement points. The results are shown in Figure 4 , Figure 5 and Figure 6 . (Since the monitoring data of measurement point up36 in label0 is incomplete, in order to facilitate subsequent model training, the data of all measurement points in label0 is truncated to 2022), it can be seen that the change trend of the monitoring sequence of each measurement point in the related measurement point group has obvious consistency; at the same time, some measurement points have obvious sharp jump, which indicates that there are abnormal values.

[0097] Figure 7 、 Figure 8 and Figure 9 are the heat maps of static proximity features and time series similarity features of the measuring points in the above three measuring point clusters, respectively.

[0098] The static proximity feature is constructed based on the "distance between measuring point sequences", and the deeper the color (or the smaller the value) in the heat map represents that the "overall numerical distribution of the monitoring sequences of the two measuring points is closer" (such as the average level of osmotic pressure value, the consistency of fluctuation range is strong), and vice versa. From the above figure, we can see that: 1. Clustering effectiveness. The static distance between most measuring points within the same cluster (such as up31 / up32 / up51 / up52 of label0, up12 / up13 / up62 / up63 of label2) is small (the color of the heat map is light or the value is low), which indicates that the grouping logic of the improved K-means clustering is reasonable, that is, the measuring points in the same cluster indeed have strong correlation in numerical level, which conforms to the engineering rule that the numerical distribution of measuring points with close spatial location and affected by common environmental factors (such as reservoir water level, geological conditions) is more consistent; 2. Accurate identification of local differences within the cluster. There are a few measuring points within the cluster that have a large static distance from other measuring points (such as up65 and up11 of label1), which reflects that these measuring points may deviate from the mainstream of the cluster due to local environmental differences (such as different anti-seepage structures, different sensor installation depths), providing a basis for "screening effective proximity" in subsequent anomaly detection; Then, the time series similarity feature is constructed based on the "Pearson correlation coefficient", and the deeper the color (or the larger the absolute value of the coefficient) in the heat map represents that the time series change synchronization of the monitoring sequences of the two measuring points is stronger (such as the rhythm of the osmotic pressure value rising / falling over time is consistent); the coefficient close to 1 is "positive synchronization", close to -1 is "negative synchronization", and close to 0 is "no correlation". From the above figure, we can see that: 1. Quantify the dynamic coupling within the cluster. The Pearson correlation coefficients between most measuring points within the same cluster are close to 1 (such as up12 and up13 of label2, up64 and up66), which indicates that the time series trends in the monitoring sequences of these measuring points are highly synchronized, for example, when the reservoir water level rises, the osmotic pressure values of the measuring points in the cluster all rise synchronously, which confirms the physical mechanism that spatially related measuring points are affected by common driving factors (such as reservoir water level fluctuations) and have consistent time series responses, which is also a key basis for the T-LOF algorithm module in the subsequent part of the paper to judge anomalies based on the synchronization of the proximity time series; 2. Distinguish between normal differences and potential anomalies: the correlation coefficients between a few measuring points within the cluster are low (such as up35 and up55 of label0), which, combined with the engineering background (such as measuring points belonging to different anti-seepage layers of the same section), belongs to normal local differences; on the contrary, if a measuring point originally has strong synchronization with its proximity, and suddenly the correlation coefficient drops sharply, it may be an anomaly signal (such as sensor failure leading to trend deviation), which provides a reference for subsequent anomaly type judgment (single measuring point anomaly / multi-measuring point collaborative anomaly).

[0099] It can be seen that the collaborative analysis of static neighborhood features (numerical consistency) and time series similarity features (trend synchronization) can avoid the limitations of single features and obtain more comprehensive coupling conclusions.

[0100] Further, taking the label0 measurement point group as an example, UP65 has multiple sharp mutation points (such as 2023-9-9, the value is 59.01m; 2022-8-12, the value is 56.11m; 2024-5-25, the value is 57.58m, etc.), as shown in the figure, it can be seen from the figure that the time points of the LOF values of up65 and up56 exceeding the threshold value are consistent with the time points of the actual monitoring sequence appearing jump, and the time of the abnormality is different. It is proved that the measurement point does not exist multiple measurement point collaborative abnormality, and it is a single measurement point abnormality, that is, it is generally caused by a single sensor or the environmental factors near the sensor. Figure 10

[0101] The repair result diagram obtained after the data repair model of the application is used to repair the data is as shown in Figure 11 The repair effect is evaluated by comparing the distribution of the original data after removing the abnormal values with the distribution of the data repaired by the two repair methods. Kullback-Leibler (KL) divergence is taken as an index to measure the difference between the two distributions. The smaller the KL divergence value is, the more similar the two distributions are. The final calculation result shows that the KL divergence calculation result of the method proposed in the application is 1.624633, and the KL divergence calculation result of the method of directly performing linear interpolation repair is 1.813083, which means that the data distribution filled by the algorithm framework proposed in the application is more similar to the original data distribution after removing the abnormal values, that is, in the repair of up65 data, the algorithm framework proposed in the application has a relatively better effect.

[0102] The adaptive multi-measurement point spatial coupling dam data management system for long hydrological sequences provided by the application is described below, and the adaptive multi-measurement point spatial coupling dam data management system described below can be correspondingly referred to the adaptive multi-measurement point spatial coupling dam data management method described above.

[0103] As shown in Figure 12 The adaptive multi-measurement point spatial coupling dam data management system for long hydrological sequences provided by the application includes a clustering module 1201, a division module 1202, a determination module 1203 and a repair module 1204. ​The clustering module 1201 is configured to use the improved K-means clustering method to cluster the preprocessed dam time series monitoring data, so as to divide the dam measuring points into a plurality of measuring point clusters according to the clustering result, and determine the double coupling characteristics of the measuring points in each measuring point cluster, wherein the double coupling characteristics include a static neighbor characteristic representing the static numerical closeness of the measuring point to the remaining measuring points in the cluster and a time series similarity characteristic matrix representing the time series data change synchronization of the measuring point to the remaining measuring points in the cluster. Considering that there is significant spatial coupling between monitoring measuring points of different sections and different positions of the dam, for example, the monitoring sequence change trend of the seepage pressure measuring points of the same section has synchronization due to the influence of the same hydrogeological condition, in the present application, whether the monitoring data anomaly is caused by the dam body structure problem is judged through the synchronization.

[0104] Specifically, first, the time series monitoring data of each measuring point of the dam is acquired, and in this embodiment, the time series seepage pressure monitoring data is acquired for illustration.

[0105] In order to eliminate the magnitude difference of different measuring points and ensure the consistency of the calculation data format, first, the original time series monitoring data of each measuring point is collected to obtain the original dam time series monitoring data, and the data dimension is “measuring point number-time-original seepage pressure value”, and then the original time series monitoring data of each measuring point is standardized.

[0106] After the standardization processing, the mean value of the monitoring data of each measuring point is unified to 0 and the standard deviation is unified to 1, so that the magnitude interference caused by the installation elevation and range difference of different measuring points is effectively avoided.

[0107] The preprocessed dam time series monitoring data includes the preprocessed time series monitoring data of each measuring point of the dam, which is used as the input of the improved K-means clustering module to obtain the division result of the measuring point cluster.

[0108] Specifically, compared with the conventional K-means clustering method, the improved K-means clustering method in this embodiment can accurately identify the spatial and temporal relationship between measuring points by constructing a graph structure of the dam measuring points, and consider the spatial coupling relationship and time series synchronization when obtaining the initial centroid, so that the monitoring data of the measuring points corresponding to the finally selected initial centroid has similar time series change and uniform spatial distribution.

[0109] In the present application, the improved K-means clustering method is used to divide the measuring points of the dam into a plurality of measuring point clusters, and the double coupling characteristics of each measuring point are output at the same time, including the static neighbor characteristic and the time series similarity characteristic.

[0110] Specifically, for each measuring point, the overall numerical closeness of the monitoring sequence of the measuring point to the other measuring points in the cluster is calculated, for example, the Euclidean distance between the measuring point and the other measuring points in the cluster is calculated as the static neighbor characteristic: ; In the formula, is the measuring point i and other measuring points in the cluster to which the measuring point belongs j , represents the static neighbor feature value of the measuring point i at the first t moment, n represents the number of moments, and the smaller the value, the closer the static value distribution of the measuring point is.

[0111] Optionally, the Pearson correlation coefficient is used to quantify the synchronization of the time series change between the measuring points to obtain a time series similarity feature: ; In the formula, is the time series similarity feature value of the measuring point i and other measuring points j in the cluster to which the measuring point i belongs, is the normalized sequence mean of the measuring point i, is the normalized sequence mean of the measuring point j, The closer to 1, the stronger the synchronization of the time series trend of the two measuring points.

[0112] In general, in the clustering stage, the invention outputs the classification result of the measuring point cluster and the double-coupling feature of each measuring point, which is used for subsequent anomaly identification. The static neighbor feature provides a range of associated measuring points for anomaly identification, and the time series similarity feature provides a coupling basis for anomaly judgment, thereby realizing the reuse of features in the framework and the cooperation of modules.

[0113] The division module 1202 is used for determining the effective neighbor set corresponding to each measuring point based on the double-coupling feature of all measuring points in the cluster to which each measuring point belongs, and using the effective neighbor set of the measuring point as a calculation reference to perform traversal calculation on all measuring points by using the local outlier factor method, so as to divide the normal measuring points and the abnormal measuring points; The invention also designs a double-feature T-LOF anomaly identification module for determining the type of the measuring point based on the classification result of the measuring point cluster and the double-coupling feature of each measuring point. The double-feature T-LOF anomaly identification module is improved based on the traditional local anomaly factor (LOF) algorithm in view of the strong time series and the easy mutation of the dam monitoring data.

[0114] Specifically, considering that the dam seepage pressure data is affected by rainfall and water level changes, there is a "weekly fluctuation" characteristic. Before screening the effective neighbors, a sliding window with a size of 14 days and a step of 1 day is set, and the complete standardized time series data of all measuring points, that is, the preprocessed dam time series monitoring data, is divided into a plurality of continuous event windows, denoted as The windowing processing ensures the ability to capture short-term mutation anomalies and avoids misjudgment caused by single time point judgment.

[0115] On this basis, the target measurement points in each window are screened based on their double coupling characteristics to obtain the corresponding effective neighbor set of each target measurement point i . i N i .

[0116] The effective neighbor needs to meet both the static neighbor condition and the time sequence similarity condition, so as to ensure that the neighbor measurement point and the target measurement point have strong correlation, thereby avoiding the judgment deviation caused by the random selection of neighbors in the traditional LOF algorithm.

[0117] On this basis, the local outlier value (T-LOF value) of the target measurement point is calculated by using the local outlier factor method with the introduction of the time sequence trend characteristics, taking the effective neighbor set of the target measurement point as the calculation reference.

[0118] After the local outlier values (T-LOF values) of all measurement points are calculated in the above manner, the abnormality determination threshold is determined according to the following three principles: the normal measurement points in each measurement point cluster without abnormal record periods are selected, the T-LOF values of these normal measurement points in 100 normal windows are calculated, and then the mean value and the standard deviation of the T-LOF values of the normal measurement points are determined to determine the abnormality determination threshold .

[0119] If the T-LOF value of a measurement point is greater than TH , the measurement point is determined to be an abnormal measurement point, otherwise, it is determined to be a normal measurement point.

[0120] The determination module 1203 is configured to compare and evaluate the synchronization deviation value of the abnormal measurement point and its effective neighbor based on the time sequence similarity features of the abnormal measurement point and its effective neighbor, and determine the type of the abnormal measurement point as a multi-measurement-point cooperative anomaly if the synchronization deviation value is greater than a preset synchronization threshold, otherwise, as a single-measurement-point anomaly. Further, for the obtained abnormal measurement point, the application determines whether the behavior of the abnormal measurement point deviates from the behavior of its effective neighbor by evaluating the consistency of the time sequence similarity features of the abnormal measurement point and its effective neighbor.

[0121] The synchronization deviation value represents the deviation degree of the time sequence similarity features of the abnormal measurement point and its effective neighbor, and the greater the synchronization deviation value, the more likely it is that the current abnormal measurement point deviates from its effective neighbor set, and the more likely it is that the anomaly of this measurement point is caused by the anomaly of its effective neighbor set, that is, the multi-measurement-point anomaly caused by the local leakage of the dam body and other structural problems.

[0122] ​The smaller the synchronization deviation value is, the higher the synchronization between the effective neighbors of the current abnormal measuring point is, and the more the abnormality of the current abnormal measuring point is caused by individual abnormality such as sensor failure, that is, the abnormal value of the current abnormal measuring point is caused by noise rather than structural problem of the dam body.

[0123] The preset synchronization threshold is determined according to historical data experience, the type of the abnormal measuring point with the synchronization deviation value greater than the preset synchronization threshold is marked as multi-measuring-point cooperative abnormality, which represents that the dam body structure needs to be checked, otherwise, it is marked as single-measuring-point abnormality, and the data of the single-measuring-point abnormality needs to be repaired to obtain dam monitoring data with higher accuracy.

[0124] The repair module 1204 is configured to repair the data of the measuring point with the type of single-measuring-point abnormality.

[0125] In a feasible implementation, the data of the measuring point marked as single-measuring-point abnormality is first deleted from the preprocessed dam time-series monitoring data, and then the remaining data is input into the data repair model.

[0126] Optionally, the data repair model can be an existing machine learning model constructed based on long short-term memory network, gated recurrent unit, convolutional neural network and Transformer, which predicts the monitoring data of a future period through the monitoring data of a historical period to realize data repair.

[0127] The improved K-means clustering method and the LOF method are used to form an abnormal data recognition and repair full link coupled with space-time relationship, without manual connection, the improved K-means clustering method is used to fuse static neighbors and time-series similarity features to accurately mine measuring point correlation clusters, so that the sensor failure is avoided to be misjudged as dam problem in subsequent abnormality identification, and the engineering decision risk is further reduced.

[0128] Figure 13 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 13As shown, the electronic device can include a processor 1310, a communications interface 1320, a memory 1330, and a communications bus 1340, wherein the processor 1310, the communications interface 1320, and the memory 1330 complete mutual communication through the communications bus 1340. The processor 1310 can invoke a logical instruction in the memory 1330 to execute a multi-measurement-point spatial coupling dam data management method adapted to a long sequence of water conservancy, which includes: using an improved K-means clustering method to cluster preprocessed dam time series monitoring data, to divide dam measurement points into multiple measurement point clusters according to the clustering results, and determine the double coupling characteristics of the measurement points in each measurement point cluster, the double coupling characteristics including a static neighbor feature representing the static numerical closeness of the measurement point to the remaining measurement points in the cluster and a time series similarity feature matrix representing the time series data change synchronization of the measurement point to the remaining measurement points in the cluster; based on the double coupling characteristics of all measurement points in the cluster to which each measurement point belongs, determine the effective neighbor set corresponding to each measurement point, and take the effective neighbor set of the measurement point as the calculation reference, and use the local outlier factor method to calculate all measurement points, thereby dividing normal measurement points and abnormal measurement points; based on the time series similarity feature comparison of the abnormal measurement point and its effective neighbor, evaluate the synchronization deviation value of the abnormal measurement point and its effective neighbor, and determine the type of the abnormal measurement point as a multi-measurement-point collaborative anomaly if the synchronization deviation value is greater than a preset synchronization threshold, otherwise, determine it as a single-measurement-point anomaly; data repair is performed on the measurement point data of the type single-measurement-point anomaly.

[0129] In addition, the logical instructions in the memory 1330 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0130] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program can be executed by a processor to enable a computer to perform the method for processing multi-point spatially coupled dam data of long hydrological sequence, which comprises: using an improved K-means clustering method to cluster the preprocessed dam time series monitoring data, so as to divide the dam monitoring points into multiple monitoring point clusters according to the clustering results, and determine the double coupling characteristics of the monitoring points in each monitoring point cluster, wherein the double coupling characteristics comprise a static neighbor characteristic representing the static value closeness of the monitoring point to the remaining monitoring points in the cluster and a time series similarity feature matrix representing the time series data change synchronization of the monitoring point to the remaining monitoring points in the cluster; determining the effective neighbor set of each monitoring point based on the double coupling characteristics of all monitoring points in the cluster to which the monitoring point belongs, and taking the effective neighbor set of the monitoring point as the calculation reference, and using the local outlier factor method to perform traversal calculation on all monitoring points, so as to divide the normal monitoring points and the abnormal monitoring points; comparing and evaluating the synchronization deviation of the abnormal monitoring point and its effective neighbor based on the time series similarity characteristics of the abnormal monitoring point and its effective neighbor, and determining the type of the abnormal monitoring point as a multi-point collaborative anomaly if the synchronization deviation is greater than a preset synchronization threshold, otherwise, determining the type of the abnormal monitoring point as a single-point anomaly; and performing data repair on the monitoring point data of the type of single-point anomaly.

[0131] In another aspect, the present application also provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program can be executed by a processor to implement the method for processing multi-point spatially coupled dam data of long hydrological sequence, which comprises: using an improved K-means clustering method to cluster the preprocessed dam time series monitoring data, so as to divide the dam monitoring points into multiple monitoring point clusters according to the clustering results, and determine the double coupling characteristics of the monitoring points in each monitoring point cluster, wherein the double coupling characteristics comprise a static neighbor characteristic representing the static value closeness of the monitoring point to the remaining monitoring points in the cluster and a time series similarity feature matrix representing the time series data change synchronization of the monitoring point to the remaining monitoring points in the cluster; determining the effective neighbor set of each monitoring point based on the double coupling characteristics of all monitoring points in the cluster to which the monitoring point belongs, and taking the effective neighbor set of the monitoring point as the calculation reference, and using the local outlier factor method to perform traversal calculation on all monitoring points, so as to divide the normal monitoring points and the abnormal monitoring points; comparing and evaluating the synchronization deviation of the abnormal monitoring point and its effective neighbor based on the time series similarity characteristics of the abnormal monitoring point and its effective neighbor, and determining the type of the abnormal monitoring point as a multi-point collaborative anomaly if the synchronization deviation is greater than a preset synchronization threshold, otherwise, determining the type of the abnormal monitoring point as a single-point anomaly; and performing data repair on the monitoring point data of the type of single-point anomaly.

[0132] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0133] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0134] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for managing dam data with multiple spatially coupled measurement points adapted to long hydraulic data sequences, characterized in that, include: An improved K-means clustering method was used to cluster the preprocessed dam time-series monitoring data to divide the dam monitoring points into multiple monitoring point clusters based on the clustering results. The dual coupling features of the monitoring points within each cluster were determined. The dual coupling features include static nearest neighbor features that characterize the static numerical closeness between the monitoring point and other monitoring points in the cluster, and a time-series similarity feature matrix that characterizes the synchronicity of the time-series data changes between the monitoring point and other monitoring points in the cluster. Based on the dual coupling characteristics of all measurement points within the cluster to which each measurement point belongs, the effective nearest neighbor set corresponding to each measurement point is determined. Using the effective nearest neighbor set of the measurement point as the calculation benchmark, the local outlier factor method is used to traverse and calculate all measurement points, thereby dividing normal measurement points into abnormal measurement points. Based on the temporal similarity features of abnormal measurement points and their effective neighbors, the synchronization deviation value between the abnormal measurement points and their corresponding effective neighbors is evaluated. Abnormal measurement points with synchronization deviation values ​​greater than the preset synchronization threshold are identified as multi-measurement point collaborative abnormalities; otherwise, they are identified as single-measurement point abnormalities. Perform data repair on measurement point data of type single measurement point anomaly.

2. The method for managing multi-measuring-point spatially coupled dam data adapted to long hydraulic sequences according to claim 1, characterized in that, The step of using an improved K-means clustering method to cluster the preprocessed dam time-series monitoring data, and dividing the dam monitoring points into multiple clusters based on the clustering results, specifically includes: A weighted undirected graph is constructed based on the preprocessed dam time-series monitoring data. Each measuring point is used as a graph node, and the preprocessed measuring point data of each measuring point is used as a feature of the graph node. A connection is added between two graph nodes that are spatially adjacent or temporally similar as an edge between the graph nodes, and the edge weight value is determined. Calculate the weighted degree centrality of each node, and based on the weighted degree centrality and spatial distribution uniformity, select a preset number of candidate centroids with high weighted degree centrality and uniform spatial distribution. Based on the composite distance of fusion feature distance, graph topology distance and temporal distance, each measurement point is assigned to the cluster of the initial centroid that is closest to its composite distance. The centroid is updated iteratively to obtain a preset number of measurement point clusters based on the centroid obtained by the iteration.

3. The method for managing multi-measuring-point spatially coupled dam data adapted to long hydraulic sequences according to claim 2, characterized in that, The step of constructing a weighted undirected graph based on the preprocessed dam time-series monitoring data specifically includes: Each measurement point is defined as a graph node, and the preprocessed measurement point data of each measurement point is used as the feature of the graph node. If the Euclidean distance between any two measuring points is less than the first preset distance threshold, then it is determined that the two measuring points are spatially adjacent. If the Pearson correlation coefficient of the time-series monitoring data between any two measurement points is greater than the preset correlation threshold, then it is determined that the two measurement points have temporal similarity. Add a line to two graph nodes that are spatially adjacent or temporally similar as an edge between the graph nodes, and determine the edge weight value based on the Euclidean distance and Pearson correlation coefficient between the two measurement points, so that the larger the edge weight value, the stronger the spatiotemporal coupling between the two measurement points.

4. The method for managing multi-measuring-point spatially coupled dam data adapted to long hydraulic engineering sequences according to claim 2, characterized in that, The step of selecting a predetermined number of candidate centroids with high weighted degree centrality and uniform spatial distribution based on weighted degree centrality and spatial distribution uniformity specifically includes: Sort the nodes by weighted degree centrality in descending order to generate a list of candidate nodes; The node with the highest weighted degree centrality is fixed as the first candidate centroid. The node is traversed sequentially from the list of candidate nodes, and the shortest path between the node and each selected candidate centroid is calculated. If the shortest path between the node and each selected candidate centroid is not less than the second preset distance threshold, the node is fixed as the new candidate centroid. Otherwise, skip the node and continue checking the next node in the candidate node list until a preset number of candidate centroids are obtained.

5. The method for managing multi-measuring-point spatially coupled dam data adapted to long hydraulic sequences according to claim 1, characterized in that, The step of determining the effective nearest neighbor set for each measuring point based on the dual coupling characteristics of all measuring points within the cluster to which each measuring point belongs specifically includes: Within the cluster to which each measurement point belongs, measurement points whose static nearest neighbor features are less than a preset nearest neighbor threshold and whose temporal similarity features are less than a preset empirical threshold are identified as their effective nearest neighbors, and an effective nearest neighbor set for each measurement point is constructed. The preset nearest neighbor threshold is half of the average value of all static nearest neighbor features within the cluster of measuring points to which the current measuring point belongs.

6. The method for managing multi-measuring-point spatially coupled dam data adapted to long hydraulic engineering sequences according to claim 1, characterized in that, The synchronization deviation value C The following formula is used for calculation: ； In the formula, Denotes the effective nearest neighbor set, which is the measurement point. i With effective neighbors j Temporal similarity features between them For measuring points i The mean of temporal similarity features with all its effective nearest neighbors.

7. The method for managing multi-measuring-point spatially coupled dam data adapted to long hydraulic engineering sequences according to claim 1, characterized in that, The steps for data repair of measurement point data of type single measurement point anomaly specifically include: In the preprocessed dam time-series monitoring data, the monitoring point data of type single monitoring point anomaly are removed, and then input into the data repair model to obtain the repaired dam time-series monitoring data output by the data repair model; The data repair model is based on the Bi-LSTM model and trained using historical time-series monitoring data of the dam.

8. A multi-measuring-point spatially coupled dam data management system adapted to long hydraulic data sequences, characterized in that, include: The clustering module is used to cluster the preprocessed dam time-series monitoring data using an improved K-means clustering method. Based on the clustering results, the dam monitoring points are divided into multiple monitoring point clusters, and the dual coupling features of the monitoring points within each cluster are determined. The dual coupling features include static nearest neighbor features that characterize the static numerical closeness between the monitoring point and other monitoring points in the cluster, and a time-series similarity feature matrix that characterizes the synchronicity of the time-series data changes between the monitoring point and other monitoring points in the cluster. The partitioning module is used to determine the effective nearest neighbor set for each measuring point based on the dual coupling characteristics of all measuring points in the cluster to which each measuring point belongs. Then, using the effective nearest neighbor set of the measuring point as the calculation benchmark, the local outlier factor method is used to traverse and calculate all measuring points to divide them into normal measuring points and abnormal measuring points. The determination module is used to evaluate the synchronization deviation value between the abnormal measurement point and its effective nearest neighbor based on the temporal similarity features. The abnormal measurement point with a synchronization deviation value greater than the preset synchronization threshold is determined as a multi-measurement point collaborative anomaly; otherwise, it is determined as a single measurement point anomaly. The repair module is used to repair data from measurement points that are classified as single-point anomalies.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the multi-measuring-point spatially coupled dam data governance method adapted to long hydraulic sequences as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the multi-measuring-point spatially coupled dam data management method adapted to long hydraulic sequences as described in any one of claims 1 to 7.

Images