Water conservancy project monitoring data early warning method and system
By using clustering algorithms and multimodal fusion early warning models, combined with physical sensors and video surveillance data, the problem of frequent sensor anomalies in water conservancy projects was solved, and efficient risk warning was achieved.
Patent Information
- Application Number
- CN202510824128.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-19
- Publication Date
- 2025-09-26
AI Technical Summary
In modern water conservancy projects, sensor data anomalies occur frequently and the processing workload is large, resulting in low efficiency in monitoring data processing and difficulty in achieving accurate and real-time risk warnings.
A clustering algorithm is used to extract marginalized time series data, which is combined with video surveillance visual data. The comprehensive safety risk index is calculated through a multimodal fusion early warning model to perform dynamic graded early warning.
It improves the efficiency of monitoring data processing, reduces repetitive work in irrelevant time periods, and realizes accurate and real-time risk warning for water conservancy projects.
Smart Images
Figure CN120708028A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of water conservancy project early warning, and in particular to a water conservancy project monitoring data early warning method and system. Background Art
[0002] Modern water conservancy projects face daunting challenges, including multi-source heterogeneous monitoring data, complex coupling of risk factors, and highly nonlinear disaster evolution. Accurately and in real time identifying subtle risk indicators amidst massive amounts of noisy monitoring data has become a lifeline for ensuring long-term project safety and preventing catastrophic accidents.
[0003] Traditional monitoring relies too much on point sensors buried in key locations, and combines them with real-time monitoring data for synchronous data comparison. For example, if the dam stress changes, the water level change needs to be captured through monitoring data, and this change can positively feedback the dam stress. At this time, it shows that the sensor value monitoring of the dam stress is accurate. In the actual processing process, in order to ensure the monitoring effect, when sensor data anomalies occur, it is often necessary to disassemble the monitoring data frame by frame and feedback the sensor abnormal data based on the change status of adjacent frames. However, after the sensor data anomaly occurs, its abnormal state lasts for a short time, and the amount of monitored data is relatively large. In addition, sensor data will inevitably have problems such as missing, drifting, and abnormal interference in long-term operation, resulting in frequent occurrence of abnormal data, which increases the workload of monitoring data processing and is not conducive to subsequent early warning work.
[0004] In order to address the above problems, there is an urgent need for a water conservancy project monitoring data early warning method that can perform adaptive monitoring and extraction. Summary of the Invention
[0005] The purpose of the present invention is to provide a water conservancy project monitoring data early warning method and system to solve the problems raised in the above background technology.
[0006] To achieve the above objectives, one of the objectives of the present invention is to provide a water conservancy project monitoring data early warning method, comprising the following steps: S1. Obtain multi-dimensional heterogeneous monitoring data for dams and water conservancy hubs, where the multi-dimensional heterogeneous monitoring data includes the following data: Physical sensor time series data: including seepage pressure, deformation displacement, and stress and strain; Video surveillance visual data: real-time video data of key areas; S2. Simulate the timing diagram for the physical sensor timing data, use the clustering algorithm to extract the marginalized timing data, and locate the key timing range; S3. According to the key timing range, formulate the key frame extraction range of the video surveillance visual data, calculate the time interval between each key timing range, and formulate the unit interval of key frame extraction; S4. Obtain the marginalized time series data in the key time series range and the characteristic values of the corresponding key frames, perform state trend mapping, and obtain the mapping results.
[0007] S5. Input the mapping results into the pre-trained multimodal fusion early warning model to perform calculations to obtain a comprehensive safety risk index that characterizes the current state of the project; S6. Determine the warning level based on the comparison between the comprehensive security risk index and the preset dynamic graded warning threshold.
[0008] As a further improvement of this technical solution, the method for extracting marginalized time series data in S2 includes the following steps: S2.1. Obtain the monitoring values corresponding to different time points based on the time sequence diagram and mark the coordinates; S2.2. Summarize the monitoring values corresponding to each time point within the monitoring period; S2.3. Select cluster center values for the aggregated monitoring values and establish clusters based on the selected cluster center values; S2.4. Calculate the distance between the remaining monitoring values and the center value of each cluster, and assign the monitoring values to the cluster with the closest distance, completing the first clustering work; S2.5. Calculate the average value for each cluster and use the average value as the cluster center value for the next round of clustering. Calculate the distance between each monitored value and the corresponding cluster center value and assign each monitored value to the cluster with the closest distance, completing the second round of clustering. S2.6. Repeat the above steps until the number of monitoring values in each cluster does not change.
[0009] As a further improvement of the present technical solution, the method for locating the critical timing range in S2 includes the following steps: S2.7. Establish monitoring thresholds; S2.8. Compare the number of monitoring values in each cluster; The number of clusters marked as not less than the monitoring threshold is normal; Extract the number clusters below the monitoring threshold and mark them as irregular number clusters; S2.9. Sort the monitoring values of the unconventional number clusters, obtain the time periods with the same trend changes based on the sorting results, and mark the time periods with the same trend changes as the key time series range.
[0010] As a further improvement of the present technical solution, the method for obtaining the time period of the same trend change according to the sorting results in S2.9 includes the following steps: S2.9.1. Define the range of normal variation; S2.9.2. Determine the range of variation in unconventional number clusters; When the monitoring value changes in a cluster are within the normal range, it is marked as a normal cluster; When the monitoring value changes in the cluster are not within the regular change range, the irregular cluster state is maintained.
[0011] As a further improvement of the present technical solution, the method for determining the key frame extraction range of the video surveillance visual data in S3 includes the following steps: S3.1. Aggregate and process each unconventional number cluster and plan the processing order according to the number of monitoring values in the unconventional number cluster; S3.2. Process a single unconventional number cluster according to the planned processing sequence and extract the key time series range in the unconventional number cluster; S3.3. Obtain the starting change time point and the ending change time point of the key timing range, and introduce the starting change time point and the ending change time point divided in each key timing range into the video stream in the video surveillance visual data, and divide the staged video stream according to the starting change time point and the ending change time point as the key frame extraction range.
[0012] As a further improvement of the present technical solution, the method for formulating the unit interval amount for key frame extraction in S3 includes the following steps: S3.4, calculating the time intervals of adjacent time points within the extraction range of each key frame; S3.5. Obtain each time interval and compare them, and use the smallest time interval as the interval time between adjacent key frames in each key frame extraction range.
[0013] As a further improvement of the present technical solution, the method for performing state trend mapping in S4 includes the following steps: S4.1. Extracting one frame of image from the staged video stream according to the interval time between the divided adjacent key frames; S4.2. Feed the image into a lightweight YOLOv5 model, which is trained to identify visual feature vectors in the image. S4.3. Calculate the visual feature vectors of adjacent key frames in sequence, obtain the change trend of each visual feature vector in the current stage-by-stage video stream, and compare them with the physical sensor timing data in the key timing range; If the state trends of the two are the same, it indicates that the mapping relationship between the two is a positive mapping. At this time, the physical sensor time series data is the actual monitored exogenous abnormal data; If the status trends of the two are the same, it indicates that the physical sensor time series data at this time is the actual monitored endogenous abnormal data.
[0014] As a further improvement of this technical solution, the method for obtaining the comprehensive safety risk index representing the current state of the project in S5 includes the following steps: S5.1. Feed the time series data into a pre-trained LSTM network to capture the long-term dependencies in the data and output a hidden state vector H_lstm that incorporates historical change trends. S5.2. Concatenate the LSTM output H_lstm with the visual feature vector at the current moment to form a more comprehensive fused feature vector V_fused; S5.3. Use V_fused as input to import a trained XGBoost model and give the importance ranking of features. S5.4. Output a continuous value between 0 and 2, representing the comprehensive safety risk index.
[0015] The second object of the present invention is to provide a system for implementing a water conservancy project monitoring data early warning method, comprising an application layer, a platform layer, a network layer, and a perception layer; The perception layer collects physical sensor time series data from the dam collection area through configured osmometers and displacement meters, and uses drones equipped with high-definition cameras to capture visual features of different areas using remote sensing satellite navigation; The network layer is used to build a network channel between the perception layer and the platform layer, such as 5G, Internet of Things, fiber optic network and satellite communication, and use the built network channel to feedback the physical sensor time series data and video surveillance visual data of the monitoring area in real time; The platform layer is used to process physical sensor time series data and video surveillance visual data, build models, and use algorithms for data calculation and data evolution; The application layer is used to make early warning and prompt decisions based on the data processing results, and to visualize the decisions, using visualization tools such as mobile apps and visual platforms to display the results, reminding monitoring personnel to handle them in a timely manner.
[0016] Compared with the prior art, the present invention has the following beneficial effects: In the water conservancy project monitoring data early warning method and system, marginalized time series data are extracted through a clustering algorithm, and a key time series range is located. The located key time series range is used to locate the key frame extraction range and the interval time of adjacent key frames in the video surveillance visual data. The staged video stream associated with the marginalized time series data is matched according to the key frame extraction range, and key frames are extracted according to the associated interval time as an evaluation of the later state trend. This can not only ensure the temporal correlation between the physical sensor time series data and the video surveillance visual data, but also reduce the workload in the video stream processing process and avoid repeated processing of irrelevant time periods. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 is a flow chart of the overall method of the present invention; Figure 2 This is one of the timing diagrams of the present invention; Figure 3 This is the second timing diagram of the present invention; Figure 4 Schematic diagram of video surveillance visual data processing of the present invention; Figure 5 It is a flowchart of the overall system of the present invention. DETAILED DESCRIPTION
[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.
[0019] See also Figure 1 As shown, one of the purposes of the present invention is to provide a water conservancy project monitoring data early warning method, comprising the following steps: S1. Obtain multi-dimensional heterogeneous monitoring data for dams and water conservancy hubs, where the multi-dimensional heterogeneous monitoring data includes the following data: Physical sensor time series data: including seepage pressure, deformation displacement, and stress and strain; Video surveillance visual data: real-time video data of key areas; S2. Simulate the timing diagram for the physical sensor timing data, use the clustering algorithm to extract the marginalized timing data, and locate the key timing range; S3. According to the key timing range, formulate the key frame extraction range of the video surveillance visual data, calculate the time interval between each key timing range, and formulate the unit interval of key frame extraction; S4. Obtain the marginalized time series data in the key time series range and the characteristic values of the corresponding key frames, perform state trend mapping, and obtain the mapping results.
[0020] S5. Input the mapping results into the pre-trained multimodal fusion early warning model to perform calculations to obtain a comprehensive safety risk index that characterizes the current state of the project; S6. Determine the warning level by comparing the comprehensive security risk index with the preset dynamic graded warning threshold.
[0021] The specific contents are as follows: First, to collect data in real time, this solution uses the MQTT protocol to collect monitoring data every 10 minutes from two piezometers (IDs: P01 and P02) deployed in the dam foundation corridor and one displacement meter (ID: D01) at the dam crest. An example data packet is: {"deviceId": "P01", "timestamp": "1749021600", "value": 0.52}, in MPa. This captures the corresponding physical sensor time series data, namely, seepage pressure, deformation displacement, and stress and strain. For video surveillance visual data, real-time video data from key areas must be collected. Key areas are divided as follows: Combined with the influencing conditions of the time series data of various physical sensors, it is used that the flow velocity, water level and wave state of the water surface will affect the corresponding physical sensor time series data. That is, when the physical sensor time series data is abnormal, the corresponding water level will change accordingly. The change in water level can be monitored and captured through real-time video. The influencing conditions of the time series data of various physical sensors are obtained in advance, and key areas are located according to the influencing conditions. For example, a real-time H.264 video stream with a bit rate of 4Mbps and 25fps is obtained from a high-definition camera that is long-term aimed at the water gauge in the downstream flood discharge area.
[0022] In the process of analyzing the time series data of physical sensors, there are two reasons for abnormal data. One is that an abnormality occurs in the actual monitoring area, and the corresponding sensor obtains the corresponding abnormal data. This abnormal data is marked as exogenous abnormal data. The other is abnormal data generated by the sensor itself. For example, long-term maintenance causes endogenous damage to the sensor, resulting in abnormal feedback monitoring data. This abnormal data is marked as endogenous abnormal data. In order to locate and extract exogenous abnormal data and endogenous abnormal data, it is necessary to formulate a time series diagram simulation based on the monitoring data fed back by the sensor in real time, that is, a time-monitoring data change diagram. Figure 2 As shown in the figure, the horizontal axis is the time series, the feedback interval of the monitoring data, for example, the monitoring data is fed back every 10 seconds, and the vertical axis is the actual value monitored by each sensor. Due to the differences in the monitoring environment in each area, the corresponding abnormal data judgment criteria are different. This solution uses a clustering algorithm to extract marginalized time series data. The specific content is as follows: First, combine the timing diagram to obtain the monitoring values corresponding to different timing points and mark the coordinates, such as Figure 2 As shown, The monitoring value corresponding to the timing point is , the corresponding coordinates are , and summarize the monitoring values corresponding to each time point in the monitoring time period, where the monitoring time period is defined according to the current monitoring environment. For example, when the dam is in the flood period, the probability of abnormal data will increase. At this time, the monitoring time period needs to be extended, and the cluster center value of the summarized monitoring values is selected, that is, two or more monitoring values are randomly selected as cluster center values, and clusters are established with the selected cluster center values. The distance between the remaining monitoring values and the center values of each cluster is calculated, and the monitoring values are assigned to the closest cluster to complete the first clustering work. Subsequently, the average value of each cluster is calculated, and the average value in each cluster is used as the cluster center value of the new round of clustering. The distance between each monitoring value and the corresponding cluster center value is calculated, and each monitoring value is assigned to the closest cluster to complete the second clustering work. Repeat the above steps until the number of monitoring values in each cluster does not change. At this time, the monitoring quantity of each cluster is counted, that is, the number of collected monitoring values, and a monitoring quantity threshold is set. The clusters below the monitoring quantity threshold are marked as irregular clusters. The monitoring values of the irregular clusters are sorted, and the time period with the same trend change is obtained according to the sorting results. Figure 3 As shown, 、 、 、 as well as are all time points in the same unconventional number cluster, and 、 、 、 as well as is the sequential change time point, corresponding to The monitoring value is the smallest. The monitoring value of is the largest, which means - - - - This change period has the same change trend as the corresponding monitoring value, that is, as time goes on, the monitored value gradually increases. The time period with the same trend change is marked as the key time series range; Furthermore, in order to avoid trend changes within the normal range, that is, during routine monitoring, the time series data of each physical sensor will also change within a certain range, which is a regular change. Therefore, in the process of dividing unconventional number clusters, it is necessary to define a regular change range. This regular change range is defined by the monitoring personnel based on historical monitoring data. When the monitoring value changes in the number cluster are within the regular change range, it is marked as a regular number cluster.
[0023] In order to cooperate with the synchronous monitoring of unconventional clusters, it is necessary to combine the visual data of the corresponding time series range for judgment. However, due to the large amount of visual data monitoring, if the overall data is split frame by frame every time unconventional data appears, the monitoring efficiency is low and timely warning cannot be achieved. Therefore, in this solution, the key frame extraction range of video surveillance visual data is formulated according to the key time series range; In this scheme, in order to realize directional data fusion in conjunction with the physical sensor time series data and reduce the workload of video surveillance visual data processing, in the process of formulating the key frame extraction range, it is necessary to summarize and process each unconventional number cluster, and plan the processing order according to the number of monitoring values in the unconventional number cluster, that is, the smaller the number of monitoring values, the higher the corresponding processing order, and conversely, the larger the number of monitoring values, the later the corresponding processing order, and process a single unconventional number cluster according to the planned processing order, extract the key time series range in the unconventional number cluster, obtain the starting change time point and the ending change time point of the key time series range, and introduce the starting change time point and the ending change time point divided in each key time series range into the video stream in the video surveillance visual data, and divide the staged video stream according to the starting change time point and the ending change time point as the key frame extraction range, that is, when performing key frame extraction later, perform key frame extraction from the corresponding key frame extraction range; After completing the division of the key frame extraction range, it is necessary to calculate the time interval between each key timing range. In order to improve the key frame matching completion, this solution calculates the time intervals of adjacent time points in each key frame extraction range, obtains each time interval, and compares them. The smallest time interval is used as the interval time between adjacent key frames in each key frame extraction range, such as Figure 3 As shown, 、 、 、 as well as are all time points in the same unconventional number cluster. Calculate the intervals between adjacent time points to get the shortest time interval, which is 、 or If the shortest time interval is at the minimum value in each key frame extraction range, the shortest time interval in the unconventional number cluster is used as the interval time between adjacent key frames in each key frame extraction range. When extracting key frames, the key frames are extracted according to the interval time calculated above.
[0024] Furthermore, in order to obtain the mapping relationship between the physical sensor time series data and the video surveillance visual data, it is necessary to obtain the marginalized time series data in the key time series range and the characteristic values of the corresponding key frames, perform state trend mapping, and obtain the mapping results. In order to reduce the computing pressure on the central server and the network bandwidth occupation, such as Figure 4 As shown in the figure, the video stream is processed on the edge computing gateway deployed at the dam site. One frame of image is extracted from the staged video stream according to the interval time of adjacent key frames, and the image is sent to the lightweight YOLOv5 model. After training, the model can accurately identify the "water gauge" area and "water surface line" in the image; in the water gauge area, the scale numbers are further identified, and the precise water level value, such as "135.24" meters, is calculated based on the relative pixel position of the water surface line and the scale line. This value is used as a visual feature vector, and the visual feature vectors of adjacent key frames are calculated in sequence to obtain the change trend of each visual feature vector in the current staged video stream and compare it with the physical sensor in the key time range. The time series data of the physical sensor are compared. If the state trends of the two are the same, for example, the visual feature vector in the staged video stream is the water level change, and the corresponding water level change trend is gradually increasing, then the stress and strain feedback by the physical sensor also shows an increasing state within the corresponding key time series range, which means that in this time stage, the state change trend of the physical sensor time series data and the video surveillance visual data is the same. At this time, the mapping relationship between the two is a forward mapping, indicating that the physical sensor time series data at this time is the actual monitored exogenous abnormal data. On the contrary, if the state trends of the two are the same, it indicates that the physical sensor time series data at this time is the actual monitored endogenous abnormal data, that is, the abnormal data caused by the physical sensor itself.
[0025] Finally, the mapping results are input into the pre-trained multimodal fusion warning model. The specific steps are as follows: The multimodal fusion warning model includes a sequence of normalized exogenous anomaly data and endogenous anomaly data. The time series data is fed into a pre-trained LSTM network. LSTM, with its unique gating mechanism, captures long-term dependencies in the data and outputs a hidden state vector H_lstm that incorporates historical change trends. The LSTM output H_lstm is concatenated with the visual feature vector at the current moment to form a more comprehensive fused feature vector V_fused. If the state change trends of the physical sensor time series data and the video surveillance visual data are different, the sensor time series data is directly used for prediction processing; V_fused is fed into a trained XGBoost model. XGBoost is an efficient ensemble learning algorithm that consists of multiple decision trees and can handle complex nonlinear relationships and rank the importance of features. The model ultimately outputs a continuous value between 0 and 2, which is expressed as a comprehensive security risk index.
[0026] Finally, the warning level is determined by comparing the comprehensive safety risk index with the preset dynamic graded warning threshold, for example, level 1 (blue) warning: risk index ≤ 0.7; level 2 (yellow) warning: 0.7 < risk index ≤ 1.2; level 3 (orange) warning: 1.2 < risk index ≤ 1.8; level 4 (red) warning: risk index greater than 1.8.
[0027] The present invention extracts marginalized time series data through a clustering algorithm and locates the key time series range. The located key time series range is used to locate the key frame extraction range and the interval time of adjacent key frames in the video surveillance visual data. The staged video stream associated with the marginalized time series data is matched according to the key frame extraction range, and the key frame is extracted according to the associated interval time. As an evaluation of the later state trend, it can not only ensure the temporal correlation between the physical sensor time series data and the video surveillance visual data, but also reduce the workload in the video stream processing process and avoid repeated processing of irrelevant time periods.
[0028] The second object of the present invention is to provide a system for realizing a water conservancy project monitoring data early warning method, such as Figure 5 As shown, it includes application layer, platform layer, network layer and perception layer; The perception layer collects physical sensor time series data from the dam collection area through configured piezometers and displacement meters, and uses drones equipped with high-definition cameras to capture visual features of different areas using remote sensing satellite navigation; The network layer is used to build network channels between the perception layer and the platform layer, such as 5G, the Internet of Things, fiber optic networks, and satellite communications. This network channel is used to provide real-time feedback of physical sensor time series data and video surveillance visual data in the monitored area. The platform layer is used to process physical sensor time series data and video surveillance visual data, build models, and use algorithms for data calculation and data evolution; The application layer is used to make early warning and prompt decisions based on data processing results, and to visualize the decisions. It uses visualization tools such as mobile apps and visual platforms to display the results, reminding monitoring personnel to handle them in a timely manner.
[0029] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.
Claims
1. A water conservancy project monitoring data early warning method, characterized in that: The steps include: S1. Obtain multi-dimensional heterogeneous monitoring data for dams and water conservancy hubs, where the multi-dimensional heterogeneous monitoring data includes the following data: Physical sensor time series data: including seepage pressure, deformation displacement, and stress and strain; Video surveillance visual data: real-time video data of key areas; S2. Simulate the timing diagram for the physical sensor timing data, use the clustering algorithm to extract the marginalized timing data, and locate the key timing range; S3. According to the key timing range, formulate the key frame extraction range of the video surveillance visual data, calculate the time interval between each key timing range, and formulate the unit interval of key frame extraction; S4. Obtain the marginalized time series data in the key time series range and the characteristic values of the corresponding key frames, perform state trend mapping, and obtain the mapping results.
2. S5. Input the mapping results into the pre-trained multimodal fusion early warning model for calculation to obtain a comprehensive safety risk index representing the current status of the project; S6. Determine the warning level based on the comparison between the comprehensive security risk index and the preset dynamic graded warning threshold.
3. The water conservancy project monitoring data early warning method according to claim 1, characterized in that: The method for extracting marginalized time series data in S2 includes the following steps: S2.
1. Obtain the monitoring values corresponding to different time points based on the time sequence diagram and mark the coordinates; S2.
2. Summarize the monitoring values corresponding to each time point within the monitoring period; S2.
3. Select cluster center values for the aggregated monitoring values and establish clusters based on the selected cluster center values; S2.
4. Calculate the distance between the remaining monitoring values and the center value of each cluster, and assign the monitoring values to the cluster with the closest distance, completing the first clustering work; S2.
5. Calculate the average value for each cluster and use the average value as the cluster center value for the next round of clustering. Calculate the distance between each monitored value and the corresponding cluster center value and assign each monitored value to the cluster with the closest distance, completing the second round of clustering. S2.
6. Repeat the above steps until the number of monitoring values in each cluster does not change.
4. The water conservancy project monitoring data early warning method according to claim 2, characterized in that: The method for locating the critical timing range in S2 includes the following steps: S2.
7. Establish monitoring thresholds; S2.
8. Compare the number of monitoring values in each cluster; The number of clusters marked as not less than the monitoring threshold is normal; Extract the number clusters below the monitoring threshold and mark them as irregular number clusters; S2.
9. Sort the monitoring values of the unconventional number clusters, obtain the time periods with the same trend changes based on the sorting results, and mark the time periods with the same trend changes as the key time series range.
5. The water conservancy project monitoring data early warning method according to claim 3, characterized in that: The method for obtaining the same trend change time period according to the sorting results in S2.9 includes the following steps: S2.9.
1. Define the range of normal variation; S2.9.
2. Determine the range of variation in unconventional number clusters; When the monitoring value changes in a cluster are within the normal range, it is marked as a normal cluster; When the monitoring value changes in the cluster are not within the regular change range, the irregular cluster state is maintained.
6. The water conservancy project monitoring data early warning method according to claim 1, characterized in that: The method for determining the key frame extraction range of video surveillance visual data in S3 includes the following steps: S3.
1. Aggregate and process each unconventional number cluster and plan the processing order according to the number of monitoring values in the unconventional number cluster; S3.
2. Process a single unconventional number cluster according to the planned processing sequence and extract the key time series range in the unconventional number cluster; S3.
3. Obtain the starting change time point and the ending change time point of the key timing range, and introduce the starting change time point and the ending change time point divided in each key timing range into the video stream in the video surveillance visual data, and divide the staged video stream according to the starting change time point and the ending change time point as the key frame extraction range.
7. The water conservancy project monitoring data early warning method according to claim 5, characterized in that: The method for formulating the unit interval amount for key frame extraction in S3 includes the following steps: S3.4, calculating the time intervals of adjacent time points within the extraction range of each key frame; S3.
5. Obtain each time interval and compare them, and use the smallest time interval as the interval time between adjacent key frames in each key frame extraction range.
8. The water conservancy project monitoring data early warning method according to claim 1, characterized in that: The method for performing state trend mapping in S4 includes the following steps: S4.
1. Extracting one frame of image from the staged video stream according to the interval time between the divided adjacent key frames; S4.
2. Feed the image into a lightweight YOLOv5 model, which is trained to identify visual feature vectors in the image. S4.
3. Calculate the visual feature vectors of adjacent key frames in sequence, obtain the change trend of each visual feature vector in the current stage-by-stage video stream, and compare them with the physical sensor timing data in the key timing range; If the state trends of the two are the same, it indicates that the mapping relationship between the two is a positive mapping. At this time, the physical sensor time series data is the actual monitored exogenous abnormal data; If the status trends of the two are the same, it indicates that the physical sensor time series data at this time is the actual monitored endogenous abnormal data.
9. The water conservancy project monitoring data early warning method according to claim 1, characterized in that: The method for obtaining the comprehensive safety risk index representing the current state of the project in S5 includes the following steps: S5.
1. Feed the time series data into a pre-trained LSTM network to capture the long-term dependencies in the data and output a hidden state vector H_lstm that incorporates historical change trends. S5.
2. Concatenate the LSTM output H_lstm with the visual feature vector at the current moment to form a more comprehensive fused feature vector V_fused; S5.
3. Use V_fused as input to import a trained XGBoost model and give the importance ranking of features. S5.
4. Output a continuous value between 0 and 2, representing the comprehensive safety risk index.
10. A system for implementing the water conservancy project monitoring data early warning method according to claim 1, characterized in that: Including application layer, platform layer, network layer and perception layer; The perception layer collects physical sensor time series data from the dam collection area through configured osmometers and displacement meters, and uses drones equipped with high-definition cameras to capture visual features of different areas using remote sensing satellite navigation; The network layer is used to build a network channel between the perception layer and the platform layer, such as 5G, Internet of Things, fiber optic network and satellite communication, and use the built network channel to feedback the physical sensor time series data and video surveillance visual data of the monitoring area in real time; The platform layer is used to process physical sensor time series data and video surveillance visual data, build models, and use algorithms for data calculation and data evolution; The application layer is used to make early warning and prompt decisions based on the data processing results, and to visualize the decisions, using visualization tools such as mobile apps and visual platforms to display the results, reminding monitoring personnel to handle them in a timely manner.