Water conservancy and hydropower engineering construction safety supervision system and method based on multi-source data fusion

Abstract

The invention belongs to the technical field of water conservancy and hydropower engineering, and discloses a water conservancy and hydropower engineering construction safety supervision system based on multi-source data fusion. The system comprises a multi-source sensing acquisition module, a heterogeneous data fusion processing module, a risk identification and early warning module, a safety behavior evaluation and feedback module, and a command scheduling and visualization module. According to the invention, by fusing multi-dimensional data such as image monitoring, environment sensing, personnel positioning, equipment state and the like, a space-air-ground three-dimensional sensing network is constructed, and in a high slope area, the distributed optical fiber strain sensors are linked with thermal imaging data of the unmanned aerial vehicle, so that millimeter-level deformation and temperature field abnormity can be captured in real time; a video stream is analyzed in real time by means of a YOLOv8 algorithm, illegal operation behaviors of personnel can be accurately identified, a cross-modal fusion model of a Transform architecture is combined, the system can dynamically capture potential correlation among data, and millisecond-level response to risks such as side slope landslide, equipment faults and personnel dangerous operation is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of water conservancy and hydropower engineering, and specifically relates to a water conservancy and hydropower engineering construction safety supervision system and method based on multi-source data fusion. Background Art

[0002] As the scale of water conservancy and hydropower project construction continues to expand, their safety supervision faces increasingly complex challenges. Currently, most construction safety management systems rely on a single data source, such as video surveillance or manual inspections to obtain safety information. These systems suffer from problems such as limited perception range, delayed response, and data isolation, making it difficult to support rapid early warning and dynamic scheduling of emergencies. For example, during high-slope excavation operations, traditional fixed cameras can only capture the surface activities of personnel and equipment and cannot monitor subtle deformations of deep geological structures in real time. Manual inspections not only have long time intervals and low efficiency, but are also difficult to carry out continuously in inclement weather, resulting in a large number of blind spots in hidden danger detection. At the same time, conventional systems lack the integration and analysis of environmental data, such as water level changes, geological sliding, and meteorological changes, with the status of construction equipment. For example, during the flood season, water level rise data is not combined with cofferdam load-bearing monitoring data, resulting in a disconnect between safety decisions and actual risks, which can easily lead to the omission of accident hazards.

[0003] To solve the above problems, this system introduces the concept of multi-source data fusion. By building an integrated supervision architecture for data collaborative perception, dynamic analysis and response linkage, at the data collection layer, it adopts an "air-space-ground" three-dimensional perception network, and combines new equipment such as millimeter-wave radar, fiber optic sensors, and drone thermal imaging with traditional monitoring methods to achieve millimeter-level deformation capture and full-field temperature field scanning; at the analysis layer, it uses the Transformer architecture and graph neural network to break down data barriers and explore the potential correlation between "environment-equipment-personnel"; at the application layer, it relies on digital twins and reinforcement learning technologies to realize the dynamic deduction of risk situations and autonomous optimization of decision-making rules.

[0004] The system innovatively realizes the interconnection and intelligent joint control of multi-dimensional data such as construction behavior, working environment, equipment status, personnel distribution, external disturbances, etc., thus breaking through the traditional "information island" management model, significantly improving the early warning capability and decision-making efficiency of construction safety, and providing more intelligent and reliable safety guarantees for the construction of water conservancy and hydropower projects. Summary of the Invention

[0005] The purpose of the present invention is to provide a water conservancy and hydropower project construction safety supervision system and method based on multi-source data fusion to solve the problems raised in the above background technology.

[0006] In order to achieve the above-mentioned object, the present invention provides the following technical solution: a water conservancy and hydropower project construction safety supervision system based on multi-source data fusion, comprising a multi-source perception and acquisition module, a heterogeneous data fusion processing module, a risk identification and early warning module, a safety behavior assessment and feedback module, and a command, dispatch and visualization module;

[0007] Multi-source perception and acquisition module: determines the data collected from sensors, video surveillance, personnel information, construction equipment, environment, etc., rationally deploys the acquisition equipment and connects it to the network, and preliminarily verifies the data to ensure quality;

[0008] Heterogeneous data fusion processing module: cleans, converts formats, and pre-processes time synchronization of data from different sources, selects algorithms such as Kalman filtering for fusion, stores the fused data in a distributed database, and creates indexes and backups;

[0009] Risk Identification and Early Warning Module: Builds a multi-dimensional safety assessment indicator system with thresholds and weights, uses algorithms to calculate real-time risk levels, and visualizes assessment results in charts and other forms;

[0010] Safety behavior assessment and feedback module: Set different levels of warning rules, methods and targets, monitor and trigger warnings in real time, and initiate response processes such as suspending construction or strengthening inspections based on different levels of warnings;

[0011] Command, dispatch and visualization module: provides multi-dimensional data query statistics, uses technology to analyze historical data to support decision-making and simulate optimization solutions, and conducts user management, equipment monitoring and maintenance, and system upgrade and optimization.

[0012] Preferably, the multi-source perception acquisition module includes:

[0013] (1) Determine the data type and source, and clearly collect sensors, temperature, humidity, pressure, displacement, vibration sensors, video surveillance, personnel information, construction equipment and environmental data. Various sensor nodes are connected through the NB-IoT network. For NB-IoT, its communication model can be expressed as: S = K × W × T, where S represents the data transmission volume, K is the data volume coefficient, W is the channel bandwidth, and T is the transmission time;

[0014] (2) Image data preprocessing: edge preprocessing of image data, including target detection and behavior recognition operations. In target detection, the detection model based on deep learning can be expressed as: Y = f(X; θ), where Y is the detection result, X is the input image data, and θ is the model parameter; the feature extraction formula for behavior recognition is: F = g(X), where F is the extracted feature vector and g is the feature extraction function;

[0015] (3) Edge intelligent processing: Introduce edge AI gateway to realize real-time pre-screening of data, so as to reduce the computing pressure of the platform. The decision function of data pre-screening is: D = h(F; τ), where D is the decision result, whether the screening is passed, and τ is the decision threshold; the formula for reducing the computing pressure of the platform is: ΔP = P0-P1 = α×n, where ΔP is the pressure reduction, P0 is the original computing pressure, P1 is the computing pressure after processing, α is the pressure coefficient, and n is the amount of data processed at the edge.

[0016] Preferably, the heterogeneous data fusion processing module includes:

[0017] Multimodal data preprocessing: Adaptive time series cleaning technology is used to remove redundancy and outliers in data from different sources, synchronize the time axis and unify the format to prepare for fusion. Outlier detection: Based on the dynamic threshold formula of sliding window, Thresh = μt + k × σt; where μt is the mean in the window, σt is the standard deviation, and k is the confidence coefficient, which is adaptively adjusted; Redundant data filtering: Time series similarity calculation is considered redundant when Sim>τ (threshold).

[0018] Deep fusion of multi-source data: Apply multimodal algorithms such as the attention mechanism fusion model, use Kalman filtering on sensor data, feature fusion on video and personnel data, and decision-level fusion on equipment and environment data to improve data relevance. Multimodal feature weighting: s = ∑ i w i ×f i (x); where f i (x) is the data feature of the i-th category, sensor, video, w i is the attention weight, calculated by the self-attention mechanism: Where Q is the query vector and score is the similarity function;

[0019] Converged data graph construction and storage: Build a semantic association graph of personnel, equipment, location, and task, store the fused data in a distributed database, and ensure security and availability through indexing and backup.

[0020] Preferably, the risk identification and early warning module includes:

[0021] Intelligent risk assessment model construction: Combined with the water conservancy and hydropower safety regulations, a multi-dimensional assessment index system is constructed. The CNN+LSTM deep learning model is used to quantitatively assess the risk level of images and environmental variables. CNN feature extraction: F c =σ(W c *X+b c ), where X is the input image, W c is the convolution kernel, b c is the bias, σ is the activation function, F cFor the extracted spatial features; LSTM time series analysis:

[0022] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0023] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0024]

[0025] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0026] h t =o t ⊙tanh(c t )

[0027] Where x t For environment variable timing input, h t is the hidden state, c t is the cell state, f t ,it,ot are the activation values ​​of the forget gate, input gate, and output gate respectively;

[0028] Establishment of disaster sensitivity warning model: Integrate real-time weather forecasts with historical accident data to build a disaster sensitivity warning model to achieve forward-looking risk prediction and weather-accident correlation analysis: S(t) = ∑i = 1nwi Impact(Mi(t),Hi), where Mi(t) is the real-time weather factor, wind speed, and rainfall, Hi is the accident frequency under the same historical weather conditions, wi is the weight, and S(t) is the disaster sensitivity index;

[0029] Hierarchical response and system linkage: supports the three-level response mechanism of early warning, alarm and emergency, and links with the dispatching system to realize dynamic risk disposal and resource allocation.

[0030] Preferably, the safety behavior assessment and feedback module includes:

[0031] Safety performance quantification and early warning rule optimization: Introduce a behavioral scoring model to quantify the safety performance of personnel / teams, set multi-level early warning rules based on risk level thresholds, and clarify the early warning method and individual safety performance score of the target. Where, vi is the number of individual violations, not wearing a helmet, V th,i is the threshold, w i is the weight, the weight of not wearing a helmet is 0.3;

[0032] Post-incident investigation and real-time early warning linkage: The "Safety Review" function integrates on-site images and sensor tracks to assist in tracing the source of the accident. At the same time, real-time monitoring triggers early warnings and links the response process.

[0033] Risk trend analysis and decision support: The system generates risk trend reports based on historical data, providing data support for the optimization of safety management strategies.

[0034] Preferably, the command, dispatch and visualization module includes:

[0035] BIM+GIS fusion visualization modeling: BIM and GIS fusion technology is used to display terrain, structure and construction status, build a three-dimensional visualization space model, and realize the spatial mapping of multi-dimensional data. Spatial coordinate mapping: PGIS = T·PBIM, where PBIM is the BIM model coordinate, PGIS is the geographic coordinate, and T is the coordinate transformation matrix, which includes translation, rotation, and scaling parameters;

[0036] Dynamic risk visualization and trajectory tracking: The visualization interface supports highlighting risk areas and combines personnel positioning with equipment trajectory data to achieve key personnel tracking and equipment operation path playback and trajectory data interpolation:

[0037] Based on the position P(ti) and P(tj) of the equipment / personnel at time ti and tj, interpolate and calculate the position at any time t;

[0038] Automatic generation of intelligent dispatch instructions: Based on risk assessment results and construction status, dispatch instructions are automatically generated and pushed to the operator's terminal, realizing closed-loop execution of management decisions.

[0039] Preferably, the specific steps of the water conservancy and hydropower project construction safety supervision method based on multi-source data fusion are as follows:

[0040] S1: Multi-source dynamic perception and edge intelligent preprocessing: Deploy millimeter-wave radar and other equipment to build a three-dimensional perception network. Edge nodes use YOLOv8 and adaptive filtering to process data, and use BeiDou and BIM to calibrate the spatiotemporal benchmark.

[0041] S2: Spatiotemporal semantic fusion and risk situation modeling: Use the Transformer architecture to fuse multimodal features, use GNN to build a dynamic knowledge graph to mine hidden relationships, and input the fused data into the digital twin model to deduce risk situation;

[0042] S3: Intelligent decision-making and closed-loop control execution: Use reinforcement learning to automatically optimize warning thresholds, generate disposal plans through the BIM+GIS fusion interface, and establish a three-level safety performance evaluation system to form closed-loop management.

[0043] Preferably, the specific steps of multi-source dynamic perception and edge intelligence preprocessing in S1 are as follows:

[0044] Step 1: Collaborative multi-dimensional data collection: By deploying millimeter-wave radars, fiber optic sensors, and 5G smart helmets, a three-dimensional "air-ground-air" perception network is constructed. Distributed fiber optic strain sensors are deployed in high slope areas to capture millimeter-level deformation in real time. UAVs equipped with thermal imagers are used to obtain full-field temperature field data, covering the blind spots of traditional sensors. in, is the state after filtering, K t is the Kalman gain, which adjusts the process noise covariance Qt and the observation noise covariance R in real time through the edge node t , adapt to the dynamic changes of the construction environment;

[0045] Step 2: Edge Intelligent Preprocessing: Introducing edge computing nodes and AI gateways to perform lightweight processing at the data source. YOLOv8 is used to detect violations and failure to wear protective equipment in real time on video streams. Adaptive Kalman filtering is used to remove outliers from sensor data. Data is compressed by 30%-50% before uploading to the cloud, reducing network transmission pressure.

[0046] Step 3: Unified calibration of time and space benchmarks: Establish a time and space benchmark network based on Beidou satellites, synchronize all sensors and cameras at the nanosecond level, and unify the spatial data collected by different devices into the construction coordinate system through BIM model coordinate mapping to ensure data time and space consistency.

[0047] Preferably, the specific steps of spatiotemporal semantic fusion and risk situation modeling in S2 are as follows:

[0048] Step 1: Multimodal feature association and fusion: A cross-modal fusion model is constructed using the Transformer architecture. Video features, human behavior, sensor features, equipment vibration, environmental characteristics, and meteorological data are dynamically weighted using a self-attention mechanism. When the crane is detected in operation, the model automatically increases the weight of the vibration sensor data to suppress interference from irrelevant environmental data.

[0049] Step 2: Dynamic knowledge graph construction: Based on the personnel-equipment-environment-process quadruple, a dynamic knowledge graph for construction safety is constructed. Graph neural networks (GNNs) are used to mine hidden relationships, such as the combined risk probability of "rainy season + high slope excavation + equipment overload";

[0050] Step 3: Three-dimensional risk situation simulation: The fused data is input into the digital twin model based on the physics engine to simulate the risk evolution of the construction scenario in real time. Slope displacement data and rainfall forecasts are input. The model automatically simulates the impact range of the landslide and marks high-risk areas with red voxels of different transparency, making the risk "perceptible, predictable, and explainable."

[0051] Preferably, the specific steps for executing the intelligent decision-making and closed-loop control are as follows:

[0052] Step 1: Adaptive decision rule generation: Using a reinforcement learning algorithm, PPO, the system automatically optimizes the warning threshold based on historical accident data and response results. The system learns that the probability of crane overturning increases during nighttime and wind speeds > 10 m / s. It automatically lowers the vibration warning threshold for this scenario by 15% to avoid missed alerts.

[0053] Step 2: Digital twin-driven collaborative response: In the BIM+GIS integrated 3D interface, clicking on a high-risk area automatically generates a response plan. When a Level 1 warning is triggered, the system sends an automatic shutdown command to the construction robot and simultaneously pushes a response process animation to the safety officer via AR glasses, achieving a "warning decision-making-execution" linkage within seconds.

[0054] Step 3: Quantify and close the loop of safety performance: Establish a three-level safety performance evaluation system for personnel, teams, and projects, convert violation records, response efficiency, and data into visual star ratings, and use blockchain technology to store safety data to form a closed-loop management of "collection-evaluation-improvement", promoting safety management from passive response to active optimization.

[0055] The beneficial effects of the present invention are as follows:

[0056] 1. This invention builds a three-dimensional "air-ground-air" perception network by integrating multi-dimensional data such as image monitoring, environmental sensing, personnel positioning, and equipment status. In high-slope areas, distributed fiber optic strain sensors are linked with drone thermal imaging data to capture millimeter-level deformation and temperature field anomalies in real time. With the help of the YOLOv8 algorithm, real-time analysis of video streams can accurately identify illegal operating behaviors of personnel. Combined with the cross-modal fusion model of the Transformer architecture, the system can dynamically capture potential correlations between data and achieve millisecond-level responses to risks such as slope landslides, equipment failures, and dangerous operations by personnel.

[0057] 2. The present invention realizes a fundamental change in the supervision model by constructing a complete data closed-loop link of "collection-fusion-analysis-dispatching-feedback". In the risk warning stage, the reinforcement learning algorithm automatically optimizes the warning threshold and combines the digital twin model to deduce the risk evolution in real time; after the warning is triggered, the system automatically generates a hierarchical disposal plan based on the BIM+GIS fusion interface, and issues instructions to construction robots, AR terminals and other equipment through the 5G network, synchronously and linkage dispatching personnel evacuation, equipment shutdown, and material allocation operations.

[0058] 3. The present invention decouples functions such as perception and collection, data fusion, analysis and decision-making through the use of modular design, and combines edge AI, cloud computing and 5G communication technologies to realize the collaborative architecture of "cloud brain + edge node". For example, small construction sites can deploy basic monitoring modules through lightweight edge gateways, while large projects can expand drone clusters and smart safety helmet equipment and seamlessly connect to cloud big data platforms. In addition, the system supports containerized deployment and microservice architecture, and can dynamically increase or decrease functional modules according to the progress of the project. It is compatible with the access of new equipment such as millimeter-wave radar and fiber optic sensors. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a diagram of the water conservancy and hydropower project construction safety supervision system based on multi-source data fusion of the present invention;

[0060] Figure 2 This is a flow chart of the water conservancy and hydropower project construction safety supervision method based on multi-source data fusion of the present invention. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] like Figures 1 to 2 As shown, the embodiment of the present invention provides a water conservancy and hydropower project construction safety supervision system based on multi-source data fusion, which includes a multi-source perception and acquisition module, a heterogeneous data fusion processing module, a risk identification and early warning module, a safety behavior assessment and feedback module, and a command, dispatch and visualization module:

[0063] Multi-source perception and acquisition module: determines the data collected from sensors, video surveillance, personnel information, construction equipment, environment, etc., rationally deploys the acquisition equipment and connects it to the network, and preliminarily verifies the data to ensure quality;

[0064] Heterogeneous data fusion processing module: cleans, converts formats, and pre-processes time synchronization of data from different sources, selects algorithms such as Kalman filtering for fusion, stores the fused data in a distributed database, and creates indexes and backups;

[0065] Risk Identification and Early Warning Module: Builds a multi-dimensional safety assessment indicator system with thresholds and weights, uses algorithms to calculate real-time risk levels, and visualizes assessment results in charts and other forms;

[0066] Safety behavior assessment and feedback module: Set different levels of warning rules, methods and targets, monitor and trigger warnings in real time, and initiate response processes such as suspending construction or strengthening inspections based on different levels of warnings;

[0067] Command, dispatch and visualization module: provides multi-dimensional data query statistics, uses technology to analyze historical data to support decision-making and simulate optimization solutions, and conducts user management, equipment monitoring and maintenance, and system upgrade and optimization.

[0068] The multi-source sensing acquisition module includes:

[0069] (1) Determine the data type and source, and clearly collect sensors, temperature, humidity, pressure, displacement, vibration sensors, video surveillance, personnel information, construction equipment and environmental data. Various sensor nodes are connected through the NB-IoT network. For NB-IoT, its communication model can be expressed as: S = K × W × T, where S represents the data transmission volume, K is the data volume coefficient, W is the channel bandwidth, and T is the transmission time;

[0070] (2) Image data preprocessing: edge preprocessing of image data, including target detection and behavior recognition operations. In target detection, the detection model based on deep learning can be expressed as: Y = f(X; θ), where Y is the detection result, X is the input image data, and θ is the model parameter; the feature extraction formula for behavior recognition is: F = g(X), where F is the extracted feature vector and g is the feature extraction function;

[0071] (3) Edge intelligent processing: Introduce edge AI gateway to realize real-time pre-screening of data, so as to reduce the computing pressure of the platform. The decision function of data pre-screening is: D = h(F; τ), where D is the decision result, whether the screening is passed, and τ is the decision threshold; the formula for reducing the computing pressure of the platform is: ΔP = P0-P1 = α×n, where ΔP is the pressure reduction, P0 is the original computing pressure, P1 is the computing pressure after processing, α is the pressure coefficient, and n is the amount of data processed at the edge.

[0072] Determine the collection of multi-source data such as sensors and video surveillance. Sensors are accessed through the NB-IoT network, and their communication volume follows the S=K×W×T model; perform edge pre-processing such as target detection and behavior recognition on image data; introduce edge AI gateway to pre-screen data to reduce platform pressure. Each link is supported by corresponding principles and formulas.

[0073] The heterogeneous data fusion processing module includes:

[0074] Multimodal data preprocessing: Adaptive time series cleaning technology is used to remove redundancy and outliers in data from different sources, synchronize timelines, and unify formats in preparation for fusion. Outlier detection: Based on the dynamic threshold formula of the sliding window, Thresh = μt + k × σt; where μt is the mean within the window, σt is the standard deviation, and k is the confidence coefficient, which is adaptively adjusted; Redundant data filtering: Time series similarity calculation is considered redundant when Sim>τ (threshold);

[0075] Deep fusion of multi-source data: Apply multimodal algorithms such as the attention mechanism fusion model, use Kalman filtering on sensor data, feature fusion on video and personnel data, and decision-level fusion on equipment and environment data to improve data relevance. Multimodal feature weighting: s = ∑ i w i ×f i (x); where f i (x) is the data feature of the i-th category, sensor, video, w i is the attention weight, calculated by the self-attention mechanism: Where Q is the query vector and score is the similarity function;

[0076] Converged data graph construction and storage: Build a semantic association graph of personnel, equipment, location, and task, store the fused data in a distributed database, and ensure security and availability through indexing and backup.

[0077] Multimodal data preprocessing uses adaptive cleaning technology to remove redundancy and outliers, synchronize timelines and unify formats; deep fusion of multi-source data uses multiple algorithms and attention mechanisms to improve data relevance; finally, a semantic association graph is constructed, and the fused data is stored in a distributed database to ensure security.

[0078] The risk identification and early warning module includes:

[0079] Intelligent risk assessment model construction: Combined with the water conservancy and hydropower safety regulations, a multi-dimensional assessment index system is constructed. The CNN+LSTM deep learning model is used to quantitatively assess the risk level of images and environmental variables. CNN feature extraction: F c =σ(W c *X+b c), where X is the input image, W c is the convolution kernel, b c is the bias, σ is the activation function, F c For the extracted spatial features; LSTM time series analysis:

[0080] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0081] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0082]

[0083] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0084] h t =o t ⊙tanh(c t )

[0085] Where x t For environment variable timing input, h t is the hidden state, c t is the cell state, f t ,it,ot are the activation values ​​of the forget gate, input gate, and output gate respectively;

[0086] Establishment of disaster sensitivity warning model: Integrate real-time weather forecasts with historical accident data to build a disaster sensitivity warning model to achieve forward-looking risk prediction and weather-accident correlation analysis: S(t) = ∑i = 1nwi Impact(Mi(t),Hi), where Mi(t) is the real-time weather factor, wind speed, and rainfall, Hi is the accident frequency under the same historical weather conditions, wi is the weight, and S(t) is the disaster sensitivity index;

[0087] Hierarchical response and system linkage: supports the three-level response mechanism of early warning, alarm and emergency, and links with the dispatching system to realize dynamic risk disposal and resource allocation.

[0088] The safety behavior assessment and feedback module includes:

[0089] Safety performance quantification and early warning rule optimization: Introduce a behavioral scoring model to quantify the safety performance of personnel / teams, set multi-level early warning rules based on risk level thresholds, and clarify the early warning method and individual safety performance score of the target. Where, v i is the number of individual violations, not wearing a helmet, V th,i is the threshold, w i is the weight, the weight of not wearing a helmet is 0.3;

[0090] Post-incident investigation and real-time early warning linkage: The "Safety Review" function integrates on-site images and sensor tracks to assist in tracing the source of the accident. At the same time, real-time monitoring triggers early warnings and links the response process.

[0091] Risk trend analysis and decision support: The system generates risk trend reports based on historical data, providing data support for the optimization of safety management strategies.

[0092] Build a multi-dimensional evaluation indicator system, use the CNN+LSTM model to quantify the risk level, integrate meteorological and historical data to build a disaster warning model, support a three-level response and linkage dispatch system, quantify safety performance through a behavioral scoring model, use "safety review" to integrate data to assist in traceability, and generate risk trend reports based on historical data to support decision-making.

[0093] The command, dispatch and visualization module includes:

[0094] BIM+GIS fusion visualization modeling: BIM and GIS fusion technology is used to display terrain, structure and construction status, build a three-dimensional visualization space model, and realize the spatial mapping of multi-dimensional data. Spatial coordinate mapping: PGIS = T·PBIM, where PBIM is the BIM model coordinate, PGIS is the geographic coordinate, and T is the coordinate transformation matrix, which includes translation, rotation, and scaling parameters;

[0095] Dynamic risk visualization and trajectory tracking: The visualization interface supports highlighting risk areas and combines personnel positioning with equipment trajectory data to achieve key personnel tracking and equipment operation path playback and trajectory data interpolation:

[0096] Based on the position P(ti) and P(tj) of the equipment / personnel at time ti and tj, interpolate and calculate the position at any time t;

[0097] Automatic generation of intelligent dispatch instructions: Based on risk assessment results and construction status, dispatch instructions are automatically generated and pushed to the operator's terminal, realizing closed-loop execution of management decisions.

[0098] Construct a multi-dimensional evaluation indicator system, use the CNN+LSTM model to quantify the risk level, integrate meteorological and historical data to build a disaster warning model, support a three-level response and linkage dispatch system, and each module contains corresponding principle formulas.

[0099] The specific steps of the water conservancy and hydropower project construction safety supervision method based on multi-source data fusion are as follows:

[0100] S1: Multi-source dynamic perception and edge intelligent preprocessing: Deploy millimeter-wave radar and other equipment to build a three-dimensional perception network. Edge nodes use YOLOv8 and adaptive filtering to process data, and use BeiDou and BIM to calibrate the spatiotemporal benchmark.

[0101] S2: Spatiotemporal semantic fusion and risk situation modeling: Use the Transformer architecture to fuse multimodal features, use GNN to build a dynamic knowledge graph to mine hidden relationships, and input the fused data into the digital twin model to deduce risk situation;

[0102] S3: Intelligent decision-making and closed-loop control execution: Use reinforcement learning to automatically optimize warning thresholds, generate disposal plans through the BIM+GIS fusion interface, and establish a three-level safety performance evaluation system to form closed-loop management.

[0103] Multi-source dynamic perception and edge intelligent preprocessing, deploying equipment to build a three-dimensional network, edge processing data and calibrating time and space; spatiotemporal semantic fusion and risk situation modeling, integrating multimodal features, building knowledge graphs, inputting digital twins to deduce risks; intelligent decision-making and closed-loop control execution, optimizing warning thresholds, generating disposal plans, and establishing a performance evaluation system to form a closed loop.

[0104] The specific steps of multi-source dynamic perception and edge intelligence preprocessing in S1 are as follows:

[0105] Step 1: Collaborative multi-dimensional data collection: By deploying millimeter-wave radars, fiber optic sensors, and 5G smart helmets, a three-dimensional "air-ground-air" perception network is constructed. Distributed fiber optic strain sensors are deployed in high slope areas to capture millimeter-level deformation in real time. UAVs equipped with thermal imagers are used to obtain full-field temperature field data, covering the blind spots of traditional sensors. in, is the state after filtering, K t is the Kalman gain, which adjusts the process noise covariance Qt and the observation noise covariance R in real time through the edge node t , adapt to the dynamic changes of the construction environment;

[0106] Step 2: Edge Intelligent Preprocessing: Introducing edge computing nodes and AI gateways to perform lightweight processing at the data source. YOLOv8 is used to detect violations and failure to wear protective equipment in real time on video streams. Adaptive Kalman filtering is used to remove outliers from sensor data. Data is compressed by 30%-50% before uploading to the cloud, reducing network transmission pressure.

[0107] Step 3: Unified calibration of time and space benchmarks: Establish a time and space benchmark network based on Beidou satellites, synchronize all sensors and cameras at the nanosecond level, and unify the spatial data collected by different devices into the construction coordinate system through BIM model coordinate mapping to ensure data time and space consistency.

[0108] Collaborative multi-dimensional data collection, deployment of new equipment such as millimeter-wave radar to build a "space-air-ground-ground" three-dimensional perception network to capture deformation and temperature field data; edge intelligent preprocessing, using edge computing nodes to lightweight process video and sensor data, compress the data volume and upload it; unified calibration of time and space benchmarks, using Beidou and BIM to achieve time and space synchronization and coordinate mapping of sensors and cameras.

[0109] The specific steps of spatiotemporal semantic fusion and risk situation modeling in S2 are as follows:

[0110] Step 1: Multimodal feature association and fusion: A cross-modal fusion model is constructed using the Transformer architecture. Video features, human behavior, sensor features, equipment vibration, environmental characteristics, and meteorological data are dynamically weighted using a self-attention mechanism. When the crane is detected in operation, the model automatically increases the weight of the vibration sensor data to suppress interference from irrelevant environmental data.

[0111] Step 2: Dynamic knowledge graph construction: Based on the personnel-equipment-environment-process quadruple, a dynamic knowledge graph for construction safety is constructed. Graph neural networks (GNNs) are used to mine hidden relationships, such as the combined risk probability of "rainy season + high slope excavation + equipment overload";

[0112] Step 3: Three-dimensional risk situation simulation: The fused data is input into the digital twin model based on the physics engine to simulate the risk evolution of the construction scenario in real time. Slope displacement data and rainfall forecasts are input. The model automatically simulates the impact range of the landslide and marks high-risk areas with red voxels of different transparency, making the risk "perceptible, predictable, and explainable."

[0113] Multimodal feature association and fusion: a cross-modal model is constructed through the Transformer architecture, and video, sensor, and environmental features are dynamically weighted through the self-attention mechanism to enhance the weight of related data; dynamic knowledge graph construction: a graph is constructed based on quadruple, and hidden relationships are mined using GNN; three-dimensional risk situation deduction: the fused data is input into the digital twin model to simulate risk evolution and mark high-risk areas.

[0114] The specific steps for executing intelligent decision-making and closed-loop control are as follows:

[0115] Step 1: Adaptive decision rule generation: Using a reinforcement learning algorithm, PPO, the system automatically optimizes the warning threshold based on historical accident data and response results. The system learns that the probability of crane overturning increases during nighttime and wind speeds > 10 m / s. It automatically lowers the vibration warning threshold for this scenario by 15% to avoid missed alerts.

[0116] Step 2: Digital twin-driven collaborative response: In the BIM+GIS integrated 3D interface, clicking on a high-risk area automatically generates a response plan. When a Level 1 warning is triggered, the system sends an automatic shutdown command to the construction robot and simultaneously pushes a response process animation to the safety officer via AR glasses, achieving a "warning decision-making-execution" linkage within seconds.

[0117] Step 3: Quantify and close the loop of safety performance: Establish a three-level safety performance evaluation system for personnel, teams, and projects, convert violation records, response efficiency, and data into visual star ratings, and use blockchain technology to store safety data to form a closed-loop management of "collection-evaluation-improvement", promoting safety management from passive response to active optimization.

[0118] Adaptive decision-making rule generation uses reinforcement learning algorithms to optimize warning thresholds based on historical data, such as lowering the vibration warning threshold when the wind speed is high at night; digital twins drive collaborative response, automatically generate plans by clicking on high-risk areas in the three-dimensional interface, and realize the linkage of warning decision execution; safety performance quantification closed loop, establish a three-level evaluation system, convert data into visual scores, and form closed-loop management through blockchain evidence storage.

[0119] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0120] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A water conservancy and hydropower project construction safety supervision system based on multi-source data fusion, characterized by: The system consists of a multi-source perception and acquisition module, a heterogeneous data fusion and processing module, a risk identification and early warning module, a safety behavior assessment and feedback module, and a command, dispatch, and visualization module: Multi-source perception and acquisition module: determines the acquisition of sensors, video surveillance, personnel information, construction equipment, and environmental data, rationally deploys the acquisition equipment and connects it to the network, and preliminarily verifies the data to ensure quality; Heterogeneous data fusion processing module: cleans, converts formats, and pre-processes time synchronization of data from different sources, selects the Kalman filter algorithm for fusion, stores the fused data in a distributed database, and creates indexes and backups; Risk Identification and Early Warning Module: Builds a multi-dimensional safety assessment indicator system with thresholds and weights, uses algorithms to calculate real-time risk levels, and visualizes the assessment results in charts and graphs; Safety behavior assessment and feedback module: Set different levels of warning rules, methods and targets, monitor and trigger warnings in real time, initiate and suspend construction according to different levels of warnings, and strengthen the inspection and response process; Command, dispatch and visualization module: provides multi-dimensional data query statistics, uses technology to analyze historical data to support decision-making and simulate optimization solutions, and conducts user management, equipment monitoring and maintenance, and system upgrade and optimization.

2. The water conservancy and hydropower project construction safety supervision system based on multi-source data fusion according to claim 1 is characterized by: The multi-source perception acquisition module includes: (1) Determine the data type and source, and clearly collect sensors, temperature, humidity, pressure, displacement, vibration sensors, video surveillance, personnel information, construction equipment and environmental data. Various sensor nodes are connected through the NB-IoT network. For NB-IoT, its communication model can be expressed as: S = K × W × T, where S represents the data transmission volume, K is the data volume coefficient, W is the channel bandwidth, and T is the transmission time; (2) Image data preprocessing: edge preprocessing of image data, including target detection and behavior recognition operations. In target detection, the detection model based on deep learning can be expressed as: Y = f(X; θ), where Y is the detection result, X is the input image data, and θ is the model parameter; the feature extraction formula for behavior recognition is: F = g(X), where F is the extracted feature vector and g is the feature extraction function; (3) Edge intelligent processing: Introduce edge AI gateway to realize real-time pre-screening of data, so as to reduce the computing pressure of the platform. The decision function of data pre-screening is: D = h(F; τ), where D is the decision result, whether the screening is passed, and τ is the decision threshold; the formula for reducing the computing pressure of the platform is: ΔP = P0-P1 = α×n, where ΔP is the pressure reduction, P0 is the original computing pressure, P1 is the computing pressure after processing, α is the pressure coefficient, and n is the amount of data processed at the edge.

3. The water conservancy and hydropower project construction safety supervision system based on multi-source data fusion according to claim 1 is characterized by: The heterogeneous data fusion processing module includes: Multimodal data preprocessing: Adaptive time series cleaning technology is used to remove redundancy and outliers in data from different sources, synchronize the time axis and unify the format to prepare for fusion. Outlier detection: Based on the dynamic threshold formula of sliding window, Thresh = μt + k × σt; where μt is the mean in the window, σt is the standard deviation, and k is the confidence coefficient, which is adaptively adjusted; Redundant data filtering: Time series similarity calculation is considered redundant when Sim>τ (threshold). Deep fusion of multi-source data: Apply the attention mechanism fusion model multimodal algorithm, use Kalman filtering for sensor data, feature fusion for video and personnel data, and decision-level fusion for equipment and environment data to improve data relevance, multimodal feature weighting: s = ∑ i w i ×f i (x); where f i (x) is the data feature of the i-th category, sensor, video, w i is the attention weight, calculated by the self-attention mechanism: Where Q is the query vector and score is the similarity function; Converged data graph construction and storage: Build a semantic association graph of personnel, equipment, location, and task, store the fused data in a distributed database, and ensure security and availability through indexing and backup.

4. The water conservancy and hydropower project construction safety supervision system based on multi-source data fusion according to claim 1 is characterized by: The risk identification and early warning module includes: Intelligent risk assessment model construction: Combined with the water conservancy and hydropower safety regulations, a multi-dimensional assessment index system is constructed. The CNN+LSTM deep learning model is used to quantitatively assess the risk level of images and environmental variables. CNN feature extraction: F c =σ(W c *X+b c ), where X is the input image, W c is the convolution kernel, b c is the bias, σ is the activation function, F c For the extracted spatial features; LSTM time series analysis: f t =σ(W f ·[h t-1 ,x t ]+b f ) i t =σ(W i ·[h t-1 ,x t ]+b i ) the t =σ(W o ·[h t-1 ,x t ]+b o ) h t =o t ⊙tanh(c t ) Where x t For environment variable timing input, h t is the hidden state, c t is the cell state, f t ,it,ot are the activation values ​​of the forget gate, input gate, and output gate respectively; Establishment of disaster sensitivity warning model: Integrate real-time weather forecasts with historical accident data to build a disaster sensitivity warning model to achieve forward-looking risk prediction and weather-accident correlation analysis: S(t) = ∑i = 1nwi Impact(Mi(t),Hi), where Mi(t) is the real-time weather factor, wind speed, and rainfall, Hi is the accident frequency under the same historical weather conditions, wi is the weight, and S(t) is the disaster sensitivity index; Hierarchical response and system linkage: supports the three-level response mechanism of early warning, alarm and emergency, and links with the dispatching system to realize dynamic risk disposal and resource allocation.

5. The water conservancy and hydropower project construction safety supervision system based on multi-source data fusion according to claim 1 is characterized by: The safety behavior assessment and feedback module includes: Safety performance quantification and early warning rule optimization: Introduce a behavioral scoring model to quantify the safety performance of personnel / teams, set multi-level early warning rules based on risk level thresholds, and clarify the early warning method and individual safety performance score of the target. Where, v i is the number of individual violations, not wearing a helmet, V th,i is the threshold, w i is the weight, the weight of not wearing a helmet is 0.3; Post-incident investigation and real-time early warning linkage: The "Safety Review" function integrates on-site images and sensor tracks to assist in tracing the source of the accident. It also monitors triggers early warnings in real time and links the response process. Risk trend analysis and decision support: The system generates risk trend reports based on historical data, providing data support for the optimization of safety management strategies.

6. The water conservancy and hydropower project construction safety supervision system based on multi-source data fusion according to claim 1 is characterized by: The command, dispatch and visualization module includes: BIM+GIS fusion visualization modeling: BIM and GIS fusion technology is used to display terrain, structure and construction status, build a three-dimensional visualization space model, and realize the spatial mapping of multi-dimensional data. Spatial coordinate mapping: PGIS = T·PBIM, where PBIM is the BIM model coordinate, PGIS is the geographic coordinate, and T is the coordinate transformation matrix, which includes translation, rotation, and scaling parameters; Dynamic risk visualization and trajectory tracking: The visualization interface supports highlighting risk areas and combines personnel positioning with equipment trajectory data to achieve key personnel tracking and equipment operation path playback and trajectory data interpolation: Based on the position P(ti) and P(tj) of the equipment / personnel at time ti and tj, interpolate and calculate the position at any time t; Automatic generation of intelligent dispatch instructions: Based on risk assessment results and construction status, dispatch instructions are automatically generated and pushed to the operator's terminal, realizing closed-loop execution of management decisions.

7. A method for safety supervision of water conservancy and hydropower project construction based on multi-source data fusion, characterized by: The specific steps of the water conservancy and hydropower project construction safety supervision method based on multi-source data fusion are as follows: S1: Multi-source dynamic perception and edge intelligent preprocessing: Deploy millimeter-wave radar equipment to build a three-dimensional perception network. Edge nodes use YOLOv8 and adaptive filtering to process data, and use BeiDou and BIM to calibrate the spatiotemporal reference. S2: Spatiotemporal semantic fusion and risk situation modeling: Use the Transformer architecture to fuse multimodal features, use GNN to build a dynamic knowledge graph to mine hidden relationships, and input the fused data into the digital twin model to deduce risk situation; S3: Intelligent decision-making and closed-loop control execution: Use reinforcement learning to automatically optimize warning thresholds, generate disposal plans through the BIM+GIS fusion interface, and establish a three-level safety performance evaluation system to form closed-loop management.

8. The method for safety supervision of water conservancy and hydropower project construction based on multi-source data fusion according to claim 7 is characterized by: The specific steps of multi-source dynamic perception and edge intelligence preprocessing in S1 are as follows: Step 1: Collaborative multi-dimensional data collection: By deploying millimeter-wave radars, fiber optic sensors, and 5G smart helmets, a three-dimensional "air-ground-air" sensing network is constructed. Distributed fiber optic strain sensors are deployed in high slope areas to capture millimeter-level deformation in real time. UAVs equipped with thermal imagers are used to obtain full-field temperature field data, covering the blind spots of traditional sensors. in, is the state after filtering, K t is the Kalman gain, which adjusts the process noise covariance Qt and the observation noise covariance R in real time through the edge node t , adapt to the dynamic changes of the construction environment; Step 2: Edge Intelligent Preprocessing: Introducing edge computing nodes and AI gateways to perform lightweight processing at the data source. YOLOv8 is used to detect violations and failure to wear protective equipment in real time on video streams. Adaptive Kalman filtering is used to remove outliers from sensor data. Data is compressed by 30%-50% before uploading to the cloud, reducing network transmission pressure. Step 3: Unified calibration of time and space benchmarks: Establish a time and space benchmark network based on Beidou satellites, synchronize all sensors and cameras at the nanosecond level, and unify the spatial data collected by different devices into the construction coordinate system through BIM model coordinate mapping to ensure data time and space consistency.

9. The method for safety supervision of water conservancy and hydropower project construction based on multi-source data fusion according to claim 7, characterized in that: The specific steps of spatiotemporal semantic fusion and risk situation modeling in S2 are as follows: Step 1: Multimodal feature association and fusion: A cross-modal fusion model is constructed using the Transformer architecture. Video features, human behavior, sensor features, equipment vibration, environmental characteristics, and meteorological data are dynamically weighted using a self-attention mechanism. When the crane is detected in operation, the model automatically increases the weight of the vibration sensor data to suppress interference from irrelevant environmental data. Step 2: Dynamic knowledge graph construction: Based on the personnel-equipment-environment-process quadruple, a dynamic knowledge graph for construction safety is constructed. Graph neural networks (GNNs) are used to mine hidden relationships and identify the combined risk probability of "rainy season + high slope excavation + equipment overload"; Step 3: 3D risk situation simulation: The fused data is input into a physics-based digital twin model to simulate the evolution of construction scenario risks in real time. Slope displacement data and rainfall forecasts are input, and the model automatically simulates the impact range of the landslide and marks high-risk areas with red voxels of varying transparency, making the risk "perceptible, predictable, and explainable." 10. The method for safety supervision of water conservancy and hydropower project construction based on multi-source data fusion according to claim 7, characterized in that: The specific steps for executing intelligent decision-making and closed-loop control are as follows: Step 1: Adaptive decision rule generation: Using a reinforcement learning algorithm, PPO, the system automatically optimizes the warning threshold based on historical accident data and response results. The system learns that the probability of crane overturning increases during nighttime and wind speeds > 10 m / s. It automatically lowers the vibration warning threshold by 15% in this scenario to avoid missed alerts. Step 2: Digital twin-driven collaborative response: In the BIM+GIS integrated 3D interface, clicking on a high-risk area automatically generates a response plan. When a Level 1 alert is triggered, the system sends an automatic shutdown command to the construction robot and simultaneously pushes a response process animation to the safety officer via AR glasses, achieving a "warning decision-making-execution" linkage within seconds. Step 3: Quantify safety performance in a closed loop: Establish a three-level safety performance evaluation system for personnel, teams, and projects, convert violation records, response efficiency, and data into visual star ratings, and use blockchain technology to store safety data to form a closed-loop management of "collection-evaluation-improvement", promoting safety management from passive response to active optimization.

Images