Dam safety perception fusion association method based on multi-modal space-time diagram neural network

Abstract

The invention provides a dam safety perception fusion association method based on a multi-modal space-time diagram neural network. The method comprises the following steps: dividing a dam into a plurality of structural units, and mapping various data into a three-dimensional coordinate system; a heterogeneous graph structure is defined, and a dynamic adjacency matrix is calculated based on the real-time stress gradient so as to reflect physical connection, mechanical conduction and geological association relationships among nodes; carrying out fusion modeling on multi-source data in the heterogeneous graph structure by utilizing a multi-modal space-time diagram neural network, constructing a causal inference engine based on an output result of the multi-modal space-time diagram neural network, and updating a three-level modeling system through structural equation modeling, anti-factual inference and dynamic weight to obtain the heterogeneous graph structure. According to the method, the dynamic coupling rule among the dam structure, geology and material states is excavated, cross-modal space-time fusion of manual inspection and sensor monitoring data can be realized, the early recognition capability and early warning accuracy of dam potential safety hazards are improved, and the problems of data islands and insufficient relevance in a traditional monitoring method are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of dam safety monitoring and intelligent early warning, and particularly relates to a dam safety perception fusion correlation method based on a multi-modal spatio-temporal graph neural network. BACKGROUND

[0002] As the core facility of modern water conservancy engineering, dams are widely used in key fields such as flood control, power generation, and water supply. The safety state of the dam is directly related to the stable operation of national infrastructure and the safety of people's lives and property. With the development of smart hydropower technology, dam safety monitoring has gradually evolved from traditional manual inspection to sensor monitoring and intelligent analysis. Currently, it mainly relies on distributed optical fiber sensors, piezometers and other equipment to collect local parameters, and combines unmanned aerial vehicles and manual inspection to obtain image and text information. Specifically, this technical system covers the whole process from data collection, feature extraction to state evaluation, including key links such as structure monitoring, geological analysis, and material performance evaluation. The fusion of multi-source data and dynamic modeling become the core direction to improve the early warning capability.

[0003] However, in the existing dam safety perception method, there is a lack of effective multi-dimensional spatio-temporal correlation analysis mechanism between manual inspection and sensor monitoring data. Directly using isolated modeling methods, a unified spatial semantic framework and dynamic coupling model are not constructed, which may lead to fragmented evaluation of key risk factors such as seepage anomalies and structural cracks, or unable to timely capture the non-explicit correlation of material aging and stress conduction, thereby affecting the timeliness of hidden danger identification and the accuracy of early warning. Furthermore, traditional methods have significant shortcomings in multi-modal data alignment, uncertainty quantification, and causal reasoning, making it difficult to support the active safety management needs of the dam throughout its life cycle. SUMMARY

[0004] The present application aims to at least partially solve one of the technical problems in the related art.

[0005] To achieve the above object, the first aspect of the present application proposes a dam safety perception fusion correlation method based on a multi-modal spatio-temporal graph neural network, comprising: S1, constructing a three-dimensional geological ontology coordinate system, dividing the dam into multiple structural units, and mapping sensor monitoring data, unmanned aerial vehicle inspection image data and artificial inspection text data into the three-dimensional coordinate system; S2, defining a heterogeneous graph structure containing structural unit nodes, sensor nodes and geological unit nodes, and calculating a dynamic adjacency matrix based on real-time stress gradient to reflect the physical connection, mechanical conduction and geological correlation between nodes; S3, using a multi-modal spatio-temporal graph neural network to model the multi-source data in the heterogeneous graph structure, wherein the sensor time series data is modeled by Transformer, the image data is extracted by the improved ResNet-50 to extract the crack texture features, and the text data is analyzed by the graph attention network to analyze the key entity relationship; S4, based on the output result of the multi-modal spatio-temporal graph neural network, constructing a causal reasoning engine, through a three-level modeling system of structural equation modeling, counterfactual reasoning and dynamic weight updating, to mine the dynamic coupling rules between the dam structure, geology and material state.

[0006] In an embodiment of the present application, the three-dimensional geological ontology coordinate system is constructed, the dam is divided into multiple structural units, and the sensor monitoring data, unmanned aerial vehicle inspection image data and artificial inspection text data are mapped into the three-dimensional coordinate system, further comprising: S11, the dam is divided into at least 12 types of structural units, each structural unit contains at least 5 measurement dimensions, including displacement, strain, temperature, seepage pressure and crack width; S12, a distributed fiber sensor array is deployed, with a sampling frequency not less than 200Hz, and sensor data and structural units are spatially aligned through coordinate system mapping.

[0007] In an embodiment of the present application, the heterogeneous graph structure containing structural unit nodes, sensor nodes and geological unit nodes is defined, and a dynamic adjacency matrix is calculated based on real-time stress gradient to reflect the physical connection, mechanical conduction and geological correlation between nodes, further comprising: S21, a geological permeability correction coefficient ρ_geo is introduced in the calculation of the dynamic adjacency matrix A, which is dynamically adjusted according to real-time seepage data and historical geological parameters; S22, the physical connection edge is automatically generated based on the construction drawings and structural design data of the dam, and the mechanical conduction edge is initialized based on the finite element simulation results.

[0008] In an embodiment of the present application, the multi-modal spatio-temporal graph neural network is used to model the fusion of multi-source data in the heterogeneous graph structure, wherein the sensor time series data is time series modeled by a Transformer, the image data is extracted by an improved ResNet-50 to extract crack texture features, and the text data is parsed by a graph attention network to analyze key entity relationships, and further comprising: S31, the improved ResNet-50 model introduces an attention mechanism in the feature extraction process to enhance the sensitivity to crack edges and texture changes; S32, when parsing the artificial inspection text, the graph attention network uses a BERT-based entity recognition model to extract key entities and construct a semantic relationship graph.

[0009] In an embodiment of the present application, further comprising: S5, quantifying the uncertainty of the output results of the multi-modal spatio-temporal graph neural network, using a method combining Monte Carlo Dropout and evidence deep learning to calculate cognitive uncertainty and random uncertainty respectively, and making early warning decisions according to the uncertainty threshold.

[0010] To achieve the above purpose, the second embodiment of the present application proposes a dam safety perception fusion correlation device based on a multi-modal spatio-temporal graph neural network, comprising: a three-dimensional coordinate system construction module for constructing a three-dimensional geological ontology coordinate system, dividing the dam into multiple structural units, and mapping sensor monitoring data, unmanned aerial vehicle inspection image data and artificial inspection text data into the three-dimensional coordinate system; a heterogeneous graph structure definition module for defining a heterogeneous graph structure containing structural unit nodes, sensor nodes and geological unit nodes, and calculating a dynamic adjacency matrix based on real-time stress gradients to reflect the physical connection, mechanical conduction and geological correlation between nodes; a multi-modal fusion modeling module for using a multi-modal spatio-temporal graph neural network to model the fusion of multi-source data in the heterogeneous graph structure, wherein the sensor time series data is time series modeled by a Transformer, the image data is extracted by an improved ResNet-50 to extract crack texture features, and the text data is parsed by a graph attention network to analyze key entity relationships; a causal reasoning construction module for constructing a causal reasoning engine based on the output results of the multi-modal spatio-temporal graph neural network, through a three-level modeling system of structural equation modeling, counterfactual reasoning and dynamic weight updating, to mine the dynamic coupling rules between dam structure, geology and material state.

[0011] The method and device of the embodiments of the present application realize multi-modal spatio-temporal fusion of artificial inspection and sensor monitoring data, and improve the early identification ability and early warning accuracy of dam safety hazards.

[0012] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. Attached Figure Description

[0013] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0014] Figure 1 This is a flowchart of a dam safety perception fusion and association method based on a multimodal spatiotemporal graph neural network according to an embodiment of the present invention;

[0015] Figure 2 This is an architecture diagram of a dam safety perception fusion and association method based on a multimodal spatiotemporal graph neural network according to an embodiment of the present invention;

[0016] Figure 3 This is a structural diagram of a dam safety perception fusion and correlation device based on a multimodal spatiotemporal graph neural network according to an embodiment of the present invention. Detailed Implementation

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0019] The following description, with reference to the accompanying drawings, describes a method and apparatus for dam safety perception fusion and association based on a multimodal spatiotemporal graph neural network, according to an embodiment of the present invention.

[0020] Figure 1 This is a flowchart of a dam safety perception fusion and association method based on a multimodal spatiotemporal graph neural network according to an embodiment of the present invention, such as... Figure 1 As shown, it includes:

[0021] S1. Construct a three-dimensional geological body coordinate system, divide the dam into multiple structural units, and map sensor monitoring data, UAV inspection image data, and manual inspection text data into the three-dimensional coordinate system.

[0022] Specifically, the core of this step is to construct a three-dimensional geological ontological coordinate system and to uniformly map multi-source heterogeneous dam safety perception data (including sensor monitoring data, UAV inspection image data and manual inspection text data) into this coordinate system, thereby realizing data fusion and correlation analysis in spatial and temporal dimensions.

[0023] At the technical implementation level, the construction of the three-dimensional geological ontological coordinate system is based on the dam's BIM (Building Information Modeling) or GIS (Geographic Information System) model, combined with its geological exploration data, to establish a three-dimensional spatial reference framework based on the dam's geometric structure and supported by its geological attributes. This coordinate system typically adopts a Local Cartesian Coordinate System, with the dam's foundation point as the origin, the X-axis along the dam's axis, the Y-axis perpendicular to the dam's axis, and the Z-axis vertical, ensuring the uniqueness and computability of the spatial coordinates. The dam is divided into N structural units, each with clearly defined geometric boundaries and physical properties (such as material type, thickness, stress distribution, etc.), facilitating subsequent localized modeling and analysis of the data.

[0024] In terms of parameter indicators, the structural unit division must meet spatial resolution requirements, and the unit size is usually no more than 1m. 3 To ensure high-precision modeling of critical components (such as dam cracks and seepage channels), sensor data must have a sampling frequency of ≥100Hz to capture dynamic changes; image data must use an improved ResNet-50 model to extract crack texture features and perform registration between image coordinates and the 3D coordinate system, with the registration error controlled within ±5cm; text data is analyzed using a graph attention network (GAT) to resolve key entity relationships, such as "crack location-time-severity," forming a semantic graph.

[0025] At the application level, this step is widely used in safety monitoring systems for complex structures such as concrete gravity dams and arch dams. For example, during the flood season or after an earthquake, the system can map real-time sensor data, high-resolution images taken by drones, and text information from manual inspection reports into a three-dimensional coordinate system to form a multimodal space-time data field, providing structured input for subsequent graph neural network modeling.

[0026] The technical benefits of this step are: to achieve spatial alignment and semantic fusion of multi-source data, break down traditional "data silos", provide a unified data foundation for subsequent spatiotemporal graph neural network modeling, and thus improve the perception accuracy and early warning timeliness of dam safety status.

[0027] Furthermore, S1 includes:

[0028] S11 divides the dam into at least 12 structural units, each containing at least 5 measurement dimensions, including displacement, strain, temperature, seepage pressure, and crack width.

[0029] Specifically, this step aims to achieve refined modeling and multi-source data fusion of the dam's structural state by dividing the dam into at least 12 structural units and configuring at least 5 measurement dimensions (including displacement, strain, temperature, seepage pressure, and crack width) for each unit. In some implementations, the division of structural units is based on the dam's geometry, material properties, stress zones, and geological conditions to ensure that each unit is representative in terms of mechanical behavior and monitoring requirements. For example, a concrete gravity dam can be divided into dam body sections (such as the foundation, heel, toe, crest, and spillway), with each section further subdivided into multiple units based on its structural complexity, totaling no fewer than 12 units, to cover stress concentration areas, seepage-sensitive areas, and crack-prone areas in key locations.

[0030] Each structural unit requires the collection of five key monitoring parameters: displacement is measured with millimeter-level accuracy using GNSS or a total station, with a sampling frequency of no less than 100Hz; strain is achieved using a fiber optic grating (FBG) sensor with an accuracy of ±1με; temperature is acquired by a distributed temperature sensing (DTS) system with a spatial resolution of 1m and an accuracy of ±0.5℃; seepage pressure is measured using a piezometer with a range of 0–500kPa and an accuracy of ±1%FS; and crack width is extracted using a high-resolution camera mounted on a UAV combined with image processing algorithms (such as an improved ResNet-50) with an accuracy of up to 0.1mm. These parameters collectively constitute a multidimensional feature vector, which is used for subsequent spatiotemporal graph neural network modeling.

[0031] This step is applicable to the full life-cycle safety monitoring of dams, especially during flood season, after earthquakes, or during material aging stages. Through high-density, multi-dimensional data acquisition, it enables real-time perception and anomaly identification of the structural condition. For example, in the coupled analysis of seepage pressure anomalies and crack propagation, this classification method can effectively capture the differences in local and overall mechanical responses, improving the spatiotemporal resolution of early warning.

[0032] This step, through structured partitioning and multi-dimensional parameter acquisition, constructs a monitoring data foundation with spatial semantics, providing high-quality input for subsequent graph neural network modeling and causal inference, and significantly improving the accuracy and systematic nature of dam safety status perception.

[0033] S12 is deployed using a distributed fiber optic sensor array with a sampling frequency of no less than 200Hz, and spatial alignment of sensor data with structural units is achieved through coordinate system mapping.

[0034] Specifically, in the specific implementation steps of this invention, a distributed fiber optic sensor array is deployed, and spatial alignment of sensor data with structural units is achieved through coordinate system mapping. This is a fundamental step in constructing a multimodal spatiotemporal graph neural network (MSTGNN) and plays a crucial technical supporting role. The core of this step lies in acquiring dynamic response data of the dam structure through a high-density, high-precision sensor network and spatially mapping it with structural units in a three-dimensional geological ontological coordinate system, thereby achieving spatiotemporal consistency fusion of multi-source heterogeneous data.

[0035] Distributed fiber optic sensor arrays typically employ OTDR (Optical Time Domain Reflectometry) or OFDR (Optical Frequency Domain Reflectometry) based sensing technologies. By laying sensing fibers along key parts of the dam (such as the dam body, foundation, and cutoff wall), continuous monitoring of multi-dimensional physical parameters such as temperature, strain, and vibration is achieved. In some implementations, the sensor spacing can be controlled between 0.5m and 2m to ensure spatial resolution meets the requirements for structural health monitoring. The sampling frequency is set to no less than 200Hz to capture transient response characteristics, such as seismic waves and stress fluctuations caused by sudden changes in water level, conforming to the sampling requirements for dynamic structural monitoring in the IEEE 1451 standard.

[0036] The sampling frequency of sensor data directly affects the accuracy and response speed of time-series modeling. A sampling frequency of 200Hz can effectively cover the dynamic frequency range of the dam structure under typical operating conditions (usually 0.1Hz to 100Hz), ensuring data integrity and continuity. Simultaneously, a three-dimensional geological ontological coordinate system needs to be established during spatial alignment. This coordinate system is typically built based on BIM (Building Information Modeling) or GIS (Geographic Information System), with millimeter-level accuracy required to support accurate modeling of adjacency relationships between nodes in subsequent graph neural networks.

[0037] This step is applicable to real-time safety monitoring of complex structures such as concrete gravity dams and arch dams. For example, deploying fiber optic arrays in the dam foundation area, combined with drone inspection images and manual inspection text records, can enable early identification of potential hazards such as crack propagation and seepage changes. Through coordinate system mapping, sensor nodes and structural units achieve a one-to-one correspondence in the spatial dimension, providing structured input for subsequent graph attention networks (GAT) to extract spatial dependencies.

[0038] The technical advantage of this step lies in its ability to effectively solve the problems of data silos and spatial alignment errors in traditional monitoring methods by deploying sensors with high sampling frequency and high spatial resolution, combined with a precise spatial mapping mechanism. Its innovation lies in dynamically associating continuous distributed sensor data with discrete structural units, providing a reliable spatial benchmark for multimodal data fusion and causal inference, thereby significantly improving the accuracy of dam safety status prediction and the timeliness of early warning.

[0039] S2 defines a heterogeneous graph structure containing structural unit nodes, sensor nodes, and geological unit nodes, and calculates a dynamic adjacency matrix based on real-time stress gradient to reflect the physical connections, mechanical transmission, and geological relationships between the nodes.

[0040] Specifically, this step aims to construct a heterogeneous graph structure containing structural unit nodes, sensor nodes, and geological unit nodes, and calculate a dynamic adjacency matrix based on real-time stress gradients to comprehensively characterize the physical connections, mechanical transmission, and geological relationships among various elements within the dam. In some implementations, this heterogeneous graph structure is defined as G = (V, E, A), where the node set V includes three types of entities: structural unit nodes (such as concrete blocks, supporting structures, etc.), sensor nodes (such as distributed fiber optic sensors, piezometers, etc.), and geological unit nodes (such as rock layers, fault zones, seepage channels, etc.). The edge set E consists of three types of edges: physical connection edges (representing the geometric connections between structural units), mechanical transmission edges (representing the transmission paths of mechanical parameters such as stress and strain), and geological relationship edges (representing the spatial or functional relationships between geological units and structures or sensors).

[0041] At the parameter level, the construction of the dynamic adjacency matrix A depends on the calculation of the real-time stress gradient σ. Specifically, the stress distribution of each structural element is obtained through finite element analysis or distributed fiber optic sensing data, and the stress gradient between adjacent elements is calculated as the weight of the mechanical transmission edge. The geological permeability correction coefficient ρ... geo This is used to adjust the connection strength between geological unit nodes, and its value range is usually between [0,1], calibrated based on geological exploration data and historical seepage models. In practical applications, this graph structure can be deployed on edge computing nodes or cloud servers to receive data streams from sensor networks and inspection systems in real time, enabling dynamic updates.

[0042] From a technical perspective, this step uses graph neural networks (GNNs) to model the complex spatial topology of the dam, mapping multi-source heterogeneous data to a unified graph representation space. This enhances the model's ability to understand the coupling relationships between structural, geological, and monitoring data. Its core value lies in breaking down traditional "data silos," achieving structured fusion of multi-dimensional data, and providing a high-fidelity, dynamically updated graph topology foundation for subsequent time-series modeling and anomaly detection. This significantly improves the accuracy of dam safety status perception and the timeliness of early warnings.

[0043] Furthermore, S2 includes:

[0044] S21, the calculation of the dynamic adjacency matrix A introduces a geological permeability correction coefficient ρ_geo, which is dynamically adjusted based on real-time seepage data and historical geological parameters.

[0045] Specifically, in the construction of the dynamic adjacency matrix A of this invention, a geological permeability correction coefficient ρ_geo is introduced to accurately model the dynamic coupling relationship between the seepage state inside the dam structure and geological parameters. This correction coefficient ρ_geo is not a fixed constant, but is dynamically adjusted based on real-time seepage monitoring data and historical geological parameters (such as permeability coefficient, porosity, rock structure, etc.), thereby enhancing the graph neural network's ability to perceive seepage anomalies inside the dam.

[0046] In some implementations, ρ_geo is calculated based on time-series data from seepage sensors (sampling frequency ≥ 100Hz) output by a multi-source data fusion module and historical permeability distributions in a geological ontology database. Specifically, by constructing a regression model between seepage state and geological parameters, a sliding time window (e.g., a window length of 24 hours) is used to fit the relationship between the seepage pressure change rate ΔP and the geological permeability K online, thereby dynamically updating the value of ρ_geo. The value range of ρ_geo is typically limited to [0.8, 1.2] to reflect the nonlinear response characteristics of geological conditions under different operating conditions.

[0047] Furthermore, the update formula for the dynamic adjacency matrix A is: Where A_base is a static adjacency matrix based on physical connectivity. This represents the element-wise multiplication operation. The introduction of ρ_geo(t) enables the graph structure to reflect the impact of changes in geological permeability on the seepage path in real time, thereby improving the model's accuracy in identifying potential risks such as seepage failure and leakage anomalies.

[0048] In practical applications, this step is particularly suitable for seepage-sensitive structures such as concrete gravity dams or earth-rock dams. Under extreme conditions such as sudden drops in water level or concentrated rainfall, by dynamically adjusting ρ_geo, the model can more accurately capture the spatial diffusion characteristics and temporal evolution trend of the seepage field, providing crucial support for early warning of seepage anomalies.

[0049] In terms of technical effects, the introduction of ρ_geo significantly improves the adaptability of graph neural networks to geological heterogeneity, reduces modeling bias caused by fixed geological parameter assumptions, and enhances the spatiotemporal consistency of multimodal data fusion, thereby improving the accuracy and robustness of dam safety state prediction.

[0050] S22, the physical connection edge is automatically generated based on the dam's construction drawings and structural design data, and the mechanical transmission edge is initialized based on the finite element simulation results.

[0051] Specifically, in the steps of this invention, "the physical connection edge is automatically generated based on the dam's construction drawings and structural design data, and the mechanical transmission edge is initialized based on the finite element simulation results" is a key foundational step in constructing a multimodal spatiotemporal graph neural network (MSTGNN). Its technical implementation involves multiple dimensions such as structural modeling, graph representation learning, and simulation data fusion.

[0052] At the technical implementation level, the generation of physical connection edges relies on the dam's BIM (Building Information Modeling) or CAD construction drawings. Through the geometric topological relationships in the structural design data, the dam is divided into N structural units, and adjacency relationships between these units are established. This process typically employs structural meshing technology, combined with the API interfaces of modeling software such as AutoCAD or Revit, to extract connection information between units and construct the edge set E_phys in the graph structure. The edge weights can be quantified based on the contact area between units, material stiffness, or connection type (such as rebar connections or concrete pouring joints) to reflect the degree of physical coupling of the structure.

[0053] The initialization of mechanical transmission edges is based on finite element simulation results. Simulation platforms such as ANSYS, ABAQUS, or COMSOL are used to obtain mechanical response data of the dam under different working conditions, including stress, strain, and displacement. In some implementations, the nodal force transmission paths in the simulation results can be mapped to the edge set E_mech in the graph structure. The edge weights are determined by the stress gradient or strain energy density between nodes, and are usually normalized to ensure the values ​​are within the [0,1] interval. For example, a stress gradient threshold σ_thres = 0.5 MPa / m can be set. Mechanical transmission edges are only established when the stress change between adjacent elements exceeds this threshold, thereby improving the sparsity and computational efficiency of the graph structure.

[0054] At the parameter level, the generation of physical connection edges must meet the following requirements: structural unit partitioning accuracy ≥ 0.1m, and edge density controlled between 0.8 and 1.2, to avoid excessively sparse or redundant graph structures. The initialization of mechanical transmission edges must ensure that the simulation model's mesh size ≤ 0.5m and time step Δt ≤ 1h, to guarantee the temporal and spatial resolution of the mechanical response. Simultaneously, the update frequency of the dynamic adjacency matrix A should be synchronized with the monitoring data, typically set to once per hour.

[0055] At the application level, this step is suitable for health status modeling of complex structures such as concrete gravity dams and arch dams, playing a particularly crucial role in multi-source data fusion and anomaly propagation path identification. By mapping the physical structure and mechanical response to a graph structure, it provides structured input for subsequent spatiotemporal graph neural network modeling, thereby enabling dynamic perception and prediction of the overall safety status of the dam.

[0056] The technical effect of this step is that, through structured modeling and simulation data-driven graph initialization, the modeling accuracy of graph neural networks for the spatial structure and mechanical behavior of dams is significantly improved, providing a reliable topological foundation for multimodal data fusion and causal inference, and enhancing the model's sensitivity to structural anomalies and interpretability.

[0057] S3, a multimodal spatiotemporal graph neural network is used to fuse and model the multi-source data in the heterogeneous graph structure. Specifically, sensor time-series data is modeled temporally using a Transformer, image data has crack texture features extracted using an improved ResNet-50, and text data has key entity relationships parsed using a graph attention network.

[0058] Specifically, in this invention, the step of "using a multimodal spatiotemporal graph neural network to fuse and model the multi-source data in the heterogeneous graph structure" is the core link in realizing intelligent perception of dam safety status. This step achieves multi-dimensional and multimodal modeling of the dam structural status by constructing a heterogeneous graph neural network architecture that fuses sensor time-series data, image data, and text data, thereby improving the accuracy and real-time performance of safety assessment.

[0059] This method first abstracts the physical structure of the dam into a heterogeneous graph structure G = (V, E, A), where the node set V includes structural unit nodes, sensor nodes, and geological unit nodes, and the edge set E covers physical connection edges, mechanical transmission edges, and geologically related edges. The dynamic adjacency matrix A is updated based on the real-time stress gradient to reflect the dynamic changes in the structural state. Sensor time-series data is modeled using a Transformer model, whose self-attention mechanism can capture long-period dependencies, making it suitable for high-frequency (≥100Hz) multi-dimensional (dimension d≥5) time-series signals acquired by distributed fiber optic sensors. For image data, an improved ResNet-50 network structure is adopted, introducing dilated convolution and multi-scale feature fusion modules to enhance the extraction capability of crack texture features, especially suitable for identifying tiny cracks in UAV inspection images. Text data is semantically modeled using a Graph Attention Network (GAT) to extract key entities and their relationships from the inspection report, constructing a semantic graph to supplement the semantic information of the structural state.

[0060] Regarding parameters, the Transformer model has a hidden layer dimension of 512, 8 attention heads, and a maximum sequence length of 256. The improved version of ResNet-50 introduces a hole ratio d=2 in stage 4 to expand the receptive field. The attention coefficients of GAT are calculated using the LeakyReLU activation function, with a node feature dimension of 256 and 4 attention heads. These parameter settings ensure that the model possesses good expressive power and generalization performance when handling high-dimensional, heterogeneous data.

[0061] In practical applications, this step is suitable for real-time safety assessment of complex structures such as concrete gravity dams and arch dams. It is particularly effective in early warning scenarios such as abnormal seepage and crack propagation, as it can effectively integrate multi-source data to improve the timeliness and accuracy of hazard identification. Through multimodal modeling, the system can achieve dynamic perception and causal reasoning of the structural state, providing data-driven decision support for the entire lifecycle management of dams.

[0062] This step significantly improves the efficiency of multi-source data fusion, reduces feature alignment errors, and enhances the model's adaptability to complex operating conditions. Simultaneously, by introducing an attention mechanism, the model possesses good interpretability, helping to identify key factors affecting safety status, thereby improving the reliability and engineering applicability of the prediction results.

[0063] Furthermore, S3 includes:

[0064] S31, the improved ResNet-50 model introduces an attention mechanism during feature extraction to enhance sensitivity to crack edges and texture variations.

[0065] Specifically, in this invention, the improved ResNet-50 model incorporates an attention mechanism during feature extraction to enhance the model's sensitivity to crack edges and texture variations, thereby improving the accuracy and robustness of dam surface defect identification. This step belongs to the image feature extraction submodule within the multimodal data fusion module, primarily processing high-resolution image data acquired by UAV inspections.

[0066] At the technical implementation level, this invention inserts a Multi-Channel Attention Module (MCAM) after the convolutional layers of ResNet-50. This module consists of two parts: Channel Attention and Spatial Attention. Channel Attention generates channel weights through Global Average Pooling (GAP) and fully connected layers, emphasizing channels that respond strongly to crack features. Spatial Attention extracts spatial feature maps through convolutional operations and generates spatial weight maps using the Sigmoid function, highlighting the spatial distribution of crack regions. Furthermore, optionally, an SENet (Squeeze-and-Excitation Network) structure can be connected after the fourth residual block of ResNet-50 to adaptively adjust the importance of feature channels and improve the model's ability to perceive subtle texture changes.

[0067] Regarding parameter metrics, in the channel weight calculation of the attention module, the channel compression ratio of the fully connected layer is set to 16, meaning that if the number of input channels is C, then the number of intermediate layer channels is C / 16. The spatial attention module uses 3×3 convolutional kernels for feature extraction with a stride of 1 and padding of 1 to maintain consistent feature map size. The input image size of the ResNet-50 model is uniformly 224×224×3, and image preprocessing follows the ImageNet standard (mean [0.485, 0.456, 0.406], standard deviation [0.229, 0.224, 0.225]) to ensure the model's generalization ability.

[0068] In application scenarios, this improved model is deployed in the UAV inspection image processing workflow to extract the texture and edge features of cracks on the dam surface in real time. Combined with time-series data from distributed fiber optic sensors, it enables dynamic correlation analysis between crack development and structural stress changes, improving early warning capabilities.

[0069] The technical effect of this step is that by introducing an attention mechanism, the model's ability to perceive crack edges and texture changes is significantly enhanced, the interference of background noise is reduced, and the focus and discriminative power of feature expression are improved, thus providing high-quality visual feature input for subsequent multimodal fusion and anomaly detection.

[0070] S32, when parsing manually inspected text, the graph attention network uses a BERT-based entity recognition model to extract key entities and construct a semantic relationship graph.

[0071] Specifically, in this invention, the Graph Attention Network (GAN) extracts key entities and constructs a semantic relationship graph when parsing manually inspected text. This is one of the core steps in achieving multi-source data fusion and intelligent perception. This step, through the synergy of Natural Language Processing (NLP) and Graph Neural Networks (GNNs), transforms unstructured text into a structured semantic graph, thereby enhancing the semantic understanding and causal reasoning capabilities of the dam's safety status.

[0072] At the technical implementation level, firstly, the BERT-based entity recognition model employs a pre-training + fine-tuning strategy, using a BERT model pre-trained on a Chinese corpus (such as BERT-Base-Chinese) as its basic architecture. The model is fine-tuned on a manually inspected text dataset, which includes labeled dam structural entities (such as "cracks," "seepage points," and "concrete spalling"), environmental parameters (such as "rainfall" and "temperature"), and operational behaviors (such as "inspection" and "reinforcement"). During entity recognition, the model captures contextual semantics through a bidirectional Transformer structure and combines CRF (Conditional Random Field) or Span-based methods to jointly decode entity boundaries and types, achieving an accuracy rate of over 92% (F1-score ≥ 0.92).

[0073] Furthermore, the identified key entities are mapped to a predefined ontology structure, forming a node set V. The graph attention network then calculates the semantic relationship weights between entities through a self-attention mechanism, constructing an edge set E. Specifically, node feature vectors are aggregated through a multi-head attention mechanism, and the weight of each edge is jointly calculated by the semantic similarity between entities and the context co-occurrence frequency. The attention coefficient α_ij = softmax(LeakyReLU(a^T[W h_i||Wh_j])), where a is a learnable attention parameter, W is the feature transformation matrix, and h_i and h_j are the embedding vectors of entities i and j, respectively.

[0074] In application scenarios, this step is mainly used to process textual information in manual inspection reports, such as the location of cracks, water seepage, and structural anomalies recorded by inspectors. Through entity recognition and graph construction, the system can make implicit information in the text explicit, providing structured input for subsequent graph neural network modeling, thereby achieving semantic alignment and fusion of multimodal data.

[0075] The technical value of this step lies in its ability to effectively extract key information from inspection texts and establish semantic relationships through BERT's semantic understanding capabilities and the spatial modeling capabilities of graph attention mechanisms. This significantly improves the accuracy and interpretability of dam safety status perception, laying a solid foundation for subsequent causal reasoning and anomaly early warning.

[0076] S4. Based on the output of the multimodal spatiotemporal graph neural network, a causal reasoning engine is constructed. Through a three-level modeling system of structural equation modeling, counterfactual reasoning, and dynamic weight update, the dynamic coupling law between the dam structure, geology, and material state is explored.

[0077] Specifically, this step aims to construct an engine with causal reasoning capabilities based on the output of a multimodal spatiotemporal graph neural network (MSTGNN) to deeply explore the dynamic coupling patterns between dam structure, geological conditions, and material states. This causal reasoning engine employs a three-tiered modeling system: structural equation modeling (SEM), counterfactual reasoning, and dynamic weight updating, to achieve a leap from correlation to causation analysis.

[0078] In some implementations, structural equation modeling is used as the first-level modeling approach to extract potential causal paths between variables from historical monitoring and inspection data. By constructing a causal graph model that includes key variables such as structural unit displacement, seepage pressure, crack width, and material elastic modulus, and combining it with maximum likelihood estimation or Bayesian structural learning algorithms (such as GES and PC algorithms), quantitative modeling of causal relationships between variables is achieved. The path coefficients between variables in the model must meet the significance test (p < 0.05), and the AIC or BIC criteria are used for model selection to ensure the rationality and stability of the causal structure.

[0079] Furthermore, the second-level modeling introduces a counterfactual reasoning mechanism to identify the root causes of specific anomalous events (such as abnormal seepage or crack propagation). By constructing an intervention model (do-calculus) and a counterfactual hypothesis space, combined with the high-dimensional feature vector output by MSTGNN, the counterfactual results under the assumed conditions are calculated, thereby locating the root cause of the anomaly. For example, if the seepage pressure abnormally increases at a certain moment, counterfactual reasoning can assess whether this pressure change would still occur without changes in geological permeability or material degradation, thus achieving causal attribution.

[0080] Optionally, the third-level modeling continuously optimizes the path weights in the causal graph through a dynamic weight update mechanism. This mechanism, based on the uncertainty assessment results output by MSTGNN (such as an uncertainty index combining Monte Carlo Dropout and evidence-based deep learning), employs online learning strategies (such as online gradient descent or Bayesian updates) to adjust the edge weights in the causal graph, ensuring the model can adapt to the time-varying characteristics of the dam's structural state. The weight update frequency can be set according to the sampling period of the monitoring data (such as hourly or daily) to maintain the model's timeliness and accuracy.

[0081] This step plays a crucial role in the overall technical solution. By using a causal inference engine, it transforms the prediction results of MSTGNN into interpretable causal relationships, which not only improves the credibility of dam safety status prediction but also provides a scientific basis for hazard tracing and risk intervention. Its technical value lies in: realizing causal modeling of multi-source heterogeneous data, breaking through the limitations of traditional correlation analysis; supporting dam status assessment under dynamic evolution, and enhancing the system's adaptability to complex operating conditions.

[0082] Furthermore, it also includes S5, which quantifies the uncertainty of the output of the multimodal spatiotemporal graph neural network by using a combination of Monte Carlo Dropout and evidence deep learning to calculate cognitive uncertainty and random uncertainty respectively, and makes early warning decisions based on the uncertainty threshold.

[0083] Specifically, in the uncertainty quantification step of this invention, a method combining Monte Carlo Dropout (MCD) and Bayesian Evidence Deep Learning (EDL) is used to perform dual uncertainty assessment on the output of the Multimodal Spatiotemporal Graph Neural Network (MSTGNN), thereby improving the reliability and interpretability of dam safety early warning. In some implementations, this step first enables a random sampling mechanism in the Dropout layer of the neural network, performing T forward propagations (typically T≥50) during the inference phase, randomly discarding some neurons in each propagation to simulate the posterior distribution of the model parameters, thereby estimating epistemic uncertainty, i.e., the model's ability to perceive uncertainty in the input data. Furthermore, combined with the evidence deep learning framework, the output probability is modeled by introducing a Dirichlet distribution to calculate aleatoric uncertainty, reflecting the noise or irreducibility uncertainty of the data itself.

[0084] In practical applications, this step is deployed on the edge computing nodes or cloud servers of the dam safety monitoring system to perform real-time uncertainty assessment of the structural state prediction results output by MSTGNN. For example, in the detection of seepage anomalies in concrete gravity dams, if the cognitive uncertainty exceeds a set threshold (e.g., 0.15), it indicates that the model's judgment of the current state has a large fluctuation in confidence, requiring the triggering of an expert review mechanism; if the random uncertainty is higher than 0.2, it indicates that the input data contains significant noise or anomalies, and data cleaning or sensor calibration procedures should be initiated.

[0085] The technical value of this step lies in effectively identifying the credibility boundary of prediction results by quantifying the uncertainty of model predictions. This allows for the introduction of a dynamic threshold mechanism in early warning decision-making, avoiding false alarms and missed alarms. In dam safety monitoring scenarios, this method can significantly improve early risk identification capabilities, provide more evidence-based decision support for operation and maintenance personnel, and enhance the robustness and adaptability of the system.

[0086] The dam safety perception fusion and association method based on multimodal spatiotemporal graph neural network of this invention improves the efficiency of multi-source data fusion and feature alignment accuracy, enhances the early warning capability of dam safety hazards, realizes intelligent correlation analysis and dynamic modeling of sensor monitoring and inspection data, and improves the accuracy of safety assessment and the credibility of decision-making.

[0087] Furthermore, the method of the embodiments of the present invention will be described in conjunction with the accompanying drawings. To achieve the above functions, the present invention adopts the following technical architecture design:

[0088] To address the shortcomings of existing methods and resolve issues such as data silos and lack of correlation, this invention designs an innovative solution to address the lack of correlation between manual inspection and sensor monitoring. It combines these two data sources using data fusion, machine learning, and deep learning techniques. This combines structured sensor data with unstructured inspection records. Since inspection records are typically text, images, or videos, natural language processing and computer vision techniques are used for parsing.

[0089] The specific implementation involves designing a multimodal spatiotemporal graph neural network that combines time-series data from sensors and multimodal data (text and images) from inspections. The physical structure of the dam is represented using a graph structure, and time-series modeling is performed using Transformer. Contrastive learning is employed to enhance the model's robustness. Simultaneously, uncertainty quantification is introduced to improve prediction reliability, and historical cases are linked through a knowledge graph to enhance early warning capabilities.

[0090] In terms of multimodal learning, a graph neural network is used to integrate data from different sources into a single model, meeting the requirements of multimodal learning through data fusion. The dam's structure is represented as a graph, with sensors as nodes and connections as edges, to better capture spatial relationships.

[0091] For dam safety status prediction, potential fault analysis, and anomaly detection, it is necessary to combine the time series prediction model Transformer with multimodal data.

[0092] Regarding the data labeling issue, expert evaluations contained in the inspection records are used as labels to train the model. Semi-supervised learning is employed, depending on the amount of labeled data.

[0093] In addition, this method incorporates an attribution function and an attention mechanism, enabling the model to explain which factors led to specific prediction results.

[0094] Meanwhile, this method system can process data in real time, and accelerate response speed by utilizing edge computing or distributed processing.

[0095] In one embodiment of the present invention, a dam safety perception method based on a multimodal spatiotemporal graph neural network (MSTGNN) is proposed, such as... Figure 2 As shown, it may include the following steps:

[0096] Establish a multimodal data fusion module:

[0097] A three-dimensional geological ontological coordinate system is constructed, dividing the dam into N structural units. Each unit integrates distributed fiber optic sensor time-series data, ensuring a sampling frequency ≥100Hz and a measurement dimension d≥5. For the visual features of UAV inspection, an improved ResNet-50 model is used to extract crack texture features. For the semantic graph of manual inspection, the key entity relationships in the inspection report are analyzed through a graph attention network.

[0098] Building a spatiotemporal graph neural network architecture:

[0099] Define a heterogeneous graph structure G = (V, E, A), where the node set V contains three types of entities: structural unit nodes, sensor nodes, and geological unit nodes; the edge set E contains three types of edges: physical connection edges, mechanical transmission edges, and geologically related edges; and the dynamic adjacency matrix A is calculated based on real-time stress gradients.

[0100]

[0101] In the formula, ρ_geo represents the geological permeability correction coefficient.

[0102] Uncertainty quantification algorithm:

[0103] A dual uncertainty assessment method combining Monte Carlo Dropout and evidence-based deep learning is employed:

[0104] Cognitive uncertainty calculation:

[0105]

[0106] Calculation of stochastic uncertainty:

[0107]

[0108] Where K is the number of state categories and T is the number of Monte Carlo samplings.

[0109] Design a causal reasoning engine:

[0110] A three-level causal reasoning model is constructed. First, causal relationships in historical data are modeled based on structural equation modeling. Then, counterfactual reasoning is applied to locate the root causes of anomalies. Finally, the weights of the causal graph are dynamically updated.

[0111] The algorithm for locating the root cause of anomalies using counterfactual reasoning is as follows:

[0112]

[0113] The algorithm for dynamically updating cause-effect graph weights is as follows:

[0114]

[0115] One possible implementation is as follows: For example, in the safety assessment of concrete gravity dams, firstly, a distributed fiber optic sensor array is deployed, with a sampling frequency of 200Hz; then, the drone inspection path planning parameters are configured, such as a flight altitude of 50m and an overlap rate of 80%; finally, the spatiotemporal graph neural network parameters are initialized, including the number of spatial attention heads, the temporal convolutional layer dilation factor d = 2^l (where l is the number of layers), the training period, and the uncertainty threshold. This can improve the efficiency of multi-source data fusion, reduce feature alignment errors, enhance early warning capabilities (seepage anomaly identification time is advanced by tens of hours), and improve decision-making reliability.

[0116] In summary, this method constructs a three-dimensional ontological coordinate system that integrates sensor time-series data, multimodal inspection records, and geological ontological features; designs a spatiotemporal graph neural network with a dynamic adjacency matrix; and implements dual uncertainty quantification by combining Monte Carlo Dropout with evidence-based deep learning.

[0117] To achieve the above embodiments, such as Figure 3 As shown, this embodiment also provides a dam safety perception fusion and correlation device 10 based on a multimodal spatiotemporal graph neural network, including:

[0118] The three-dimensional coordinate system construction module 100 is used to construct a three-dimensional geological body coordinate system, divide the dam into multiple structural units, and map sensor monitoring data, UAV inspection image data and manual inspection text data to the three-dimensional coordinate system.

[0119] The heterogeneous graph structure definition module 200 is used to define a heterogeneous graph structure that includes structural unit nodes, sensor nodes and geological unit nodes, and calculates a dynamic adjacency matrix based on real-time stress gradient to reflect the physical connection, mechanical transmission and geological correlation between each node.

[0120] The multimodal fusion modeling module 300 is used to fuse and model multi-source data in the heterogeneous graph structure using a multimodal spatiotemporal graph neural network. The sensor time series data is modeled temporally using Transformer, the image data is extracted with crack texture features using an improved ResNet-50, and the text data is parsed with a graph attention network to resolve key entity relationships.

[0121] The causal reasoning construction module 400 is used to construct a causal reasoning engine based on the output of the multimodal spatiotemporal graph neural network. Through a three-level modeling system of structural equation modeling, counterfactual reasoning, and dynamic weight update, it explores the dynamic coupling law between the dam structure, geology, and material state.

[0122] Furthermore, the three-dimensional coordinate system construction module is also used for:

[0123] The dam is divided into at least 12 structural units, each containing at least 5 measurement dimensions, including displacement, strain, temperature, seepage pressure, and crack width.

[0124] A distributed fiber optic sensor array is deployed with a sampling frequency of no less than 200Hz, and spatial alignment of sensor data with structural units is achieved through coordinate system mapping.

[0125] Furthermore, the heterogeneous graph structure definition module is also used for:

[0126] In the calculation of the dynamic adjacency matrix A, a geological permeability correction coefficient ρ_geo is introduced, and ρ_geo is dynamically adjusted according to real-time seepage data and historical geological parameters.

[0127] The physical connection edges are automatically generated based on the dam's construction drawings and structural design data, while the mechanical transmission edges are initialized based on finite element simulation results.

[0128] Furthermore, the multimodal fusion modeling module is also used for:

[0129] The improved ResNet-50 model introduces an attention mechanism during feature extraction to enhance sensitivity to crack edges and texture variations.

[0130] When the graph attention network parses the manually inspected text, it uses a BERT-based entity recognition model to extract key entities and construct a semantic relationship graph.

[0131] Furthermore, it also includes:

[0132] The uncertainty quantification module is used to quantify the uncertainty of the output results of the multimodal spatiotemporal graph neural network. It adopts a method combining Monte Carlo Dropout and evidence deep learning to calculate cognitive uncertainty and random uncertainty respectively, and makes early warning decisions based on the uncertainty threshold.

[0133] According to embodiments of the present invention, a dam safety perception fusion and correlation device based on a multimodal spatiotemporal graph neural network improves the efficiency of multi-source data fusion and feature alignment accuracy, enhances the early warning capability of dam safety hazards, realizes intelligent correlation analysis and dynamic modeling of sensor monitoring and inspection data, and improves the accuracy of safety assessment and the credibility of decision-making.

[0134] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0135] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A dam safety perception fusion and association method based on a multimodal spatiotemporal graph neural network, characterized in that, include: S1. Construct a three-dimensional geological body coordinate system, divide the dam into multiple structural units, and map sensor monitoring data, UAV inspection image data, and manual inspection text data into the three-dimensional coordinate system; S2 defines a heterogeneous graph structure containing structural unit nodes, sensor nodes, and geological unit nodes, and calculates a dynamic adjacency matrix based on real-time stress gradient to reflect the physical connection, mechanical transmission, and geological correlation between nodes. S3, Multimodal spatiotemporal graph neural network is used to fuse and model the multi-source data in the heterogeneous graph structure. The sensor time series data is modeled temporally using Transformer, the image data is extracted with crack texture features using an improved ResNet-50, and the text data is parsed with a graph attention network to resolve key entity relationships. S4. Based on the output of the multimodal spatiotemporal graph neural network, a causal reasoning engine is constructed. Through a three-level modeling system of structural equation modeling, counterfactual reasoning, and dynamic weight update, the dynamic coupling law between the dam structure, geology, and material state is explored.

2. The method as described in claim 1, characterized in that, The construction of a three-dimensional geological coordinate system, which divides the dam into multiple structural units and maps sensor monitoring data, UAV inspection image data, and manual inspection text data to the three-dimensional coordinate system, also includes: S11 divides the dam into at least 12 structural units, each containing at least 5 measurement dimensions, including displacement, strain, temperature, seepage pressure, and crack width. S12 is deployed using a distributed fiber optic sensor array with a sampling frequency of no less than 200Hz, and spatial alignment of sensor data with structural units is achieved through coordinate system mapping.

3. The method as described in claim 1, characterized in that, The definition includes a heterogeneous graph structure of structural unit nodes, sensor nodes, and geological unit nodes, and calculates a dynamic adjacency matrix based on real-time stress gradients to reflect the physical connections, mechanical transmission, and geological relationships between nodes. It also includes: S21, the calculation of the dynamic adjacency matrix A introduces a geological permeability correction coefficient ρ_geo. ρ_geo is dynamically adjusted based on real-time seepage data and historical geological parameters; S22, the physical connection edge is automatically generated based on the dam's construction drawings and structural design data, and the mechanical transmission edge is initialized based on the finite element simulation results.

4. The method as described in claim 1, characterized in that, The method of using a multimodal spatiotemporal graph neural network to fuse and model multi-source data in the heterogeneous graph structure further includes: S31, The improved ResNet-50 model introduces an attention mechanism during feature extraction to enhance sensitivity to crack edges and texture variations; S32, when parsing manually inspected text, the graph attention network uses a BERT-based entity recognition model to extract key entities and construct a semantic relationship graph.

5. The method as described in claim 1, characterized in that, Also includes: S5. The output of the multimodal spatiotemporal graph neural network is subjected to uncertainty quantification. A method combining Monte Carlo Dropout and evidence deep learning is used to calculate cognitive uncertainty and random uncertainty respectively, and a warning decision is made based on the uncertainty threshold.

6. A dam safety perception fusion and correlation device based on a multimodal spatiotemporal graph neural network, characterized in that, include: The three-dimensional coordinate system construction module is used to construct a three-dimensional geological body coordinate system, divide the dam into multiple structural units, and map sensor monitoring data, UAV inspection image data, and manual inspection text data to the three-dimensional coordinate system. The heterogeneous graph structure definition module is used to define a heterogeneous graph structure that includes structural unit nodes, sensor nodes, and geological unit nodes, and calculates a dynamic adjacency matrix based on real-time stress gradient to reflect the physical connection, mechanical transmission, and geological correlation between nodes. The multimodal fusion modeling module is used to fuse and model multi-source data in the heterogeneous graph structure using a multimodal spatiotemporal graph neural network. The sensor time series data is modeled temporally using Transformer, the image data is extracted with crack texture features using an improved ResNet-50, and the text data is parsed with a graph attention network to resolve key entity relationships. The causal reasoning construction module is used to build a causal reasoning engine based on the output of the multimodal spatiotemporal graph neural network. Through a three-level modeling system of structural equation modeling, counterfactual reasoning, and dynamic weight update, it explores the dynamic coupling law between the dam structure, geology, and material state.

7. The apparatus as claimed in claim 6, characterized in that, The three-dimensional coordinate system construction module is also used for: The dam is divided into at least 12 structural units, each containing at least 5 measurement dimensions, including displacement, strain, temperature, seepage pressure, and crack width. A distributed fiber optic sensor array is deployed with a sampling frequency of no less than 200Hz, and spatial alignment of sensor data with structural units is achieved through coordinate system mapping.

8. The apparatus as claimed in claim 6, characterized in that, The heterogeneous graph structure definition module is also used for: In the calculation of the dynamic adjacency matrix A, a geological permeability correction coefficient ρ_geo is introduced, and ρ_geo is dynamically adjusted according to real-time seepage data and historical geological parameters. The physical connection edges are automatically generated based on the dam's construction drawings and structural design data, while the mechanical transmission edges are initialized based on finite element simulation results.

9. The apparatus as claimed in claim 6, characterized in that, The multimodal fusion modeling module is also used for: The improved ResNet-50 model introduces an attention mechanism during feature extraction to enhance sensitivity to crack edges and texture variations. When the graph attention network parses the manually inspected text, it uses a BERT-based entity recognition model to extract key entities and construct a semantic relationship graph.

10. The apparatus as claimed in claim 6, characterized in that, Also includes: The uncertainty quantification module is used to quantify the uncertainty of the output results of the multimodal spatiotemporal graph neural network. It adopts a method combining Monte Carlo Dropout and evidence deep learning to calculate cognitive uncertainty and random uncertainty respectively, and makes early warning decisions based on the uncertainty threshold.

Images