A knowledge graph-based dam safety hazard propagation path analysis method and system

Abstract

The present invention provides a method and system for analyzing the propagation path of dam safety hazards based on a knowledge graph, the method comprising the following steps: based on the historical operation status data of the dam and the information resources related to dam safety obtained from the outside, as well as the engineering and technical documents formed by the dam in the planning, design, construction and operation stages, the historical safety hazards of the dam are evaluated, and the historical hazards and their characteristics are identified; according to the historical safety hazard evaluation results, a knowledge graph is constructed, and the structural units, historical hazards and their relationships of the dam are represented in the form of nodes and edges; based on the real-time operation status data of the dam, the node attributes and relationships of the knowledge graph are updated, and the current safety hazard propagation path of the dam is analyzed using the updated knowledge graph. The present invention realizes the comprehensiveness and accuracy of dam hazard analysis.

Description

Technical Field

[0001] The present invention belongs to the technical field of dam safety monitoring, and specifically relates to a method and system for analyzing the propagation path of dam safety hazards based on a knowledge graph. Background Art

[0002] As an important water conservancy project facility, the safety of dams is directly related to the safety of life and property of residents in the downstream area and the stability of the ecological environment. However, as the service life of dams increases, external environmental changes (such as climate change and natural disasters) are becoming increasingly severe, which puts higher requirements on dam safety monitoring. Existing dam monitoring and hazard assessment technologies face the following major technical problems in practical applications:

[0003] 1. Single source of truth and information isolation

[0004] Current dam safety monitoring systems mostly rely on single-mode sensor data, such as strain sensors, vibration sensors or environmental monitoring sensors. Although these data can reflect local operating conditions, they cannot provide comprehensive hazard identification and risk assessment. For example, strain sensors can detect structural deformation, but it is difficult to reflect the comprehensive impact of temperature changes on structural stress. The isolation of different data sources makes it difficult for existing systems to achieve hazard analysis from a global perspective.

[0005] 2. Lack of dynamic correlation analysis and transmission path modeling

[0006] In the existing technology, the assessment and analysis of dam hidden dangers are usually based on static data and fixed model methods. This method cannot capture the dynamic propagation characteristics of hidden dangers in the dam structure. For example, when a hidden danger occurs in a dam unit, its impact path and propagation speed on surrounding units cannot be effectively modeled by traditional methods. In addition, the impact range and propagation risk of hidden dangers have not been quantified and expressed in the overall structure, making it difficult for managers to accurately identify potential key nodes and hidden danger propagation chains.

[0007] 3. Lack of comprehensive utilization of historical data and external information resources

[0008] During the operation of the dam, a large amount of historical operating status data, hidden danger records, design and construction documents, etc. have been accumulated. These data are of great reference value for hidden danger assessment, but the existing system usually lacks effective tools and means to conduct comprehensive analysis. In addition, external resources related to dam safety (such as academic research, technical standards, patent data and accident cases) can provide important cutting-edge reference information, but the dynamic acquisition and fusion analysis capabilities of these data are still blank in the existing technology. This lack of utilization of historical and external data limits the intelligence level of the existing hidden danger analysis system.

[0009] 4. Insufficient structured and dynamic data management capabilities

[0010] As a complex engineering system, the dam has complex physical connections and mechanical relationships between its various structural units, but the existing system lacks efficient tools to express these relationships. At the same time, there is a lack of unified expression and association modeling capabilities between real-time monitoring data, historical data and external data, which makes it difficult to mine and utilize the potential value of the data. In particular, when the external environment and real-time data change, the system cannot dynamically update the hidden danger assessment model to adapt to the new operating status. Summary of the invention

[0011] The purpose of the present invention is to address the deficiencies of the above-mentioned background technology and to provide a method and system for analyzing the propagation path of dam safety hazards based on a knowledge graph to achieve comprehensiveness and accuracy of hazard analysis.

[0012] The technical solution adopted by the present invention is: a method for analyzing the propagation path of dam safety hazards based on a knowledge graph, comprising the following steps:

[0013] Based on the historical operating status data of the dam and the information resources related to dam safety obtained from the outside, as well as the engineering and technical documents generated during the planning, design, construction and operation stages of the dam, the historical safety hazard assessment of the dam is carried out to identify the historical hazards and their characteristics;

[0014] A knowledge graph is constructed based on the historical safety hazard assessment results, representing the dam's structural units, historical hazards and their relationships in the form of nodes and edges;

[0015] Based on the real-time operating status data of the dam, the node attributes and relationships of the knowledge graph are updated, and the updated knowledge graph is used to analyze the current safety hazard propagation path of the dam.

[0016] In the above technical solution, the historical operating status data and real-time operating status data of the dam include but are not limited to data collected by physical quantity sensors, vibration sensors, visual sensor data, audio sensors and environmental sensors deployed on the dam.

[0017] In the above technical solution, the information resources include: news reports, academic papers, patent data, technical standards, specifications and laws and regulations related to dam safety.

[0018] In the above technical solution, the historical and real-time operating status data of the dam are preprocessed for subsequent steps; the preprocessing process includes: using standardized formulas for various data sensors to eliminate dimensional differences and aligning timestamps of data from different sensors; fusing data from different sensors according to certain weights; and dynamically adjusting weights during the fusion process based on sensor errors.

[0019] In the above technical scheme, the process of obtaining information resources related to dam safety from the outside includes: obtaining a large amount of external resource data through network search and data crawling; establishing an inverted index and a B+ tree index for the external resource data; using index and parallel query technology to retrieve the external resource data to find the target data; caching the frequently accessed data sources and query results; weighted fusion of the target data and the dam operation status data and dynamically adjusting the weight according to the data characteristics to form a comprehensive information resource data set.

[0020] In the above technical scheme, the engineering and technical documents generated during the planning, design, construction and operation stages of the dam are pre-processed for use in subsequent steps; the pre-processing process includes: building a knowledge document library based on the engineering and technical documents generated during the planning, design, construction and operation stages of the dam; dividing the long documents in the knowledge document library into text blocks; using a text embedding model to convert the text blocks into digital vectors, capturing the semantic information of the text, and forming a vector database; the vector database is used to retrieve text blocks related to the query and provide the most relevant information.

[0021] In the above technical solution, the process of assessing the historical safety hazards of the dam and identifying the historical hazards and their characteristics includes:

[0022] Build and update BIM models: Based on engineering technical documents, build a BIM model of the dam and extract attribute data from it to assign to each structural unit; associate historical operating status data, maintenance records, stress history and other information to the corresponding parts of the BIM model; based on external information resources, incorporate the latest technical standards, specifications, laws and regulations, accident cases and academic research into the BIM model, and update risk assessment parameters and hidden danger identification models;

[0023] Use BIM models to identify historical hazards: Use the geometry and material information of the BIM model to conduct finite element analysis, fluid mechanics analysis, heat conduction analysis, and soil mechanics analysis to simulate the response of the dam under various working conditions and identify potential hazard areas and features, including stress concentration areas, areas that may be subject to excessive water pressure or scour, areas that may crack due to excessive temperature stress, and foundation areas that may experience settlement or landslides;

[0024] Hazard verification and risk assessment: Compare the identified hazard areas with the historical operating status data and historical hazard records in engineering technical documents to verify the accuracy of the model analysis results; and classify the identified historical hazards according to their characteristics;

[0025] Visualization and model update: The identified potential danger areas are highlighted in the BIM model and the potential danger characteristics are marked; as new operating data and information resources are acquired, the BIM model and various analysis models are continuously updated.

[0026] In the above technical solution, the process of constructing and updating the knowledge graph and analyzing the propagation path of the dam's safety hazards includes:

[0027] Definition and attribute assignment of nodes and edges: define each structural unit of the dam as a node and assign attributes; node attributes include: basic physical information, historical hidden danger records, detection data and maintenance logs; define the physical connection and mechanical relationship between nodes as structural connection edges and assign attributes;

[0028] Establish the relationship between nodes and edges: Establish the edges between nodes based on the historical operation status data, engineering technical documents and geometric information in the BIM model;

[0029] Data storage and connectivity calculation: store node and edge information in the graph database; calculate the nodes and edges in the graph database to obtain the connectivity and importance scores of the nodes;

[0030] Knowledge graph update: Based on the real-time collected dam operation status data and new external information resources, the health status and risk level of the nodes in the knowledge graph, as well as the weight and impact probability of the edges are updated;

[0031] Hidden danger propagation analysis and update: Using the updated node and edge attributes and connectivity calculation results, analyze the path and probability of hidden dangers propagating from one structural unit to another;

[0032] Construct hidden danger impact matrix and identify key nodes: Based on the hidden danger propagation probability, construct a matrix representing the hidden danger correlation between nodes; combine the hidden danger impact matrix and graph database to calculate the shortest path from the source node to other nodes of the hidden danger; identify the nodes with important influence in the hidden danger propagation process according to the importance score of the node and its role in the hidden danger propagation process.

[0033] The present invention provides a dam safety hazard propagation path analysis system based on knowledge graph, comprising: a historical hazard identification module, a knowledge graph construction module and a hazard propagation path analysis module;

[0034] The historical hazard identification module is used to evaluate the historical safety hazards of the dam and identify the historical hazards and their characteristics based on the historical operating status data of the dam and the information resources related to the dam safety obtained from the outside, as well as the engineering and technical documents formed during the planning, design, construction and operation stages of the dam.

[0035] The knowledge graph construction module is used to construct a knowledge graph based on the historical safety hazard assessment results, representing the structural units of the dam, historical hazards and their relationships in the form of nodes and edges;

[0036] The hidden danger propagation path analysis module is used to update the node attributes and relationships of the knowledge graph based on the real-time operating status data of the dam, and use the updated knowledge graph to analyze the safety hazard propagation path of the dam.

[0037] The present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for analyzing the propagation path of dam safety hazards based on a knowledge graph described in the above technical solution is implemented.

[0038] The beneficial effect of the present invention is that a closed-loop information flow and feedback mechanism is formed between the steps of the method in the present invention. The basic data is provided by the dam operation status data collection, the external information calculation data enriches the information source, and the engineering and technical documents provide support for the historical background. By deeply analyzing the above data, a knowledge graph is constructed to analyze the potential spread of hidden dangers. The present invention converts complex data into understandable information, integrates all information, forms a comprehensive risk assessment, and provides scientific decision-making support for dam management.

[0039] Furthermore, the present invention collects data from multiple sensors (such as physical quantity sensors, vibration sensors, visual sensors, etc.), covering various aspects of the dam's operating status information and enhancing the comprehensiveness of monitoring. The present invention provides high-quality data input for subsequent hidden danger assessment and knowledge graph construction, improving the accuracy and reliability of analysis results.

[0040] Furthermore, the present invention dynamically introduces external information resources related to dam safety (such as technical standards, specifications, patent data, etc.), expanding the field of view of hidden danger analysis. The present invention improves the adaptability of the knowledge graph to new hidden danger patterns, helps to continuously update the hidden danger analysis model, and enhances the foresight of the technical solution.

[0041] Furthermore, the present invention proposes specific data preprocessing methods (such as dimensional difference elimination, time alignment and dynamic weight adjustment) to improve the fusion quality of multi-source data. Dynamic adjustment of data weights can enhance the system's adaptability to different data qualities and reduce analysis bias caused by noise or errors.

[0042] Furthermore, the present invention improves the retrieval efficiency of external information resources by establishing an inverted index and a B+ tree index, and using parallel query technology; realizes the caching and dynamic fusion of high-frequency access data, effectively reducing the time cost of external data acquisition, while ensuring the freshness of the data and the accuracy of the analysis results.

[0043] Furthermore, the present invention effectively improves the efficiency and accuracy of large-scale document processing by processing long documents of engineering technical documents (such as dividing them into text blocks and vectorized representation); and constructs a knowledge document library and a vector database, providing efficient data query support for subsequent hidden danger analysis and knowledge graph construction.

[0044] Furthermore, the present invention integrates the geometric information, material properties and historical records of the dam through a hidden danger assessment method based on the BIM model, providing accurate basic data for hidden danger analysis; using multiple models such as finite element analysis and fluid mechanics analysis, it can identify potential hidden danger areas in multiple dimensions, enhancing the comprehensiveness and accuracy of hidden danger identification; hidden danger verification and risk grading and visualization methods improve the intuitiveness and credibility of hidden danger assessment, providing clear decision-making support for dam management.

[0045] Furthermore, the present invention realizes comprehensive modeling of structural units, hidden danger events and propagation paths through the definition and attribute assignment of nodes and edges; constructs a hidden danger propagation matrix and dynamically updates the knowledge graph, which can identify hidden danger propagation paths and key nodes, providing an accurate basis for prevention and control measures; data storage and connectivity calculation methods further improve analysis efficiency and provide technical support for subsequent propagation path analysis.

[0046] Furthermore, the present invention realizes historical hidden danger identification, knowledge graph construction and hidden danger propagation path analysis independently through modular design, which enhances the flexibility and scalability of the system. The hidden danger propagation path analysis module can dynamically respond to changes in real-time data and improve the timeliness and dynamism of hidden danger analysis. The structured system design improves the feasibility of the overall solution and provides an intelligent tool for dam safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is the overall architecture diagram of the present invention;

[0048] Figure 2 It is a flow chart of multi-modal monitoring data collection and processing inside the dam;

[0049] Figure 3 It is a flow chart of data collection and processing of external information resources based on network search;

[0050] Figure 4 It is a flow chart for preprocessing and retrieval of engineering technical documents;

[0051] Figure 5 It is a dam hazard assessment flow chart based on BIM model, mathematical and physical model and their hybrid model;

[0052] Figure 6 It is a flowchart for analyzing the propagation path of dam hidden dangers based on knowledge graph and graph database. DETAILED DESCRIPTION

[0053] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments to facilitate a clear understanding of the present invention, but they do not constitute a limitation on the present invention.

[0054] like Figure 1 As shown, the present invention provides a dam safety hazard propagation path analysis method based on knowledge graph, comprising the following steps:

[0055] Based on the historical operating status data of the dam and the information resources related to dam safety obtained from the outside, as well as the engineering and technical documents generated during the planning, design, construction and operation stages of the dam, the historical safety hazard assessment of the dam is carried out to identify the historical hazards and their characteristics;

[0056] A knowledge graph is constructed based on the historical safety hazard assessment results, representing the dam's structural units, historical hazards and their relationships in the form of nodes and edges;

[0057] Based on the real-time operating status data of the dam, the node attributes and relationships of the knowledge graph are updated, and the updated knowledge graph is used to analyze the propagation path of the dam's safety hazards.

[0058] The principle of the present invention is further explained below with reference to specific embodiments.

[0059] like Figure 1 As shown, the present invention provides a dam safety monitoring and assessment system based on a multi-modal large model, comprising the following modules:

[0060] Module 1: Real-time perception module of multi-modal dam internal monitoring data, which is used for collecting and processing multi-modal monitoring data inside the dam, obtaining real-time and historical operating status data, and generating a comprehensive real-time monitoring view of the dam health status.

[0061] Monitoring data include physical quantities (deformation, seepage, stress, strain and temperature, etc.), images (photos, videos and point clouds, etc.), vibrations (strong earthquake monitoring, etc.), audio frequencies (metal structure monitoring), environment (meteorology, earthquake, hydrology, etc.), etc.

[0062] The real-time monitoring view of dam health status is a data visualization interface that comprehensively displays the current safety status and structural health of the dam. The real-time monitoring view of dam health status displays the following data:

[0063] Real-time data overview: displays various real-time monitoring data of the dam, including stress, displacement, temperature, seepage and other physical quantities.

[0064] Status indicators: Health indicators calculated from various monitoring data, such as safety factor, risk level, etc., help quickly determine the overall health of the dam.

[0065] Anomaly detection: Real-time display of abnormal data or warning signals detected by sensors, helping managers to identify potential safety hazards in a timely manner.

[0066] Visualization charts: Use graphs and charts to intuitively display data change trends, such as stress change curves, displacement distribution diagrams, etc., to facilitate the identification of abnormal patterns. Regional heat maps: Use heat maps to show the health status of different parts of the dam, with color depth representing different risk levels or abnormalities in monitoring data.

[0067] Historical comparison: Provides comparison of historical monitoring data to help analyze changes between current and past states and identify long-term trends and potential problems.

[0068] Combining this information, the real-time monitoring view of the dam health status provides managers with a clear and intuitive tool to help them make quick and scientific decisions to ensure the safe operation of the dam.

[0069] Module 2: Real-time perception module of external resource data based on network search, which is used to obtain information resources related to dam safety around the world through automated web crawler technology.

[0070] Specifically, a large amount of external resource data is obtained through network search and data crawling; an inverted index and B+ tree index are established for the external resource data; index and parallel query technology are used to retrieve external resource data to find the target data; frequently accessed data sources and query results are cached; the target data and dam operation status data are weightedly fused and the weights are dynamically adjusted according to the data characteristics to form a comprehensive information resource data set.

[0071] Information resource data related to dam safety includes news reports on reservoir dam monitoring, safety assessment, monitoring and early warning, and maintenance and reinforcement around the world, as well as the latest academic papers, patents, laws, regulations, standards and other scientific and technological resources. These data are combined with multimodal sensor data to enhance the ability to assess the overall safety status of dams. The introduction of external resource data improves the comprehensiveness of decision-making and provides rich background information and reference for subsequent analysis modules.

[0072] Module three, the RAG-based historical data retrieval module, builds a knowledge document library based on the engineering and technical documents generated during the planning, design, construction and operation stages of the dam; divides the long documents in the knowledge document library into text blocks; uses a text embedding model to convert text blocks into digital vectors, captures the semantic information of the text, and forms a vector database; the vector database is used to retrieve text blocks related to the query and provide the most relevant information.

[0073] The dam knowledge document library covers a variety of important data types, including survey and design, construction records, daily dam operation data, hidden danger records, log records, hydraulic gate and turbine unit operation and maintenance records, and dam repair and reinforcement records, covering detailed information on all aspects of dam construction, operation, maintenance and repair.

[0074] To ensure that this information can be retrieved and used efficiently and accurately, the third step of the module first uses embedding technology to vectorize the historical data and convert the text information into digital vectors to facilitate subsequent calculations and comparisons. This vectorization not only retains the semantic information of the data, but also improves the efficiency of data processing.

[0075] During the retrieval process, after the user enters the query keywords, the system first vectorizes these keywords to generate a query vector. Then, the system uses the stored vectors in the vector database to find the historical data vector closest to the query vector through similarity calculation (such as cosine similarity). This process can quickly identify documents or records related to user needs.

[0076] To further improve the relevance of the search results, the system then uses reranking technology. Based on the initial search, this technology re-ranks the results and optimizes the sorting order of the search results by introducing more contextual information and features (such as the importance of the document, historical search frequency, etc.).

[0077] Specifically, reranking may use machine learning models to score preliminary results, taking into account the relevance of the document to the query, the quality of the document, and the user's historical preferences, to ensure that users can quickly find the most appropriate information when querying, thereby providing strong support for subsequent risk assessment and decision-making.

[0078] The knowledge document library supports subsequent data analysis and provides historical context for the system, helping to more accurately analyze current monitoring data. The historical data of the dam not only provides key references for the analysis of real-time monitoring data, but also adds necessary historical background to the assessment process, making the dam safety assessment more comprehensive and accurate.

[0079] Module 4: Construct a hidden danger assessment module based on BIM model, mathematical model, physical model and mathematical-physical hybrid model to evaluate the structural strength and stability of the dam based on real-time monitoring data and historical data, and obtain the location and type of hidden dangers.

[0080] Mathematical models, physical models and mathematical-physical hybrid models include finite element analysis models, fluid mechanics models, heat conduction models and soil mechanics models.

[0081] The finite element analysis model is used to identify stress concentration areas through finite element analysis; the fluid mechanics model is used to identify areas that may suffer structural damage due to excessive water pressure or water scouring through fluid mechanics analysis; the heat conduction model is used to identify areas where temperature changes affect structural safety through thermal stress distribution analysis; the soil mechanics model is used to identify areas where settlement or landslides may occur through soil mechanics analysis.

[0082] Specifically, Module 4 builds the BIM model of the dam based on engineering technical documents, and extracts attribute data from it to assign to each structural unit; associates historical operating status data, maintenance records, stress history and other information to the corresponding parts of the BIM model; based on externally acquired information resources, incorporates the latest technical standards, specifications, laws and regulations, accident cases and academic research into the BIM model, and updates risk assessment parameters and hidden danger identification models;

[0083] Use the geometry and material information of the BIM model to conduct finite element analysis, fluid mechanics analysis, heat conduction analysis, and soil mechanics analysis to simulate the response of the dam under various working conditions and identify potential risk areas and features, including stress concentration areas, areas that may be subject to excessive water pressure or scour, areas that may crack due to excessive temperature stress, and foundation areas that may experience settlement or landslides;

[0084] Compare the identified potential danger areas with the historical operating status data and historical potential danger records in engineering and technical documents to verify the accuracy of the model analysis results; and classify the risks of the identified historical potential dangers according to their characteristics;

[0085] The identified potential danger areas will be highlighted in the BIM model and the potential danger characteristics will be marked; as new operating data and information resources are acquired, the BIM model and various analysis models will be continuously updated.

[0086] Module 4 integrates BIM (Building Information Modeling) models, mathematical models, physical models, and hybrid models of mathematical and physical models, combined with internal monitoring data, external resource data, and historical data to achieve dynamic assessment of dam safety hazards. The core of this module is to integrate multiple models to provide comprehensive and scientific analysis results to support dam safety management and decision-making. By integrating these models, the system can obtain the hidden dangers of the dam and the location of the hidden dangers.

[0087] Module 5: Hidden danger propagation path analysis module based on knowledge graph and graph database, which is used to construct knowledge graph and graph database, analyze the correlation between dam structure nodes according to real-time monitoring data and historical data to identify related hidden dangers, and realize the analysis and early warning of dam hidden danger propagation path, and obtain related hidden danger information and hidden danger propagation path.

[0088] Specifically, module five defines each structural unit of the dam as a structural unit node and assigns attributes; defines the physical connection and mechanical relationship between structural units as a structural connection edge and assigns attributes;

[0089] Establish the relationship between nodes and edges: Establish the edges between the nodes of the structural units based on the historical operation status data, engineering technical documents and geometric information in the BIM model;

[0090] Store the node and edge information in the graph database; calculate the nodes and edges in the graph database to obtain the node connectivity and importance scores;

[0091] Based on the real-time collected dam operation status data and new external information resources, the health status and risk level of the nodes in the knowledge graph, as well as the weight and impact probability of the edges, are updated;

[0092] Using the updated node and edge attributes and the results of connectivity calculation, the path and probability of hidden dangers propagating from one structural unit to another are analyzed;

[0093] Based on the probability of hidden danger propagation, a matrix representing the hidden danger correlation between nodes is constructed; the hidden danger impact matrix and graph database are combined to calculate the shortest path from the source node to other nodes of the hidden danger; according to the importance score of the node and its role in the hidden danger propagation process, the nodes with important influence in the hidden danger propagation process are identified.

[0094] Since the structure of the dam is composed of multiple interconnected components, an abnormality in any node may trigger a chain reaction, which in turn affects the safety of the entire structure. The knowledge graph represents the various structural units of the dam and their relationships in a graphical way, allowing the system to clearly display the connections and influences between nodes. As the underlying support of the knowledge graph, the graph database can efficiently store and query complex structural information. By abstracting the structural units of the dam into nodes in the graph, the attributes of each node not only contain basic physical information, but also integrate multi-dimensional data such as historical hidden danger records, detection data, and maintenance logs. This integration of information enables the system to quickly retrieve relevant nodes and analyze possible associated hidden dangers when hidden dangers occur.

[0095] In addition, by using the powerful path calculation capabilities of the graph database, the system can accurately analyze the path of hidden danger propagation. When an abnormality occurs at a certain node, the system can quickly calculate other nodes that the hidden danger may affect and identify the shortest path for the hidden danger to propagate.

[0096] By combining knowledge graphs with graph databases, we can not only systematically analyze the structural relevance of the dam, but also comprehensively identify potential associated hazards and their locations, providing more reliable data support and decision-making basis for ensuring dam safety. Through this multi-level analysis method, the dam monitoring system can achieve efficient linkage from early warning to decision-making, ensuring the overall health and safety of the dam.

[0097] The dam safety hazard propagation path analysis method based on knowledge graph, which is implemented by the dam safety hazard propagation path analysis based on knowledge graph, includes the following steps:

[0098] 1) The acquisition and processing of multi-modal monitoring data inside the dam is realized by the real-time perception block of multi-modal dam internal monitoring data (module 1). Figure 2 As shown, the specific technical details and implementation methods are as follows:

[0099] 1.1) Deployment and data collection of multimodal sensors

[0100] Multimodal sensors include physical quantity sensors, vibration sensors, visual sensors, audio sensors, and environmental sensors to obtain multimodal data. Physical quantity sensors include deformation sensors, seepage sensors, stress strain sensors, temperature sensors, etc.; visual sensors include cameras, drones, etc.; environmental sensors include meteorological sensors, seismic sensors, hydrological sensors, etc.

[0101] Real-time data acquired by multimodal sensors are transmitted to the central processing system via wireless networks and stored in a unified database for data processing, data fusion and analysis of multimodal large models.

[0102] 1.2) Standardization and synchronization of multi-dimensional heterogeneous data

[0103] (1) Standardization can eliminate the dimensional differences between data and ensure that data from different modalities can be compared and integrated in the same dimension. The formula for data standardization is:

[0104] ;

[0105] in, D i For i The raw data of the sensors, μ For the i The average value of sensor data, For the i The standard deviation of the sensor data, D norm For iThe data of each sensor is normalized. After normalization, all data are converted into standardized data with a mean of zero and a standard deviation of one, thus ensuring the relative consistency between data from different sensors.

[0106] (2) Timestamp alignment: Use the spatiotemporal synchronization algorithm to align the timestamps of data from different sensors to ensure that all modal data are compared and analyzed in the same time dimension. The formula for time synchronization is:

[0107] ;

[0108] in, T sync is the timestamp after synchronization, T i is the timestamp of each sensor, n is the number of sensors. The algorithm ensures the consistency of the data in the time dimension, especially in the time synchronization between vibration, audio and visual data.

[0109] 1.3) Data fusion and error processing

[0110] The data fusion of various sensors adopts the multi-modal weighted fusion method. The formula for data fusion is:

[0111] ;

[0112] in, D i From i The data of the sensors, w i For the i The system adjusts the weights of the sensor data. w i To optimize the fused data so that it can more accurately reflect the actual status of the dam.

[0113] Taking into account the measurement accuracy of different sensors and the errors that may be introduced by transmission delays. Therefore, the error calculation formula is used to evaluate the accuracy of data fusion E f :

[0114] ;

[0115] This formula is used to calculate the error value generated during the fusion process of different modal data. If the data of a certain sensor produces a large error after being fused with the data of other sensors, this module will dynamically adjust the data weight of the sensor. w i , to reduce errors and improve the accuracy of overall monitoring.

[0116] 2) Obtaining external resource data related to dam safety monitoring and assessment based on network search is realized by the real-time perception module of external resource data based on network search (module 2), including the collection and processing of external resource data based on network search, which is realized through automated network crawler technology. Figure 3 As shown in the figure, the external resource data collection and processing process based on network search, the specific technical details and implementation methods are as follows:

[0117] 2.1) Network data acquisition, including the following main steps:

[0118] (1) Index establishment and retrieval: Establish an inverted index structure and use a B+ tree index structure. The retrieved data is sorted based on factors such as relevance and timeliness to ensure that users get the most useful information. The ranking score formula is:

[0119]

[0120] in, CTR is the click rate, Relevance For information P With query Q The relevance of Freshness is the timestamp of information release, α, β, and γ are CTR, Relevance, Freshness Corresponding weights. Click-through rate (CTR) is obtained by recording the number of impressions of each search result and the number of clicks by users. When a user performs a search, the system automatically counts these data to calculate the CTR value. Relevance can be determined by manual annotation, user feedback, and machine learning models. Experts evaluate the relevance of each document and adjust it based on the user's click behavior and satisfaction feedback. In addition, machine learning models can automatically score by analyzing features. The acquisition of freshness depends on the timestamp of the record. The system evaluates the freshness of the information based on the difference between the current time and the time when the record was created. Through these methods, the system can comprehensively judge the quality and relevance of each search result. Through this ranking algorithm, the module can quickly obtain and utilize the latest information related to dam safety.

[0121] (2) Data caching and updating: Use a caching mechanism and regularly update the cached data. To improve retrieval efficiency, use a caching mechanism to cache frequently accessed data in local storage, such as specific standards and regulations, to reduce repeated queries; regularly update cached data to ensure the timeliness of the data.

[0122] (3) Parallel query and distributed computing: A distributed indexing solution is used to distribute external data across multiple servers or nodes for processing. When querying, tasks are assigned to multiple servers for parallel processing, thereby reducing the load pressure on a single server, reducing query response time, and further improving query efficiency.

[0123] 2.2) Network data fusion and application. By acquiring network data in real time, such as global patent information, international standards, laws and regulations, and the latest academic papers, the system can combine these external data with internal sensor data. First, external data provides the latest technical and normative references for dam safety assessment; second, sensor data provides real-time monitoring of dam health status. The system integrates these two types of data, analyzes potential risks and hidden dangers, and generates comprehensive assessment reports to support intelligent decision-making and ensure the safe operation and management of dams. For example: by acquiring global patent data in real time, the system can identify the latest monitoring and reinforcement technologies and help managers choose appropriate technical solutions for dam maintenance. By retrieving international standards and laws and regulations, the system can help dam operators adjust operating procedures, ensure compliance, and reduce potential risks caused by non-compliance. The latest academic papers and research results provide managers with rich background information to help conduct scientific risk assessment and decision-making.

[0124] 3) Obtaining historical data of dams, including building a knowledge base of basic dam data and performing efficient retrieval through vector database, which is realized by module 3. Figure 4 As shown, the specific technical details and implementation methods are as follows:

[0125] 3.1) Build a knowledge base for basic dam information. The knowledge base for basic dam information includes dam survey, design and construction data, dam daily operation data, operation log records, fault log records, hidden danger records, turbine unit operation and maintenance records, hydraulic gate operation and maintenance data, dam repair and reinforcement records, etc. These documents cover detailed information on various aspects of dam construction, operation, maintenance and repair, and constitute an important basis for system analysis and decision support.

[0126] 3.2) Text segmentation. The documents in the dam basic information knowledge base are segmented into blocks to obtain text blocks. Text blocks represent different chapters, paragraphs or sentences of a document. The purpose of segmentation is to divide longer documents into smaller fragments for subsequent processing and calculation, and to improve processing speed and data operability while maintaining semantic integrity. Each text block contains meaningful semantic units, ensuring that the subsequent information extraction step can obtain high-quality and accurate content.

[0127] 3.3) Text embedding. The text block is processed by the text embedding model, and the text block is converted into a digital vector to obtain the text block vector. All the text block vectors constitute the vector database. The semantic information of the text is represented by the vector, and the vector database finally obtained is the knowledge base of the dam basic information.

[0128] Text embedding is achieved through machine learning algorithms. Text embedding models include Word2Vec, GloVe, and the BERT model based on deep learning. The model maps similar words or sentences into similar vector spaces by learning contextual relationships in a large-scale corpus. The closer two text blocks are semantically, the closer they are in the vector space. Through text embedding, the system can capture subtle differences and hidden semantic information in the text, providing a basis for information extraction and correlation analysis.

[0129] 3.4) Information extraction and retrieval based on vector database.

[0130] Information extraction and retrieval are achieved through the nearest neighbor search algorithm, which can quickly find the text block vector closest to the query vector and return the corresponding original text content. The nearest neighbor search algorithm includes the K nearest neighbor algorithm or the approximate nearest neighbor search algorithm. Law .

[0131] In practical applications, this function can quickly find records or knowledge related to the current problem from a large amount of documents. For example, if the system needs to check the maintenance records of a specific dam, it can extract the maintenance and repair data related to the dam from the dam basic information knowledge base for risk assessment or decision support.

[0132] 4) Based on the real-time monitoring data and historical dam data, dynamic assessment and analysis of dam safety hazards are carried out to determine the type and location of the hazards, which is achieved by module 4. Figure 5 As shown in the figure, the specific technical details and implementation methods of the dam safety hazard assessment process are as follows:

[0133] 4.1) Construction of a refined BIM (Building Information Modeling) model of the dam. The BIM model uses three-dimensional visualization to represent the geometric information, material properties, construction history and maintenance records of the dam, providing accurate basic information for the analysis of mathematical models, physical models and mathematical-physical hybrid models.

[0134] For example, the system can use the BIM model to identify vulnerable parts of the dam or areas with a historical maintenance record. These areas are often the focus of assessment and may pose potential safety hazards.

[0135] 4.2) Analysis of dam safety hazards integrating mathematical models, physical models and mathematical-physical hybrid models: By combining the calculation results of BIM models and mathematical models, physical models and mathematical-physical hybrid models, a comprehensive safety assessment of the dam is conducted, and potential hazard areas, types and locations of hazards are accurately located.

[0136] Mathematical models, physical models and mathematical-physical hybrid models include finite element models, fluid mechanics models, heat conduction models and soil mechanics models.

[0137] The BIM model provides geometric information, material properties, historical records, and foundation data for the calculation of BIM models and mathematical models, physical models, and mathematical-physical hybrid models. The three-dimensional geometric structure of the BIM model is composed of multiple structural units. The geometric information (including volume, shape, and position) of each structural unit is discretized and transferred to the finite element model, fluid mechanics model, and heat conduction model for analysis to improve the accuracy of simulation calculations. Material properties include the strength, elastic modulus, and thermal conductivity of concrete, which are used for mechanical analysis of the dam. Historical records include the historical maintenance records and stress history information of the dam, which are used to identify areas where stress concentration has occurred or where repairs have been carried out, so as to better determine whether there are repeated hidden dangers in these areas.

[0138] Types of hidden dangers of dams include but are not limited to stress concentration, water scouring, cracks caused by thermal stress, and unstable foundation.

[0139] Stress concentration hazards: Finite element analysis models are used to simulate the stress and deformation distribution of the dam under different load conditions. The geometric information and material property data provided by the BIM model are discretized into multiple structural units, and the stress changes of these units under static loads (such as water pressure) and dynamic loads (such as earthquakes or floods) are calculated. Through stress tensor and deformation calculations, stress concentration areas are identified. These areas are often weak links in the structure and may cause cracks, deformations, or even failures. For example, when a part of the dam is subjected to excessive pressure or stress, the finite element analysis model can identify the location and, combined with the historical records of the BIM model, determine whether it is outside the safety threshold.

[0140] Water scour hazards: The fluid mechanics model is combined with the BIM model to analyze the dynamic impact of water flow on the dam. The fluid mechanics model simulates the interaction between water flow and the dam structure based on the dam body contour and boundary conditions provided in the BIM model, and predicts areas that may suffer structural damage due to excessive water pressure or water scour. The structural geometry information provided by the BIM model provides accurate boundary conditions for fluid mechanics analysis, allowing the fluid mechanics model to simulate real water flow behavior, thereby more accurately identifying water pressure sensitive areas.

[0141] Temperature change hazards: The heat conduction model simulates the thermal stress distribution of the dam under different temperature conditions, combined with the material properties and structural positions provided by the BIM model, to identify the expansion or contraction of materials due to temperature changes. Under extreme temperature conditions (such as extremely cold or hot climates), the dam material will produce thermal stress, resulting in cracks or deformation. By simulating and analyzing the stress distribution caused by temperature changes, the system can identify the affected parts and determine their stability.

[0142] Hidden dangers of unstable foundation: The soil mechanics model evaluates the stability of the dam foundation by analyzing the interaction between the dam and the foundation soil. The foundation data provided by the BIM model is combined with the soil mechanics model to evaluate the bearing capacity and sliding risk of the foundation. By analyzing the structural instability that may be caused by foundation settlement or sliding, potential hidden dangers of the dam foundation are identified, especially areas where settlement, landslides or collapses may occur, thereby ensuring the safety of the dam foundation. The common soil model is the Mohr-Coulomb model, which can evaluate the supporting effect of the soil on the dam and analyze the stability of the soil under extreme conditions.

[0143] 4.3) Hidden danger marking: mark the hidden dangers analyzed in the above models on the BIM model and determine the specific location of the hidden dangers in the dam structure.

[0144] The BIM model not only provides geometric and material information, but also helps to accurately locate the location of hidden dangers through its 3D visualization function.

[0145] Hidden danger marking: When mathematical, physical models and their hybrid models identify potential hidden dangers, these hidden dangers are marked on the corresponding structural elements of the BIM model. For example, when finite element analysis identifies a stress concentration area, the area is highlighted with a color on the BIM model to quickly identify the location of the hidden danger.

[0146] Hidden danger location: The specific location of hidden dangers in the dam structure is determined through the 3D BIM model. The 3D BIM model can view different areas of the dam from multiple perspectives and understand the spatial distribution of hidden dangers. Ensure the accurate location of hidden dangers to facilitate on-site maintenance and treatment.

[0147] 5) Based on the historical data and real-time monitoring data of the dam, the hidden dangers are identified and the propagation paths of the hidden dangers in the dam are traced, which is realized by module five. Figure 6 As shown in the figure, the dam hazard propagation path analysis process, the specific technical details and implementation methods are as follows:

[0148] 5.1) Construct dam knowledge graph and graph database.

[0149] The structure of the dam is abstracted as a graph G = (V, E), where VIt is a collection of nodes, representing the various structural units of the dam, including the dam body, spillway, foundation, etc. E It is a set of edges, which represents the connection relationship between structural units. Among them, the node contains the attribute information of the corresponding structure, including historical test data, material strength and stress distribution, etc. v The properties of are represented as a vector: A ( v ) = [historical test data, material strength, stress distribution]. Edges contain the physical connection of the structure and related attribute information. v With Node j Connected Edges e vj The properties are expressed as: W ( e vj )=[material strength, force transfer relationship]. The hidden danger propagation relationship is a dynamic attribute, which is closely related to the state of the node (such as the probability of hidden danger occurrence) and the propagation weight of the edge (such as the probability of hidden danger impact, the hidden danger propagation intensity). This type of information usually needs to be dynamically generated through real-time calculation or historical data analysis, rather than a fixed static attribute of the edge. Therefore, only static attributes (material strength and force transfer relationship) are listed in the initial definition. In this embodiment, the main purpose of constructing edge attributes is to describe the physical and mechanical relationships between structural units to support basic structural analysis (such as stress transfer). The hidden danger propagation relationship is more complex and may be divided into a separate calculation module instead of being the default static attribute of the edge.

[0150] The node set and edge set are stored in the graph database, which can efficiently manage the structural data and provide basic information for the subsequent hidden danger propagation analysis. This structured data storage method makes data query and analysis more efficient and can provide accurate information in a short time. The graph database contains node sets, edge sets, and node connectivity, which is used to indicate the degree of association between a structural unit and other structural units.

[0151] Node connectivity LJ ( v ) is used to measure the node v The relative importance of the node in the dam structure specifically reflects the degree of connection between the node and other structural units. v The connectivity of a node can be used to determine its position and role in the overall structure of the dam. If a node has a high connectivity, it means that it is connected to multiple other nodes, indicating that the node may have an important function or be subjected to greater stress in the structure. v Connectivity LJ ( v ) can be expressed as:

[0152] ;

[0153] in, A jv is an element in the adjacency matrix, representing a node j Is it related to the node v Connected, n is the total number of nodes in the knowledge graph.

[0154] The adjacency matrix is ​​a matrix used to represent the graph structure, which can clearly show the connection relationship between nodes. If the connectivity of a node is high, it means that the node is in a critical position in the dam structure and may have a greater impact on the overall safety of the dam.

[0155] The Pagerank algorithm is used to evaluate the relative importance of each node in the dam. The calculation formula is:

[0156] ;

[0157] in, PR ( v ) is a node v The relative importance score of B v For the node v The set of connected nodes, L ( u ) is a node u The number of outgoing edges, d is the damping factor (usually around 0.85), n is the total number of nodes. This score can help the system identify critical nodes in the dam structure, especially when the dam is under stress or hidden dangers, these critical nodes may be the parts that need the most attention.

[0158] 5.2) Identification of associated hazards and analysis of hazard propagation. Identify possible associated hazards near the hazard locations determined in step 4). The core of the knowledge graph is indeed composed of nodes and edges. Nodes represent entities (such as structural units of dams, hazards, etc.), while edges represent the relationships between nodes (such as connections, impacts, etc.). However, the richness and functionality of the knowledge graph are not only reflected in the connections between nodes and edges, but also include attribute information related to nodes and edges.

[0159] Node attributes: Each node can carry rich attribute information, such as historical test data, material strength, stress distribution, etc. These attributes provide important background for understanding the characteristics and status of the node. For example, the material strength and historical test data of a structural unit of a dam can help evaluate the safety of the unit.

[0160] Edge attributes: Edge attributes describe the relationship characteristics between nodes, such as force transmission relationship or connection strength. These attributes help analyze the mutual influence between different structural units, especially when hidden dangers occur, it is crucial to understand how forces are transmitted in the structure.

[0161] Information integration: Knowledge graphs can integrate information from different sources, such as historical hidden danger records, maintenance logs, and expert advice. This information can complement nodes and edges to help build a more comprehensive hidden danger network. For example, the location of a hidden danger may be related to a specific state of a node in the historical record, and this relationship can be described by an edge in the knowledge graph.

[0162] Hidden danger propagation analysis: By analyzing the characteristics of nodes and their attributes and edges, the knowledge graph can identify the associations between hidden dangers and trace the paths of hidden danger propagation within the dam. This propagation analysis not only relies on the connectivity of nodes and edges, but also comprehensively considers attribute information to more accurately assess the impact of hidden dangers on the entire structure.

[0163] Therefore, the value of the knowledge graph lies in its ability to provide a deep understanding and analysis of complex structures through rich attribute information of nodes and edges, thereby effectively supporting hidden danger identification and propagation analysis.

[0164] To analyze the spread of hidden dangers, the system constructs a hidden danger impact matrix. The hidden danger impact matrix is ​​a structured data table used to represent the hidden danger correlation between different structural units of the dam. In this matrix, rows and columns represent each node, and the value of each cell reflects the intensity or risk level of hidden danger propagation between two nodes. These values ​​can be calculated based on factors such as historical hidden danger records, material strength and stress distribution of nodes. By analyzing the hidden danger impact matrix, managers can identify high-risk nodes and formulate timely prevention and repair measures accordingly, thereby effectively reducing potential risks and ensuring the safe operation of the dam. By analyzing this matrix, managers can identify high-risk nodes and take timely measures.

[0165] The shortest path calculation based on the graph database is used to analyze the propagation path of hidden dangers in the dam structure. The calculation formula of the shortest path is:

[0166] ;

[0167] in, d ( u , v ) represents a node u and nodes v The shortest path distance between w It is on the path e i The weight of kis the number of edges on the path. In dam monitoring, the weight setting of edges is determined based on several key factors. First, physical distance is an important consideration. The shorter the distance between connected nodes, the lower the weight of the edge is usually set, because the potential danger is more likely to spread over a short distance. In addition, the strength of force transmission will also affect the weight. The higher the strength and stiffness of the connecting material, the greater the resistance to the spread of hidden dangers, and the corresponding weight can be lower. Historical hidden danger data is also important. If a path has often experienced hidden danger propagation in the past, the weight of the edge should be adjusted lower to reflect the higher risk. Environmental factors such as rainfall, temperature changes, and material properties also affect the weight setting. By comprehensively considering these factors, a reasonable weight can be set for each edge, making the shortest path calculation more accurate and ensuring the scientificity and effectiveness of the hidden danger propagation analysis. The possibility and speed of hidden danger propagation from one node to other nodes are calculated by the shortest path algorithm.

[0168] In addition, the graph database also supports complex path traversal queries, which can efficiently traverse the relevant paths in the dam structure according to the conditions set by the user, and provide detailed information on the path (such as the status of each node and the attributes of the edge). Path traversal query methods include using recursive traversal algorithms to query nodes v With Node u All paths between P ( u , v ), and evaluate the importance of the path based on the edge weight w and node status on the path. In hidden danger propagation analysis, node status refers to the health or safety status of each node at a specific moment, reflecting its current safety and potential risks. Node status usually includes safety levels, such as "safe", "warning" or "dangerous", which is evaluated through real-time monitoring data (such as stress, displacement and temperature). In addition, if a node has been identified as having hidden dangers, its status will include the type of hidden danger, severity and possible impact range. Historical monitoring records and maintenance logs will also affect the status of the node, especially nodes that have had hidden dangers. Environmental factors, such as rainfall or earthquakes, can also affect the node status and change its safety. By comprehensively evaluating the node status, the system can effectively predict the risk of hidden danger propagation and provide an important basis for management decisions.

[0169] If a hidden danger occurs in a node, the system will calculate the impact of the hidden danger on its adjacent nodes and predict the possible scope of the hidden danger through the propagation model. The probability of hidden danger propagation can be expressed by the following formula:

[0170] ;

[0171] in, F ( u , v ) is a hidden danger slave nodeu Propagate to nodes v The probability of P ( u ) is a node u The probability of hidden dangers occurring, d ( u , v ) is a node u and nodes v The shortest path distance between d 0 is the influence radius set by the system. This formula shows that the possibility of hidden dangers spreading decreases as the distance between nodes increases. Based on the propagation prediction results, the system can trigger an early warning mechanism. Especially when key nodes are threatened, the system will automatically issue a warning to prompt dam managers to conduct further inspections or take preventive measures.

[0172] Get Node P ( u ) The probability of hidden dangers can be achieved through a variety of methods. First, historical data analysis is an effective way to estimate the probability by counting the frequency of hidden dangers that occurred at the node in the past and the ratio of the total number of monitoring times. In addition, real-time monitoring data can also provide important information. The health status of the node can be evaluated by using data such as stress, displacement, and temperature monitored by sensors, combined with the set safety threshold, to determine the possibility of hidden dangers. Building a statistical model is also a common method to predict the probability of hidden dangers through a variety of influencing factors (such as material properties, environmental conditions, etc.). Finally, expert evaluation can provide a basis for probability and conduct in-depth analysis of node characteristics combined with professional knowledge. Through these methods, the system can dynamically update the probability of hidden dangers at the node and provide accurate support for the prediction of hidden danger propagation.

[0173] 5.3) The dam updates the knowledge graph and graph database. The data of nodes and edges in the knowledge graph and graph database are continuously updated incrementally as the monitoring data is updated, which is achieved by the incremental update algorithm.

[0174] The node status information is updated according to the real-time sensor data, including the structural health status, stress conditions, etc. At the same time, the attributes of the edges are also dynamically adjusted. For example, if the force transmission in certain areas is abnormal during structural detection, the weight of the corresponding edge is adjusted, and the shortest path and node importance are recalculated.

[0175] By introducing an incremental update algorithm, only the changed nodes and edges are updated, which greatly improves the system's response speed and operating efficiency.

[0176] In actual application, the calculation results obtained in the above steps can be used to predict dam hidden dangers and conduct risk assessment, and then adjust the alarms and thresholds of relevant sensors, which specifically include the following steps:

[0177] 6) Pre-training and fine-tuning of the multimodal large model: Pre-train and fine-tune the multimodal large model so that it can learn to obtain risk assessment results, warning thresholds and corresponding warning mechanisms based on the real-time monitoring data, external resource data, dam historical data, hidden danger locations and types, and dam hidden danger propagation paths. The technical details and implementation methods of pre-training and fine-tuning of the multimodal large model are as follows:

[0178] 6.1) Dataset construction and preprocessing.

[0179] Dataset construction: collect historical multimodal monitoring data (from module 1), external resource data (from module 2), dam historical data (from module 3), hidden danger locations and types (from module 4), hidden danger propagation paths (from module 5), and form input samples based on the above data: sensor data deployed on the dam, historical hidden danger records, engineering and technical documents formed in the planning, design, construction and operation stages of the dam, information resources related to dam safety, geometry and material information of the BIM model, timestamp and spatial location information.

[0180] Data preprocessing: including data cleaning (removing invalid data, noise and outliers caused by sensor failure), format unification (such as equal-interval processing of time series data, vectorization of text data through word segmentation and keyword extraction), alignment and synchronization (time synchronization of multimodal data based on timestamps, and alignment of sensor data with BIM models through spatial coordinates) and normalization processing (Min-Max Scaling is used for continuous data, and one-hot encoding is performed on categorical data).

[0181] Data labeling: Create labels for each sample, including a binary label of whether a hidden danger event has occurred (0: not occurred, 1: occurred), the probability of hidden danger event occurrence (calculated by counting the ratio of the number of hidden danger occurrences under specific conditions to the total number of occurrences of the condition), and the type and impact of hidden danger events, as auxiliary labels for the event classification task. The calculation data of the data labels come from the historical data of the dam (from module 3), the location and type of hidden dangers (from module 4), and the propagation path of hidden dangers (from module 5).

[0182] Data augmentation: When data is insufficient or unbalanced, data augmentation techniques can be used to improve the generalization ability of the model, including cutting long time series into short time windows to extract more features, jointly sampling multimodal data to generate new samples, and using simulation methods to generate synthetic samples of sensor data or hidden danger propagation paths, thereby enriching the diversity and completeness of the data set.

[0183] Modal feature encoding: The Transformer model requires serialized vectorized feature input, so each modal data is feature encoded: sensor data uses sliding windows to extract time series features, visual data uses pre-trained CNN (such as ResNet or ViT) to extract image features, text data uses pre-trained language models (such as BERT) to generate embedded vectors, BIM model data extracts geometric and material properties and digitizes them, and spatiotemporal information encodes timestamps and spatial positions into numerical vectors. These features are processed separately by independent encoders to provide the model with a fusion basis for multi-modal features.

[0184] Data is organized as Transformer input format: The Transformer architecture requires that multimodal features be organized in sequence form. Each sample constitutes a sequence, in which each element corresponds to a feature vector of a modality (such as modality 1 features, modality 2 features, modality 3 features, ... modality 1 features, modality 2 features, modality 3 features, ... modality 1 features, modality 2 features, modality 3 features, ...). The fusion of modal features is achieved through the Transformer's multi-head self-attention mechanism, generating a joint representation with contextual relevance, which provides a basis for the comprehensive analysis of multimodal data.

[0185] Save training set: To facilitate efficient training and validation of the model, the training set needs to be saved in a format suitable for batch loading (such as .csv, .json or .tfrecord), and the data set needs to be divided according to certain rules. Usually, the data is divided into training set and validation set in chronological order or random proportion, for example, 80% of the data is used for training and 20% of the data is used for validation, so as to ensure the scientificity and accuracy of model training and evaluation.

[0186] 6.2) Model fine-tuning and pre-training

[0187] Based on the above-established data set, the multimodal large model uses the Transformer's multi-head self-attention mechanism to fuse the features of each modality. These data from different sources are aggregated into the Transformer model for joint analysis, enabling the model to provide information support from multiple dimensions when capturing hidden dangers. Through the joint processing of multimodal data, the system can comprehensively assess the health of the dam based on abnormal audio signals combined with surface change information in images.

[0188] The loss function for pre-training the deep learning model based on the Transformer architecture is as follows:

[0189] ;

[0190] in, y iis the actual label, is the result predicted by the model, N is the total number of samples, and θ is the model parameter. This loss function is used to evaluate the difference between the model prediction results and the actual results.

[0191] In order to reduce the loss value of the model and optimize the parameters of the model, the gradient descent algorithm is introduced. The formula is as follows:

[0192] ;

[0193] Among them, θ is the learning rate, is the gradient of the loss function to the parameter. By continuously updating the model parameters θ, the model is gradually optimized so that it can process and analyze the monitoring data of the dam more accurately.

[0194] In order to prevent the model from overfitting during training, L2 regularization technology is introduced to maintain the generalization ability of the model. Even when facing unseen data, the model can make accurate predictions. The formula is as follows:

[0195] ;

[0196] Where λ is a regularization parameter, which is used to control the complexity of the model and prevent the model from being overly dependent on training data.

[0197] 6.3) Continuous learning and optimization of the model: Through online learning mode, the model is continuously updated with new monitoring data to maintain its sensitivity to environmental changes.

[0198] The key to online learning is the rapid response to new data and the dynamic adjustment of model weights. For example, when floods or extreme weather occur, the latest monitoring data is used to quickly update model parameters and improve the model's ability to respond to emergencies. At the same time, the system's closed-loop feedback mechanism allows each prediction result to be compared with the actual data, from which errors are obtained and the model is adjusted to maintain stable high performance during long-term monitoring.

[0199] In the process of continuous learning and optimization of the model, the error refers to the difference between the model prediction result and the actual observation value. This difference can be quantified in many ways. For example, the prediction error is the deviation between the model's output of the input data and the true label. The loss function is used to comprehensively evaluate this error, such as the commonly used loss functions such as mean square error and cross entropy. In online learning, the system compares each model prediction result with the latest monitoring data through a closed-loop feedback mechanism to calculate the feedback error. This process not only helps the model adjust weights and parameters, but also decomposes the error into deviation and variance to further optimize the model performance. Through continuous monitoring and analysis of errors, the system can improve its ability to respond to emergencies in real time and ensure scientific and effective decision-making support for dam safety management.

[0200] Through continuous optimization, the model weights and parameters are constantly adjusted according to the latest data to ensure that it can always provide scientific and effective decision-making support for dam safety management in various complex environments.

[0201] 7) Comprehensive dam safety early warning and response based on multimodal big model: Input the real-time monitoring data, the type and location of hidden dangers, the historical data of the dam related to the hidden dangers, the propagation path of the hidden dangers in the dam, and the external resource data related to the hidden dangers into the pre-trained multimodal big model, and analyze the dam historical data and real-time external resource data to obtain the probability of occurrence of hidden dangers in the dam, the degree of impact of hidden dangers, and the early warning threshold, so as to output the risk assessment results and the corresponding early warning mechanism.

[0202] The comprehensive early warning and response mechanism for dam safety based on multi-modal large models has the following specific technical details and implementation methods:

[0203] 7.1) The multimodal large model predicts the probability of dam hidden dangers by integrating real-time monitoring data, external resource data, and historical data P failure , and then calculate the possible impact of dam hidden dangers C impact , thereby generating the final risk assessment results.

[0204] (1) Probability of hidden dangers occurring P failure : The multimodal large model uses real-time monitoring data from multimodal sensors, including water flow pressure, vibration, temperature, displacement and other information, to analyze the current structural status of the dam.

[0205] These data reflect the operation status of the dam under different environmental conditions and provide rich feature vectors for the multimodal big model. Through deep learning algorithms, the multimodal big model can identify potential hidden dangers and calculate their probability of occurrence. P failureIn addition to real-time monitoring data from internal sensors, the multimodal big model also makes full use of external resource data and dam historical data to improve the accuracy of prediction. Through comprehensive analysis of these multi-dimensional data, the multimodal big model can more accurately determine the probability of hidden dangers caused by dam maintenance records and other factors, and dynamically adjust P failure The value of .

[0206] External resource data is obtained through online search, providing cutting-edge reference for risk prediction of multimodal big models. For example, accident case analysis around the world can help multimodal big models identify potential hazards similar to the current status of the dam, thereby adjusting the risk more accurately. P failure .

[0207] The historical data of the dam comes from the dam basic information knowledge base built by RAG (Retrieval Enhanced Generation). By analyzing the historical records, the multimodal big model can capture the long-term operation trend, maintenance history and hidden danger distribution of the dam, and combine these data to predict the probability of the current hidden danger. For example, if the historical records show that some areas have repeatedly had hidden dangers, the multimodal big model can increase the probability of hidden dangers in these areas. P failure value.

[0208] (2) Impact C impact :Multimodal large model evaluates the possible impact of dam hidden dangers C impact , assess the potential consequences of the risk. Impact C impact It depends not only on the probability of occurrence of hidden dangers, but also on the degree of threat posed by the hidden dangers to the key parts of the dam. The multimodal large model calculates the possible damage caused by each type of hidden danger by analyzing historical hidden danger records and dam maintenance data. C impact The calculation formula is:

[0209] ;

[0210] Among them, I i represents the impact of the i-th hidden danger event on the dam, is the probability of occurrence of this type of hidden danger event, and n represents the number of hidden danger event types. This formula integrates the risk levels of multiple hidden dangers and assigns different weights to each hidden danger. The model derives a comprehensive impact value based on factors such as structural damage, economic losses, and ecological impacts that these hidden dangers may bring. By quantifying the impact of various hidden dangers, managers can make preventive and response preparations in advance.

[0211] (3) Comprehensive risk scoreR： By calculating the probability of hidden dangers occurring P failure The degree of impact C impact Combined to derive a comprehensive risk score R :

[0212]

[0213] in, α p The probability of hidden dangers occurring P failure The weight of β c For the degree of impact C impact Weight. α p and β c There are many ways to obtain the weights. First, the weight values ​​can be initially set with the help of expert experience and industry standards to ensure their rationality. Secondly, collect user feedback and the opinions of managers to understand the importance of the probability of hidden dangers and the degree of impact to decision-making in actual operations, so as to make corresponding adjustments. In addition, through A / B testing, different weight combinations can be used in different scenarios, their effects can be compared, and the best configuration can be found. Using machine learning models is also an effective method. By training the model, the weights are automatically optimized to make them more in line with actual needs. Finally, statistical analysis of historical data can help evaluate the changing trends of the probability of hidden dangers and the degree of impact, thereby providing data support for weight setting. Through these methods, the comprehensive risk score will be more accurate and can provide an effective decision-making basis for the safety management of the dam.

[0214] The output of the multimodal large model directly affects the comprehensive risk score R The final value of the dam can provide data support for the risk management of the dam. In this way, managers can have a more comprehensive understanding of the potential risks and formulate reasonable risk response strategies accordingly. R When the value is high, the system may recommend increasing the monitoring frequency, strengthening the maintenance of the dam, or preparing emergency measures.

[0215] The multimodal big model analyzes multimodal data to calculate the probability and impact of hidden dangers, providing accurate and intelligent support for dam safety risk assessment. The comprehensive risk score generated by the system not only helps identify potential hidden dangers, but also provides a scientific basis for dam managers to ensure timely and reasonable risk response strategies.

[0216] 7.2) Dam safety risk warning triggering. The multimodal large model dynamically adjusts the warning threshold by comprehensively analyzing internal sensor data, external resource data and historical data. The specific adjustment mechanism is as follows:

[0217] (1) Threshold adjustment based on model output: When the multimodal large model predicts the probability of occurrence of a certain type of hidden danger P failure When the set threshold (for example, 70%) is exceeded, the system will actively lower the alarm threshold of the sensor related to the hidden danger to improve the sensitivity of monitoring. The degree of reduction is determined according to the degree to which the probability of the hidden danger exceeds the set threshold. For example, for every 5% increase in the probability of the hidden danger, the alarm threshold of the relevant sensor is reduced by 2% to 5%. Specifically, when the probability of hidden danger is high, the threshold is lowered; when the probability of hidden danger is low and the external environment is stable, the threshold can be moderately increased to reduce false alarms. For example, if the model predicts that the probability of structural stress hidden dangers in a certain area reaches 80%, which exceeds the set threshold by 10%, the system will reduce the alarm threshold of the stress sensor in the area by 5% to 10% to capture stress changes more sensitively.

[0218] (2) Threshold adjustment in response to external environmental data: When external environmental forecasts (such as heavy rain, earthquakes, etc.) show that the dam may have an impact, the system dynamically adjusts the alarm thresholds of relevant sensors according to the degree of environmental impact. When heavy rain is expected, the system lowers the alarm thresholds of seepage sensors and water level sensors. The reduction can be adjusted according to the rainfall level (moderate rain, heavy rain, heavy rain), for example, by 5% to 15%. In the case of earthquake warning, the system adjusts the alarm thresholds of ground stress and vibration sensors according to the magnitude of the earthquake and the distance between the epicenter and the dam. The greater the magnitude and the closer the distance, the greater the threshold reduction, which may be 10% to 20%. By lowering the threshold, the sensitivity of the sensor to anomalies caused by changes in the external environment is increased, ensuring that potential risks can be discovered in a timely manner.

[0219] (3) Threshold optimization based on historical data: The system regularly analyzes the deviation between sensor data and historical hidden danger records and dynamically adjusts the warning threshold. When the data of a certain sensor repeatedly triggers an alarm when it is 5% to 10% higher than the set threshold, the system determines that the current threshold may be too sensitive or insensitive. Depending on the degree of deviation, the threshold is appropriately raised or lowered, and the adjustment range is generally between 5% and 10%. For example, if historical data shows that the incidence of a certain type of hidden danger increases under specific environmental conditions (such as continuous high temperature), the system will lower the threshold of the relevant sensor during this period to improve monitoring sensitivity.

[0220] (4) Threshold anomaly detection and adaptive optimization: The system monitors changes in sensor data in real time and uses anomaly detection algorithms to dynamically adjust the warning threshold. When sensor data fluctuates abnormally (such as drastic changes in a short period of time) and does not conform to historical trends, the system identifies it as an anomaly. If the anomaly persists for more than a preset time (for example, 30 minutes), the system adjusts the threshold appropriately based on historical data and external information, with the adjustment range being 5% to 10%. When data anomalies are caused by equipment aging or external interference, the threshold is adjusted to avoid false positives or false negatives.

[0221] (5) Comprehensive evaluation and threshold adjustment: The system can set different weights for the above adjustment mechanisms, comprehensively consider the probability of hidden dangers output by the model, changes in the external environment and historical data, and determine the direction and magnitude of threshold adjustment through weighted calculation. In high-risk scenarios, when the probability of hidden dangers is high and the external environment is unfavorable, the threshold is lowered by up to 20%, and the monitoring frequency is increased. In low-risk scenarios, when the probability of hidden dangers is low and the environment is stable, the threshold can be moderately increased by 5% to 10% to reduce false alarms and improve system efficiency. The adjustment of the threshold is directly based on the probability of hidden dangers output by the model. The higher the probability, the more the threshold is lowered and the more sensitive the monitoring system is. This dynamic adjustment mechanism ensures the accuracy and flexibility of the early warning system.

[0222] 7.3) Dam safety risk early warning response mechanism. The dam safety risk early warning response mechanism is a key system to ensure that managers can quickly take effective countermeasures when facing potential risks. This mechanism relies on the output of a multimodal large model, combined with external resource data and dam historical data, to provide accurate and dynamic early warning response strategies for dam management. Through the network search function to obtain the latest global standards and specifications, academic papers and laws and regulations, as well as the dam basic information knowledge base built based on RAG technology, the system can establish a complete early warning response mechanism to ensure the scientificity and timeliness of decision-making.

[0223] The multimodal large model ensures that dam monitoring and emergency strategies meet international standards by analyzing the latest technical standards and specifications retrieved by module 2. For example, updates on emergency response to floods and earthquakes and installation requirements for monitoring equipment will be promptly incorporated into the early warning mechanism to help managers adjust emergency measures according to these standards to ensure their compliance and scientificity. Academic research provides the latest results for dam risk assessment, structural reinforcement, etc., supporting managers to formulate protection plans more scientifically and improve the effectiveness of emergency measures. In addition, the system can obtain the latest laws and regulations to ensure that dam managers comply with relevant laws in emergency responses such as disaster warning and evacuation to avoid legal risks.

[0224] The multimodal big model identifies high-risk areas based on historical maintenance and reinforcement records, and prioritizes monitoring these weak points in emergencies. Hidden danger logs and operation logs help the system analyze past hidden dangers and how they were handled, thereby providing response strategies for early warning responses. For example, if some areas have hidden dangers due to similar environmental conditions, the multimodal big model will propose corresponding emergency plans based on historical data, such as increasing monitoring frequency or implementing preventive reinforcement, to ensure that managers can respond quickly and reduce potential losses.

[0225] By combining external resource data obtained through online search functions with historical dam data provided by the RAG knowledge base, the multimodal large model ultimately forms a comprehensive risk warning response mechanism. The system can provide accurate early warning signals when the dam is facing risks, and combine standards and specifications, academic research, laws and regulations, and historical maintenance and reinforcement data to provide comprehensive emergency decision-making support for managers.

[0226] This mechanism ensures that managers can not only formulate emergency plans based on the latest standards, specifications, laws and regulations, but also identify high-risk areas and hidden danger transmission paths based on the historical operation data of the dam, and accurately formulate protection strategies. In an emergency, the system can dynamically adjust the response plan based on real-time analysis of multi-source data to ensure the scientificity, timeliness and compliance of emergency measures, and effectively ensure the safety and reliability of the dam.

[0227] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

Claims

1. A dam safety hazard propagation path analysis method based on knowledge graph, characterized by: The following steps are involved: Based on the historical operating status data of the dam and the information resources related to dam safety obtained from the outside, as well as the engineering and technical documents generated during the planning, design, construction and operation stages of the dam, the historical safety hazard assessment of the dam is carried out to identify the historical hazards and their characteristics; A knowledge graph is constructed based on the historical safety hazard assessment results, representing the dam's structural units, historical hazards and their relationships in the form of nodes and edges; Based on the real-time operating status data of the dam, the node attributes and relationships of the knowledge graph are updated, and the updated knowledge graph is used to analyze the current propagation path of the dam's safety hazards; The process of assessing the historical safety hazards of a dam and identifying the historical hazards and their characteristics includes: Build and update BIM models: Based on engineering technical documents, build a BIM model of the dam and extract attribute data from it to assign to each structural unit; associate historical operating status data, maintenance records, and stress history information with the corresponding parts of the BIM model; based on external information resources, incorporate the latest technical standards, specifications, laws and regulations, accident cases, and academic research into the BIM model, and update risk assessment parameters and hidden danger identification models; Use BIM models to identify historical hazards: Use the geometry and material information of the BIM model to conduct finite element analysis, fluid mechanics analysis, heat conduction analysis, and soil mechanics analysis to simulate the response of the dam under various working conditions and identify potential hazard areas and features, including stress concentration areas, areas that may be subject to excessive water pressure or scour, areas that may crack due to excessive temperature stress, and foundation areas that may experience settlement or landslides; Hazard verification and risk assessment: Compare the identified hazard areas with the historical operating status data and historical hazard records in engineering technical documents to verify the accuracy of the model analysis results; and classify the identified historical hazards according to their characteristics; Visualization and model update: Highlight the identified potential danger areas in the BIM model and mark the potential danger characteristics; continuously update the BIM model and various analysis models as new operation data and information resources are acquired; The process of building and updating the knowledge graph and analyzing the propagation path of dam safety hazards includes: Definition and attribute assignment of nodes and edges: define each structural unit of the dam as a node and assign attributes; node attributes include: basic physical information, historical hidden danger records, detection data and maintenance logs; define the physical connection and mechanical relationship between nodes as structural connection edges and assign attributes; Establish the relationship between nodes and edges: Establish the edges between nodes based on the historical operation status data, engineering technical documents and geometric information in the BIM model; Data storage and connectivity calculation: store node and edge information in the graph database; calculate the nodes and edges in the graph database to obtain the connectivity and importance scores of the nodes; Knowledge graph update: Based on the real-time collected dam operation status data and new external information resources, the health status and risk level of the nodes in the knowledge graph, as well as the weight and impact probability of the edges are updated; Hidden danger propagation analysis and update: Using the updated node and edge attributes and connectivity calculation results, analyze the path and probability of hidden dangers propagating from one structural unit to another; Construct hidden danger impact matrix and identify key nodes: Based on the hidden danger propagation probability, construct a matrix representing the hidden danger correlation between nodes; combine the hidden danger impact matrix and graph database to calculate the shortest path from the source node to other nodes of the hidden danger; identify the nodes with important influence in the hidden danger propagation process according to the importance score of the node and its role in the hidden danger propagation process.

2. A method according to claim 1, characterized in that: The historical operating status data and real-time operating status data of the dam include but are not limited to data collected by physical quantity sensors, vibration sensors, visual sensor data, audio sensors and environmental sensors installed on the dam.

3. A method according to claim 1, characterized in that: The information resources include: news reports, academic papers, patent data, technical standards, specifications and laws and regulations related to dam safety.

4. A method according to claim 2, characterized in that: The historical and real-time operating status data of the dam are preprocessed for subsequent steps; the preprocessing process includes: using standardized formulas for various data sensors to eliminate dimensional differences, aligning timestamps of data from different sensors; fusing data from different sensors according to certain weights; and dynamically adjusting weights during the fusion process based on sensor errors.

5. A method according to claim 2, characterized in that: The process of obtaining information resources related to dam safety from the outside includes: obtaining a large amount of external resource data through network search and data crawling; establishing inverted index and B+ tree index for external resource data; using index and parallel query technology to retrieve external resource data to find target data; caching frequently accessed data sources and query results; weighted fusion of target data and dam operation status data and dynamically adjusting weights according to data characteristics to form a comprehensive information resource data set.

6. A method according to claim 1, characterized in that: Pre-process the engineering and technical documents generated during the planning, design, construction and operation stages of the dam for subsequent steps; The preprocessing process includes: building a knowledge document library based on the engineering and technical documents generated during the planning, design, construction and operation stages of the dam; dividing the long documents in the knowledge document library into text blocks; using a text embedding model to convert the text blocks into digital vectors, capturing the semantic information of the text, and forming a vector database; the vector database is used to retrieve text blocks related to the query and provide the most relevant information.

7. A dam safety hazard propagation path analysis system based on knowledge graph, characterized in that: Used to implement the method according to any one of claims 1 to 6, comprising: a historical hidden danger identification module, a knowledge graph construction module and a hidden danger propagation path analysis module; The historical hazard identification module is used to evaluate the historical safety hazards of the dam and identify the historical hazards and their characteristics based on the historical operating status data of the dam and the information resources related to the dam safety obtained from the outside, as well as the engineering and technical documents formed during the planning, design, construction and operation stages of the dam. The knowledge graph construction module is used to construct a knowledge graph based on the historical safety hazard assessment results, representing the structural units of the dam, historical hazards and their relationships in the form of nodes and edges; The hidden danger propagation path analysis module is used to update the node attributes and relationships of the knowledge graph based on the real-time operating status data of the dam, and use the updated knowledge graph to analyze the safety hazard propagation path of the dam.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the knowledge graph-based dam safety hazard propagation path analysis method described in any one of claims 1 to 6.

Images