Power plant safety situation visualization system and method fusing digital twin and big data

Abstract

The application belongs to the technical field of intelligent power plant safety management and control, and specifically discloses a power plant safety situation visualization system and method fusing digital twinning and big data. The system comprises: a multi-modal data dynamic alignment and fusion module for outputting high-precision synchronized unified data frame sequences; a safety situation twinning model construction and updating module receiving the unified data frame sequences, driving power plant three-dimensional dynamic digital twinning model updating according to an incremental construction and verification mechanism; a big data correlation analysis and prediction engine running an analysis model based on a dynamic causal event graph, outputting a structured early warning report; and a hierarchical correlation type visualization mapping and presentation module generating an interactive and traceable panoramic safety situation view. The application can solve the problems of time and space mismatch caused by clock asynchronization and non-uniform coordinate system of multi-source heterogeneous data, lack of dynamic causal event graph model construction, and automatic identification and deduction of risk transmission links.

Description

Technical Field

[0001] This application belongs to the field of smart power plant safety management and control technology, specifically involving a power plant safety situation visualization system and method that integrates digital twins and big data. Background Technology

[0002] During the construction of smart power plants, production sites are generally equipped with video surveillance, sensor networks, personnel positioning systems, and various business management platforms to collect multimodal data, including video streams, time-series data, trajectory information, and event logs. To utilize this data to perceive the safety status of power plant operations, existing technologies typically integrate different data sources using timestamp alignment or preset logical rules, and construct digital twins based on 3D modeling tools, focusing on the visual display of equipment appearance and the static correlation of some status parameters. For safety monitoring and status presentation, threshold alarm engines are often used, combined with two-dimensional charts, alarm lists, and independent video footage to provide alarm information to management personnel.

[0003] However, this type of processing method struggles to address the spatiotemporal mismatch caused by differences in acquisition frequency, network latency, clock drift, and inconsistencies in coordinate systems in multi-source heterogeneous data. This results in inaccurate alignment of video behavior, positioning coordinates, and sensor readings in both time and space, affecting the reliability of fusion analysis. Digital twin models only perform simple mapping from data to attributes, lacking proactive verification and reverse correction mechanisms for input data quality. Errors or conflicts in the source data directly lead to inaccurate model representations, limiting their credibility in actual security management. Security monitoring still primarily relies on pushing and displaying isolated alarms, failing to construct dynamic analysis models that can characterize the causal relationships between equipment, personnel, and the environment. This makes it difficult to automatically identify risk transmission links from a large number of alarms and deduce accident evolution scenarios, and also prevents the intuitive fusion of risk paths and equipment status in a three-dimensional spatial view. Consequently, there is a lack of effective support for a comprehensive grasp of the security situation and rapid decision-making. Furthermore, existing systems mostly remain at the passive response stage of monitoring and alarms, lacking proactive prediction and tracing analysis capabilities based on a combination of historical operating patterns and real-time data, making it difficult to meet the actual needs for early detection, early warning, and early handling of security risks. Summary of the Invention

[0004] This application addresses the technical problems of insufficient safety situation awareness and decision support capabilities caused by the difficulty of achieving high-precision spatiotemporal alignment of multi-source heterogeneous data, reliable driving and reverse verification of digital twin models, and automatic inference and intuitive presentation of risk causal links in existing technologies. It provides a power plant safety situation visualization system and method that integrates digital twins and big data.

[0005] To achieve the above objectives, this application adopts the following technical solution: The first aspect of this application is a power plant safety situation visualization system that integrates digital twins and big data, including a multimodal data dynamic alignment and fusion module, a safety situation twin model construction and updating module, a big data correlation analysis and prediction engine, and a hierarchical correlation visualization mapping and presentation module. The multimodal data dynamic alignment and fusion module is used to acquire real-time video streams from the power plant, sensor time-series streams, personnel positioning streams, and event log streams, and to dynamically align them to obtain a unified data frame sequence that is synchronized in time and space. The security situation twin model construction and update module is used to drive and continuously update the three-dimensional dynamic digital twin model of the power plant according to the unified data frame sequence to obtain real-time status data. The big data correlation analysis and prediction engine is used to acquire historical operating data of the power plant, and to perform graph calculation and time series analysis based on the historical operating data and unified data frame sequence to obtain structured analysis results. The structured analysis results include safety event correlation, real-time risk level and predictive early warning signal. The hierarchical and relational visualization mapping and presentation module is used to dynamically render the structured analysis results and real-time status data to a three-dimensional twin scene, generating an interactive panoramic view of the security situation.

[0006] In some implementations, the multimodal data dynamic alignment and fusion module is specifically used for: A set of virtual entity anchor points are predefined in the digital twin space, and each virtual entity anchor point is associated with a physical entity and its standard three-dimensional coordinates; The observation information related to the physical entity corresponding to the virtual entity anchor point is identified by the power plant's real-time video stream, sensor time-series stream, personnel positioning stream, and event log stream respectively; and the observation information of different data streams belonging to the same virtual entity anchor point is associated. For observation information associated with the same virtual entity anchor point, the timestamp difference and spatial projection deviation between different data streams are calculated. Using the standard three-dimensional coordinates as a reference, the Kalman filter algorithm is used to estimate and dynamically compensate the local clock offset and spatial registration offset of each data stream online to obtain a unified data frame sequence.

[0007] In some implementations, the security posture twin model construction and update module is specifically used for: Obtain power plant design drawings and equipment ledgers, and construct a static twin basis based on the power plant design drawings and equipment ledgers. The static twin basis includes a geometric model, topological relationships, and basic attributes. The unified data frame sequence is input into the static twin base to drive dynamic updates, forming the three-dimensional dynamic digital twin model of the power plant; wherein, the state data in the unified data frame sequence is directly mapped to model attributes, the event and behavior data in the unified data frame sequence are extracted into twin behavior fragments through pattern recognition, and the twin behavior fragments are stored in the model knowledge base. Preset physical rules and business logic rules: When the state of the power plant's three-dimensional dynamic digital twin model driven by the unified data frame sequence conflicts with the physical rules or business logic rules, conflict information is generated and sent to the multimodal data dynamic alignment and fusion module as a data quality feedback signal.

[0008] In some implementations, the big data correlation analysis and prediction engine is specifically used for: A dynamic causal event graph model is constructed. The nodes of the dynamic causal event graph model are composed of devices, alarm types, personnel roles, and environmental areas. The edge weights between nodes represent the strength of causal influence between nodes. Event pairs are extracted from the historical operational data and unified data frame sequence. The temporal co-occurrence relationship, transition probability and information theory index of the event pairs are analyzed, and the edge weights are dynamically calculated and updated to obtain the security event association relationship. Time series analysis is performed on the unified data frame sequence, and the current risk status is assessed by combining the security event correlations. Real-time risk levels and predictive warning signals are generated, and the security event correlations, real-time risk levels, and predictive warning signals are used as structured analysis results.

[0009] In some implementations, the hierarchical relational visualization mapping and presentation module is specifically used for: Based on the structured analysis results, risk transmission paths are dynamically drawn in the 3D twin scene, and alarm information and parameter information associated with the nodes of the risk transmission path are linked and bound. At the same time, the real-time status data is dynamically rendered to the 3D twin scene. The interactive panoramic view of the security situation is generated by fusing and presenting the real-time status data within the same view, and by extracting short-term historical trajectories and long-term statistical data from the unified data frame sequence.

[0010] The second aspect of this application provides a method for visualizing the safety situation of power plants by integrating digital twins and big data, implemented based on the aforementioned system for visualizing the safety situation of power plants by integrating digital twins and big data, including: The system acquires real-time video streams from the power plant, sensor time-series streams, personnel location streams, and event log streams, and performs dynamic alignment to obtain a unified data frame sequence that is synchronized in time and space. Based on the unified data frame sequence, the three-dimensional dynamic digital twin model of the power plant is driven and continuously updated to obtain real-time status data; Historical operating data of the power plant is acquired. Based on the historical operating data and the unified data frame sequence, graph calculation and time series analysis are performed to obtain structured analysis results, which include safety event correlations, real-time risk levels, and predictive early warning signals. The structured analysis results and real-time status data are dynamically rendered onto a 3D twin scene to generate an interactive panoramic view of the security situation.

[0011] In some implementations, the process of acquiring real-time video streams from the power plant, sensor time-series streams, personnel location streams, and event log streams, and dynamically aligning them to obtain a unified data frame sequence with spatiotemporal synchronization, includes: A set of virtual entity anchor points are predefined in the digital twin space, and each virtual entity anchor point is associated with a physical entity and its standard three-dimensional coordinates; The observation information related to the physical entity corresponding to the virtual entity anchor point is identified by the power plant's real-time video stream, sensor time-series stream, personnel positioning stream, and event log stream respectively; and the observation information of different data streams belonging to the same virtual entity anchor point is associated. For observation information associated with the same virtual entity anchor point, the timestamp difference and spatial projection deviation between different data streams are calculated. Using the standard three-dimensional coordinates as a reference, the Kalman filter algorithm is used to estimate and dynamically compensate the local clock offset and spatial registration offset of each data stream online to obtain a unified data frame sequence.

[0012] In some implementations, the Kalman filtering algorithm is used to perform online estimation and dynamic compensation of the local clock offset and spatial registration parameters of each data stream, including: For a data stream corresponding to observation information belonging to the same virtual entity anchor point, a calibration state vector to be estimated is defined. The calibration state vector includes the clock offset of the data stream relative to the time reference corresponding to the standard three-dimensional coordinates, and the spatial registration offset of the observation coordinates of the data stream relative to the standard three-dimensional coordinates. Set the initial estimate of the calibration state vector, predict the calibration state vector at the next moment before obtaining the new observation coordinates of the data stream each time, calculate the Kalman gain after obtaining the new observation coordinates, and use the mapping relationship between the new observation coordinates and the predicted calibration state vector to calculate the predicted observation value of the virtual entity anchor point position. The actual observed value of the new observation coordinates is compared with the predicted observed value to obtain the observation residual. The observation residual and the Kalman gain are used to update the calibration state vector to obtain the estimated clock offset and the estimated spatial registration offset at the current time. Based on the estimated clock offset and spatial registration offset, the data stream is timestamped and its coordinates are corrected to obtain a unified data frame sequence.

[0013] A third aspect of this application is a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed by the power plant safety posture visualization method integrating digital twins and big data.

[0014] A fourth aspect of this application is a computer program product comprising a computer program that, when executed by a processor, implements the aforementioned method for visualizing the power plant safety situation by integrating digital twins and big data.

[0015] Compared with the prior art, this application has the following beneficial effects: In the multimodal data dynamic alignment and fusion module, this application acquires and dynamically aligns real-time video streams, sensor time-series streams, personnel positioning streams, and event log streams from the power plant, resulting in a unified data frame sequence that is spatiotemporally synchronized. This allows heterogeneous data from different acquisition frequencies, coordinate systems, and clock sources to be unified under the same spatiotemporal reference, solving the problem of high-precision fusion of multi-source data due to spatiotemporal mismatch. The safety situation twin model construction and update module drives and continuously updates the power plant's three-dimensional dynamic digital twin model based on this unified data frame sequence, obtaining real-time status data. This ensures that the digital twin model is no longer a static display or simple mapping, but is continuously driven by spatiotemporally aligned real-time data, guaranteeing a reliable correspondence between the model's state and the actual operating state of the power plant. The big data correlation analysis and prediction engine acquires historical operating data and combines it with the unified data frame sequence for graph calculation and time series analysis, obtaining structured analysis results containing safety event correlations, real-time risk levels, and predictive early warning signals. This enables the system to automatically mine causal relationships between events from massive amounts of data and achieve risk prediction, changing the passive mode of existing technologies that can only push isolated alarms. The hierarchical and relational visualization mapping and presentation module dynamically renders structured analysis results and real-time status data to a 3D twin scene, generating an interactive panoramic view of the security situation. This allows risk transmission paths, equipment status, and early warning information to be presented intuitively in a unified 3D space, providing managers with visual support for direct interaction and traceability decision-making. Attached Figure Description

[0016] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 A schematic diagram of the structure of a power plant safety situation visualization system that integrates digital twins and big data, provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] like Figure 1 As shown, this application provides a power plant safety situation visualization system that integrates digital twins and big data, including a multimodal data dynamic alignment and fusion module, a safety situation twin model construction and update module, a big data correlation analysis and prediction engine, and a hierarchical correlation visualization mapping and presentation module. The multimodal data dynamic alignment and fusion module is used to acquire real-time video streams from the power plant, sensor time-series streams, personnel positioning streams, and event log streams, and to dynamically align them to obtain a unified data frame sequence that is synchronized in time and space. Specifically, the system simultaneously receives real-time video streams from various areas of the power plant, DCS and sensor time-series data streams, personnel positioning trajectory streams, and event log streams from the safety management system. First, a set of virtual entity spatiotemporal anchor points bound to physical entities (e.g., specific equipment, key points on inspection routes) is predefined within the digital twin space. For each frame of input data, target detection and feature matching techniques are used to associate the target bounding box, positioning coordinates, and sensor-assigned equipment in the video with the nearest virtual anchor point in real time. For multimodal observation data associated with the same anchor point (e.g., the image of the same equipment in the video at a certain moment, sensor readings, and the positions of surrounding personnel), the system calculates the timestamp differences and spatial projection deviations between the data streams. Using a Kalman filter algorithm, based on the standard spatiotemporal coordinates of the anchor points, the system recursively estimates the local clock offset and spatial registration offset of each data stream online, and dynamically compensates and corrects the original data stream accordingly, ultimately outputting a unified data frame sequence that is spatiotemporally synchronized.

[0019] The security situation twin model construction and update module is used to drive and continuously update the three-dimensional dynamic digital twin model of the power plant according to the unified data frame sequence to obtain real-time status data. Specifically, the unified data frame sequence is input into the safety situation twin model construction and update module. The module constructs an initial three-dimensional static twin base based on the power plant BIM model. The data frames continuously drive model updates: equipment status parameters (temperature and pressure) are directly mapped to the attribute values ​​and visualization status (color changes) of the corresponding equipment in the three-dimensional model; personnel positioning data drives the virtual character to move in the scene; event logs are extracted into standard "twin behavior fragments" after pattern recognition and stored in the knowledge base; at the same time, the module has built-in physical rule constraints (energy conservation and material flow) and business logic rules (operation ticket sequence, area access permissions); when the model status driven by the input data conflicts with the rule base (for example, the data shows that personnel appear in a physically inaccessible area), the module immediately triggers a consistency alarm and starts a tracing procedure. This procedure transmits the conflict information back to the data alignment and fusion module as a feedback signal for evaluating the data fusion quality or triggering the realignment of specific data streams, forming an intelligent closed loop from data to model-driven and from model to data verification. The big data correlation analysis and prediction engine is used to acquire historical operating data of the power plant, and to perform graph calculation and time series analysis based on the historical operating data and unified data frame sequence to obtain structured analysis results. The structured analysis results include safety event correlation, real-time risk level and predictive early warning signal. Specifically, the big data correlation analysis and prediction engine simultaneously connects to historical operational databases and real-time unified data frame sequences. The core of the engine is a dynamic causal event graph model, whose nodes consist of equipment entities, alarm types, personnel roles, and environmental areas. By analyzing the temporal co-occurrence, transition probabilities, and information entropy changes of event pairs in historical data frame sequences, the weights of edges between nodes are dynamically calculated and updated, forming a safety knowledge graph characterizing the intensity of causal influence. During real-time analysis, newly occurring safety events (such as the #3 pump bearing temperature exceeding limits) are used as input, and multi-hop reasoning or random walk simulations are performed in the causal graph to quickly identify potential risk transmission links, such as temperature exceeding limits, increased vibration, seal leakage, fire risk, critical vulnerable nodes, and possible evolution scenarios. The engine outputs structured risk path reports and predictive early warning signals. The hierarchical and relational visualization mapping and presentation module is used to dynamically render the structured analysis results and real-time status data to a three-dimensional twin scene, generating an interactive panoramic view of the security situation.

[0020] Specifically, the hierarchical and relational visualization mapping and presentation module receives real-time status from the twin model, risk path reports and early warning signals from the analysis engine. This module follows three mapping principles: 1) Spatial layering principle: Dynamically determining information granularity based on the user's perspective level (plant-wide, workshop, equipment level). For example, the plant-wide perspective displays personnel heatmaps and major alarm aggregation icons, while the equipment-level perspective displays detailed parameter curves; 2) Logical association binding principle: Intelligently binding non-spatially directly corresponding analysis results, i.e., risk transmission paths, to the 3D scene. For example, dynamically drawing risk paths with highlighted flowing lines along the spatial direction of relevant equipment and pipelines, with detailed alarms and parameters associated with path nodes displayed as expandable floating panels; 3) Temporal fusion principle: Fuding and presenting the current status (model color), recent trajectories (personnel movement trails), and long-term statistics (regional violation heatmaps) within the same view. Ultimately, a 3D panoramic safety situation view integrating real-time monitoring, historical backtracking, risk perspective, and interactive query is generated, allowing users to conduct in-depth exploration through clicking, dragging, and zooming.

[0021] In one embodiment of this application, in conjunction with the above system architecture, this application also provides a method for visualizing the safety status of power plants by integrating digital twins and big data, including: S1: Acquire real-time video streams from the power plant, sensor time-series streams, personnel location streams, and event log streams, and dynamically align them to obtain a unified data frame sequence that is synchronized in time and space. S2, based on the unified data frame sequence, drive and continuously update the three-dimensional dynamic digital twin model of the power plant to obtain real-time status data; S3, acquire historical operating data of the power plant, perform graph calculation and time series analysis based on the historical operating data and unified data frame sequence, and obtain structured analysis results, including safety event correlation, real-time risk level and predictive early warning signal; S4. The structured analysis results and real-time status data are dynamically rendered into a three-dimensional twin scene to generate an interactive panoramic view of the security situation.

[0022] Specifically, acquiring real-time video streams from the power plant, sensor time-series streams, personnel location streams, and event log streams, and dynamically aligning them to obtain a unified data frame sequence for spatiotemporal synchronization includes: A set of virtual entity anchor points are predefined in the digital twin space, and each virtual entity anchor point is associated with a physical entity and its standard three-dimensional coordinates; The observation information related to the physical entity corresponding to the virtual entity anchor point is identified by the power plant's real-time video stream, sensor time-series stream, personnel positioning stream, and event log stream respectively; and the observation information of different data streams belonging to the same virtual entity anchor point is associated. For observation information associated with the same virtual entity anchor point, the timestamp difference and spatial projection deviation between different data streams are calculated. Using the standard three-dimensional coordinates as a reference, the Kalman filter algorithm is used to estimate and dynamically compensate the local clock offset and spatial registration offset of each data stream online to obtain a unified data frame sequence.

[0023] Specifically, during the system initialization phase, based on the power plant's 3D design drawings, equipment layout, and process flow, a set of virtual entity spatiotemporal anchor points are pre-established in the digital twin virtual space. Each anchor point is associated with a specific physical entity object (e.g., the temperature measurement point of the high-pressure cylinder bearing of turbine #1, inspection point A on the 12-meter platform of boiler #2, and the entrance / exit of the main control room), and is assigned a long-term, stable, and accurate standard 3D coordinate. These coordinates do not originate directly from any real-time data stream but are based on the actual ground location calibrated by the physical design, serving as a unified and stable absolute reference benchmark for the spatiotemporal alignment of all subsequent data streams. When the original data frames of each data stream (video stream, positioning stream, and sensor stream) are input, the system performs feature extraction and matching in parallel. Video stream processing: Target detection algorithms are run on video frames to identify targets such as device outlines and personnel. Using pre-calibrated camera parameters (intrinsic and extrinsic parameters) and 3D scene registration information, the targets in the detection frame are projected into 3D space to obtain 3D observation coordinates. Nearest neighbor search is used to match these observation coordinates with a predefined virtual entity anchor point library to determine the most likely associated anchor point. Location stream processing: Two-dimensional or 3D coordinates reported by personnel positioning terminals, such as UWB and Bluetooth beacons, are also directly matched with the virtual entity anchor point library using nearest neighbor matching to associate them with specific anchor points. Sensor stream processing: Based on the physical installation location log of sensors, sensor IDs are directly associated with specific virtual entity anchor points. The binding relationship is established so that each data packet (video target, positioning point, sensor reading) is marked with its corresponding virtual entity anchor point ID after association. The system maintains a cache with the virtual entity anchor point ID as the key. When different modal observation data belonging to the same anchor point ID arrive successively within a short time window, such as video observations, temperature sensor readings, and positioning points of surrounding personnel from the same device at a certain moment, the system initiates alignment calculations, including: time deviation calculation: extracting the local timestamp carried by each data packet and calculating their relative time difference; spatial deviation calculation: for data with coordinate information (video projection coordinates, positioning coordinates), calculating its Euclidean distance from the standard coordinates of the associated anchor point as the spatial projection deviation; after using a card... The Mann filter algorithm estimates and dynamically compensates for the local clock offset and spatial registration parameters of each data stream online. All data streams are then calibrated to a unified spatiotemporal framework based on the standard coordinates of virtual entity anchor points and the system's global time. The system then packages different modal data that are at the same time (accurate to the same millisecond-level time window after calibration) and associated with the same or adjacent anchor points—namely, the calibrated video target coordinates, sensor readings, and personnel positions—into a structured unified data frame, assigns it a precise global timestamp and spatial reference information, and outputs it to downstream modules. This process is continuously performed online, adaptively tracking and eliminating spatiotemporal deviations caused by network latency, clock drift, etc., to ensure the accuracy of the fused data.

[0024] The Kalman filter algorithm is used to perform online estimation and dynamic compensation of the local clock offset and spatial registration parameters of each data stream, including: For the association with the first The first virtual entity anchor point There are data streams, and their calibration state vectors to be estimated are defined as follows: ,in, For the first The clock offset of each data stream relative to the global standard time. For the first Spatial registration offset of each data stream observation coordinate relative to the standard coordinates of the anchor point For transpose; When in the Local timestamps of each data stream At that moment, the target and the first The virtual entity anchor points are associated, and the observed coordinates are obtained. The observations are relative to the global standard time. The actual location below The relationship between them can be modeled as follows: ,in, From local time From the moment The actual three-dimensional coordinates observed in each data stream. To observe noise; Set the initial estimate of the state vector based on the initial calibration or prior information. and its error covariance matrix Before each new observation is obtained, based on the assumption that the calibration parameters change slowly over time, the state estimate and covariance at the next time step are predicted using a process model; after obtaining the new observation, the Kalman gain is calculated; using the observation model, the predicted calibration state is substituted to calculate the state estimate for the next time step. A virtual entity anchor point at Predicted observations at time and location; The observed values ​​are compared with the predicted observed values ​​to obtain the observation residuals. The observation residuals and Kalman gain are then used to update the state estimate and covariance. The latest estimate and Applied to raw data streams.

[0025] Specifically, for each independent data stream (e.g., the video stream numbered CAM_01, or the UWB positioning stream in region ZONE_A), a dedicated Kalman filter is established, and the state vector of the filter is set as follows: ,in, Let $\mathbf$ be the offset of the data stream clock relative to the system's global standard time. The spatial registration offset of the observation coordinate system of the data stream relative to the global coordinate system of the digital twin is to be estimated; based on the initial calibration or historical statistical data after equipment installation, the initial estimate of the state vector and its uncertainty, i.e., the error covariance matrix, is given. Before receiving new observation data, the Kalman filter first performs a prediction step. Based on the reasonable assumption that clock offset and spatial offset change slowly over a short time scale, a process model is established. The simplest constant-value model is typically used, assuming that the state vector itself remains unchanged between adjacent time steps, but is superimposed with a small Gaussian process noise. Therefore, the prediction equation is:

[0026]

[0027] in, and They represent based on the previous time step. Information, for the current moment Prior predictions of states and their covariances The process noise covariance matrix is ​​a small positive definite matrix used to model the slow, small drifts that may occur in the state, ensuring the filter's ability to track changes. When data stream Local clock A new observation data packet is generated at any time, and this data packet is successfully associated with the virtual entity anchor point. At this point, the filter enters the update step, specifically: let the observed target coordinates be... This observation is considered to be in global standard time. (clock offset) (One of the state variables to be estimated) At time , for the anchor point The actual location is measured with noise. Establish an observation model: ,in, For observation noise, it is usually assumed to be zero-mean Gaussian white noise, and its covariance matrix is: This reflects the observation accuracy of the data stream (e.g., a specific camera, a positioning base station). It can be obtained through high-precision reference data streams (such as laser scanning), static models of the device, or by inference from the state of other aligned data streams. In this embodiment, it is an anchor point for a virtual entity. Standard coordinates (Statically invariant or provided by a high-precision model) as an approximation of its true position; Kalman gain This determines the extent to which the predicted state should be corrected by the current observations, and its calculation formula is as follows: ,in, It is the observation model for the state vector The Jacobian matrix (i.e., the observation matrix) in this model, Because the observations are directly related to the spatial offset Related, but with a weak instantaneous relationship to time offset (can be determined by...) The derivatives with respect to time are correlated, but for simplicity they are often treated as independent, or estimated through multiple anchor point observations. ; State updates include: calculating observation residuals ,in, For predicting the state Spatial offset portion; Update state estimate Update error covariance: Thus, the posterior optimal estimate of the current state vector and its uncertainty have been obtained. .

[0028] In one embodiment of this application, the latest estimated and Applied to the raw data stream, including: For each data packet subsequently input into the data stream, its timestamp is corrected to... and correct its observation coordinates to The corrected data packets are then output as part of a unified data frame sequence.

[0029] Specifically, the latest estimates and It is applied to real-time compensation of the raw input following the data stream. After compensation, the timestamp of the data packet is aligned with the system's global time base, and its coordinates are corrected to a unified spatial coordinate system with the standard coordinates of the virtual entity anchor point as a reference. Thus, it can be directly used to generate a high-quality unified data frame sequence. This filtering and compensation process runs continuously online, enabling the system to adaptively track and eliminate spatiotemporal deviations caused by factors such as network latency fluctuations, device clock drift, and sensor installation micro-movements.

[0030] In one embodiment of this application, the incremental construction and verification mechanism for the security posture-oriented twin model includes: Based on power plant design drawings and equipment ledgers, a static twin basis including geometry, topology and basic attributes is constructed; The unified data frame sequence is continuously input to drive model updates; for state data, it is directly mapped to model attributes; for event and behavior data, it is extracted into twin behavior fragments through pattern recognition and stored in the model knowledge base in the form of reusable skills. When physical rules and business logic rules are set, and a conflict arises between the model state driven by input data and the rule base, a tracing procedure is triggered to transmit the conflict information back to the multimodal data dynamic alignment and fusion module as a feedback signal for it to evaluate its own data fusion quality or trigger realignment.

[0031] Specifically, based on the power plant's design drawings (BIM model), equipment information model (P&ID diagram), and asset ledger, an initial three-dimensional static digital twin base is constructed. This base includes: a geometric and topological model: an accurate three-dimensional building structure, equipment shape model, and a topological network reflecting physical connections (pipeline connections, electrical wiring); and a basic attribute library: associating static attributes with each entity model (such as pumps, valves, and switchgear), including equipment ID, name, model, rated parameters (rated pressure, temperature), design location, and system to which it belongs. Spatial index structure: Establish efficient spatial indexes such as octrees or BSP trees to quickly realize spatial queries, coordinate associations, and subsequent collision detection and spatial analysis of virtual entity anchor points; The unified data frame sequence from the alignment and fusion module is received and parsed in real time, driving the twin model to dynamically update and accumulate knowledge: for the real-time monitoring data (sensor readings) carried in the data frame, the corresponding device entity in the base model is located according to its associated virtual entity anchor point ID. The numerical values ​​(temperature, pressure, vibration amplitude) are directly mapped to the dynamic attributes of the equipment entity. The mapping implementation steps are as follows: During the twin model construction phase, a set of mappable dynamic attribute channels are predefined for each equipment entity. These channels correspond one-to-one with the monitoring points in the equipment ledger and are associated through configuration files or databases. When a unified data frame is received, the system parses the data entries in the frame. The system completes the matching through the following links: Based on the virtual entity anchor point ID, the corresponding equipment entity is located in the twin model; Based on the sensor ID, the corresponding channel is found in the pre-configured dynamic attribute channels of the entity; After successful matching, the system performs numerical updates, state deduction, and visualization synchronization. Simultaneously, based on preset thresholds (alarm values, danger values), the visualization status of the driving device model in the 3D scene changes, such as changing the model color (green, yellow, red), triggering alarm flashing icons, or updating the suspended numerical panel; for discrete events such as event logs and video analysis results (not wearing a safety helmet, unauthorized entry into the area) in the data frame, a rule engine or lightweight pattern recognition algorithm is used to extract semantically meaningful twin behavioral fragments by combining their temporal and spatial context; for example, a series of events such as personnel entering area A, triggering a high temperature alarm, and personnel quickly leaving can be identified as a complete high-temperature area avoidance behavior fragment; The extracted behavioral fragments, along with their triggering conditions, associated entities, process data snapshots, and other information, are stored in the model knowledge base in the form of structured skills. Each skill can be retrieved and reused for subsequent behavior prediction, procedure compliance verification, or virtual training for new employees. While the model is being updated, the system's built-in rule engine works in parallel to verify the consistency of the model's state after the update: Physical rule constraints: Define basic physical law constraints, such as the conservation of mass, which includes the principle that the total inlet flow of a pipe equals the total outlet flow; the conservation of energy, which includes the balance between heat generation and dissipation in equipment; and the mutual exclusion of states, which means that equipment cannot be in both operating and maintenance states at the same time. Business logic rules: Define the power plant's safety production procedures and operating logic. Hot work must maintain a safe distance from flammable materials. Gas detection must be performed before entering a confined space and the results must be qualified. Equipment isolation operations must follow the prescribed electrical / mechanical isolation sequence. The rule engine continuously monitors dynamic attributes and events in the twin model. When a conflict is detected, such as when the twin model shows that a valve is closed, but the associated flow sensor data frame shows a continuous flow, or when it shows that a person has entered an area marked as a confined space without gas detection, a consistency conflict alarm is immediately generated. Based on the timestamp and entity information in the conflict alarm, the system traces back to the original unified data frame that triggered the model state, and can further trace back to the original multimodal data packet (video clip, original sensor readings, etc.) that generated the data frame; the conflict alarm and the traced original data information are packaged into a quality feedback signal and transmitted back to the multimodal data dynamic alignment and fusion module; after receiving the feedback signal, the alignment and fusion module starts the self-test process.

[0032] The core of the big data correlation analysis and prediction engine is a dynamic causal event graph model. Specifically, the nodes of the graph consist of devices, alarm types, personnel roles, and environmental areas. The edge weights of the graph represent the strength of causal influence between nodes, which are dynamically calculated and updated by analyzing the temporal co-occurrence, transition probability, and information theory indicators of event pairs in historical unified data frame sequences. In real-time analysis, newly occurring security events are used as input, and random walks or community diffusion simulations are performed in the dynamic causal event graph to quickly identify potential risk transmission links, key vulnerable nodes, and accident evolution scenarios, outputting a structured risk path report.

[0033] Specifically, a unified node system covering four major categories of entities will be constructed, including: Equipment nodes: Key equipment or subsystems (such as #1 steam turbine, main feedwater system) are nodes, and their status is characterized by the quantification of associated sensor data (excessive temperature, excessive vibration). Alarm type node: Standardize alarm codes and descriptions across the entire plant (e.g., FIRE_ALARM, GAS_LEAK), with each type as an independent node; Personnel Role Nodes: Based on organizational structure and work permits, define role nodes (inspectors, maintenance work supervisors, outsourced personnel), and their behavior is quantified by video analysis and access control logs (failure to inspect along the route, unauthorized operation). Environmental area nodes: Divide physical or logical areas (hydrogen station, coal conveying corridor, #1 unit control area), and their status is determined by the comprehensive data of the sensor cluster in the area (low oxygen content, high dust concentration). The system maintains a dynamic causal event graph, where directed edges exist between nodes, and the weights of the edges are... This represents the causal influence strength from node A to node B. The weights are calculated using a combination of offline historical learning and online rolling updates. Temporal co-occurrence analysis: Within a sliding window of historical event sequences, it counts the occurrences of events at node B after an event at node A, with a time delay. The frequency of occurrence within a given time frame is used to quantify the association strength beyond random co-occurrence using methods such as time-shifted mutual information or improved Granger causality tests; transition probability estimation: based on historical event sequences, the conditional probability of event B occurring within a subsequent specific time window is calculated given that event A has occurred. This probability is used as a basic indicator of causal strength; information theory indicator fusion: combining indicators such as transfer entropy to quantify the information flow from A to B, reducing spurious associations, for example, calculating transfer entropy. This measures the degree to which knowing the history of A reduces the uncertainty in predicting the future of B. Based on the above indicators, the initial weights are calculated as follows: ,in, This is the normalized transition entropy value from node A to node B. This is a causality score derived from Granger causality tests. , and To ensure adjustable weighting coefficients, the system periodically (hourly) recalculates weights and updates the graph using recent 24-hour event data, enabling the model to adapt to changes in system operating conditions. Random walk and community diffusion simulations are performed, recording the most probable paths from downstream high-risk nodes back to seed nodes. These paths, linked by a series of high-weight edges, are extracted as potential risk transmission links, such as hydrogen leaks at hydrogen stations, ventilation system malfunctions, flammable gas accumulation, electrical room sparks, and fires / explosions. The engine summarizes the simulation results and automatically generates a structured risk path report, including: triggering events, key transmission links, impact range and key nodes, scenario evolution summary, and recommended measures.

[0034] The hierarchical-associative visualization mapping and presentation module follows the following principles in its mapping logic: Spatial layering principle: Dynamically determine the granularity and type of visualized information based on the user's perspective hierarchy; Logical association binding principle: For information output by the analysis engine that is not directly spatially corresponding, a main line binding and branch line radiation method is used for visualization; Temporal fusion principle: Real-time status, short-term history and long-term statistics are presented in the same view.

[0035] Specifically, the system establishes a three-tiered core view hierarchy and dynamically switches between levels based on user actions (such as zooming, panning, and selection): Plant-wide view: Triggered when the user's viewpoint height exceeds a set threshold (e.g., 100 meters), or when the plant-wide overview mode is selected; a simplified plant outline and main building blocks are used as the background; real-time personnel gathering areas are displayed in the form of a dynamic heat map, with color depth representing density; all real-time alarms across the plant are aggregated by region, with each region displaying an aggregated alarm icon and its quantity, which can be clicked to drill down; plant-wide KPIs are displayed in a fixed area of ​​the screen, such as the current total number of alarms, the number of major risk operations, and the total number of safe operating days of the plant; all equipment details, parameter panels, and specific alarm lists are hidden. Workshop / System-level View: Triggered when the viewpoint is focused on a workshop or process system, or when a specific area / building is clicked from the plant-wide view; Displays a complete, detailed 3D model of the area, including major equipment, pipelines, and platforms; Equipment is highlighted in different colors (green / yellow / red) according to its real-time status (normal / warning / fault); Specific alarm icons (flame, exclamation mark) are directly attached to the equipment or location where the alarm occurred; Displays simplified models or point markers of personnel in the area, possibly with role identification (distinguished by the color of safety helmets); Hides internal equipment structures, non-critical small instruments, and overly detailed parameter curves; Equipment / Location-Level View: The trigger condition is clicking or selecting a specific piece of equipment or dashboard in the workshop view; a close-up view is taken centered on the equipment, possibly showing its cross-sectional structure or internal components; its real-time data curve, historical trend chart, current reading, and units are displayed floating next to the equipment; the equipment's maintenance records, last maintenance time, and associated work order number are displayed; all current and historical alarm entries related to the equipment are listed; other unrelated equipment and area information is hidden; for the structured risk path report output by the big data analytics engine, the module performs the following binding and rendering: the system parses the node sequence (equipment A, area B, system C) in the path report and locates these nodes in the digital twin scenario. The corresponding 3D entity or logical region center point; if the node represents an entity with a clear spatial orientation (pipe, conveyor belt), a path is generated along the 3D model centerline of that entity; if the node represents a device or region, a smooth Bézier curve or straight line is generated between them as a connection; along the generated geometric path, a highlighted flowing line is drawn in real time, which has a flowing particle or light strip effect, and the flow direction represents the assumed direction of risk transmission (from A to B and then to C). The line color can be set according to the overall risk level of the path (red represents high risk, orange represents medium risk); near each node (device or region) of the path, a semi-transparent floating information panel is generated.

[0036] This panel is collapsed by default, displaying only the node name and key status icons. When the user hovers the mouse over or clicks on a path line or node, all floating panels associated with that node expand simultaneously. Within the expanded panels, all detailed information related to that node is displayed in categories, such as triggered alarms, real-time parameters, handling suggestions, and related documents. In the same visualization view, past, present, and future time-dimensional information is integrated, including real-time status, short-term history, and long-term statistics.

[0037] In one embodiment of the present invention, the logical relationship binding principle is specifically as follows: the risk transmission path output by big data correlation analysis and prediction is dynamically drawn in the three-dimensional twin scene as a highlighted flowing line along the spatial direction of the relevant physical equipment or pipeline, and the detailed alarms and parameter information associated with the path nodes are associated and bound in the form of an expandable floating panel.

[0038] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements a method for visualizing the safety status of a power plant by integrating digital twins and big data.

[0039] In one embodiment, a computer program product is provided, the computer program product including a computer program, which, when executed by a processor, implements a method for visualizing the safety status of a power plant by integrating digital twins and big data.

[0040] This invention, through the construction of a virtual entity spatiotemporal anchoring method and an online Kalman filter estimation algorithm, can automatically compensate for clock skew and spatial deviations between different data streams, achieving millisecond-level and centimeter-level accuracy synchronization of cross-modal data within a unified spatiotemporal framework. Through an incremental construction and verification mechanism, the digital twin model can not only be driven by data in real time but also perform reverse verification of input data using built-in physical and business rules. When conflicts are detected, source tracing can be triggered and feedback can be sent to the data fusion layer, forming an intelligent closed loop of data-driven model and model verification data, significantly improving the reliability of the entire system output. Reliability and robustness; Based on a dynamic causal event graph model, it can automatically learn and update the causal relationships between events from massive historical and real-time data, realize the automatic identification and deduction of risk transmission links, enable the system to see potential systemic risks from isolated alarms, and achieve early prediction and root cause location of risks, transforming passive response into proactive prevention and control; Through a hierarchical-associative visualization mapping engine, it seamlessly integrates complex risk analysis results (such as transmission paths) with three-dimensional twin scenes, presenting them in an intuitive form such as highlighted streamlines and floating panels, greatly improving the efficiency of situational awareness and the accuracy of decision-making.

[0041] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0042] The above-described embodiments are merely preferred embodiments of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A power plant safety situation visualization system integrating digital twins and big data, characterized in that: It includes a multimodal data dynamic alignment and fusion module, a security situation twin model construction and update module, a big data correlation analysis and prediction engine, and a hierarchical correlation visualization mapping and presentation module; The multimodal data dynamic alignment and fusion module is used to acquire real-time video streams from the power plant, sensor time-series streams, personnel positioning streams, and event log streams, and to dynamically align them to obtain a unified data frame sequence that is synchronized in time and space. The security situation twin model construction and update module is used to drive and continuously update the three-dimensional dynamic digital twin model of the power plant according to the unified data frame sequence to obtain real-time status data. The big data correlation analysis and prediction engine is used to acquire historical operating data of the power plant, and to perform graph calculation and time series analysis based on the historical operating data and unified data frame sequence to obtain structured analysis results. The structured analysis results include safety event correlation, real-time risk level and predictive early warning signal. The hierarchical and relational visualization mapping and presentation module is used to dynamically render the structured analysis results and real-time status data to a three-dimensional twin scene, generating an interactive panoramic view of the security situation.

2. The power plant safety situation visualization system integrating digital twins and big data as described in claim 1, characterized in that, The multimodal data dynamic alignment and fusion module is specifically used for: A set of virtual entity anchor points are predefined in the digital twin space, and each virtual entity anchor point is associated with a physical entity and its standard three-dimensional coordinates; The observation information related to the physical entity corresponding to the virtual entity anchor point is identified by the power plant's real-time video stream, sensor time-series stream, personnel positioning stream, and event log stream respectively; and the observation information of different data streams belonging to the same virtual entity anchor point is associated. For observation information associated with the same virtual entity anchor point, the timestamp difference and spatial projection deviation between different data streams are calculated. Using the standard three-dimensional coordinates as a reference, the Kalman filter algorithm is used to estimate and dynamically compensate the local clock offset and spatial registration offset of each data stream online to obtain a unified data frame sequence.

3. The power plant safety situation visualization system integrating digital twins and big data as described in claim 1, characterized in that, The security posture twin model construction and update module is specifically used for: Obtain power plant design drawings and equipment ledgers, and construct a static twin basis based on the power plant design drawings and equipment ledgers. The static twin basis includes a geometric model, topological relationships, and basic attributes. The unified data frame sequence is input into the static twin base to drive dynamic updates, forming the three-dimensional dynamic digital twin model of the power plant; wherein, the state data in the unified data frame sequence is directly mapped to model attributes, the event and behavior data in the unified data frame sequence are extracted into twin behavior fragments through pattern recognition, and the twin behavior fragments are stored in the model knowledge base. Preset physical rules and business logic rules: When the state of the power plant's three-dimensional dynamic digital twin model driven by the unified data frame sequence conflicts with the physical rules or business logic rules, conflict information is generated and sent to the multimodal data dynamic alignment and fusion module as a data quality feedback signal.

4. The power plant safety situation visualization system integrating digital twins and big data as described in claim 1, characterized in that, The big data correlation analysis and prediction engine is specifically used for: A dynamic causal event graph model is constructed. The nodes of the dynamic causal event graph model are composed of devices, alarm types, personnel roles, and environmental areas. The edge weights between nodes represent the strength of causal influence between nodes. Event pairs are extracted from the historical operational data and unified data frame sequence. The temporal co-occurrence relationship, transition probability and information theory index of the event pairs are analyzed, and the edge weights are dynamically calculated and updated to obtain the security event association relationship. Time series analysis is performed on the unified data frame sequence, and the current risk status is assessed by combining the security event correlations. Real-time risk levels and predictive warning signals are generated, and the security event correlations, real-time risk levels, and predictive warning signals are used as structured analysis results.

5. The power plant safety situation visualization system integrating digital twins and big data as described in claim 1, characterized in that, The hierarchical, relational visualization mapping and presentation module is specifically used for: Based on the structured analysis results, risk transmission paths are dynamically drawn in the 3D twin scene, and alarm information and parameter information associated with the nodes of the risk transmission path are linked and bound. At the same time, the real-time status data is dynamically rendered to the 3D twin scene. The interactive panoramic view of the security situation is generated by fusing and presenting the real-time status data within the same view, and by extracting short-term historical trajectories and long-term statistical data from the unified data frame sequence.

6. A method for visualizing the safety status of power plants by integrating digital twins and big data, implemented based on the power plant safety status visualization system integrating digital twins and big data as described in any one of claims 1 to 5, characterized in that, include: The system acquires real-time video streams from the power plant, sensor time-series streams, personnel location streams, and event log streams, and performs dynamic alignment to obtain a unified data frame sequence that is synchronized in time and space. Based on the unified data frame sequence, the three-dimensional dynamic digital twin model of the power plant is driven and continuously updated to obtain real-time status data; Historical operating data of the power plant is acquired. Based on the historical operating data and the unified data frame sequence, graph calculation and time series analysis are performed to obtain structured analysis results, which include safety event correlations, real-time risk levels, and predictive early warning signals. The structured analysis results and real-time status data are dynamically rendered onto a 3D twin scene to generate an interactive panoramic view of the security situation.

7. The power plant safety situation visualization method integrating digital twins and big data according to claim 6, characterized in that, The process of acquiring real-time video streams from the power plant, sensor time-series streams, personnel location streams, and event log streams, and dynamically aligning them to obtain a unified data frame sequence with spatiotemporal synchronization, includes: A set of virtual entity anchor points are predefined in the digital twin space, and each virtual entity anchor point is associated with a physical entity and its standard three-dimensional coordinates; The observation information related to the physical entity corresponding to the virtual entity anchor point is identified by the power plant's real-time video stream, sensor time-series stream, personnel positioning stream, and event log stream respectively; and the observation information of different data streams belonging to the same virtual entity anchor point is associated. For observation information associated with the same virtual entity anchor point, the timestamp difference and spatial projection deviation between different data streams are calculated. Using the standard three-dimensional coordinates as a reference, the Kalman filter algorithm is used to estimate and dynamically compensate the local clock offset and spatial registration offset of each data stream online to obtain a unified data frame sequence.

8. The power plant safety situation visualization method integrating digital twins and big data according to claim 7, characterized in that, The Kalman filter algorithm is used to perform online estimation and dynamic compensation of the local clock offset and spatial registration parameters of each data stream, including: For a data stream corresponding to observation information belonging to the same virtual entity anchor point, a calibration state vector to be estimated is defined. The calibration state vector includes the clock offset of the data stream relative to the time reference corresponding to the standard three-dimensional coordinates, and the spatial registration offset of the observation coordinates of the data stream relative to the standard three-dimensional coordinates. Set the initial estimate of the calibration state vector, predict the calibration state vector at the next moment before obtaining the new observation coordinates of the data stream each time, calculate the Kalman gain after obtaining the new observation coordinates, and use the mapping relationship between the new observation coordinates and the predicted calibration state vector to calculate the predicted observation value of the virtual entity anchor point position. The actual observed value of the new observation coordinates is compared with the predicted observed value to obtain the observation residual. The observation residual and the Kalman gain are used to update the calibration state vector to obtain the estimated clock offset and the estimated spatial registration offset at the current time. Based on the estimated clock offset and spatial registration offset, the data stream is timestamped and its coordinates are corrected to obtain a unified data frame sequence.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in claims 6 to 8 for the power plant safety situation visualization method integrating digital twins and big data.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the power plant safety situation visualization method that integrates digital twins and big data as described in claims 6 to 8.

Images