Hydropower plant three-dimensional model construction management system based on digital twinning

Abstract

This invention relates to the field of digital twin technology and discloses a 3D model construction and management system for hydropower plants based on digital twins. The system includes: a data acquisition module that acquires exterior images of the plant area through oblique photography by drones, acquires indoor video streams through high-definition cameras, and integrates access control records, SAP work orders, and equipment operation data; a 3D modeling module that constructs a real-world model of the plant area and core indoor areas, creating a digital twin scene with a unified coordinate system; a fusion positioning module that tracks personnel via video, compensates for blind spots, generates continuous personnel trajectories, and identifies violations; a visualization module that dynamically labels personnel locations, equipment status, and alarm information in the 3D scene, generating heat maps and risk four-color maps; and an intelligent linkage module that uses a rule engine to automatically prohibit entry into abnormal areas, track videos, and push alarms. By establishing a full-scene digital model of the hydropower plant, the efficiency of hydropower plant management and control is improved.

Description

Technical Field

[0001] This invention relates to the field of digital twin technology, and more specifically, to a management system for constructing and managing three-dimensional models of hydropower plants based on digital twins. Background Technology

[0002] As a crucial component of the power system, the safe operation of hydropower plants directly impacts grid stability and energy supply. With the increasing complexity of hydropower equipment and stricter safety requirements, traditional two-dimensional monitoring systems and personnel management models are no longer sufficient to meet the sophisticated management needs of modern hydropower plants. Currently, 3D modeling of hydropower plants primarily employs a single oblique photogrammetry technique, which, while capable of accurately reproducing the plant's exterior, lacks sufficient precision in modeling core indoor areas (such as generator and turbine floors), failing to meet the demands of sophisticated equipment-level management. Traditional manual modeling methods are time-consuming and labor-intensive, and struggle to realistically reflect the on-site environment. While 3D Gaussian sputtering technology has seen some application in computer graphics in recent years, there are no reports of its application to complex industrial indoor scenes in hydropower plants.

[0003] Furthermore, existing personnel positioning systems in hydropower plants mostly rely on purely visual solutions. These systems are susceptible to damage from complex environments such as high humidity, drastic light changes, and metal structure obstructions, making it easy for targets to be lost due to occlusion or lighting variations. Consequently, the continuity and accuracy of positioning are difficult to guarantee. Moreover, most existing systems only provide visual displays and lack the ability to deeply integrate and analyze multi-source data and make intelligent decisions. When abnormal events occur, they cannot achieve coordinated responses such as automatic access control, video tracking, and alarm push notifications, leaving safety management still reliant on manual intervention and resulting in slow response times. Additionally, hydropower plants have multiple independently operating business systems, such as SAP systems, industrial television systems, and access control systems, with varying data formats and inconsistent timestamps. This makes real-time fusion and spatiotemporal alignment difficult, hindering the real-time performance and accuracy of the digital twin system. Summary of the Invention

[0004] This application provides a digital twin-based 3D model construction and management system for hydropower plants, addressing the comprehensive technical problems in traditional hydropower plant management, such as isolated multi-source data, insufficient accuracy and efficiency of 3D modeling, visual blind spots in personnel positioning, reliance on manual and delayed safety control, and lack of automated linkage for abnormal events. By integrating UAV oblique photography, 3D Gaussian Splatting, and LiDAR point cloud technologies, it achieves rapid construction and coordinate unification of high-precision digital twin scenes for both indoor and outdoor areas of the plant. Based on the DeepSORT algorithm combined with Bayesian fusion of access control and work order data, it effectively compensates for visual tracking blind spots, generating continuous and accurate personnel trajectories. Furthermore, through a visualization module, it renders personnel heatmaps and risk four-color maps in real time, and based on a rule engine, it automatically locks access control in abnormal areas, switches video tracking, and pushes alarms, significantly improving the accuracy of safety control and operational response efficiency of the hydropower plant.

[0005] To achieve the above objectives, this invention provides a digital twin-based 3D model construction and management system for hydropower plants, comprising: The data acquisition module acquires external images of the factory area through oblique photography by drones, acquires indoor video streams through high-definition cameras, and integrates access control records, SAP work orders, and equipment operation data. The 3D modeling module constructs a real-world model of the factory area and the core indoor areas, integrates LiDAR point clouds to ensure accuracy, and builds a digital twin scene with a unified coordinate system. The fusion positioning module uses the DeepSORT algorithm to track people in video, integrates access control and work order data to compensate for visual blind spots, generates continuous personnel trajectories, and identifies violations. The visualization module dynamically marks personnel locations, equipment status, and alarm information in a 3D scene, generating heat maps and four-color risk maps; The intelligent linkage module establishes spatial associations between personnel, work orders, and equipment, and uses a rule engine to automatically prohibit entry into abnormal areas, track videos, and send alarms.

[0006] Furthermore, the data acquisition module specifically includes: The data acquisition module consists of a UAV oblique photography acquisition unit, a video acquisition unit, and an IoT and business data access unit, wherein: The oblique photography acquisition unit is configured to acquire multi-angle images of the hydropower plant area and its surrounding environment using drones. The video acquisition unit is configured to acquire real-time video stream data through high-definition cameras deployed in the core indoor area and key outdoor areas. The IoT and business data access unit is configured to access access control card swipe records through standard industrial protocols, obtain work order information and personnel permissions by calling the SAP system through the RFC interface, collect equipment operating parameters through IoT sensors, and achieve low-latency transmission and unified storage of multi-source data using high-performance message queues and time-series databases.

[0007] Furthermore, it possesses the adaptability to replace image transmission in network-constrained environments, specifically including: When the network bandwidth is detected to be lower than the preset threshold or there is a traffic limit, the video stream acquisition mode will be automatically switched to the image replacement transmission mode. In the image replacement transmission mode, the system extracts keyframe images from the video stream at configurable time intervals, converting the continuous video stream into a discrete image sequence for transmission.

[0008] Furthermore, the integration of lidar point clouds ensures accuracy, specifically including: Based on the factory exterior images and indoor video streams, obtain lidar point cloud data; Using the lidar point cloud data as a geometric reference, a high-precision three-dimensional feature point set is extracted; The outdoor 3D model generated by UAV oblique photography is iteratively registered with the lidar point cloud to calculate the geometric distortion deviation of the outdoor 3D model relative to the lidar point cloud. Based on the geometric distortion deviation, the outdoor 3D model is subjected to non-rigid deformation correction so that the surface of the corrected outdoor 3D model is highly consistent with the lidar point cloud. The corrected outdoor 3D model is then merged with the LiDAR point cloud of the core indoor area. The unified output is a fusion model with LiDAR point cloud accuracy, which is then embedded into the digital twin scene of the unified coordinate system.

[0009] Furthermore, based on the geometric distortion deviation, the outdoor 3D model is subjected to non-rigid deformation correction, specifically including: Using the high-precision feature point set of the lidar point cloud as control points, the three-dimensional deviation vector between the corresponding points of the outdoor three-dimensional model is calculated, and the deviation vector field is constructed. Thin-plate spline interpolation is performed on the deviation vector field to generate a continuous deformation function covering the entire model, and the smoothness of deformation is ensured with the goal of minimizing bending energy; The offset is calculated by substituting the vertex coordinates of the model mesh into the deformation function. At the same time, the curvature change rate of the model surface is extracted, and the deformation stiffness weight of the high curvature region is dynamically adjusted to suppress feature distortion. The adjusted stiffness weights are introduced into the thin plate spline interpolation regularization term, the deformation function is re-optimized and the vertex displacement is driven, so that the matching error between the corrected model and the lidar point cloud is less than the preset threshold.

[0010] Furthermore, the corrected outdoor 3D model is fused with the LiDAR point cloud of the core indoor area, specifically including: Identify the overlapping edge area between the outdoor 3D model and the indoor LiDAR point cloud, and establish corresponding matching point pairs within the overlapping edge area; Based on the matching point pairs, the iterative nearest point algorithm is used for fine registration to eliminate coordinate deviations in the edge region; A weighted average algorithm is used for geometric fusion within the junction area, where the weight is dynamically determined based on the distance between the matching point and the junction boundary, with higher weights for closer points. The merged edge area is then re-gridded and smoothed.

[0011] Furthermore, the integration of access control and work order data compensates for visual blind spots, specifically including: When a person is lost due to video tracking due to occlusion or changes in lighting, obtain the person's access control card swipe records in the most recent time to determine the area where the person last appeared. At the same time, obtain the work task information of the person in the SAP work order system, and extract the work area and equipment location that should be in the current time period; By employing a Bayesian estimation method, the location constraints of access control records are integrated with the work area permissions determined by work order information to calculate the position of personnel within the visual blind spot; Based on the fusion results, interpolation compensation is performed on the lost personnel trajectories to restore the continuity of the trajectories, and identity association and trajectory stitching are completed when the personnel reappear in the video field of view.

[0012] Furthermore, the system dynamically labels personnel locations, equipment status, and alarm information within the 3D scene, generating heatmaps and four-color risk maps, specifically including... The system can mark personnel locations and identity information in real time in a 3D scene, dynamically display operating parameters and status at corresponding locations on the equipment model, and highlight alarm signals with audio-visual prompts. A dynamic heat map is generated based on real-time personnel distribution data. At the same time, a four-color safety risk map is integrated to perform risk classification and real-time rendering of each area of ​​the factory. The risk status update delay is no more than 5 seconds. It supports users in switching perspectives, cross-sectional viewing, and data drilling operations in 3D scenes, and allows for customized visualization views based on different roles.

[0013] Furthermore, the four-color safety risk map is integrated to perform real-time risk classification and rendering for each area of ​​the factory, specifically including: By accessing equipment operating status, personnel location data, access control records, and work order information, a multi-source risk factor database is constructed. A dynamic risk assessment model is constructed based on the analytic hierarchy process (AHP) to calculate a comprehensive risk score by weighting the personnel density, equipment defect level, and operational risk coefficient of each region. Based on the comprehensive risk score, each area of ​​the factory is divided into four risk levels: red, orange, yellow, and blue, and is rendered and covered in real time in the 3D scene with the corresponding colors. The risk status is dynamically updated based on changes in multi-source data, with an update delay of no more than 5 seconds, and an early warning is automatically triggered when the risk level increases.

[0014] Furthermore, based on the rule engine, automatic access control to abnormal areas, video tracking, and alarm push notifications are implemented, specifically including: Pre-set multi-condition judgment rules for personnel permissions, area density, and equipment status; monitor abnormal events in real time. When unauthorized entry or excessive crowding is detected, the system automatically sends a command to the access control system to lock the entrance to the relevant area. Simultaneously lock onto cameras surrounding the abnormal area for continuous video tracking, and push alarm information and video footage to the monitoring terminal.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: It realizes full-scene digital twin and intelligent safety management and control of hydropower plants. At the data acquisition level, it uses UAV oblique photography, high-definition video, access control records, SAP work orders and equipment operation data, and adopts an image-based transmission mechanism to ensure data continuity in network-constrained environments. At the 3D modeling level, it uses LiDAR point clouds as a high-precision geometric benchmark, and effectively eliminates the geometric distortion of the oblique photography model through feature point extraction, non-rigid deformation correction and edge fusion techniques, achieving seamless connection between the outdoor real-scene model and the indoor point cloud, and constructing a digital twin base that meets the unified high-precision requirements of indoor and outdoor environments. At the fusion positioning level, it uses the DeepSORT algorithm for video personnel tracking, integrates access control and work order data, and uses Bayesian estimation to compensate for visual blind spots, achieving continuous and accurate personnel trajectory generation and violation identification in complex environments, significantly improving the intelligent level of power plant safety management and emergency response capabilities. Attached Figure Description

[0016] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1A schematic diagram of the structure of the management system for constructing a three-dimensional model of a hydropower plant based on digital twins is shown in an embodiment of the present invention. Detailed Implementation

[0017] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0018] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0019] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0020] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0021] The following is a description of preferred embodiments of the present invention in conjunction with the accompanying drawings.

[0022] like Figure 1 As shown, embodiments of the present invention disclose a 3D model construction and management system for hydropower plants based on digital twins, including: 1. Data acquisition module: acquires external images of the factory area through drone oblique photography, acquires indoor video streams through high-definition cameras, and integrates access control records, SAP work orders, and equipment operation data; In some embodiments of the present invention, the data acquisition module specifically includes: The data acquisition module consists of a UAV oblique photography acquisition unit, a video acquisition unit, and an IoT and business data access unit, wherein: The oblique photography acquisition unit is configured to acquire multi-angle images of the hydropower plant area and its surrounding environment using drones. The video acquisition unit is configured to acquire real-time video stream data through high-definition cameras deployed in the core indoor area and key outdoor areas. The IoT and business data access unit is configured to access access control card swipe records through standard industrial protocols, obtain work order information and personnel permissions by calling the SAP system through the RFC interface, collect equipment operating parameters through IoT sensors, and achieve low-latency transmission and unified storage of multi-source data using high-performance message queues and time-series databases.

[0023] In this embodiment, the ability to adapt to image replacement transmission in network-limited environments specifically includes: When the network bandwidth is detected to be lower than the preset threshold or there is a traffic limit, the video stream acquisition mode will be automatically switched to the image replacement transmission mode. In the image replacement transmission mode, the system extracts keyframe images from the video stream at configurable time intervals, converting the continuous video stream into a discrete image sequence for transmission.

[0024] In this embodiment, a high-performance message queue and a time-series database are used to achieve low-latency transmission and unified storage of multi-source data. The high-performance message queue, as the core of the data bus, is responsible for interfacing with access control records, SAP work orders, IoT sensor data, and video keyframes. Through asynchronous decoupling and peak-shaving mechanisms, it ensures millisecond-level reliable transmission during peak data concurrency periods, avoiding data loss or system blockage. Simultaneously, it supports parallel subscription and consumption by multiple functional modules, achieving low-coupling and efficient collaboration between modules. The time-series database optimizes the storage of high-throughput, time-sensitive data such as device operating parameters, personnel location trajectories, and access control time series. Through efficient compression algorithms and time partitioning strategies, it reduces storage costs and improves query response speed, supporting historical trajectory playback, status trend analysis, and dynamic risk calculation.

[0025] 2. 3D modeling module: Constructs a real-world model of the factory area and the core indoor area, integrates LiDAR point cloud to ensure accuracy, and builds a digital twin scene with a unified coordinate system; In this embodiment, the fusion of lidar point clouds to ensure accuracy specifically includes: Based on the factory exterior images and indoor video streams, obtain lidar point cloud data; Using the lidar point cloud data as a geometric reference, a high-precision three-dimensional feature point set is extracted; The outdoor 3D model generated by UAV oblique photography is iteratively registered with the lidar point cloud to calculate the geometric distortion deviation of the outdoor 3D model relative to the lidar point cloud. Based on the geometric distortion deviation, the outdoor 3D model is subjected to non-rigid deformation correction so that the surface of the corrected outdoor 3D model is highly consistent with the lidar point cloud. The corrected outdoor 3D model is then merged with the LiDAR point cloud of the core indoor area. The unified output is a fusion model with LiDAR point cloud accuracy, which is then embedded into the digital twin scene of the unified coordinate system.

[0026] In this embodiment, the extraction of a high-precision three-dimensional feature point set specifically includes: Voxel filtering downsampling is performed on lidar point cloud data to reduce point cloud density while preserving geometric features; Calculate the normal vector and rate of curvature change for each point, and select points whose rate of curvature change exceeds a preset threshold as candidate feature points; Non-maximum suppression is applied to candidate feature points, and the points with the largest curvature change in the local region are retained as the initial feature point set; The iterative nearest point algorithm is used to perform multi-view consistency verification on the initial feature point set, and outliers with matching errors exceeding the threshold are removed to ensure the geometric accuracy and distribution uniformity of the feature points.

[0027] In this embodiment, calculating the three-dimensional deviation vector between the corresponding points of the outdoor three-dimensional model specifically includes: For each lidar feature point, the nearest neighbor point is searched on the surface of the outdoor 3D model as the initial corresponding point; The initial corresponding points are filtered based on the similarity between the angle between the normal vectors and the curvature, and erroneous corresponding points with mismatched geometric features are eliminated. A two-way consistency check is performed on the selected corresponding point pairs to ensure that the correspondence between the lidar points and the model points is that they are the nearest neighbors. Based on the finally determined corresponding point pairs, the three-dimensional spatial deviation vector from the model point to the lidar point is calculated to form a sparse deviation vector field.

[0028] In this embodiment, the outdoor 3D model is subjected to non-rigid deformation correction based on the geometric distortion deviation, specifically including: Using the high-precision feature point set of the lidar point cloud as control points, the three-dimensional deviation vector between the corresponding points of the outdoor three-dimensional model is calculated, and the deviation vector field is constructed. Thin-plate spline interpolation is performed on the deviation vector field to generate a continuous deformation function covering the entire model, and the smoothness of deformation is ensured with the goal of minimizing bending energy; The offset is calculated by substituting the vertex coordinates of the model mesh into the deformation function. At the same time, the curvature change rate of the model surface is extracted, and the deformation stiffness weight of the high curvature region is dynamically adjusted to suppress feature distortion. The adjusted stiffness weights are introduced into the thin plate spline interpolation regularization term, the deformation function is re-optimized and the vertex displacement is driven, so that the matching error between the corrected model and the lidar point cloud is less than the preset threshold.

[0029] In this embodiment, the corrected outdoor 3D model is fused with the LiDAR point cloud of the indoor core area, specifically including: Identify the overlapping edge area between the outdoor 3D model and the indoor LiDAR point cloud, and establish corresponding matching point pairs within the overlapping edge area; Based on the matching point pairs, the iterative nearest point algorithm is used for fine registration to eliminate coordinate deviations in the edge region; A weighted average algorithm is used for geometric fusion within the junction area, where the weight is dynamically determined based on the distance between the matching point and the junction boundary, with higher weights for closer points. The merged edge area is then re-gridded and smoothed.

[0030] The beneficial effects of the above technical solution are as follows: By using LiDAR point cloud data as a high-precision geometric benchmark, a three-dimensional feature point set is extracted and the geometric distortion deviation of the oblique photogrammetry model is calculated using iterative nearest-point registration. Then, through non-rigid deformation correction based on thin plate spline interpolation, combined with curvature adaptive weight adjustment, the surface of the outdoor three-dimensional model is made to highly match the LiDAR point cloud. At the same time, the corrected outdoor model is fused with the indoor LiDAR point cloud, and the coordinate deviation is eliminated through fine registration and a weighted average algorithm is used to achieve a smooth transition. Finally, a fused model with LiDAR point cloud accuracy is output, which solves the geometric deformation problem caused by lens distortion, image matching error and other factors in the oblique photogrammetry model, significantly improves the geometric accuracy of the outdoor model, and achieves seamless connection between the outdoor model and the indoor point cloud, ensuring that the digital twin model of the whole scene meets the high-precision requirements of both indoor and outdoor environments.

[0031] 3. Fusion positioning module: Based on the DeepSORT algorithm, it performs video personnel tracking, integrates access control and work order data to compensate for visual blind spots, generates continuous personnel trajectories, and identifies violations; In this embodiment, the integration of access control and work order data to compensate for visual blind spots specifically includes: When a person is lost due to video tracking due to occlusion or changes in lighting, obtain the person's access control card swipe records in the most recent time to determine the area where the person last appeared. At the same time, obtain the work task information of the person in the SAP work order system, and extract the work area and equipment location that should be in the current time period; By employing a Bayesian estimation method, the location constraints of access control records are integrated with the work area permissions determined by work order information to calculate the position of personnel within the visual blind spot; Based on the fusion results, interpolation compensation is performed on the lost personnel trajectories to restore the continuity of the trajectories, and identity association and trajectory stitching are completed when the personnel reappear in the video field of view.

[0032] 4. Visualization module: Dynamically marks personnel locations, equipment status, and alarm information in a 3D scene, generating heat maps and four-color risk maps; In this embodiment, personnel locations, equipment status, and alarm information are dynamically labeled in a 3D scene to generate heatmaps and risk four-color maps, specifically including... The system can mark personnel locations and identity information in real time in a 3D scene, dynamically display operating parameters and status at corresponding locations on the equipment model, and highlight alarm signals with audio-visual prompts. A dynamic heat map is generated based on real-time personnel distribution data. At the same time, a four-color safety risk map is integrated to perform risk classification and real-time rendering of each area of ​​the factory. The risk status update delay is no more than 5 seconds. It supports users in switching perspectives, cross-sectional viewing, and data drilling operations in 3D scenes, and allows for customized visualization views based on different roles.

[0033] In this embodiment, a four-color safety risk map is integrated to perform real-time risk classification and rendering for each area of ​​the factory, specifically including: By accessing equipment operating status, personnel location data, access control records, and work order information, a multi-source risk factor database is constructed. A dynamic risk assessment model is constructed based on the analytic hierarchy process (AHP) to calculate a comprehensive risk score by weighting the personnel density, equipment defect level, and operational risk coefficient of each region. Based on the comprehensive risk score, each area of ​​the factory is divided into four risk levels: red, orange, yellow, and blue, and is rendered and covered in real time in the 3D scene with the corresponding colors. The risk status is dynamically updated based on changes in multi-source data, with an update delay of no more than 5 seconds, and an early warning is automatically triggered when the risk level increases.

[0034] 5. Intelligent linkage module: Establishes spatial association between personnel, work orders, and equipment, and realizes automatic entry restriction, video tracking, and alarm push based on rule engine.

[0035] In this embodiment, automatic access control to abnormal areas, video tracking, and alarm push notifications are implemented based on a rule engine, specifically including: Pre-set multi-condition judgment rules for personnel permissions, area density, and equipment status; monitor abnormal events in real time. When unauthorized entry or excessive crowding is detected, the system automatically sends a command to the access control system to lock the entrance to the relevant area. Simultaneously lock onto cameras surrounding the abnormal area for continuous video tracking, and push alarm information and video footage to the monitoring terminal.

[0036] In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0037] Although the invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the embodiments disclosed in this invention can be combined with each other in any way. The fact that not all of these combinations are described in this specification is merely for the sake of brevity and resource conservation.

[0038] It will be understood by those skilled in the art that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A digital-twin-based three-dimensional model construction management system for a hydropower plant, characterized in that, include: The data acquisition module acquires external images of the factory area through oblique photography by drones, acquires indoor video streams through high-definition cameras, and integrates access control records, SAP work orders, and equipment operation data. The 3D modeling module constructs a real-world model of the factory area and the core indoor areas, integrates LiDAR point clouds to ensure accuracy, and builds a digital twin scene with a unified coordinate system. The fusion positioning module uses the DeepSORT algorithm to track people in video, integrates access control and work order data to compensate for visual blind spots, generates continuous personnel trajectories, and identifies violations. The visualization module dynamically marks personnel locations, equipment status, and alarm information in a 3D scene, generating heat maps and four-color risk maps; The intelligent linkage module establishes spatial associations between personnel, work orders, and equipment, and uses a rule engine to automatically prohibit entry into abnormal areas, track videos, and send alarms.

2. The digital-twin-based hydropower plant three-dimensional model construction management system according to claim 1, characterized in that, The data acquisition module specifically includes: The data acquisition module consists of a UAV oblique photography acquisition unit, a video acquisition unit, and an IoT and business data access unit, wherein: The oblique photography acquisition unit is configured to acquire multi-angle images of the hydropower plant area and its surrounding environment using drones. The video acquisition unit is configured to acquire real-time video stream data through high-definition cameras deployed in the core indoor area and key outdoor areas. The IoT and business data access unit is configured to access access control card swipe records through standard industrial protocols, obtain work order information and personnel permissions by calling the SAP system through the RFC interface, collect equipment operating parameters through IoT sensors, and achieve low-latency transmission and unified storage of multi-source data using high-performance message queues and time-series databases.

3. The digital-twin-based hydropower plant three-dimensional model construction management system according to claim 2, characterized in that, It possesses the adaptability to image-based transmission in network-constrained environments, specifically including: When the network bandwidth is detected to be lower than the preset threshold or there is a traffic limit, the video stream acquisition mode will be automatically switched to the image replacement transmission mode. In the image replacement transmission mode, the system extracts keyframe images from the video stream at configurable time intervals, converting the continuous video stream into a discrete image sequence for transmission.

4. The digital-twin-based hydropower plant three-dimensional model construction management system according to claim 1, characterized in that, Fusion of LiDAR point cloud data ensures accuracy, specifically including: Based on the factory exterior images and indoor video streams, obtain lidar point cloud data; Using the lidar point cloud data as a geometric reference, a high-precision three-dimensional feature point set is extracted; The outdoor 3D model generated by UAV oblique photography is iteratively registered with the lidar point cloud to calculate the geometric distortion deviation of the outdoor 3D model relative to the lidar point cloud. Based on the geometric distortion deviation, the outdoor 3D model is subjected to non-rigid deformation correction so that the surface of the corrected outdoor 3D model is highly consistent with the lidar point cloud. The corrected outdoor 3D model is then merged with the LiDAR point cloud of the core indoor area. The unified output is a fusion model with LiDAR point cloud accuracy, which is then embedded into the digital twin scene of the unified coordinate system.

5. The digital-twin-based hydropower plant three-dimensional model construction management system according to claim 4, characterized in that, Based on the geometric distortion deviation, the outdoor 3D model is subjected to non-rigid deformation correction, specifically including: Using the high-precision feature point set of the lidar point cloud as control points, the three-dimensional deviation vector between the corresponding points of the outdoor three-dimensional model is calculated, and the deviation vector field is constructed. Thin-plate spline interpolation is performed on the deviation vector field to generate a continuous deformation function covering the entire model, and the smoothness of deformation is ensured with the goal of minimizing bending energy; The offset is calculated by substituting the vertex coordinates of the model mesh into the deformation function. At the same time, the curvature change rate of the model surface is extracted, and the deformation stiffness weight of the high curvature region is dynamically adjusted to suppress feature distortion. The adjusted stiffness weights are introduced into the thin plate spline interpolation regularization term, the deformation function is re-optimized and the vertex displacement is driven, so that the matching error between the corrected model and the lidar point cloud is less than the preset threshold.

6. The digital-twin-based hydropower plant three-dimensional model construction management system according to claim 4, characterized in that, The corrected outdoor 3D model is then fused with the LiDAR point cloud of the core indoor area, specifically including: Identify the overlapping edge area between the outdoor 3D model and the indoor LiDAR point cloud, and establish corresponding matching point pairs within the overlapping edge area; Based on the matching point pairs, the iterative nearest point algorithm is used for fine registration to eliminate coordinate deviations in the edge region; A weighted average algorithm is used for geometric fusion within the junction area, where the weights are dynamically determined based on the distance between the matching point and the junction boundary. The merged edge area is then re-gridded and smoothed.

7. The hydropower plant 3D model construction and management system based on digital twins according to claim 1, characterized in that, Integrating access control and work order data to compensate for visual blind spots, specifically including: When a person is lost due to video tracking due to occlusion or changes in lighting, obtain the person's access control card swipe records in the most recent time to determine the area where the person last appeared. At the same time, obtain the work task information of the person in the SAP work order system, and extract the work area and equipment location that should be in the current time period; By employing a Bayesian estimation method, the location constraints of access control records are integrated with the work area permissions determined by work order information to calculate the position of personnel within the visual blind spot; Based on the fusion results, interpolation compensation is performed on the lost personnel trajectories to restore the continuity of the trajectories, and identity association and trajectory stitching are completed when the personnel reappear in the video field of view.

8. The hydropower plant 3D model construction and management system based on digital twins according to claim 1, characterized in that, Dynamically label personnel locations, equipment status, and alarm information in a 3D scene to generate heatmaps and four-color risk maps, specifically including... The system can mark personnel locations and identity information in real time in a 3D scene, dynamically display operating parameters and status at corresponding locations on the equipment model, and highlight alarm signals with audio-visual prompts. A dynamic heat map is generated based on real-time personnel distribution data. At the same time, a four-color safety risk map is integrated to perform risk classification and real-time rendering of each area of ​​the factory. The risk status update delay is no more than 5 seconds. It supports users in switching perspectives, cross-sectional viewing, and data drilling operations in 3D scenes, and allows for customized visualization views based on different roles.

9. The hydropower plant 3D model construction and management system based on digital twins according to claim 8, characterized in that, The system integrates a four-color safety risk map to perform real-time risk classification and rendering for various areas of the factory, specifically including: By accessing equipment operating status, personnel location data, access control records, and work order information, a multi-source risk factor database is constructed. A dynamic risk assessment model is constructed based on the analytic hierarchy process (AHP) to calculate a comprehensive risk score by weighting the personnel density, equipment defect level, and operational risk coefficient of each region. Based on the comprehensive risk score, each area of ​​the factory is divided into four risk levels: red, orange, yellow, and blue, and is rendered and covered in real time in the 3D scene with the corresponding colors. The risk status is dynamically updated based on changes in multi-source data, with an update delay of no more than 5 seconds, and an early warning is automatically triggered when the risk level increases.

10. The hydropower plant 3D model construction and management system based on digital twins according to claim 1, characterized in that, Based on a rule engine, automatic entry restrictions, video tracking, and alarm push notifications are implemented for abnormal areas, specifically including: Pre-set multi-condition judgment rules for personnel permissions, area density, and equipment status; monitor abnormal events in real time. When unauthorized entry or excessive crowding is detected, the system automatically sends a command to the access control system to lock the entrance to the relevant area. Simultaneously lock onto cameras surrounding the abnormal area for continuous video tracking, and push alarm information and video footage to the monitoring terminal.

Images