Digital twin modeling and dynamic visualization method based on multi-source heterogeneous data fusion

Abstract

The invention discloses a digital twin modeling and dynamic visualization method based on multi-source heterogeneous data fusion, which comprises the following steps: firstly, constructing a multi-source data access engine, and standardly integrating heterogeneous data such as GIS, IoT, BIM, video monitoring and the like; secondly, realizing rapid three-dimensional modeling by adopting aerial photography of an unmanned aerial vehicle and an AI algorithm, and optimizing rendering performance in combination with an LOD technology; and finally, dynamic visualization and interaction are realized through a spatio-temporal data engine and a cloud rendering technology. According to the method, the problems of scenic spot data islands, low modeling efficiency and large-scale scene rendering lagging are solved, the real-time performance and decision-making efficiency of scenic spot management are remarkably improved, and total-factor scene support is provided for smart tourism.

Description

Technical Field

[0001] The present invention belongs to the field of smart city and digital twin technology, and specifically relates to a digital twin modeling and dynamic visualization method based on multi-source heterogeneous data fusion. Background Art

[0002] Currently, the development of smart scenic spots faces significant technical bottlenecks. First, data silos and heterogeneity are prominent. Scenic spot management involves multiple data sources, including GIS geographic information (using the WGS84 coordinate system), BIM building models (using local coordinate systems), IoT devices (using multiple protocols such as Zigbee, LoRa, and NB-IoT), and video surveillance. However, format incompatibilities, coordinate system conflicts, and protocol differences make data integration difficult, making it difficult to ensure real-time performance. For example, some scenic spot management systems rely solely on GIS data, failing to integrate BIM and GIS coordinate systems and thus unable to construct a globally unified spatial model. Second, 3D modeling efficiency and accuracy are insufficient. Traditional methods rely on manual matching of feature points from drone aerial photography, which is time-consuming and prone to errors. Existing LOD (level of detail) technology fails to address the needs of dynamic scenarios, resulting in blurry models at long distances or missing details at close range, impacting the user experience. Finally, dynamic visualization capabilities are weak. While existing technologies (such as commercial ArcGIS software) support static model display, they lack a spatiotemporal data engine and cannot link dynamic data (such as visitor flow and equipment status), resulting in insufficient real-time situational awareness. Furthermore, when rendering large-scale scenes, traditional methods often experience lags and delays due to high memory usage and uneven distribution of computing resources. Furthermore, because they fail to consider spatiotemporal characteristics, dynamic data updates lag, making it difficult to support real-time decision-making. Therefore, a smart scenic area digital twin approach is urgently needed that can efficiently integrate multi-source heterogeneous data, achieve dynamic spatiotemporal correlation, and optimize rendering performance. This approach can overcome the limitations of existing technologies and improve real-time management, accuracy, and user experience. Summary of the Invention

[0003] The purpose of the present invention is to solve the shortcomings existing in the prior art, and to propose a digital twin modeling and dynamic visualization method based on multi-source heterogeneous data fusion. This method aims to achieve accurate mapping and real-time management of all-factor scenes in smart scenic areas by constructing a multi-source data access engine, AI-driven fast three-dimensional modeling technology and dynamic visualization interaction solutions, and promote the digitalization and intelligent transformation of scenic area management. This method integrates multi-source heterogeneous data such as GIS, BIM, IoT through a unified interface protocol, combines drone oblique photography with deep learning algorithms to achieve high-precision modeling, and associates static triangulated network models with dynamic information based on a spatiotemporal data engine, and finally achieves low-latency visualization interaction through cloud rendering and edge computing technology, thereby better serving the sustainable development needs of smart city construction.

[0004] The technical solution for achieving the purpose of the present invention is: A digital twin modeling and dynamic visualization method based on multi-source heterogeneous data fusion includes the following steps: 1) Build a multi-source data access engine, using standardized access for GIS vector data (SHP), image data (DOM), terrain data (DEM), IoT real-time data (MQTT / HTTP), and BIM / CAD design data; 2) Complete 3D modeling based on drone oblique photography and AI algorithms, including GNSS positioning, feature point matching, and LOD layered loading; 3) Design a spatiotemporal data engine to perform spatiotemporal calibration and correlation between static data (buildings, roads) and dynamic data (passenger flow, equipment) in 3D modeling; 4) Cloud rendering technology is used to achieve dynamic visualization of multiple terminals, and the WebRTC protocol is used to push low-latency video streams.

[0005] The steps of 3D modeling based on drone oblique photography and AI algorithm in step 2) specifically include: 2.1 Acquire multi-angle images based on drone oblique photography and optimize the position accuracy of images by combining inertial navigation systems; 2.2 Use convolutional neural network (CNN) to automatically match feature points of images and generate high-precision triangulated network models; 2.3 Dynamically adjust the level of detail (LOD) of the triangulated mesh model according to the view distance, and maintain a stable frame rate of ≥45FPS.

[0006] The steps of designing the spatiotemporal data engine in step 3) specifically include: 3.1 Timestamp Alignment: Standardize the timestamps of dynamic data (passenger flow, equipment status) to ensure synchronization with static data; 3.2 Spatial coordinate system conversion: converting geographic coordinates and local coordinate systems in static data from different sources into a global coordinate system: using the seven-parameter method to convert the local coordinate system in the static data into a global coordinate system and using the Gauss-Krüger projection to convert the geographic coordinates (latitude and longitude) in the static data into plane coordinates; 3.3 Data Association Modeling: Establish associations between static data (buildings, roads) and dynamic data using unique identifiers (UUIDs) or relational databases (MySQL). Assign unique UUIDs to static triangulated network models (e.g., buildings, roads), and bind dynamic data (e.g., sensor readings) to the static triangulated network models using the UUIDs. Create a relational database table (e.g., MySQL) to store the association between model ID, sensor ID, timestamp, and data value. 3.4 Real-time data update: Receive dynamic data in real time through the message queue (Kafka) and update the association relationship in the spatiotemporal data engine.

[0007] In step 4), the steps of implementing multi-terminal dynamic visualization using cloud rendering technology specifically include: 4.1 Build a full-factor scenario service that supports multi-dimensional data aggregation analysis of heat maps and migration maps; calculate passenger flow density in real time based on Spark Streaming and generate heat maps; analyze tourist movement paths through a graph database (Neo4j) and generate migration maps; 4.2 Use tile technology to divide the full-factor scene, load the area within the field of view on demand, and reduce memory usage; 4.3 Process real-time video streams through edge computing nodes with response delay ≤ 100ms.

[0008] This approach aims to achieve accurate mapping and real-time management of all-factor scenarios in smart scenic areas by building a multi-source data access engine, AI-driven rapid 3D modeling technology, and dynamic visualization interaction solutions. This approach addresses existing issues such as data silos, low modeling efficiency, and insufficient rendering performance, and promotes the digital and intelligent transformation of scenic area management. This approach integrates heterogeneous data from multiple sources, such as GIS, BIM, and IoT, through a unified interface protocol. It combines drone oblique photography with deep learning algorithms to achieve high-precision modeling. Based on a spatiotemporal data engine, it associates static triangulated mesh models with dynamic information. Ultimately, it achieves low-latency visualization interaction through cloud rendering and edge computing technologies, thereby better serving the sustainable development needs of smart city construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a flow chart of an embodiment; Figure 2 This is a flowchart of three-dimensional modeling based on drone oblique photography and AI algorithm in the embodiment; Figure 3 A flowchart of the spatiotemporal data engine design in the embodiment; Figure 4 This is a flow chart of cloud rendering technology used in the embodiment to achieve dynamic visualization of multiple terminals. DETAILED DESCRIPTION

[0010] The present invention will be further described below with reference to the accompanying drawings and embodiments, but the present invention is not limited thereto. Example

[0011] Reference Figure 1 , a digital twin modeling and dynamic visualization method based on multi-source heterogeneous data fusion, including the following steps: 1) Build a multi-source data access engine, using standardized access that supports GIS vector data (SHP), image data (DOM), terrain data (DEM), IoT real-time data (MQTT / HTTP), and BIM / CAD design data; 2) Complete 3D modeling based on drone oblique photography and AI algorithms, including GNSS positioning, feature point matching, and LOD layered loading; 3) Design a spatiotemporal data engine to perform spatiotemporal calibration and correlation between static data (buildings, roads) and dynamic data (passenger flow, equipment) in 3D modeling; 4) Cloud rendering technology is used to achieve dynamic visualization of multiple terminals, and the WebRTC protocol is used to push low-latency video streams.

[0012] like Figure 2 The three-dimensional modeling steps based on drone oblique photography and AI algorithm in step 2) specifically include: 2.1 Acquire multi-angle images based on drone oblique photography and optimize the position accuracy of images by combining inertial navigation systems; 2.2 Use convolutional neural network (CNN) to automatically match feature points of images and generate high-precision triangulated network models; 2.3 Dynamically adjust the level of detail (LOD) of the triangulated mesh model according to the view distance, and maintain a stable frame rate of ≥45FPS.

[0013] like Figure 3 The steps of designing the spatiotemporal data engine in step 3) specifically include: 3.1 Timestamp Alignment: Standardize the timestamps of dynamic data (passenger flow, equipment status) to ensure synchronization with static data; 3.2 Spatial Coordinate System Conversion: The geographic coordinates in static data from different sources and the local coordinate system of the BIM are converted to the global coordinate system (CGCS2000). The local coordinate system in the static data is converted to the global coordinate system using the seven-parameter method and the geographic coordinates in the static data are converted to planar coordinates using the Gauss-Krüger projection. In this case, the geographic coordinates are the GIS WGS84 coordinates. 3.3 Data Association Modeling: Establish associations between static data (buildings, roads) and dynamic data using unique identifiers (UUIDs) or relational databases (MySQL). Assign unique UUIDs to static triangulated network models (e.g., buildings, roads), and bind dynamic data (e.g., sensor readings) to the static triangulated network models using the UUIDs. Create a relational database table (e.g., MySQL) to store the association between model ID, sensor ID, timestamp, and data value. 3.4 Real-time data update: Receive dynamic data in real time through the message queue (Kafka) and update the association relationship in the spatiotemporal data engine.

[0014] like Figure 4 In step 4), the steps of implementing multi-terminal dynamic visualization using cloud rendering technology specifically include: 4.1 Build a full-factor scenario service that supports multi-dimensional data aggregation analysis such as heat maps and migration maps; calculate passenger flow density in real time based on Spark Streaming and generate heat maps; analyze tourist movement paths using a graph database (Neo4j) to generate migration maps; 4.2 Use tile technology to divide the full-factor scene, load the area within the field of view on demand, and reduce memory usage; 4.3 Process real-time video streams through edge computing nodes with response delay ≤ 100ms.

Claims

1. A digital twin modeling and dynamic visualization method based on multi-source heterogeneous data fusion, characterized by: The following steps are involved: 1) Build a multi-source data access engine that supports standardized access to GIS vector data, image data, terrain data, IoT real-time data, and BIM / CAD design data; 2) Complete 3D modeling based on drone oblique photography and AI algorithms, including GNSS positioning, feature point matching, and LOD layered loading; 3) Design a spatiotemporal data engine to perform spatiotemporal calibration and association between static and dynamic data in 3D modeling; 4) Cloud rendering technology is used to achieve dynamic visualization of multiple terminals, and the WebRTC protocol is used to push low-latency video streams.

2. The digital twin modeling and dynamic visualization method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: The steps of 3D modeling based on drone oblique photography and AI algorithm in step 2) specifically include: 2.1 Acquire multi-angle images based on drone oblique photography and optimize the position accuracy of images by combining inertial navigation systems; 2.2 Use convolutional neural network to automatically match the feature points of the image and generate a high-precision triangulated network model; 2.3 Dynamically adjust the level of detail of the triangulated network model according to the viewing distance, and maintain a stable frame rate of ≥45FPS.

3. The digital twin modeling and dynamic visualization method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: The steps of designing the spatiotemporal data engine in step 3) specifically include: 3.1 Timestamp Alignment: Standardize the timestamps of dynamic data to ensure synchronization with static data; 3.2 Spatial coordinate system conversion: convert geographic coordinates and local coordinate systems in static data from different sources into a global coordinate system: use the seven-parameter method to convert the local coordinate system in static data into a global coordinate system and convert the geographic coordinates in static data into plane coordinates through Gauss-Krüger projection; 3.3 Data Association Modeling: Establish an association between static and dynamic data through a unique identifier (UUID) or a relational database. Assign a unique UUID to the static triangulated network model, and bind the dynamic data to the static triangulated network model through the UUID. Establish a relational database table to store the association between "model ID-sensor ID-timestamp-data value"; 3.4 Real-time data update: Receive dynamic data in real time through the message queue and update the association relationship in the spatiotemporal data engine.

4. The digital twin modeling and dynamic visualization method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: In step 4), the steps of implementing multi-terminal dynamic visualization using cloud rendering technology specifically include: 4.1 Build a full-factor scenario service that supports multi-dimensional data aggregation analysis of heat maps and migration maps; calculate passenger flow density in real time based on Spark Streaming and generate heat maps; analyze tourist movement paths using the graph database Neo4j and generate migration maps; 4.2 Use tile technology to divide the full-factor scene, load the area within the field of view on demand, and reduce memory usage; 4.3 Process real-time video streams through edge computing nodes with response delay ≤ 100ms.

Images