Digital twin real-time modeling method and system based on laser radar and image fusion

Abstract

The invention discloses a digital twinborn real-time modeling method and system based on laser radar and image fusion, and belongs to the technical field of digital twinborn model construction.The method comprises the steps that laser radar point cloud, images and inertial measurement unit correction data are synchronously collected, noise reduction and segmentation are conducted on point cloud data, image distortion removal and feature extraction are conducted, and a digital twinborn model is obtained. Fusing the point cloud and the image data, performing registration and combination on the point cloud and the image data, generating a depth map, constructing a three-dimensional environment model in real time through SLAM, dynamically updating the model, performing rendering and visual output on the data of the three-dimensional environment model, and completing digital twin real-time modeling. The method combines the three-dimensional point cloud and image information of the laser radar, improves the precision of object recognition and environment perception, guarantees the real-time updating of the three-dimensional environment model through SLAM and deep learning, supports the efficient reconstruction of a dynamic scene, achieves the low-delay data transmission and visualization, and improves the real-time performance and stability of the three-dimensional environment model.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital twin model construction, and particularly relates to a digital twin real-time modeling method and system based on the fusion of lidar and images. Background Art

[0002] Digital Twin, as a technology that uses physical world data to construct virtual models in real time, can be widely applied in fields such as industrial automation, intelligent transportation, and environmental monitoring. Existing digital twin systems mainly rely on single-sensor data (such as only using cameras or lidar), resulting in insufficient real-time performance, limited accuracy, and poor environmental adaptability. Although lidar (LiDAR) has high-precision 3D perception capabilities, it has limitations in terms of the loss of object surface color and texture information. Image sensors can provide rich visual information, but their depth information acquisition ability is weak and they are easily affected by light. A single sensor is difficult to meet the requirements of high-precision and real-time intelligent detection and modeling. Therefore, how to efficiently fuse lidar and image information to achieve real-time and high-precision digital twin modeling and intelligent detection is an issue that urgently needs further research and optimization.

[0003] Currently, the modeling of digital twins usually relies on a single sensor (such as a camera or lidar). In this way, there may be some problems, including poor environmental adaptability: image processing is easily affected by light and weather, while lidar point clouds lack texture information. Low data fusion efficiency: the complexity of multi-source data synchronization and registration algorithms is high, making it difficult to achieve real-time updates. Insufficient dynamic scene modeling: the existing dynamic capture accuracy of moving objects or complex structures is limited. In summary, in the current modeling of digital twins, the adaptability to different environments is poor, the data fusion efficiency is not high, and the data capture accuracy for dynamic scenes is limited. Summary of the Invention

[0004] The present invention provides a digital twin real-time modeling method and system based on the fusion of lidar and images, aiming to solve the problems of poor adaptability to different environments, low data fusion efficiency, and limited data capture accuracy for dynamic scenes in the current digital twin modeling.

[0005] To achieve the above object, the present invention adopts the following technical solutions: The present invention provides a digital twin real-time modeling method based on the fusion of lidar and images, including the following steps: S1. Synchronously collect point cloud data, image data, and motion attitude data through a sensor group to generate synchronized lidar point cloud data, image data, and inertial measurement unit correction data; Among them, the sensor group uses a lidar, an industrial camera, and an inertial measurement unit; S2. Denoise and geometrically segment the lidar point cloud data to generate denoised and segmented point cloud data; perform distortion removal and feature extraction on the synchronized image data to generate a distortion-removed image and image feature data; S3. Project the denoised and segmented point cloud data onto the distortion-removed image to generate an initial depth map; complete depth completion of the initial depth map based on the image feature data to generate a dense depth map; S4. Based on the dense depth map, the denoised and segmented point cloud data, and the inertial measurement unit calibration data, construct a three-dimensional environment model in real time through SLAM, and the three-dimensional environment model is dynamically updated according to the input of the sensor; S5. Render and visually output the data of the three-dimensional environment model through a low-latency communication protocol to complete real-time digital twin modeling.

[0006] In some embodiments, in S1, the synchronous data acquisition specifically includes: achieving time synchronization between the lidar and the industrial camera through time synchronization, and completing spatial alignment through spatial calibration.

[0007] Further, in S1, the time synchronization includes: GPS clock synchronization or a hardware trigger, and the spatial calibration uses a calibration algorithm to obtain the rotation matrix and translation vector between the lidar coordinate system and the camera coordinate system.

[0008] In some embodiments, in S2, the denoising and geometric segmentation processing: uses a filtering algorithm to remove the point cloud noise in the lidar point cloud data, and geometrically segments the point cloud in the lidar point cloud data through a geometric shape segmentation algorithm.

[0009] Further, in S2, the distortion removal and feature extraction processing of the synchronized image data includes: using an image processing algorithm to correct the camera lens distortion, and using the SIFT algorithm or the ORB algorithm to extract the key feature points in the image data.

[0010] In some embodiments, in S3, the depth completion of the initial depth map based on the image feature data includes: using a depth completion algorithm in a convolutional neural network to fill the missing areas of the initial depth map to generate a dense depth map.

[0011] Further, in S3, the following formula (1) is used to project the denoised and segmented point cloud data onto the distortion-removed image: (1); where, is the camera internal parameter matrix, is the transformation matrix from the lidar to the industrial camera.

[0012] In some embodiments, in S4, SLAM includes the LIO-SAM algorithm or the ORB-SLAM3 algorithm, and combines inertial measurement unit correction data to achieve real-time construction and dynamic update of the three-dimensional environment model.

[0013] In some embodiments, in S5, the low-latency communication protocol includes ROS2, and the visualization engine includes the Unity3D engine.

[0014] The present invention also provides a digital twin real-time modeling system based on lidar and image fusion. The system includes a data acquisition and synchronization module, a data processing module, a multimodal fusion module, a digital twin modeling module, and a real-time feedback module, wherein: The data acquisition and synchronization module is used to synchronously collect point cloud data, image data, and motion attitude data through a lidar, an industrial camera, and an inertial measurement unit, and generate synchronized lidar point cloud data, image data, and inertial measurement unit correction data; The data processing module is used to perform noise reduction and geometric segmentation processing on the lidar point cloud data to generate noise-reduced and segmented point cloud data; perform distortion removal and feature extraction processing on the synchronized image data to generate a distortion-removed image and image feature data; The multimodal fusion module is used to project the noise-reduced and segmented point cloud data onto the distortion-removed image to generate an initial depth map; perform depth completion on the initial depth map based on the image feature data to generate a dense depth map; The digital twin modeling module is used to, based on the dense depth map, the noise-reduced and segmented point cloud data, and the inertial measurement unit correction data, construct a three-dimensional environment model in real time through SLAM, and the three-dimensional environment model is dynamically updated according to the input of the sensor; The real-time feedback module is used to render and visually output the data of the three-dimensional environment model through a low-latency communication protocol to complete digital twin real-time modeling.

[0015] Compared with the prior art, the digital twin real-time modeling method and system based on lidar and image fusion of the present invention have the following beneficial effects: A digital twin real-time modeling method based on lidar and image fusion. Through data synchronization acquisition and data preprocessing, combined with multi-modal data fusion for digital twin modeling and dynamic update, and finally real-time visual feedback to form a complete digital twin real-time model. The present invention synchronously acquires data through lidar, cameras, and inertial measurement units (IMUs) to ensure time and space alignment, avoiding model deviations caused by data misalignment. The data quality is improved through point cloud noise reduction and segmentation, and the usability of visual information is enhanced through image distortion correction and feature extraction. In addition, the present invention combines the depth accuracy of point clouds and the texture details of images to generate a more complete depth map, making up for the deficiencies of single sensors. Real-time environment modeling is achieved through SLAM technology, and dynamic updates are made to adapt to scene changes. Low-latency transmission and rendering ensure that users can immediately perceive the environmental state. The synchronous acquisition of the present invention provides an alignment basis for subsequent fusion, and the high-quality data after preprocessing improves the fusion accuracy; the dense depth map generated by fusion directly supports SLAM modeling, and the dynamic update ability further depends on the real-time feedback of IMU correction data; the visualization module intuitively presents the modeling results, forming a closed-loop feedback, which has better practicability. Description of the Drawings

[0016] The accompanying drawings in the specification are used to provide a further understanding of the present invention and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0017] Figure 1 It is a schematic flow chart of a digital twin real-time modeling method based on lidar and image fusion of the present invention; Figure 2 It is a schematic diagram of the flow principle of a digital twin real-time modeling method based on lidar and image fusion of the present invention; Figure 3 It is a schematic diagram of the architecture of a digital twin real-time modeling system based on lidar and image fusion of the present invention. Detailed Embodiments

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and illustrated herein can be arranged and designed in various different configurations.

[0019] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0020] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0021] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper", "lower", "horizontal", "inner", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the invention product is usually placed during use, it is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation of the present invention. In addition, terms such as "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0022] In addition, if the term "horizontal" appears, it does not mean that the component is required to be absolutely horizontal, but it can be slightly inclined. For example, "horizontal" only means that its direction is more horizontal relative to "vertical", and does not mean that the structure must be completely horizontal, but it can be slightly inclined.

[0023] In the description of the embodiments of the present invention, it should also be noted that unless otherwise clearly specified and limited, if terms such as "set", "installed", "connected", "coupled" are understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0024] How to efficiently fuse lidar and image information to achieve real-time and high-precision digital twin modeling and intelligent detection.

[0025] As Figure 1 and Figure 2 shown, the present invention provides a digital twin real-time modeling method based on lidar and image fusion, including the following steps: S1. Synchronously collect point cloud data, image data, and motion attitude data through a sensor group to generate synchronized lidar point cloud data, image data, and inertial measurement unit calibration data; Among them, the sensor group adopts lidar, industrial cameras, and inertial measurement units; S2. Denoise and geometrically segment the lidar point cloud data to generate denoised and segmented point cloud data; perform distortion removal and feature extraction on the synchronized image data to generate undistorted images and image feature data; S3. Project the denoised and segmented point cloud data onto the undistorted image to generate an initial depth map; perform depth completion on the initial depth map based on the image feature data to generate a dense depth map; S4. Based on the dense depth map, the denoised and segmented point cloud data, and the inertial measurement unit calibration data, construct a three-dimensional environment model in real time through SLAM, and the three-dimensional environment model is dynamically updated according to the input of the sensors; S5. Render and visually output the data of the three-dimensional environment model through a low-latency communication protocol to complete real-time digital twin modeling.

[0026] The real-time digital twin modeling method of the present invention combines the high-precision three-dimensional point cloud of lidar with the rich texture information of images to improve the accuracy of object recognition and environmental perception. Through SLAM and deep learning technologies, it ensures that the environment model can be updated in real time and supports the efficient reconstruction of dynamic scenes. Through a low-latency communication protocol, low-latency data transmission and visualization are achieved, ensuring the real-time performance and stability of the constructed model, making it applicable to different scenarios and improving the applicability.

[0027] In some embodiments, the real-time digital twin modeling method of the present invention based on lidar and image fusion: (1) Data acquisition and synchronization; The present invention synchronously acquires point cloud and image data using lidar and cameras. Data alignment is ensured through time synchronization (e.g., GPS clock synchronization, trigger synchronization) and spatial calibration (such as the transformation matrix of the camera and lidar coordinate systems).

[0028] Specifically: First, the point cloud data scanned by the lidar and the images captured by the camera are stored separately. Then, using the synchronization module, the consistency of the two data in space and time is ensured through calibration algorithms (such as the checkerboard method or the rotation and translation transformation of the lidar and the camera).

[0029] (2) Data processing; Point cloud denoising and segmentation in the present invention: Filtering algorithms (such as Voxel Grid filter or Gaussian filter) are used to remove noise from the point cloud, and the environment is segmented through geometric shape-based segmentation algorithms (such as Region Growing, K-means clustering).

[0030] Furthermore, for image undistortion and feature extraction in the present invention, an image processing algorithm (such as the undistortion algorithm in OpenCV) is used to correct the distortion of the camera lens. Feature extraction is performed through algorithms such as SIFT and ORB to extract important feature points in the image, providing key features for subsequent data fusion.

[0031] (3) Multi-modal data fusion; In the present invention, the lidar point cloud is projected onto the image plane to generate a depth map, and lidar and image information are combined through feature-level fusion (for example: early or late fusion in a convolutional neural network (CNN)) to provide richer three-dimensional depth information.

[0032] In the present invention, the fusion algorithm uses CSPN++ (depth completion algorithm) to generate a dense depth map to complete the missing depth parts in the image.

[0033] Furthermore, in the present invention, the lidar point cloud is projected onto the image plane, and the formula (1) used is as follows: (1); Wherein, is the camera intrinsic matrix, is the transformation matrix from the lidar to the industrial camera.

[0034] (4) Digital twin modeling; The present invention uses SLAM technology: LIO-SAM (Lidar Inertial Odometry) and ORB-SLAM3 (feature-based SLAM algorithm) are used to achieve dynamic reconstruction of the three-dimensional environment, and the virtual model is updated through real-time feedback.

[0035] Furthermore, the real-time three-dimensional environment model of the present invention will be updated in real time according to sensor inputs to ensure that the virtual model reflects the dynamic changes in the physical world. In some practical applications, for example, it can be applied to vehicle driving or object movement, etc.

[0036] (5) Real-time feedback; The present invention realizes low-latency real-time communication through ROS2 (Robot Operating System2), transmits sensor data to the computing unit, and feeds it back to Unity3D in real time for visual display. Scene rendering is performed through the Unity3D engine, and the generated three-dimensional model and detection results are visualized to display the real-time environment status and detection information.

[0037] The present invention also provides a digital twin real-time modeling system based on lidar and image fusion. The system includes a data acquisition and synchronization module and a computing unit. The computing unit includes a data processing module, a multi-modal fusion module, a digital twin modeling module, and a real-time feedback module, wherein: The data acquisition and synchronization module is used to synchronously acquire point cloud data, image data, and motion attitude data through a lidar, an industrial camera, and an inertial measurement unit, and generate synchronized lidar point cloud data, image data, and inertial measurement unit calibration data; The data processing module is used to perform noise reduction and geometric segmentation processing on the lidar point cloud data to generate denoised and segmented point cloud data; perform distortion removal and feature extraction processing on the synchronized image data to generate undistorted images and image feature data; The multi-modal fusion module is used to project the denoised and segmented point cloud data onto the undistorted image to generate an initial depth map; perform depth completion on the initial depth map based on the image feature data to generate a dense depth map; The digital twin modeling module is used to construct a three-dimensional environment model in real time based on the dense depth map, the denoised and segmented point cloud data, and the inertial measurement unit calibration data through SLAM, and the three-dimensional environment model is dynamically updated according to the input of the sensor; The real-time feedback module is used to render and visually output the data of the three-dimensional environment model through a low-latency communication protocol to complete real-time digital twin modeling.

[0038] In the real-time digital twin modeling system of the present invention, a lidar (LiDAR), an industrial camera (Camera), and an inertial measurement unit (IMU) form a sensor group; the computing unit, as the core of data processing, is connected to each module. Through the visualization platform: the final output module is used to display the modeling results.

[0039] The sensor module (LiDAR, Camera, IMU) of the present invention transmits data to the data acquisition and synchronization module. The data acquisition and synchronization module transmits the synchronized data to the data processing module. The data processing module preprocesses the data and then transmits it to the multi-modal fusion module. The multi-modal fusion module transmits the processed data to the digital twin modeling module. The digital twin modeling module transmits the modeling result to the real-time feedback module. The real-time feedback module transmits the final result to the visualization platform, and finally completes real-time digital twin modeling.

[0040] As Figure 3 shown, in some embodiments, in the real-time digital twin modeling system based on lidar and image fusion of the present invention, the hardware part includes: Lidar (LiDAR): Used for high-precision three-dimensional point cloud data acquisition, with long-distance and high-precision measurement capabilities, and can provide depth information of the environment.

[0041] Industrial camera: Used to collect two-dimensional images in the environment and provide rich visual information. An industrial camera with a relatively high resolution and frame rate can be selected, which is suitable for real-time data processing.

[0042] IMU (Inertial Measurement Unit): It is used to measure the acceleration, angular velocity and attitude of the device in order to correct sensor data and provide more accurate dynamic behavior data.

[0043] Computing unit: It can be an embedded system (such as NVIDIA Jetson) or a high-performance server, responsible for data processing, model inference and feedback output.

[0044] The software part includes: Data acquisition and synchronization module: Synchronously acquire data from different sensors to ensure the time synchronization and spatial alignment of lidar point clouds and image data.

[0045] Data processing module: It includes noise reduction and segmentation of point cloud data, image undistortion and feature extraction to ensure the quality of input data.

[0046] Multi-modal fusion module: Fuse lidar and image data, register and combine point cloud and image data through algorithms to generate a more accurate depth map.

[0047] Digital twin modeling module: Realize the real-time three-dimensional reconstruction of the environment through SLAM technology (such as LIO-SAM, ORB-SLAM3) and dynamically update the virtual model.

[0048] Real-time feedback module: Achieve low-latency data transmission through ROS2 and perform real-time visualization rendering and feedback through Unity3D.

[0049] The digital twin real-time modeling method based on lidar and image fusion of the present invention realizes the complementarity of depth information and visual information by combining the high-precision three-dimensional point cloud data of lidar and the texture information of image sensors. Lidar can work stably under low light and complex weather conditions, while image sensors provide rich texture and color details. The fusion of the two effectively makes up for the deficiencies of single sensors and significantly improves the environmental adaptability and data accuracy of the system.

[0050] The digital twin real-time modeling method of the present invention uses data fusion to ensure the consistency of lidar point clouds and image data in time and space through time synchronization and spatial calibration. Through SLAM technology (such as LIO-SAM and ORB-SLAM3), the system can construct and dynamically update a three-dimensional virtual model in real time, supporting the efficient reconstruction of dynamic scenes. In addition, the system quickly visualizes the fused data through low-latency communication technology (such as ROS2) and real-time rendering engine (such as Unity3D), providing users with real-time environmental status and detection information.

[0051] In addition, by integrating lidar and image data, the present invention significantly improves the environmental perception ability of the digital twin real-time modeling system. The system can more accurately identify and classify objects, reducing misjudgments and missed detections. In complex environments (such as urban streets, industrial factories, etc.), the integrated data can more comprehensively perceive the environment and provide more reliable detection results.

[0052] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and does not impose any formal limitations on the present invention. Any person skilled in the art can smoothly implement the present invention according to the instructions and the above description. Slight changes, modifications, and equivalent variations made by using the disclosed technical content are all equivalent embodiments of the present invention. At the same time, any equivalent changes, modifications, and evolutions made to the above embodiments based on the substantial technology of the present invention still fall within the protection scope of the technical solutions of the present invention.

Claims

1. A digital twin real-time modeling method based on laser radar and image fusion, characterized in that: The steps include: S1. Synchronously collect point cloud data, image data and motion posture data through a sensor group to generate synchronized laser radar point cloud data, image data and inertial measurement unit correction data; Among them, the sensor group uses lidar, industrial cameras and inertial measurement units; S2, performing noise reduction and geometric segmentation processing on the laser radar point cloud data to generate noise-reduced and segmented point cloud data; performing dedistortion and feature extraction processing on the synchronized image data to generate a dedistorted image and image feature data; S3, projecting the denoised and segmented point cloud data onto the dedistorted image to generate an initial depth map; performing depth completion on the initial depth map based on the image feature data to generate a dense depth map; S4, based on the dense depth map, the point cloud data after noise reduction and segmentation, and the inertial measurement unit correction data, a three-dimensional environment model is constructed in real time through SLAM, and the three-dimensional environment model is dynamically updated according to the input of the sensor; S5. Render and visualize the data of the three-dimensional environment model through a low-latency communication protocol to complete real-time modeling of the digital twin.

2. The digital twin real-time modeling method based on laser radar and image fusion according to claim 1 is characterized in that: In S1, synchronous data collection specifically includes: achieving time synchronization between the laser radar and the industrial camera through time synchronization, and completing spatial alignment through spatial calibration.

3. The digital twin real-time modeling method based on laser radar and image fusion according to claim 2 is characterized in that: In S1, time synchronization includes: GPS clock synchronization or hardware trigger, and spatial calibration uses a calibration algorithm to obtain the rotation matrix and translation vector between the laser radar coordinate system and the camera coordinate system.

4. The digital twin real-time modeling method based on laser radar and image fusion according to claim 1 is characterized in that: In S2, the noise reduction and geometric segmentation processing: a filtering algorithm is used to remove the point cloud noise in the laser radar point cloud data, and a geometric shape segmentation algorithm is used to perform geometric segmentation on the point cloud in the laser radar point cloud data.

5. The digital twin real-time modeling method based on laser radar and image fusion according to claim 4 is characterized in that: In S2, the dedistortion and feature extraction processing of the synchronized image data includes: using an image processing algorithm to correct the camera lens distortion, and using a SIFT algorithm or an ORB algorithm to extract key feature points in the image data.

6. The digital twin real-time modeling method based on laser radar and image fusion according to claim 1 is characterized in that: In S3, performing depth completion on the initial depth map based on the image feature data includes: using a depth completion algorithm in a convolutional neural network to fill in missing areas of the initial depth map to generate a dense depth map.

7. The digital twin real-time modeling method based on laser radar and image fusion according to claim 6 is characterized in that: In S3, the point cloud data after noise reduction and segmentation is projected onto the dedistorted image using the following formula (1): （1）； in, is the camera intrinsic parameter matrix, Transformation matrix from laser radar to industrial camera.

8. The digital twin real-time modeling method based on laser radar and image fusion according to claim 1 is characterized in that: In the S4, SLAM includes LIO-SAM algorithm or ORB-SLAM3 algorithm, which is combined with inertial measurement unit correction data to realize real-time construction and dynamic update of three-dimensional environment model.

9. The digital twin real-time modeling method based on laser radar and image fusion according to claim 1 is characterized in that: In the S5, the low-latency communication protocol includes ROS2, and the visualization engine includes the Unity3D engine.

10. The system according to any one of claims 1 to 9 on which the digital twin real-time modeling method based on laser radar and image fusion is based is characterized in that: The system includes a data acquisition and synchronization module and a computing unit, wherein the computing unit includes a data processing module, a multimodal fusion module, a digital twin modeling module and a real-time feedback module, wherein: The data acquisition and synchronization module is used to synchronously acquire point cloud data, image data and motion posture data through laser radar, industrial camera and inertial measurement unit, and generate synchronized laser radar point cloud data, image data and inertial measurement unit correction data; The data processing module is used to perform noise reduction and geometric segmentation processing on the laser radar point cloud data to generate noise-reduced and segmented point cloud data; perform dedistortion and feature extraction processing on the synchronized image data to generate dedistorted images and image feature data; The multimodal fusion module is used to project the denoised and segmented point cloud data onto the dedistorted image to generate an initial depth map; perform depth completion on the initial depth map based on the image feature data to generate a dense depth map; The digital twin modeling module is used to build a 3D environment model in real time through SLAM based on dense depth maps, point cloud data after noise reduction and segmentation, and inertial measurement unit correction data. The 3D environment model is dynamically updated according to the input of the sensor; The real-time feedback module is used to render and visualize the data of the three-dimensional environment model through a low-latency communication protocol to complete the real-time modeling of the digital twin.

Images