A digital twin model construction method, system, medium, equipment and product

Abstract

The present invention relates to the field of digital twin technology, and discloses a method, system, medium, equipment and product for constructing a digital twin model, including: obtaining image data and point cloud data of a mountain scenic area; extracting features of the image data and point cloud data using a deep learning model, respectively, to obtain image features and point cloud features; based on the image features and point cloud features, obtaining a matching matrix by minimizing the distance between the image data and the point cloud data after registration, and obtaining a conversion matrix by minimizing the square sum of the distance between the converted point cloud data and the image data based on the matching matrix; for each point in the point cloud data, finding the corresponding pixel in the image data according to the matching matrix, converting the color information of the corresponding pixel using the conversion matrix, and assigning it to the point cloud data to obtain a digital twin model of the mountain scenic area. No large number of labels are required for training the neural network, and the speed of constructing the digital twin model is accelerated.

Description

Technical Field

[0001] The present invention relates to the field of digital twin technology, and more specifically, to a method, system, medium, equipment and product for constructing a digital twin model. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] At present, most digital twin model construction methods on the market are designed for cities or plain areas. When faced with complex terrain such as mountainous scenic areas, they often show their limitations. These limitations are mainly manifested in insufficient data collection accuracy and point cloud density, as well as poor adaptability of equipment to complex terrain. They are unable to meet the needs of high-quality digital twin model construction in mountainous scenic areas, limiting the breadth and effectiveness of their applications in complex terrain.

[0004] Mountain scenic areas pose great challenges to data collection and processing due to their complex terrain and undulating topography. Accurate modeling requires high-precision data collection technology and complex data processing algorithms. In addition, the environment of mountain scenic areas is changeable, such as cloudy, foggy and other severe weather conditions, which have a significant impact on the quality and efficiency of data collection, and increase the difficulty of obtaining accurate digital twin models and knowledge point data.

[0005] Integrating multiple professional equipment such as lidar, panoramic cameras, monocular cameras and high-precision inertial navigation into the vehicle-mounted platform system not only requires solving the problems of physical installation between devices and stability in complex terrain, but also makes it difficult to achieve high-precision, high-point cloud density data collection; more importantly, it faces the technical challenge of synchronizing and fusing large amounts of data from different devices. The type, format and timestamp of the data collected by each device may be different. How to effectively synchronize and fuse these data for subsequent processing and analysis is the key to achieving a high-quality digital twin model.

[0006] However, most of the existing digital twin model construction methods rely on machine learning neural network models, that is, multi-source data is directly input into the neural network model to obtain a digital twin model. This method is only applicable to small scenes such as cities or plains. For complex scenes such as mountain scenic spots, it is difficult to train the neural network model due to the lack of labeled data; moreover, mountain scenic spots are large in scope and large in data volume, so the speed of machine learning neural network model construction for digital twin models is slow. Summary of the invention

[0007] In order to solve the above problems, the present invention provides a method, system, medium, equipment and product for constructing a digital twin model. It only uses a deep learning model to extract features of images and point clouds, and performs image and point cloud alignment and point cloud coloring by obtaining matching matrices and transformation matrices. It only requires a pre-trained deep learning model, and does not require a large number of labels for neural network training. Moreover, it speeds up the construction of the digital twin model.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] A first aspect of the present invention provides a method for constructing a digital twin model, comprising:

[0010] Acquire image data and point cloud data of mountain scenic areas;

[0011] For image data and point cloud data, deep learning models are used to extract features to obtain image features and point cloud features;

[0012] Based on the image features and the point cloud features, after obtaining a matching matrix by minimizing the distance between the registered image data and the point cloud data, a conversion matrix is ​​obtained by minimizing the sum of squares of the distances between the converted point cloud data and the image data based on the matching matrix;

[0013] For each point in the point cloud data, the corresponding pixel in the image data is found according to the matching matrix. The color information of the corresponding pixel is converted using the transformation matrix and assigned to the point cloud data to obtain a digital twin model of the mountain scenic area.

[0014] Furthermore, the distance between the image data and the point cloud data after minimization can be expressed as: , where M is the matching matrix, is the element in the i-th row and j-th column of the matching matrix, is the distance metric function, is the feature of the i-th pixel in the image data, is the feature of the jth point in the point cloud data.

[0015] Furthermore, the minimization of the sum of squares of distances between the converted point cloud data and the image data is expressed by the formula: ,in, T represents the transformation matrix, is the feature of the i-th point in the point cloud data, The image data The features of pixels, Represents the pixel in the image data found in the matching matrix M that matches the i-th point in the point cloud data.

[0016] Furthermore, it also includes: for the digital twin model, using multi-level detail technology to simplify, using high dynamic range imaging technology to enhance lighting, using instanced rendering technology to reduce rendering batches and compressing the digital twin model data.

[0017] Furthermore, it also includes: aligning the point cloud data with the Beidou coordinate system.

[0018] Furthermore, it also includes: using Beidou timestamp to align the timestamps of image data and point cloud data.

[0019] A second aspect of the present invention provides a digital twin model construction system, comprising:

[0020] A data acquisition module, which is configured to: acquire image data and point cloud data of a mountain scenic area;

[0021] A feature extraction module is configured to: extract features from image data and point cloud data using a deep learning model to obtain image features and point cloud features;

[0022] A matching module, configured to: based on the image features and the point cloud features, obtain a matching matrix by minimizing the distance between the registered image data and the point cloud data, and then, based on the matching matrix, obtain a transformation matrix by minimizing the sum of squares of the distances between the transformed point cloud data and the image data;

[0023] The model building module is configured as follows: for each point in the point cloud data, the corresponding pixel in the image data is found according to the matching matrix, the color information of the corresponding pixel is converted using the conversion matrix, and then assigned to the point cloud data to obtain a digital twin model of the mountain scenic area.

[0024] The third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor. When the program is executed by the processor, the steps in a digital twin model construction method as described above are implemented.

[0025] The fourth aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, the steps in a digital twin model construction method as described above are implemented.

[0026] A fourth aspect of the present invention provides a computer program product or a computer program, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the above-described method for constructing a digital twin model.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] The present invention only uses a deep learning model to extract features from images and point clouds of mountain scenic areas. By obtaining matching matrices and transformation matrices, it performs image and point cloud registration and point cloud coloring. It only requires a pre-trained deep learning model, and does not require a large number of labels for neural network training. Moreover, it speeds up the construction of the digital twin model.

[0029] The present invention optimizes the digital twin model. The optimized digital twin model is not only suitable for the management and planning of mountain scenic spots, but also can support various applications such as virtual tourism and environmental monitoring, and has broad social and economic value.

[0030] The present invention integrates advanced equipment such as a high-precision inertial navigation system, a three-dimensional laser scanner, a high-resolution camera, and a panoramic camera into a mobile platform, thereby enabling efficient and high-precision data collection in complex terrain. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which constitute a part of the specification of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but do not constitute limitations of the present invention.

[0032] Figure 1 This is a flow chart of a method for constructing a digital twin model according to Embodiment 1 of the present invention;

[0033] Figure 2 A hardware connection diagram of a vehicle-mounted measurement system according to a first embodiment of the present invention;

[0034] Figure 3 is a structural diagram of a vehicle-mounted measurement system according to a first embodiment of the present invention;

[0035] Figure 4 FIG. 1 is a schematic diagram of the installation of the vehicle-mounted measurement system according to the first embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0037] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0038] In the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention is further described below with reference to the accompanying drawings and embodiments.

[0039] Embodiment 1

[0040] The purpose of this embodiment is to provide a method for constructing a digital twin model.

[0041] This embodiment provides a method for constructing a digital twin model. Figure 1 As shown, the following steps are included:

[0042] Step 1: Integration and installation of vehicle-mounted measurement system.

[0043] like Figure 3 As shown, the core of the vehicle-mounted measurement system is to integrate a variety of high-precision measurement equipment on a mobile platform. The mobile platform is a vehicle equipped with a special mounting frame 6 to ensure that all measurement equipment is stably installed on the top of the vehicle. Figure 2 As shown, these measuring devices include but are not limited to a high-precision inertial navigation system (high-precision inertial navigation system, Novatel IMU-ISA-100C) 5, a first 3D laser scanner (RIEGL VZ2000i) 1, a second 3D laser scanner (RIEGL VUX-1HA22) 2, a wheel speed meter 13, a high-resolution camera (PHASE IXA180) 3 and a panoramic camera (LADYBUG 6) 4.

[0044] Among them, Figure 4 As shown, the special mounting frame 6 is made of high-strength lightweight alloy material and is designed as an adjustable multi-point support structure to adapt to different models of vehicles and equipment. The mounting frame 6 includes a main bracket 7 and multiple auxiliary antenna fixing arms 10, each of which is equipped with a shockproof device to reduce the impact of vibration during driving on the Beidou signal receiving antenna.

[0045] The main bracket 7 is fixed to the vehicle top interface or luggage rack 12 through a special quick lock 9, and the auxiliary antenna fixing arm 10 is connected to the main bracket 7 through a rotating joint 11. The rotating joint 11 allows the auxiliary antenna fixing arm 10 to freely adjust the angle in three-dimensional space. The auxiliary antenna fixing arm 10 is used to install the Beidou signal receiving antenna. Each measuring device is connected to the main bracket 7 through a customized adapter plate 8, and the customized adapter plate 8 is designed with a fine-tuning mechanism to achieve precise positioning of the measuring device.

[0046] Among them, the anti-vibration device uses high-performance shock-absorbing materials and spring shock-absorbing structures to effectively absorb the vibration generated during vehicle driving. In particular, for high-precision inertial navigation systems and 3D laser scanners, the installation position is selected near the center of gravity of the vehicle to minimize the error caused by movement.

[0047] Step 2: Data collection.

[0048] While the vehicle is moving, each measuring device works synchronously to collect data in its respective area.

[0049] The inertial navigation system provides accurate real-time position and attitude information of the vehicle-mounted measurement system, provides reference coordinates for data collected by other devices, records timestamp information, and helps complete the matching and fusion of multi-dimensional data. The high-precision positioning information provided by the Beidou system ensures accurate spatial positioning in the complex terrain of mountainous scenic areas.

[0050] The laser scanner is responsible for collecting 3D laser point cloud data of the surrounding environment of the mountain scenic area. It is used for high-precision, high-point cloud density terrain and object data collection, covering the data collection of buildings on the side of the vehicle and the rear of the vehicle's forward direction. Specifically, the first 3D laser scanner (RIEGL VZ2000i) 1 is used to obtain point cloud data such as shops on the side of the vehicle (abbreviated as point cloud on the side of the vehicle's forward direction). The scanner has a high point density and can effectively obtain information about shops along the way; the second 3D laser scanner (RIEGL VUX-1HA22) 2 is used to obtain road information on the rear side of the vehicle's forward direction (abbreviated as point cloud behind the vehicle's forward direction). Because an inertial navigation system is installed, both sets of point cloud data have location information, and the two sets of point clouds can be automatically matched based on the location coordinate information.

[0051] The camera collects high-resolution image data of mountain scenic areas. Among them, the panoramic camera collects panoramic photo data, provides 360-degree panoramic images, provides all-round color information for point cloud data, enhances visual effects, and realizes panoramic image acquisition. The high-resolution camera collects high-resolution photo data (monocular photo data) of the landscape (such as street scenes, shops, etc.) on both sides of the road in the mountain scenic area (i.e., on the side of the vehicle's travel direction), collects high-quality image data, and is used for color processing and texture mapping of point cloud data such as shops in the mountain scenic area on the side of the vehicle's travel direction, enhancing the realism of the digital twin model.

[0052] The wheel speed meter collects vehicle speed information, that is, the forward speed information of the vehicle-mounted measurement system.

[0053] exist Figure 2In the figure, COM is a serial port, and the panoramic image data, monocular photo data, and point cloud data are all directly connected to the computer through the serial port of the computer; RS232 is an inertial navigation data transmission protocol, which mainly transmits timestamp information to ensure the position accuracy of the data; CAN is a control signal used to obtain the forward speed of the vehicle; PPS is a pulse per second, which is used for the three-dimensional laser scanner and inertial navigation position registration to provide position information for the point cloud data.

[0054] The collection of these data is carried out synchronously, ensuring the consistency of the data in time and space.

[0055] Step 3: The data collected synchronously by each measuring device are uploaded to the data processing unit (DPU) for data fusion and data processing.

[0056] Data fusion is one of the key steps of the present invention, which involves combining data from different sources (such as point cloud data and image data) to generate a richer and more accurate digital twin model. This step requires complex algorithms, such as image to point cloud registration, color information mapping, etc. Through data fusion, point cloud data can be colored to improve the realism and visualization of the model.

[0057] Step 301: timestamp alignment.

[0058] In this embodiment, Beidou timestamps are used to align the timestamps of point cloud data (side point cloud in the direction of vehicle travel or rear point cloud in the direction of vehicle travel), panoramic photo data, and monocular photo data.

[0059] Step 302, the data processing unit also obtains the static differential data collected by the GNSS (Global Navigation Satellite System) base station, and performs POS (airborne positioning and attitude system) combined solution on the vehicle position and attitude information, vehicle speed information and the static differential data, and obtains the vehicle moving trajectory data and its timestamp information, which can be used for data fusion and registration between multiple devices.

[0060] Step 303: Data synchronization.

[0061] Based on the vehicle's trajectory data and its timestamp information, point cloud data (point cloud on the side of the vehicle's direction of travel or point cloud behind the vehicle's direction of travel), panoramic photo data and monocular photo data are synchronized.

[0062] Data synchronization ensures that data from different devices can accurately correspond to the same time point and spatial location. This step is crucial for subsequent data fusion.

[0063] Step 303: data preprocessing.

[0064] The purpose of data preprocessing is to prepare and optimize data and lay a solid foundation for data fusion and processing.

[0065] First, the point cloud data P (the side point cloud in the direction of vehicle travel or the rear point cloud in the direction of vehicle travel) and the image data I (the panoramic photo data or the monocular photo data) are cleaned (such as removing noise) and corrected (such as correcting camera distortion) to improve data quality.

[0066] Wherein, the distortion correction function is applied to the image data I , is the correction parameter.

[0067] Among them, the image data I adopts the Beidou coordinate system.

[0068] Secondly, the point cloud data is aligned with the BeiDou coordinate system in the inertial navigation system, and a point cloud data conversion process is introduced, which can ensure seamless integration and compatibility between the point cloud data and the BeiDou coordinate system in the inertial navigation system.

[0069] During the conversion process, the coordinates of the point cloud data in the scanning coordinate system ( X p , Y p , Z p ) and the coordinates in the BeiDou coordinate system ( X BD , Y BD , Z BD ). The coordinate conversion formula from point cloud data to the Beidou inertial navigation coordinate system is: ,but In the formula, α , β and γ is the rotation angle from the scanning coordinate system to the inertial navigation Beidou coordinate system, △ a ,△ b and c It is the eccentricity between the scanning coordinate system and the inertial navigation Beidou coordinate system.

[0070] Step 302: feature extraction and point cloud and image registration.

[0071] Extract features from the preprocessed point cloud data and image data and match (register) them to facilitate data fusion. It should be noted that the point cloud data and image data to be matched need to have the same corresponding trajectory data and their timestamp information.

[0072] (1) Image data I , perform image feature extraction and obtain image features : .in, is the feature extraction function based on the first deep learning model, is the first deep learning model parameter.

[0073] (2) Point cloud data I , extract point cloud features and obtain point cloud features : .in, is the feature extraction function of the second deep learning model for point cloud, is the second deep learning model parameter.

[0074] Wherein, both the first deep learning model and the second deep learning model adopt pre-trained neural networks.

[0075] (3) Feature matching is performed based on image features and point cloud features to achieve point cloud and image registration.

[0076] First, the matching matrix is ​​obtained by minimizing the distance between the registered image data and the point cloud data. That is, by solving the formula , and get the matching matrix M. Where M is the matching matrix, is the element in the i-th row and j-th column in the matching matrix, which represents the correspondence between the feature points in the image data and the feature points in the point cloud data. is the distance metric function between features, is the feature of the i-th pixel in the image data, is the feature of the jth point in the point cloud data.

[0077] After the matching matrix M determines the correspondence between the image data and the point cloud data, these correspondences can be used to solve an optimal transformation matrix T, which is used to transform (or align) the point cloud data space to the image data space, or vice versa.

[0078] Finding the optimal transformation matrix T usually involves using an optimization algorithm to minimize a suitably defined loss function, usually the sum of the squared distances between the transformed point cloud data and the image data, expressed as: ,in, is the feature of the i-th point in the point cloud data, The image data The features of pixels, Represents the pixel in the image data found in the matching matrix M that matches the i-th point in the point cloud data.

[0079] In this embodiment, an optimization algorithm (such as gradient descent method, least square method, etc.) is used to minimize the loss function L(T) to solve the optimal transformation matrix T. The optimization goal is to find T that minimizes the loss function, that is, to find the best transformation matrix to best align the point cloud data with the image data. The minimum and optimal iteration method is expressed as: .

[0080] (4) Color processing of point cloud data to achieve three-dimensional reconstruction of mountain scenic areas.

[0081] Once the optimal transformation matrix T is obtained, the color information in the image data can be mapped to the corresponding point cloud data to achieve colorization of the point cloud. Specifically, for each point i in the point cloud data, find its corresponding position in the image data according to the matching matrix M. , and then the color information of the corresponding position is converted using the transformation matrix T and assigned to the point cloud to obtain three-dimensional color point cloud data.

[0082] It should be noted that monocular photos have higher pixels and are mainly used to obtain color photos of shops on the side of the vehicle's direction of travel, providing color information for RIEGL VZ2000i; panoramic equipment is mainly used to collect road panoramic image data, mainly for road point cloud coloring, and assisting in the side shop point cloud coloring. That is, after the side point cloud of the vehicle's direction of travel is registered with the panoramic photo data and the monocular photo data respectively, the monocular photo data and the panoramic photo data are used to color the side point cloud of the vehicle's direction of travel; after the rear point cloud of the vehicle's direction of travel is registered with the panoramic photo data, the panoramic photo data is used to color the rear point cloud of the vehicle's direction of travel.

[0083] The three-dimensional color point cloud data corresponding to different timestamps and travel trajectory data are combined to obtain a digital twin model of the mountain scenic area, thereby integrating data from different sources into a unified, high-quality digital twin model.

[0084] Step 4: Optimization and application of digital twin model.

[0085] After the generated digital twin model is initially completed, it needs to be optimized to improve its application value. Optimization includes simplification of the digital twin model (reducing unnecessary details to increase processing speed) and quality improvement (such as increasing texture resolution).

[0086] The optimized digital twin model of mountain scenic spots can be applied to multiple fields, including but not limited to planning and management of mountain scenic spots, virtual tourism, environmental monitoring, disaster prevention and assessment, etc. These applications can not only improve the management efficiency of mountain scenic spots and the experience of tourists, but also provide valuable data resources for scientific researchers to support the goals of environmental protection and sustainable development.

[0087] Specifically, the optimization steps of the digital twin model of the mountain scenic area are as follows:

[0088] (1) Model simplification: Multi-level detail (LOD) technology is used to simplify the digital twin model. The level of detail of the digital twin model is dynamically adjusted according to the user's perspective and the position of the digital twin model in the scene to optimize rendering performance while maintaining visual effects;

[0089] (2) Quality improvement: High-resolution processing of the digital twin model texture, including the use of High Dynamic Range Imaging (HDRI) technology to enhance the lighting effect, and the use of detail enhancement technology to enhance the texture details of the digital twin model surface, making the digital twin model more realistic;

[0090] (3) Performance optimization: Improve the rendering efficiency of digital twin models by culling invisible parts of the digital twin model (frustum culling and occlusion culling techniques) and using instanced rendering technology to reduce rendering batches;

[0091] (4) Data compression: Compress the digital twin model data to reduce storage space and increase loading speed. Use efficient data compression algorithms, such as point cloud data compression technology based on wavefront coding and image data compression technology based on texture mapping.

[0092] The present embodiment provides a method for constructing a digital twin model, which only uses a deep learning model to extract features of images and point clouds of mountain scenic areas. By obtaining matching matrices and transformation matrices, the image and point cloud are aligned and the point clouds are colored. Only a pre-trained deep learning model is required, and a large number of labels are not required for training neural networks. Moreover, the speed of constructing digital twin models of mountain scenic areas is accelerated.

[0093] The digital twin model construction method provided in this embodiment solves the limitations of existing digital twin model construction methods in data collection in complex terrain, especially mountainous scenic spots, overcomes the technical challenges of establishing accurate digital twin models of mountainous scenic spots, and provides a new solution for the digital and intelligent management of mountainous scenic spots.

[0094] A digital twin model construction method provided in this embodiment can achieve accurate measurement and recording of various geographic information such as terrain, landforms, vegetation, etc. in a complex mountain scenic environment, and construct a highly realistic and detailed digital twin model.

[0095] A digital twin model construction method provided in this embodiment integrates multiple high-precision measurement devices on a mobile platform (such as a vehicle) to achieve high-precision, all-round data collection and analysis of mountain scenic spots, thereby constructing a high-quality digital twin model. The model is specially designed for the complex terrain of mountain scenic spots, which solves the application limitations in such environments, improves the accuracy and efficiency of data collection, and provides a new technical means for the digital management and research of mountain scenic spots.

[0096] The digital twin model construction method provided in this embodiment integrates multi-source data such as positioning data from the Beidou high-precision inertial navigation system and vehicle wheel speed data in addition to conventional point cloud data and image data, aligns the point cloud data and image data, and realizes point cloud coloring. The integration of multi-source data not only improves the accuracy of the digital twin model, but also enhances the adaptability and responsiveness of the digital twin model to environmental changes.

[0097] A digital twin model construction method provided in this embodiment can perform efficient and high-precision data collection in complex terrain by integrating advanced equipment such as a high-precision inertial navigation system, a three-dimensional laser scanner, a high-resolution camera, and a panoramic camera into a mobile platform.

[0098] A digital twin model construction method provided in this embodiment adopts advanced data processing algorithms to achieve efficient synchronization and precise fusion of different types of data, thereby improving the quality and authenticity of the digital twin model.

[0099] This embodiment provides a digital twin model construction method, which optimizes the digital twin model. The optimized digital twin model is not only suitable for the management and planning of mountain scenic spots, but also can support virtual tourism, environmental monitoring and other applications, and has broad social and economic value.

[0100] The digital twin model construction method provided in this embodiment not only solves the limitations of the existing technology in data accuracy, equipment stability, data processing, etc., but also expands the application scope of digital twin technology in complex terrains such as mountain scenic spots. With the further development and application of technology, it is expected to make important contributions to the sustainable development of mountain scenic spots, environmental protection, scientific research and other fields.

[0101] Embodiment 2

[0102] The purpose of this second embodiment is to provide a digital twin model construction system, including a vehicle-mounted measurement system and a data processing unit (DPU).

[0103] The core of the vehicle-mounted measurement system is to integrate a variety of high-precision measurement equipment on a mobile platform, which is a vehicle equipped with a special mounting frame to ensure that all measurement equipment is stably installed on the top of the vehicle. These measurement equipment include but are not limited to a high-precision inertial navigation system (high-precision inertial navigation system, Novatel IMU-ISA-100C), a first 3D laser scanner (RIEGL VZ2000i), a second 3D laser scanner (RIEGL VUX-1HA22), a wheel speed meter, a high-resolution camera (PHASE IXA180) and a panoramic camera (LADYBUG 6).

[0104] The inertial navigation system provides accurate real-time position and attitude information of the vehicle-mounted measurement system, provides reference coordinates for data collected by other devices, records timestamp information, and helps complete the matching and fusion of multi-dimensional data. The high-precision positioning information provided by the Beidou system ensures accurate spatial positioning in the complex terrain of mountainous scenic areas.

[0105] The laser scanner is responsible for collecting 3D laser point cloud data of the surrounding environment of the mountain scenic area. It is used for high-precision, high-point cloud density terrain and object data collection, covering the data collection of buildings on the side of the vehicle and the rear of the vehicle's forward direction. Specifically, the first 3D laser scanner (RIEGL VZ2000i) is used to obtain point cloud data such as shops on the side of the vehicle (abbreviated as point cloud on the side of the vehicle's forward direction). The scanner has a high point density and can effectively obtain information about shops along the way; the second 3D laser scanner (RIEGL VUX-1HA22) is used to obtain road information on the rear side of the vehicle's forward direction (abbreviated as point cloud behind the vehicle's forward direction). Because the inertial navigation system is installed, both sets of point cloud data have location information, and the two sets of point clouds can be automatically matched according to the location coordinate information.

[0106] The camera collects high-resolution image data of the mountain scenic area. Among them, the panoramic camera collects panoramic photo data, provides 360-degree panoramic images, provides all-round color information for the point cloud data, enhances the visual effect, and realizes panoramic image acquisition. The high-resolution camera collects high-resolution photo data (monocular photo data) of the landscape on both sides of the road in the mountain scenic area (such as street scenes, shops, etc.), collects high-quality image data, and is used for color processing and texture mapping of point cloud data such as shops in the mountain scenic area on the side of the vehicle's travel direction, enhancing the realism of the digital twin model.

[0107] The wheel speed meter collects vehicle speed information, that is, the forward speed information of the vehicle-mounted measurement system.

[0108] The special mounting frame is made of high-strength lightweight alloy material and is designed as an adjustable multi-point support structure to adapt to different models of vehicles and equipment. The mounting frame includes a main bracket and multiple auxiliary antenna fixing arms, each of which is equipped with a shockproof device to reduce the impact of vibration during driving on the Beidou signal receiving antenna.

[0109] The main bracket is fixed to the top interface or luggage rack of the vehicle through a special quick lock, and the auxiliary antenna fixing arm is connected to the main bracket through a rotating joint. The rotating joint allows the auxiliary antenna fixing arm to freely adjust the angle in three-dimensional space. The auxiliary antenna fixing arm is used to install the Beidou signal receiving antenna. Each measuring device is connected to the main bracket through a customized adapter plate, which is designed with a fine-tuning mechanism to achieve precise positioning of the measuring device.

[0110] Among them, the anti-vibration device uses high-performance shock-absorbing materials and spring shock-absorbing structures to effectively absorb the vibration generated during vehicle driving. In particular, for high-precision inertial navigation systems and 3D laser scanners, the installation position is selected near the center of gravity of the vehicle to minimize the error caused by movement.

[0111] The data processing unit (DPU) includes a data acquisition module, a timestamp alignment module, a data preprocessing module, a feature extraction module, a matching module, a model building module, and a model optimization module.

[0112] A data acquisition module, which is configured to: acquire image data (panoramic photo data or monocular photo data) and point cloud data of a mountain scenic area;

[0113] A timestamp alignment module is configured to: use Beidou timestamp to align the timestamps of image data and point cloud data;

[0114] A data synchronization module is configured to: synchronize point cloud data (side point cloud in the direction of vehicle travel or rear point cloud in the direction of vehicle travel), panoramic photo data and monocular photo data based on the vehicle travel trajectory data and its timestamp information;

[0115] A data preprocessing module, which is configured to: align the point cloud data with the Beidou coordinate system;

[0116] A feature extraction module is configured to: extract features from image data and point cloud data using a deep learning model to obtain image features and point cloud features;

[0117] A matching module, configured to: based on the image features and the point cloud features, obtain a matching matrix by minimizing the distance between the registered image data and the point cloud data, and then, based on the matching matrix, obtain a transformation matrix by minimizing the sum of squares of the distances between the transformed point cloud data and the image data;

[0118] The model building module is configured as follows: for each point in the point cloud data, the corresponding pixel in the image data is found according to the matching matrix, the color information of the corresponding pixel is converted using the conversion matrix, and then assigned to the point cloud data to obtain a digital twin model of the mountain scenic area.

[0119] A model optimization module is configured to: simplify the digital twin model using multi-level detail technology, enhance lighting using high dynamic range imaging technology, reduce rendering batches using instanced rendering technology, and compress the digital twin model data.

[0120] It should be noted here that each module in this embodiment corresponds to each step in Example 1 one by one, and the specific implementation process is the same, which will not be repeated here.

[0121] Embodiment 3

[0122] This embodiment provides a computer-readable storage medium on which a computer program is stored. The program is executed by a processor. When the program is executed by the processor, the steps in the digital twin model construction method described in the above embodiment 1 are implemented.

[0123] Embodiment 4

[0124] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, the steps in the digital twin model construction method described in the above embodiment 1 are implemented.

[0125] Embodiment 5

[0126] This embodiment provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the method for building a digital twin model described in the first embodiment.

[0127] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0128] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for constructing a digital twin model, characterized in that: include: Integrate a variety of high-precision measurement equipment into a mobile platform. The measurement equipment includes but is not limited to a high-precision inertial navigation system, a first three-dimensional laser scanner, a second three-dimensional laser scanner, a wheel speed meter, a high-resolution camera and a panoramic camera. Obtain image data and point cloud data of mountain scenic areas. Use Beidou timestamps to align the timestamps of image data and point cloud data. Obtain static differential data collected by a global navigation satellite system base station. Perform POS combination calculation on vehicle position and posture information, vehicle speed information and static differential data to obtain vehicle travel trajectory data and its timestamp information. Based on the vehicle travel trajectory data and its timestamp information, synchronize point cloud data, panoramic photo data and monocular photo data. Image data I uses Beidou coordinate system. Image data I is panoramic photo data or monocular photo data. Align point cloud data with Beidou coordinate system in the inertial navigation system. The mobile platform is a vehicle equipped with a special mounting frame to ensure that all measuring equipment is stably installed on the top of the vehicle. The mounting frame includes a main bracket and multiple auxiliary antenna fixing arms. Each auxiliary antenna fixing arm is equipped with a shockproof device. The main bracket is fixed to the top interface of the vehicle or the luggage rack through a quick lock. The auxiliary antenna fixing arm is connected to the main bracket through a rotating joint. The rotating joint allows the auxiliary antenna fixing arm to freely adjust the angle in three-dimensional space. The auxiliary antenna fixing arm is used to install the Beidou signal receiving antenna. Each measuring device is connected to the main bracket through a customized adapter plate. The customized adapter plate is designed with a fine-tuning mechanism to achieve precise positioning of the measuring equipment. For high-precision inertial navigation systems and three-dimensional laser scanners, the installation position is selected near the center of gravity of the vehicle to minimize errors caused by movement; For image data and point cloud data, deep learning models are used to extract features to obtain image features and point cloud features; For image data I , perform image feature extraction and obtain image features : ,in, is the feature extraction function based on the first deep learning model, It is the first deep learning model parameter for point cloud data P , extract point cloud features and obtain point cloud features : ,in, is the feature extraction function of the second deep learning model for point cloud, is a parameter of a second deep learning model; wherein both the first deep learning model and the second deep learning model adopt a pre-trained neural network; Based on the image features and the point cloud features, after obtaining a matching matrix by minimizing the distance between the registered image data and the point cloud data, a conversion matrix is ​​obtained by minimizing the sum of squares of the distances between the converted point cloud data and the image data based on the matching matrix; The distance between the image data and the point cloud data after minimization registration is expressed as: , where M is the matching matrix, is the element in the i-th row and j-th column of the matching matrix, is the distance metric function, is the feature of the i-th pixel in the image data, is the feature of the jth point in the point cloud data; The minimization of the sum of square distances between the transformed point cloud data and the image data is expressed by the formula: ,in, T represents the transformation matrix, is the feature of the i-th point in the point cloud data, The image data The features of pixels, represents the pixel in the image data that matches the i-th point in the point cloud data found in the matching matrix M; For each point in the point cloud data, the corresponding pixel in the image data is found according to the matching matrix, the color information of the corresponding pixel is converted using the transformation matrix, and then assigned to the point cloud data to obtain a digital twin model of the mountain scenic area; for the digital twin model, multi-level detail technology is used to simplify it, high dynamic range imaging technology is used to enhance lighting, instantiated rendering technology is used to reduce rendering batches, and the digital twin model data is compressed.

2. A digital twin model construction system, characterized in that: include: A data acquisition module is configured to: integrate a variety of high-precision measurement equipment into a mobile platform, the measurement equipment includes but is not limited to a high-precision inertial navigation system, a first three-dimensional laser scanner, a second three-dimensional laser scanner, a wheel speed meter, a high-resolution camera and a panoramic camera, obtain image data and point cloud data of a mountain scenic area, use a Beidou timestamp to align the timestamps of the image data and the point cloud data, obtain static differential data collected by a global navigation satellite system base station, perform POS combination calculation on the vehicle position and posture information, the vehicle speed information and the static differential data, obtain the vehicle travel trajectory data and its timestamp information, synchronize the point cloud data, the panoramic photo data and the monocular photo data based on the vehicle travel trajectory data and its timestamp information, the image data I adopts the Beidou coordinate system, the image data I is the panoramic photo data or the monocular photo data, and align the point cloud data with the Beidou coordinate system in the inertial navigation system; The mobile platform is a vehicle equipped with a special mounting frame to ensure that all measuring equipment is stably installed on the top of the vehicle. The mounting frame includes a main bracket and multiple auxiliary antenna fixing arms. Each auxiliary antenna fixing arm is equipped with a shockproof device. The main bracket is fixed to the top interface of the vehicle or the luggage rack through a quick lock. The auxiliary antenna fixing arm is connected to the main bracket through a rotating joint. The rotating joint allows the auxiliary antenna fixing arm to freely adjust the angle in three-dimensional space. The auxiliary antenna fixing arm is used to install the Beidou signal receiving antenna. Each measuring device is connected to the main bracket through a customized adapter plate. The customized adapter plate is designed with a fine-tuning mechanism to achieve precise positioning of the measuring equipment. For high-precision inertial navigation systems and three-dimensional laser scanners, the installation position is selected near the center of gravity of the vehicle to minimize errors caused by movement; The feature extraction module is configured to: extract features from image data and point cloud data using a deep learning model to obtain image features and point cloud features; I , perform image feature extraction and obtain image features : ,in, is the feature extraction function based on the first deep learning model, It is the first deep learning model parameter for point cloud data P , extract point cloud features and obtain point cloud features : ,in, is the feature extraction function of the second deep learning model for point cloud, is a parameter of a second deep learning model; wherein both the first deep learning model and the second deep learning model adopt a pre-trained neural network; A matching module, configured to: based on the image features and the point cloud features, obtain a matching matrix by minimizing the distance between the registered image data and the point cloud data, and then, based on the matching matrix, obtain a transformation matrix by minimizing the sum of squares of the distances between the transformed point cloud data and the image data; The distance between the image data and the point cloud data after minimization registration is expressed as: , where M is the matching matrix, is the element in the i-th row and j-th column of the matching matrix, is the distance metric function, is the feature of the i-th pixel in the image data, is the feature of the jth point in the point cloud data; The minimization of the sum of square distances between the transformed point cloud data and the image data is expressed by the formula: ,in, T represents the transformation matrix, is the feature of the i-th point in the point cloud data, The image data The features of pixels, represents the pixel in the image data that matches the i-th point in the point cloud data found in the matching matrix M; The model building module is configured as follows: for each point in the point cloud data, the corresponding pixel in the image data is found according to the matching matrix, the color information of the corresponding pixel is converted using the transformation matrix, and then assigned to the point cloud data to obtain a digital twin model of the mountain scenic area; for the digital twin model, multi-level detail technology is used to simplify it, high dynamic range imaging technology is used to enhance lighting, instantiated rendering technology is used to reduce rendering batches, and the digital twin model data is compressed.

3. A computer-readable storage medium having a computer program stored thereon, the program being executed by a processor, characterized in that: When the program is executed by a processor, the steps in the digital twin model construction method as described in claim 1 are implemented.

4. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the digital twin model construction method as described in claim 1 are implemented.

5. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in the digital twin model construction method as claimed in claim 1.

Images