Intelligent infrastructure construction digital twinning scene construction method, device, equipment, medium and product

Abstract

The invention discloses an intelligent infrastructure construction digital twinborn scene construction method, device and equipment, a medium and a product, and relates to the field of infrastructure construction, and the method comprises the steps: obtaining multi-modal data under an infrastructure construction site; the multi-modal data comprises a point cloud set, an image set, a thermal imaging data set, a global point cloud set and a global image set; on the basis of the type of the multi-modal data, the multi-modal data are preprocessed; fusing the preprocessed multi-modal data to construct a fused data set; constructing a real-time dynamic three-dimensional model containing multi-modal information according to the fused data set; carrying out lightweight processing on the real-time dynamic three-dimensional model, and constructing a digital twinborn scene; the digital twinborn scene is used for monitoring the whole process of the infrastructure construction scene in real time, the data processing efficiency can be improved, the whole process of the infrastructure construction scene can be monitored in real time, and real-time interaction of dynamic scenes is supported.

Description

Technical Field

[0001] This application relates to the field of infrastructure construction, and in particular to a method, device, equipment, medium and product for constructing a digital twin scene for intelligent infrastructure construction. Background Art

[0002] With the continuous development of infrastructure construction, the complexity of construction scenarios and technical requirements are increasing. Traditional construction management methods often struggle to achieve efficient and accurate management and decision-making in complex engineering environments and changing construction conditions. Digital twin technology, as an emerging technology, provides a new solution for infrastructure construction process management. By constructing virtual digital models, digital twins enable real-time mapping and simulation analysis of physical entities, thereby improving the transparency and controllability of the construction process.

[0003] However, the massive amount of multi-source data involved in the construction process is complex and heterogeneous, making efficient integration difficult. This results in low data processing efficiency and impacts the real-time performance of the twin system. Furthermore, the complexity and dynamic nature of construction scenarios make their 3D models complex and bulky, reducing the efficiency of model transmission and loading, and making it difficult to support real-time interaction in dynamic scenes. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, equipment, medium and product for constructing digital twin scenes of intelligent infrastructure construction to solve the problems of low data processing efficiency and difficulty in supporting real-time interaction of dynamic scenes.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In the first aspect, the present application provides a method for constructing a digital twin scenario for intelligent infrastructure construction, comprising:

[0007] Acquire multimodal data at a construction site; the multimodal data includes a point cloud set, an image set, a thermal imaging data set, a global point cloud set, and a global image set;

[0008] Preprocessing the multimodal data based on the type of the multimodal data;

[0009] Fuse the pre-processed multimodal data to construct a fused dataset;

[0010] constructing a real-time dynamic three-dimensional model containing multimodal information based on the fused data set;

[0011] The real-time dynamic three-dimensional model is lightweighted to construct a digital twin scene; the digital twin scene is used to monitor the entire process of the infrastructure construction scene in real time.

[0012] In a second aspect, the present application provides a device for constructing a digital twin scene for intelligent infrastructure construction, comprising:

[0013] A multimodal data acquisition module is used to acquire multimodal data at the infrastructure construction site; the multimodal data includes a point cloud set, an image set, a thermal imaging data set, a global point cloud set, and a global image set;

[0014] A preprocessing module, configured to preprocess the multimodal data based on the type of the multimodal data;

[0015] The fusion module is used to fuse the pre-processed multimodal data and construct a fused dataset;

[0016] A real-time dynamic three-dimensional model construction module, configured to construct a real-time dynamic three-dimensional model containing multimodal information based on the fused data set;

[0017] A lightweight processing module is used to perform lightweight processing on the real-time dynamic three-dimensional model to construct a digital twin scene; the digital twin scene is used to monitor the entire process of the infrastructure construction scene in real time.

[0018] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-mentioned methods for constructing a digital twin scene for intelligent infrastructure construction.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements any of the above-mentioned methods for constructing a digital twin scene for intelligent infrastructure construction.

[0020] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for constructing a digital twin scene for intelligent infrastructure construction.

[0021] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0022] This application combines the spatial information of point cloud sets and global point cloud sets, the lighting conditions of image sets and global image sets, and the temperature distribution of thermal imaging data sets to form multimodal data, providing comprehensive data support for construction inspections; to further improve the fusion effect, based on the type of multimodal data, targeted preprocessing is performed separately, and the preprocessed multimodal data is fused to achieve alignment and fusion of multimodal data. This fusion method not only makes up for the limitations of a single sensor, but also provides a more comprehensive and accurate description of the environment in complex scenes, providing high-quality, unified multimodal data for the construction of digital twin scenes, achieving efficient integration, and improving data processing efficiency.

[0023] In addition, this application constructs a real-time dynamic three-dimensional model containing multimodal information by fusing data sets, and performs lightweight processing on the model to construct a digital twin scene. Due to the lightweight processing, the complex real-time dynamic three-dimensional model and volume are optimized, which improves the transmission and loading efficiency of the lightweight processed model. The lightweight processed model can reduce the designed data volume and calculation amount on the basis of accurately describing the original object information in the infrastructure construction scene, thereby improving the construction efficiency of the digital twin scene, thereby realizing real-time monitoring of the entire infrastructure construction scene process and supporting real-time interaction of dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 This is an application environment diagram of the method for constructing a digital twin scenario for intelligent infrastructure construction provided in this application;

[0026] Figure 2 A flowchart of the method for constructing a digital twin scenario for intelligent infrastructure construction provided in this application;

[0027] Figure 3 This is a schematic diagram of the structure of the device for building a digital twin scene for intelligent infrastructure construction provided in this application;

[0028] Figure 4 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0031] The embodiment of the present application provides a method for constructing a digital twin scene of intelligent infrastructure construction, which is executed by a computer device, such as Figure 1 As shown, it can be specifically executed by a computer device such as a terminal or a server alone, or it can be executed by a terminal and a server together. In the embodiment of the present application, as shown Figure 2 As shown, the method includes the following steps.

[0032] S1: Acquire multimodal data at the infrastructure construction site; the multimodal data includes a point cloud set, an image set, a thermal imaging data set, a global point cloud set, and a global image set.

[0033] S2: Preprocessing the multimodal data based on the type of the multimodal data.

[0034] S3: Fuse the preprocessed multimodal data to construct a fused dataset.

[0035] S4: Constructing a real-time dynamic three-dimensional model containing multimodal information based on the fused data set.

[0036] S5: Lightweight processing is performed on the real-time dynamic three-dimensional model to construct a digital twin scene; the digital twin scene is used to monitor the entire process of the infrastructure construction scene in real time.

[0037] In an exemplary embodiment, S1 may be replaced by the following steps.

[0038] S11: Collect a global point cloud set and a global image set by using a multimodal sensor device mounted on a drone; the multimodal sensor device includes a laser scanner, an RGB camera, and a thermal imager.

[0039] S12: Using the laser scanner to collect point cloud data of the infrastructure construction site to generate a point cloud set.

[0040] S13: Using the RGB camera to capture texture images of the infrastructure construction site to generate an image set.

[0041] S14: Using the thermal imager to capture temperature distribution and generate a thermal imaging data set.

[0042] In practical applications, the point cloud set is P, the image set is I, and the thermal imaging data set is T.

[0043] Using multimodal sensor equipment (laser scanner, RGB camera, thermal imager) and using drones equipped with multimodal sensor equipment to collect the global point cloud set P global and the global image set I global , providing high-precision, multi-modal data input to ensure the accuracy of the reconstruction results.

[0044] In an exemplary embodiment, in order to ensure the fusion effect, the following data preprocessing method is first used to improve data quality and eliminate noise and inconsistency. S2 can be replaced by the following steps.

[0045] S21: performing denoising and filtering processing on the point cloud set and the global point cloud set to generate a denoised point cloud set and a denoised global point cloud set.

[0046] S22: Perform radial distortion correction and illumination compensation on the image set and the global image set to generate a corrected image set and a corrected global image set.

[0047] S23: performing temperature range normalization processing on the thermal imaging data set to generate a normalized thermal imaging data set; the preprocessed multimodal data includes a denoised point cloud set, a denoised global point cloud set, a corrected image set, a corrected global image set, and a normalized thermal imaging data set.

[0048] In practical applications, data preprocessing is as follows:

[0049] a) Point cloud data denoising.

[0050] For point cloud sets P and P global Perform denoising and filtering to generate the denoised point cloud set P' and the denoised global point cloud set P' global , this process uses formula (1).

[0051] P'={p∈P|SD(p)<τ}(1)

[0052] Where SD(p) is the local standard deviation of point p, which is used to measure the change in point cloud density; τ is a threshold determined by experience or experiments.

[0053] b) Camera imaging object coordinate transformation.

[0054] During the camera imaging process, the object coordinates (X, Y, Z) in the three-dimensional world are projected onto the two-dimensional image plane (camera sensor plane), and the two-dimensional pixel coordinates (u, v) are calculated using formula (2).

[0055]

[0056] Among them, (X, Y, Z) is the three-dimensional coordinate of the object in the real coordinate system, (u, v) is the two-dimensional pixel coordinate projected onto the image plane, and f x and f y The focal length parameters of the camera correspond to the horizontal and vertical focal lengths, c x and c y are the coordinates of the principal point of the image, usually located at the center of the image.

[0057] In the process of converting the camera coordinate system to the image coordinate system and the pixel coordinate system to the image plane coordinate system, the image distortion will inevitably occur due to the camera assembly system error and the manufacturing error of the optical lens. For the image set I and the global image set I global ,According to the actual camera principle, radial distortion formula (3) and tangential distortion formula (4) are used to perform distortion correction.

[0058]

[0059] Where (u, v) is the ideal pixel coordinate before distortion, (u', v') is the actual pixel coordinate after distortion, r = u 2 +v 2 Indicates the distance from the pixel to the center of the image, k1, k2, k3 are radial distortion coefficients, and p1 and p2 are tangential distortion coefficients.

[0060] Radial distortion is primarily caused by the shape of the lens and usually appears as distortion at the edges of the image.

[0061] Tangential distortion is usually caused by assembly errors between the lens and the image sensor, and usually manifests as horizontal or vertical shifts of certain points in the image.

[0062] In practical applications, images often have both radial distortion and tangential distortion. It is necessary to use formula (3) and formula (4) simultaneously to correct these two distortions.

[0063] In order to keep the image consistent in visual effects and feature expression under different lighting conditions, I and I global The grayscale world hypothesis method is used for illumination compensation, and the corrected image set I' and the corrected global image set I' are generated by referring to formula (5). global .

[0064]

[0065] Among them, R, G, B are the channel values of the original image, R', G', B' are the adjusted channel values, avg_gray is the average value of gray, avg_R, avg_G, avg_B are the average values of the R, G, B channels of the image.

[0066] c) Thermal imaging data normalization.

[0067] The temperature range normalization of the thermal imaging data set T is realized by using formula (6). This can scale data of different magnitudes to the same range, eliminate magnitude differences, make data easier to compare and process, and facilitate the subsequent fusion of thermal imaging data to generate the normalized thermal imaging data set T'.

[0068]

[0069] In an exemplary embodiment, S3 may be replaced by the following steps.

[0070] S31: aligning the denoised point cloud set and the denoised global point cloud set to the same coordinate system, and registering them with the point cloud set and the global point cloud set to generate a registered point cloud set and a registered global point cloud set.

[0071] S32: Stitching the corrected image set and the corrected global image set to determine a panoramic image.

[0072] S33: Perform multi-sensor fusion on the registered point cloud set, the registered global point cloud set, the panoramic image, and the normalized thermal imaging data set to construct a fused data set.

[0073] In practical applications, multimodal data fusion specifically includes the following:

[0074] a) Point cloud data alignment.

[0075] In order to ensure the global consistency of point cloud data, point cloud sets P' and P' global Aligning to the same coordinate system is the prerequisite for multimodal data fusion. The aligned point cloud data is combined with the pre-processed data mentioned above to achieve complementary fusion.

[0076] The rotation matrix R and translation vector t are obtained by formula (7), which are used to generate the registered point cloud sets P' and P g ” lobal Be prepared.

[0077]

[0078] Among them, p iis the i-th point in the source point cloud (the i-th point of the point cloud data collected by the laser scanner at the construction site), that is, the point that needs to be registered, q i is the i-th point in the target point cloud, μ P is the centroid of the source point cloud, μ Q is the centroid of the target point cloud, p' i is the point after the centroid of the i-th point in the source point cloud, q' i is the decentralized point of the i-th point in the target point cloud. H is the covariance matrix of the source and target point clouds. Singular value decomposition is performed on the covariance matrix H to obtain the orthogonal matrices U and V and the diagonal matrix S. The result of the singular value decomposition is used to calculate the rotation matrix R. If the rotation matrix R is not orthogonal, it is necessary to adjust V = V × diag(1,1,-1) and recalculate R. In each iteration, the translation vector t is calculated based on the center of mass and the rotation matrix.

[0079] After obtaining the rotation matrix R and the translation vector t, the registered point cloud sets P' and P are generated by formula (8). g ” lobal

[0080] P"=R×P+t(8)

[0081] b) Image, point cloud, and thermal imaging data fusion.

[0082] The corrected image set I' and the corrected global image set I' global Stitched into panoramic image I' panorama Finally, the point cloud sets P' and P' global , panoramic image I' panorama And the normalized thermal imaging data set T' is used for multi-sensor fusion to generate a fused data set D, achieving high-precision data fusion and providing a unified data input for dynamic reconstruction.

[0083] In an exemplary embodiment, S4 may be replaced by the following steps.

[0084] S41: Based on the fused data set, extract image feature points from the corrected image set, and extract point cloud feature points from the registered point cloud set.

[0085] S42: Perform feature matching on the image feature points and the point cloud feature points to generate matching point pairs and construct a feature point set.

[0086] S43: Aligning the registered point cloud data of adjacent frames according to the feature point set and the posture transformation matrix; the registered point cloud data is the point cloud data in the registered point cloud set.

[0087] S44: Splice the aligned point cloud data to construct a local dynamic 3D model.

[0088] S45: Based on the local dynamic three-dimensional model, combined with the registered global point cloud set and the corrected global image set, a global three-dimensional model of the infrastructure construction site is generated.

[0089] S46: Fusing the local dynamic three-dimensional model and the global three-dimensional model to generate a real-time dynamic three-dimensional model of the infrastructure construction site.

[0090] S47: Based on the panoramic image and the normalized thermal imaging data set, temperature distribution, texture information, and construction progress marks are superimposed on the real-time dynamic three-dimensional model of the infrastructure construction site to generate a real-time dynamic three-dimensional model containing multimodal information.

[0091] In practical applications, real-time dynamic 3D reconstruction specifically includes:

[0092] (1) Feature point extraction.

[0093] Feature point extraction relies on the generated high-quality fusion data set D. These data have been pre-processed by denoising, filtering, etc. to eliminate noise and inconsistency in the data, ensure the accuracy and consistency of the data, and provide a reliable data basis for feature point extraction. The ORB algorithm is used to extract feature points from the image set I', and the ISS algorithm is used to extract feature points from the point cloud set P". The feature points are divided into image feature points F image and point cloud feature points F pointcloud .

[0094] The ORB algorithm is used to perform feature detection on the corrected image set I', extract key points in the image, and record their two-dimensional coordinates (u', v') and descriptors. In order to facilitate mathematical calculations, feature points are usually represented by homogeneous coordinates: x1 = (u1', v1', 1), and the extracted image feature points are represented as a set F image .

[0095] Use the point cloud feature detection ISS algorithm to record the three-dimensional coordinates (x, y, z) and descriptors of each key point, and represent the extracted point cloud feature points as a set F pointcloud .

[0096] The image feature points F image and point cloud feature points F pointcloud Merge to form a feature point set F.

[0097] (2) Feature matching.

[0098] Image feature points F extracted from high-quality fusion dataset D image and point cloud feature points Fpointcloud It is the basis for subsequent feature matching. Feature point extraction provides the necessary feature points and descriptors for feature matching.

[0099] The FLANN algorithm is used to calculate the distance between descriptors, match the feature points of adjacent frames, and use the ratio test to filter low-quality matches to select high-quality matching point pairs to achieve feature matching.

[0100] (3) Constructing the pose transformation matrix T i,j .

[0101] For x1 = (u1', v1', 1) and x2 = (u2', v2', 1) respectively represent the homogeneous coordinates of the feature points in the first frame image I' and the homogeneous coordinates of the feature points in the second frame image, if x1 and x2 are a set of matching point pairs and satisfy Under this condition, the basic matrix M can be estimated by using matching point pairs, and the essential matrix E = K can be calculated by the camera's intrinsic parameter matrix K. T MK, perform singular value decomposition on the essential matrix E and extract the rotation matrix R i,j and the translation vector t i,j , the pose transformation matrix T is obtained by formula (9) i,j

[0102]

[0103] (4) Align point cloud data of adjacent frames.

[0104] The pose transformation between adjacent frames is calculated by visual odometry (VO), based on the feature point set F and the pose transformation matrix T i,j . Using the pose transformation matrix T i,j The point cloud data P of adjacent frames are converted into j ” and P i ”Align.

[0105] P j ”=T i,j ·P i ”(10)

[0106] (5) Dynamic three-dimensional reconstruction including multimodal information.

[0107] Splice local point cloud data into a local dynamic 3D model M local , use M local Combined with the drone global data P g ” lobal and I' global , generate a global 3D model M of the construction site global , the global model M global With the local dynamic model Mlocal Fusion is performed to generate a real-time dynamic 3D model of the construction site M final , in the three-dimensional model M final Temperature distribution, texture information and construction progress marks are superimposed on it to generate a real-time dynamic three-dimensional model M' containing multimodal information final .

[0108] In an exemplary embodiment, S5 may be replaced by the following steps.

[0109] S51: Using a quadratic error metric algorithm to perform mesh simplification on the real-time dynamic three-dimensional model to determine an initial lightweight model.

[0110] S52: performing block processing, slice storage, and data compression on the initial lightweight model to generate a final lightweight model.

[0111] S53: Construct a digital twin scene based on the final lightweight model.

[0112] In practical applications, model lightweighting processing specifically includes:

[0113] This step primarily involves lightweighting the 3D model, using the quadric error metric (QEM) algorithm for mesh simplification, and combining block processing, slice storage, and data compression techniques to achieve lightweight processing of the 3D model of the infrastructure construction scene. This not only reduces the model's storage space and computing resource usage, but also improves processing speed and system performance, making it suitable for building digital twin scenarios. The specific steps are as follows:

[0114] (1) Calculate the error metric matrix:

[0115] For the three-dimensional model M' final Every vertex v in i , an error matrix is constructed based on its adjacent faces, which is used to evaluate the error in the subsequent vertex merging process.

[0116] For each face f, its error matrix Q i The calculation formula is as follows:

[0117]

[0118] Among them: F i is the vertex v i The set of adjacent patches, n f is the normal vector of the patch f, Represents the outer product of the normal vectors, which is used to construct the error matrix.

[0119] (2) Select the optimal edge for folding:

[0120] For each edge in the model e=(v i ,v j ), calculate its two vertices v i and v j The sum of the error metric matrix Q i +Q j , then solve the position of the new vertex v k , that is, choose so that v i and v j The edge with the smallest sum of error metric matrices is folded.

[0121] In the edge collapse operation, the new vertex v k The position of can be calculated by minimizing the error matrix:

[0122] v k =arg min v (v T (Q i +Q j )v)(12)

[0123] Where V is an orthogonal matrix in the singular value decomposition, used to calculate the rotation matrix R; v represents the vertex coordinate v in the edge collapse operation i , v j .

[0124] (3) Iterative simplification:

[0125] Repeat the edge folding operation in step (2), each time selecting the edge with the smallest folding cost (i.e., error) in the current mesh for shrinking, gradually reducing the number of vertices and faces of the model while minimizing the geometric error. When the number of vertices and faces of the model reaches the preset simplification target, stop the edge folding operation, and then generate a lightweight model M. light .

[0126] After each edge folding operation, the vertex and face information of the model is updated, the folded edges and vertices are deleted, the adjacent faces and edges of the new vertices are updated, and the error metric matrix of the new vertices is recalculated to ensure that the topological structure and geometric information of the model remain consistent.

[0127] (4) Data loading in blocks:

[0128] Through the first three steps above, the model has been lightweight, but it is still difficult to directly load and transmit the entire lightweight model for large-scale infrastructure construction scenes. Therefore, the loading and transmission efficiency of the model can be further optimized. Therefore, the data block loading technology is introduced. By light Block processing, tile storage, and data compression are performed to significantly reduce loading times.

[0129] a) Block processing:

[0130] Lightweight Model M light Perform block processing and divide the model into multiple small blocks M according to the space or topological structure light1、 M light2、 M light3 ...M lightn ,Each block contains independent vertex, face and texture information.,Block loading technology is used to load data on demand.,The client only loads the model blocks within the current view range,,reducing memory usage and loading time.

[0131] b) Slice storage:

[0132] The model data is sliced and stored, and the model's geometric data and texture data are sliced and stored by layer or view range. Local loading of only the slice data within the current view range and progressive transmission starting from low-resolution slices and gradually loading high-resolution slices are adopted to optimize data access and transmission efficiency.

[0133] Local loading: The client only loads the slice data within the current view range to reduce memory usage and loading time. The loading strategy can be expressed as:

[0134] M current ={M slice |M slice ∈ViewRange}(13)

[0135] Among them: ViewRange represents the slice collection within the current view range.

[0136] Progressive transmission: Starting from low-resolution slices, gradually loading high-resolution slices to optimize data access and transmission efficiency. This process can be expressed as:

[0137] M loaded =M low-res →M mid-res →M high-res (14)

[0138] c) Data Compression:

[0139] Use efficient compression algorithms to compress texture images and video sequence data to further reduce the amount of data. H.265 compresses texture images and video sequence data, JPEG2000 compresses single frame images and texture maps, and generates the final lightweight model M compress This reduces the amount of model data, improves transmission and loading efficiency, and supports lightweight use of client devices.

[0140] Block processing divides the model into multiple small blocks. Slice storage is further subdivided into levels of different resolutions based on each small block to achieve progressive loading. Data compression compresses the data of each small block and slice to further reduce the workload and optimize storage and transmission efficiency.

[0141] Intelligent interaction: Model lightweighting provides high-precision, lightweight data support for intelligent interaction, enabling accurate monitoring and intelligent management of the entire construction process.

[0142] Through data dimensionality reduction, model lightweighting and other processing methods, three-dimensional reconstruction is achieved in the digital space, and then data mapping interaction between virtual and real spaces is realized. The newly collected data is assigned to different modules according to different attributes for targeted processing, thereby achieving accurate monitoring of the entire process of infrastructure construction scenes, ensuring the safety of the entire construction process, and providing support for scientific management and intelligent operation and maintenance, thereby improving the intelligence level of construction scenes. The digital twin system built based on multimodal data fusion and model lightweight processing can effectively integrate multi-source heterogeneous data, optimize data flows and model structures, and greatly improve the efficiency and intelligence level of construction management while maintaining high precision and high performance. In addition, combined with Internet of Things technology, real-time collection of sensor data at the construction site further enhances the real-time and comprehensiveness of the system, providing strong support for the realization of intelligent interaction and refined management.

[0143] Based on the same inventive concept, the embodiments of the present application also provide an intelligent infrastructure construction digital twin scene construction device for implementing the above-mentioned intelligent infrastructure construction digital twin scene construction method. The implementation solution provided by this device is similar to the implementation solution described in the above-mentioned method. Therefore, the specific limitations of one or more intelligent infrastructure construction digital twin scene construction device embodiments provided below can be found in the above-mentioned limitations of the intelligent infrastructure construction digital twin scene construction method, and will not be repeated here.

[0144] In an exemplary embodiment, Figure 3 As shown, the present application provides a device for constructing a digital twin scene of intelligent infrastructure construction, including:

[0145] The multimodal data acquisition module 1 is used to acquire multimodal data at the infrastructure construction site; the multimodal data includes a point cloud set, an image set, a thermal imaging data set, a global point cloud set, and a global image set.

[0146] The preprocessing module 2 is configured to preprocess the multimodal data based on the type of the multimodal data.

[0147] Fusion module 3 is used to fuse the preprocessed multimodal data and construct a fused dataset.

[0148] The real-time dynamic three-dimensional model construction module 4 is used to construct a real-time dynamic three-dimensional model containing multimodal information according to the fused data set.

[0149] The lightweight processing module 5 is used to perform lightweight processing on the real-time dynamic three-dimensional model to construct a digital twin scene; the digital twin scene is used to monitor the entire process of the infrastructure construction scene in real time.

[0150] Multimodal data fusion of this application: Multimodal data fusion provides comprehensive data support for construction monitoring by integrating the spatial information of traditional point clouds with rich details such as images, lighting conditions, and temperature distribution. To further improve the fusion effect, the point cloud is denoised and registered, the image is distortion-corrected, and the thermal imaging data is normalized during the processing process, ultimately achieving the alignment and fusion of multimodal data. This fusion method not only makes up for the limitations of a single sensor, but also provides a more comprehensive and accurate description of the environment in complex scenes, significantly enhancing the adaptability and reliability of the system, providing high-quality, unified input data for the construction of digital twin scenes, and laying a solid foundation for construction monitoring and intelligent management.

[0151] The lightweight processing of this application: During the three-dimensional reconstruction process, by combining the QEM algorithm, block processing technology, data compression technology, etc., the size of the three-dimensional model is reduced, the geometric details of the model are retained to the greatest extent, and the three-dimensional model is optimized and improved in terms of memory usage and computational complexity. The resulting lightweight model can reduce the amount of designed data and calculation on the basis of accurately describing the original object information, thereby improving the efficiency of digital twin scene construction.

[0152] In an exemplary embodiment, a computer device is provided, such as Figure 4 As shown, the computer device can be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data for constructing digital twin scenarios of intelligent infrastructure construction. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for constructing digital twin scenarios of intelligent infrastructure construction is implemented.

[0153] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above method when executing the computer program.

[0154] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above method when executed by a processor.

[0155] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the above method when executed by a processor.

[0156] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0157] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0158] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0159] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for constructing a digital twin scene for intelligent infrastructure construction, characterized in that: include: Acquire multimodal data at a construction site; the multimodal data includes a point cloud set, an image set, a thermal imaging data set, a global point cloud set, and a global image set; Preprocessing the multimodal data based on the type of the multimodal data; Fuse the pre-processed multimodal data to construct a fused dataset; constructing a real-time dynamic three-dimensional model containing multimodal information based on the fused data set; The real-time dynamic three-dimensional model is lightweighted to construct a digital twin scene; the digital twin scene is used to monitor the entire process of the infrastructure construction scene in real time.

2. The method for constructing a digital twin scene for intelligent infrastructure construction according to claim 1, characterized in that: Acquire multimodal data at infrastructure construction sites, including: Combined with the multimodal sensor equipment carried by the UAV, a global point cloud set and a global image set are collected; the multimodal sensor equipment includes a laser scanner, an RGB camera and a thermal imager; The laser scanner is used to collect point cloud data of the infrastructure construction site to generate a point cloud set; Using the RGB camera to capture texture images of the infrastructure construction site to generate an image set; The thermal imager is used to capture temperature distribution and generate a thermal imaging data set.

3. The method for constructing a digital twin scene for intelligent infrastructure construction according to claim 1, characterized in that: Preprocessing the multimodal data based on the type of the multimodal data includes: Performing denoising and filtering processing on the point cloud set and the global point cloud set to generate a denoised point cloud set and a denoised global point cloud set; Performing radial distortion correction and illumination compensation on the image set and the global image set to generate a corrected image set and a corrected global image set; The thermal imaging data set is subjected to temperature range normalization processing to generate a normalized thermal imaging data set; the preprocessed multimodal data includes a denoised point cloud set, a denoised global point cloud set, a corrected image set, a corrected global image set, and a normalized thermal imaging data set.

4. The method for constructing a digital twin scene for intelligent infrastructure construction according to claim 3 is characterized in that: Fuse the preprocessed multimodal data to construct a fused dataset, including: Aligning the denoised point cloud set and the denoised global point cloud set to the same coordinate system, and registering the point cloud set and the global point cloud set to generate a registered point cloud set and a registered global point cloud set; Stitching the corrected image set and the corrected global image set to determine a panoramic image; The registered point cloud set, the registered global point cloud set, the panoramic image and the normalized thermal imaging data set are subjected to multi-sensor fusion to construct a fused data set.

5. The method for constructing a digital twin scene for intelligent infrastructure construction according to claim 4, characterized in that: Constructing a real-time dynamic three-dimensional model containing multimodal information based on the fused data set, specifically comprising: Extracting image feature points from the corrected image set and extracting point cloud feature points from the registered point cloud set based on the fused data set; Performing feature matching on the image feature points and the point cloud feature points to generate matching point pairs and construct a feature point set; Aligning the registered point cloud data of adjacent frames according to the feature point set and the posture transformation matrix; the registered point cloud data is the point cloud data in the registered point cloud set; Splice the aligned point cloud data to build a local dynamic 3D model; Generate a global three-dimensional model of the infrastructure construction site based on the local dynamic three-dimensional model and in combination with the registered global point cloud set and the corrected global image set; Fusion of the local dynamic three-dimensional model and the global three-dimensional model to generate a real-time dynamic three-dimensional model of the infrastructure construction site; Based on the panoramic image and the normalized thermal imaging data set, temperature distribution, texture information and construction progress marks are superimposed on the real-time dynamic three-dimensional model of the infrastructure construction site to generate a real-time dynamic three-dimensional model containing multimodal information.

6. The method for constructing a digital twin scene for intelligent infrastructure construction according to claim 1, characterized in that: The real-time dynamic three-dimensional model is lightweighted to construct a digital twin scene, specifically including: Using a quadratic error metric algorithm to perform mesh simplification on the real-time dynamic three-dimensional model to determine an initial lightweight model; The initial lightweight model is divided into blocks, sliced and stored, and data is compressed to generate a final lightweight model; A digital twin scene is constructed based on the final lightweight model.

7. A device for constructing a digital twin scene for intelligent infrastructure construction, characterized in that: include: A multimodal data acquisition module is used to acquire multimodal data at the infrastructure construction site; the multimodal data includes a point cloud set, an image set, a thermal imaging data set, a global point cloud set, and a global image set; A preprocessing module, configured to preprocess the multimodal data based on the type of the multimodal data; The fusion module is used to fuse the pre-processed multimodal data and construct a fused dataset; A real-time dynamic three-dimensional model construction module, configured to construct a real-time dynamic three-dimensional model containing multimodal information based on the fused data set; A lightweight processing module is used to perform lightweight processing on the real-time dynamic three-dimensional model to construct a digital twin scene; the digital twin scene is used to monitor the entire process of the infrastructure construction scene in real time.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and runnable on the processor, characterized in that the processor executes the computer program to implement the method for constructing a digital twin scene for intelligent infrastructure construction according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method for constructing a digital twin scene for intelligent infrastructure construction according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the method for constructing a digital twin scene for intelligent infrastructure construction according to any one of claims 1 to 6.

Images