Building digital twin three-dimensional reconstruction method and system based on large model

Abstract

The invention discloses a building digital twinning three-dimensional reconstruction method and system based on a large model, and relates to the technical field of building digital twinning and three-dimensional modeling fusion, and the method comprises the steps: laying multi-source equipment in a building, collecting building multi-angle data, and carrying out type distinguishing and structured preprocessing. An adaptive deep network model is constructed and trained to identify building features, and the robustness of the model is improved by adopting a multi-type enhancement strategy. Point cloud splicing and coordinate unification are realized by using a fusion algorithm, and twin model updating and synchronization are realized based on a multi-dimensional threshold. According to the method, high-quality input is established for a depth model; building structure feature recognition has the advantages of robustness and precision; and the twinborn model has dynamic updatable capability. The three parts form a complete closed loop, the technical bottlenecks of a traditional reconstruction method in precision, stability and renewability are finally broken through, and a new building digital twinning three-dimensional modeling path facing a complex scene is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of the integration of building digital twins and 3D modeling, and specifically to a method and system for 3D reconstruction of building digital twins based on large models. Background Art

[0002] With the integrated development of technologies such as Building Information Modeling (BIM), 3D Laser Scanning (LiDAR), and artificial intelligence, building digital twins have become an important supporting means for realizing the perception and intelligent decision-making of the entire life cycle of buildings. Digital twin models require highly accurate and real-time synchronized 3D modeling capabilities to accurately map the physical state and structural form of buildings. Traditional 3D reconstruction methods mostly rely on static modeling processes, fixed topological structures, or regular point cloud matching algorithms, and it is difficult to meet the dynamic update requirements such as building deformation monitoring and operation and maintenance early warning. In recent years, with the development of deep learning, especially large-scale multimodal neural networks, the fusion recognition ability of building images and point cloud data has been significantly improved, providing the possibility of higher accuracy and adaptive modeling for digital twin 3D reconstruction methods.

[0003] There are still many technical bottlenecks in existing building 3D reconstruction methods in terms of multi-source data stitching, structural continuity expression, and dynamic update mechanisms. On the one hand, traditional point cloud stitching techniques rely on the extraction of feature points with fixed rules and rigid registration algorithms. When facing non-uniform sampling, occluded areas, or irregular building structures, it is difficult to ensure the spatial alignment accuracy, and problems such as misregistration, stitching gaps, or local distortions often occur. On the other hand, many methods lack a twin model update mechanism based on real-time sensor feedback and cannot effectively map the small changes in the building state to the 3D model, resulting in the digital twin system "only building without updating" and being unable to support key scenarios such as structural deformation early warning and disaster damage response. In addition, current models mostly adopt static network structures, with limited ability to co-model images and point clouds, and it is difficult to fully extract spatial semantic features and structural correlation information in different modalities. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the technical problems solved by the present invention are: the existing building 3D reconstruction and digital twin synchronous modeling methods have problems such as low point cloud stitching accuracy, discontinuous structural topology, and inability to respond to physical state changes in real time, and the problems of how to achieve the unified alignment of multi-source heterogeneous data, the deep fusion expression of structural information, and the dynamic update mechanism of digital twin 3D models.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: A method for three-dimensional reconstruction of building digital twins based on large models, including deploying multi-source devices in a building to collect multi-angle data of the building, and performing type classification and structured preprocessing. An adaptive depth network model is constructed and trained to identify building features, and a multi-type enhancement strategy is adopted to improve the robustness of the model. A fusion algorithm is used to achieve point cloud stitching and coordinate unification, and the twin model is updated and synchronized based on multi-dimensional thresholds. The use of the fusion algorithm to achieve point cloud stitching and coordinate unification includes: taking the point cloud data after the enhancement strategy as the benchmark with the building center point, and uniformly transforming it to the global world coordinate system, and using an initial registration strategy based on feature point matching to roughly align different source point clouds. The iterative closest point algorithm is used for fine registration. After all the point cloud data registration is completed, weighted merging processing is performed according to the density and quality thresholds to generate a single high-density point cloud model. The Poisson surface reconstruction algorithm is used to re-estimate the global normal vector of the fused point cloud, and a triangular mesh model with a continuous and topological structure is constructed. The triangular mesh and the original point cloud establish a one-to-one mapping relationship through a mapping table.

[0007] As a preferred solution of the method for three-dimensional reconstruction of building digital twins based on large models according to the present invention, wherein: the collection of multi-angle data of the building includes deploying a laser scanner and a ground image acquisition device around the target building, and arranging scanners in different elevation and height areas. Each device records the timestamp and sampling coordinates during the collection process.

[0008] As a preferred solution of the method for three-dimensional reconstruction of building digital twins based on large models according to the present invention, wherein: the type classification and structured preprocessing includes using a statistical outlier removal algorithm to process the original data stream of the point cloud data, removing outliers whose distance from the average value exceeds 3 times the standard deviation, and compressing the point cloud data through a voxel grid filter at a resolution of 0.1 meters. The image data is organized in the order of shooting time, and gray normalization and histogram equalization processing are performed, and the texture of the key area is extracted. Different types of data are organized according to a unified spatial coordinate system, and a type marking field is established.

[0009] As a preferred solution of the building digital twin three-dimensional reconstruction method based on the large model of the present invention, wherein: the construction and training of the adaptive depth network model to identify building features includes constructing an image data set and a point cloud data set with labels based on the structured preprocessed data. The image data labels include edges, corners, and structural connection parts, and the point cloud data annotates the building floors, window areas, and edge points. An adaptive depth network model is constructed, adopting a convolutional neural network architecture with an attention mechanism module and a multi-scale extraction structure embedded, and introducing skip connections for feature compensation. The point cloud data uses a graph neural network based on a dynamic graph structure to construct an edge weight tensor to enhance the local structure preservation ability. During training, a cross-entropy loss function is used in combination with a mean square error combined loss, and the optimization algorithm adopts an Adam variable step size strategy.

[0010] As a preferred solution of the building digital twin three-dimensional reconstruction method based on the large model of the present invention, wherein: the adoption of a multi-type enhancement strategy to improve the robustness of the model includes, during the training stage, introducing affine transformation and brightness perturbation for image data to simulate different shooting conditions, and constructing a random occlusion area to simulate on-site occlusion to generate an additional sample set. Gaussian noise is applied to the point cloud data to simulate sensor errors, and at the same time, a random local point loss strategy is introduced to simulate measurement loss under occlusion. The original labels are retained for the enhanced image data samples and point cloud data samples and are processed synchronously, and then added to the training set to participate in training.

[0011] As a preferred solution of the 3D reconstruction method of building digital twin based on large model of the present invention, wherein: the implementation of point cloud stitching and coordinate unification using the fusion algorithm includes setting the center point of the building structure as the global reference origin for the point cloud data after the enhancement strategy, constructing a unified world coordinate system based on the origin, and mapping the point cloud data from different devices and perspectives to the world coordinate system through coordinate transformation. The initial registration adopts a matching strategy based on 3D feature points, extracts geometrically significant feature points for each group of point clouds, including edge points, corner points and curvature mutation points, uses a matching algorithm based on distance and normal consistency to match the corresponding relationships of feature points between adjacent point clouds, and completes the preliminary spatial alignment by calculating the optimal rigid transformation matrix between the corresponding points. After the initial registration is completed, the iterative closest point algorithm is used to perform fine registration on each group of point clouds. In each iteration, the closest point pair relationship is recalculated and the transformation matrix is continuously optimized according to the principle of error minimization until the preset error threshold is satisfied. After the fine registration is completed, all the point cloud data is spatially divided into fixed-size voxel grids, and the number of points and the mean square distance between points in each voxel are counted as density and quality evaluation indicators. If the data density in the voxel meets the threshold and the distance variance is lower than the set standard, the regional point cloud is retained, otherwise it is eliminated, and finally a dense and uniformly distributed single high-density point cloud model is generated. The Poisson surface reconstruction algorithm is used to estimate the global normal vector of the point cloud model, estimate the normal vector direction of the points in its neighborhood for each point and unify the orientation, then construct an implicit function field, construct a global integral field function through octree spatial partitioning, reconstruct a closed and continuous triangular mesh model, and record the spatial mapping relationship between each mesh surface and the original points at the same time, forming a structurally stable one-to-one mapping table.

[0012] As a preferred solution of the 3D reconstruction method of building digital twin based on large model of the present invention, wherein: the implementation of twin model update and synchronization based on multi-dimensional thresholds includes importing the constructed building 3D mesh model into the digital twin platform, performing the sensor data access step, collecting building physical state data through sensors, analyzing the building physical state data, and when the detection results of any type of sensor exceed the set threshold range in three consecutive sampling periods, performing the local reconstruction step, including re-collecting the image data and point cloud data corresponding to the area, and calling the trained deep learning model to identify the structural features of the abnormal area and then complete the local mesh update.

[0013] Another object of the present invention is to provide a building digital twin three-dimensional reconstruction system based on a large model, which can realize point cloud stitching and coordinate unification by using a fusion algorithm, and realize the update and synchronization of the twin model based on multi-dimensional thresholds, solving the problems of low point cloud stitching accuracy, discontinuous structural topology, and inability to respond to physical state changes in the existing building three-dimensional reconstruction and digital twin synchronous modeling methods, as well as the problems of how to achieve the unified alignment of multi-source heterogeneous data, the deep fusion expression of structural information, and the dynamic update mechanism of the digital twin three-dimensional model.

[0014] As a preferred embodiment of the building digital twin three-dimensional reconstruction system based on a large model according to the present invention, it includes an acquisition and preprocessing module, a neural network model establishment module, and a point cloud stitching and unification module.

[0015] The acquisition and preprocessing module is used to deploy multi-source devices in the building, collect multi-angle data of the building, and perform type differentiation and structured preprocessing.

[0016] The neural network model establishment module is used to construct and train an adaptive deep network model to identify building features, and adopt a multi-type enhancement strategy to improve the robustness of the model.

[0017] The point cloud stitching and unification module is used to realize point cloud stitching and coordinate unification by using a fusion algorithm, and realize the update and synchronization of the twin model based on multi-dimensional thresholds.

[0018] A computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the building digital twin three-dimensional reconstruction method based on a large model.

[0019] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, it implements the steps of the building digital twin three-dimensional reconstruction method based on a large model.

[0020] The beneficial effects of the present invention: The building digital twin three-dimensional reconstruction method provided by the present invention realizes the comprehensive acquisition of multi-angle and multi-modal original data of the building by deploying laser scanners and image acquisition devices around the building, and combining the space control strategy and the timestamp synchronization mechanism. Subsequently, abnormal data is removed and voxel compression is performed on the point cloud data, and gray normalization and histogram enhancement are performed on the image data, realizing noise suppression and structure strengthening of multi-source data. Finally, all data is uniformly mapped to the world coordinate system and type tags are added, realizing the standardized fusion of heterogeneous data. The purpose of this step is to provide a high-quality, structurally consistent, and time-sequence controllable data basis for the subsequent input of the deep model, avoiding problems such as structural deviation and feature loss caused by messy data in traditional three-dimensional modeling, and finally achieving the effect of improving the reconstruction accuracy and modeling stability. Brief Description of the Drawings

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0022] Figure 1 It is the overall flowchart of a method for three-dimensional reconstruction of building digital twins based on large models provided by the first embodiment of the present invention. Detailed Embodiments

[0023] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the detailed embodiments of the present invention with reference to the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] Embodiment 1, referring to Figure 1 , which is an embodiment of the present invention, provides a method for three-dimensional reconstruction of building digital twins based on large models, including: S1: Deploy multi-source devices in the building, collect multi-angle data of the building, and perform type classification and structured preprocessing.

[0025] Deploy a laser scanner and ground image acquisition devices around the target building, and deploy scanners in different elevation and height areas. Each device records the timestamp and sampling coordinates during the acquisition process. The statistical outlier removal algorithm is used to process the original data stream of the point cloud data, and the outliers with a distance exceeding 3 times the standard deviation from the average value are removed. The point cloud data is compressed at a resolution of 0.1 meters through a voxel grid filter. The image data is organized in the order of shooting time, and gray normalization and histogram equalization processing are performed to extract the texture of the key area. Different types of data are organized according to a unified spatial coordinate system, and a type marking field is established.

[0026] Deploy laser scanners and ground image acquisition devices around the target building. A preferred solution for arranging scanners in different elevation and height areas specifically includes, before arranging the devices in the surrounding area of the target building, first conducting an environmental assessment based on the elevation distribution, number of floors, and unobstructed conditions of the building, and selecting appropriate arrangement points. Place the 3D laser scanners in the open areas on the front facade, rear facade, left and right sides of the building respectively, and set up different height brackets or lifting platforms according to the building storey height to cover the complete scanning area from the ground to the roof. The ground image acquisition device includes a high-resolution industrial camera and a mobile track device with a fixed shooting trajectory. It is arranged on the building perimeter, maintaining an equal spacing arrangement to ensure that the shooting perspectives are staggered and cover all visible surfaces of the building. Before starting the acquisition, initialize the synchronization module for each device to ensure that the data timestamp and device spatial position information can be recorded in real time during the acquisition process for subsequent multi-source data alignment.

[0027] After the data acquisition is completed, perform preprocessing operations on different types of data. As the main source of spatial geometric data, the point cloud data first removes noise points through a statistical outlier removal algorithm. Based on local neighborhood analysis, calculate the average distance between each point and its K nearest neighbors, and determine the points with a distance exceeding 3 times the standard deviation of the overall average as outliers and remove them. After the removal, use a voxel grid filter to sparsify the remaining point cloud. Divide the point cloud into voxel units with a side length of 0.1 meters, and only retain one representative point in each voxel unit to compress the data volume and maintain the integrity of the geometric contour. For the image data, sort it according to the acquisition time sequence, and perform preprocessing operations on each image, including image gray normalization to eliminate exposure differences, and using histogram equalization technology to enhance the image contrast and improve the recognizability of building edges and texture features. Subsequently, extract the key area texture information in the image and mark its spatial acquisition position. After the preprocessing, both the point cloud data and the image data are transformed to the building reference coordinate system through a unified spatial coordinate, and set a type marking field for each type of data to facilitate data shunting and structural docking during the input of the subsequent deep model. After completing the above operations, enter the next stage of model training and feature recognition process.

[0028] It should be noted that the design idea of S1 is to achieve all-round and multi-angle data acquisition of the building through the arrangement of multi-source heterogeneous devices, and combine type differentiation and structured preprocessing to improve the data quality and organization efficiency. This step can ensure the standardization and consistency of the model input, and provide a high-precision data basis for subsequent in-depth recognition and 3D modeling.

[0029] S2: Construct and train an adaptive deep network model to identify building features, and adopt a multi-type enhancement strategy to improve the robustness of the model.

[0030] Construct image and point cloud datasets with labels based on the pre - processed structured data. The image data labels include edges, corners, and structural connection parts, while the point cloud data is labeled with building floors, window areas, and edge points. Construct an adaptive deep network model, which embeds an attention mechanism module and a multi - scale extraction structure on the basis of a convolutional neural network architecture, and introduces skip connections for feature compensation. For point cloud data, a graph neural network based on a dynamic graph structure is used to construct an edge - weight tensor to enhance the local structure preservation ability. During training, a cross - entropy loss function is combined with a mean - square error loss, and the optimization algorithm adopts an Adam variable - step - size strategy.

[0031] A preferred solution for constructing an adaptive deep network model, which embeds an attention mechanism module and a multi - scale extraction structure on the basis of a convolutional neural network architecture, and introduces skip connections for feature compensation, specifically includes, after completing the structured pre - processing of building images and point clouds, first constructing a training dataset containing spatial semantic labels. Among them, the image data is labeled according to the edge lines, structural corners, and connector positions, while the point cloud data generates three - dimensional space labels according to the building floors, window areas, and boundary points. Each data sample is bound with timestamp and spatial positioning information to ensure that multi - modal data can be uniformly loaded and synchronously processed during the training phase. In terms of image feature modeling, an adaptive deep convolutional neural network structure is constructed, an attention mechanism is embedded to highlight key structure responses, and a multi - scale convolutional module is used to extract image texture features under different receptive fields. At each position , calculate its weighted response tensor as follows: , where, represents the attention response value at position in the feature map. represents the main - branch convolution activation output, represents the auxiliary - branch attention response. represents the inverse Euclidean kernel. represents the fused texture response function. represents the number of attention dimensions. This expression is used to dynamically adjust the channel output intensity and highlight the key structure regions. The value range is 0 - 1, and the larger the value, the more significant the structure. is expressed as the th channel's self - response intensity in the self - attention module. and respectively represent the sampling coordinates in the horizontal (x - axis) and vertical (y - axis) directions of the image feature map.

[0032] is expressed as: , where, is the fused texture response function value, which is used to weight and extract the features of each channel at the image coordinate position (u, v) in the attention mechanism. is the number of channels of the input image. is the weight coefficient of the channel in channel fusion. is the pixel value of the channel of the image at the position. axis and axis of the second-order Laplacian directional derivative of the image at this position, which is used to reflect the local texture change rate. is the channel at the sum of the squares of the gradients, which is used to evaluate the edge strength. is the normalization parameter of this channel to control the edge response suppression range. is the total number of texture kernel sets, is the fusion weight of the th high-order texture filter. is the absolute value of the response of the th high-order texture response kernel at (u, v), which represents the mid-high frequency structure.

[0033] The value range is [0, +∞], and the larger the value, the higher the fused texture intensity at the

[0034] In the point cloud data, a preferred solution for constructing a graph neural network based on a dynamic graph structure to enhance the local structure preservation ability of the edge weight tensor specifically includes, in terms of point cloud data modeling, constructing a graph neural network based on a dynamic graph structure. First, an edge weight tensor is constructed based on the local relative vectors of the point cloud, and the edge weight of any point pair in the r-th iteration is defined as: Among them, represents the edge weight of edge in the r-th round, and represent the vectors constructed by the three-dimensional normal vectors of points and point. represents the spatial perturbation amount of point from its center point, Indicates the number of neighborhood points. This edge weight tensor is used to guide the neural network to adjust the adjacency matrix according to local structural changes, realizing topological adaptive modeling of point clouds. The value range is 0 to , the larger the value, the greater the difference in directions. Indicates the integral variable used to construct the spatial structure consistency kernel in the point cloud feature. The domain of definition is the angular interval [0, π], which is used for Gaussian kernel approximation of direction consistency. and Indicate the indices of two spatial sampling points in the point cloud, specifying the starting and ending points of the edge.

[0035] During training, the cross-entropy loss function is used in combination with the mean squared error combined loss. A preferred scheme of the Adam variable step size strategy for the optimization algorithm specifically includes, in the network training stage, designing a combined loss function to simultaneously evaluate the prediction accuracy of images and point clouds, defined as follows: , Among them, is the total training loss, and are the numbers of image and point cloud samples respectively, is the sample confidence factor, is the image label value, is the fusion of the model prediction values of the image and the point cloud, is the quantization value of image enhancement complexity, is the occlusion intensity, is the point cloud prediction error function, is the structure tensor of point j in the z dimension. The model is optimized by backpropagation through this loss function.

[0036] In the training stage, for image data, affine transformation and brightness perturbation are introduced to simulate different shooting conditions, and a random occlusion area is constructed to simulate on-site occlusion, generating an additional sample set. Gaussian noise is applied to the point cloud data to simulate sensor errors, and at the same time, a random local point loss strategy is introduced to simulate measurement loss under occlusion. The original labels are retained for the enhanced image data samples and point cloud data samples and processed synchronously, and then added to the training set to participate in training.

[0037] Furthermore, to improve the robustness of the model, multiple types of data augmentation need to be performed before training. Image data augmentation includes three types of operations: resampling the original image after affine transformation (rotation + scaling + translation) to generate new samples. Simulating different lighting conditions through random brightness perturbation. Overlaying an area occlusion map to simulate on-site occlusion and retaining the original labels at a certain ratio. The enhanced image and its original label are synchronously input into the model for training.

[0038] For point cloud data, a dual strategy enhancement of noise perturbation and sampling loss is designed. First, Gaussian random perturbations with zero mean and variance of 0.01 are added to some points to simulate laser scanning errors. Second, randomly selected block regions are executed for regional point loss in space, that is, the points within the region are directly removed to simulate the structural loss after acquisition shadows or occlusions.

[0039] During the training process, all enhanced samples and original samples are uniformly fed into the main network. The sample ratio is controlled through a dynamic batch update mechanism to ensure that the model gradually learns the stable discrimination ability for perturbations during iteration. The parameter update adopts an optimizer with momentum adaptive learning rate to dynamically adjust the step size and enhance the training stability, and finally outputs the trained model.

[0040] It should be noted that the design idea of S2 is to jointly model through an adaptive deep network model and a dynamic graph neural structure to achieve accurate extraction of multi-modal building features of images and point clouds. The attention mechanism and edge weight tensor are introduced to achieve structure perception. Multi-scale extraction and skip connections enhance the spatial hierarchical expression. Combined with multi-type perturbation enhancement strategies, the robustness of the model is improved. Compared with the existing technologies that only process single-modal or static topologies, this step still has high recognition accuracy and adaptive update ability in scenarios such as complex occlusions, structural losses, and uneven illuminations, significantly enhancing the generalization ability and engineering applicability of building information modeling.

[0041] S3: Use a fusion algorithm to achieve point cloud stitching and coordinate unification, and update and synchronize the twin models based on multi-dimensional thresholds.

[0042] The point cloud data after the enhanced strategy is set with the center point of the building structure as the global reference origin, and a unified world coordinate system is constructed based on the origin. The point cloud data from different devices and perspectives is mapped to the world coordinate system through coordinate transformation. The initial registration adopts a matching strategy based on three-dimensional feature points. Geometrically significant feature points, including edge points, corner points, and curvature mutation points, are extracted from each group of point clouds. A matching algorithm based on distance and normal consistency is used to match the corresponding relationships of feature points between adjacent point clouds. By calculating the optimal rigid transformation matrix between corresponding points, the initial spatial alignment is completed. After the initial registration is completed, the iterative closest point algorithm is used to perform fine registration on each group of point clouds. In each iteration, the relationship of the closest point pairs is recalculated, and the transformation matrix is continuously optimized according to the principle of minimizing the error until the preset error threshold is met. After the fine registration is completed, all the point cloud data is spatially divided into fixed-size voxel grids, and the number of points and the mean square distance between points in each voxel are statistically calculated as density and quality evaluation indicators. If the data density in the voxel meets the threshold and the distance variance is lower than the set standard, the regional point cloud is retained; otherwise, it is removed. Finally, a dense and uniformly distributed single high-density point cloud model is generated. The Poisson surface reconstruction algorithm is used to estimate the global normal vector of the point cloud model, estimate the normal vector direction of the points in the neighborhood of each point and unify the orientation, and then construct an implicit function field. A global integral field function is constructed through octree spatial partitioning to reconstruct a closed and continuous triangular mesh model, and at the same time, the spatial mapping relationship between each mesh surface and the original points is recorded to form a structurally stable one-to-one mapping table.

[0043] The constructed three-dimensional building mesh model is imported into the digital twin platform, and the sensor data access step is executed. The building physical state data is collected through sensors, and the building physical state data is analyzed. When the detection results of any type of sensor exceed the set threshold range in three consecutive sampling periods, the local reconstruction step is executed, including re-collecting the image data and point cloud data corresponding to the area, and calling the trained deep learning model to identify the structural features of the abnormal area and then complete the local mesh update.

[0044] A preferred solution for constructing a unified world coordinate system based on the origin and mapping the point cloud data from different devices and perspectives to the world coordinate system through coordinate transformation specifically includes taking the geometric center point of the building structure of the enhanced multi-source point cloud data as the global origin, setting a three-dimensional unified world coordinate system, and performing coordinate transformation operations on each group of point cloud data of the acquisition devices : , where: represents the original th point. represents the mapped point coordinates. is the rotation matrix. is the translation vector. represents the total number of points contained in the group of point cloud data. All point clouds enter the unified coordinate system through this transformation.

[0045] A preferred solution for initial registration using a matching strategy based on three-dimensional feature points specifically includes, for each pair of point clouds, extracting edge points and curvature mutation points as the set of significant feature points, and constructing the corresponding point pair set: , where: and are the th, nd feature points in the two groups of point clouds respectively. and are the normal vectors of the corresponding points.

[0046] After completing the initial registration, a preferred solution for fine registration of each group of point clouds using the Iterative Closest Point (ICP) algorithm specifically includes, after initial registration, using the Iterative Closest Point (ICP) algorithm for fine registration. The iterative process is as follows: , where, and are the source point and the target point in the matching point pair respectively. and are the rotation and translation to be solved. is the number of matching point pairs. represents the optimal rotation matrix obtained after the Iterative Closest Point (ICP) algorithm is executed. represents the optimal translation vector obtained after the Iterative Closest Point (ICP) algorithm is executed. Iterative update until the error converges to a set threshold less than 0.001.

[0047] A preferred solution for dividing all point cloud data into voxel grids of a fixed size according to space and statistically analyzing the number of points and the mean square distance between points in each voxel as density and quality evaluation indicators specifically includes dividing the finely registered point clouds into voxel grids, setting the voxel size to 0.1 meters, and statistically analyzing the point density and the mean square distance quality indicator between points, which are defined as follows: , where: is the number of points in the th voxel. is the voxel volume. is the th point among them. is the center of the voxel points.

[0048] Only the voxel regions that meet the conditions that the point density within the voxel is greater than the sum of the lowest density thresholds and the mean square distance quality index between points is less than the maximum quality index threshold are used to generate a dense point cloud set. In the present invention, the lowest density threshold is 400 points per cubic meter, and the maximum quality index threshold is 0.005 square meters.

[0049] An optimal solution for global normal vector estimation of a point cloud model using the Poisson surface reconstruction algorithm specifically includes performing Poisson surface reconstruction on the generated dense point cloud set, including normal estimation, implicit function construction, and mesh extraction, calculating the normal direction for each point and unifying the orientation. Construct an integral field function , expressed as: , where, is an arbitrary position in space. is the normal vector of point . is the Dirac delta function. is the point cloud distribution domain. Through an octree, is hierarchically discretized, the isosurface of is extracted, a closed triangular mesh model is generated, and the spatial correspondence between each mesh surface and the original points is recorded to construct a mapping table.

[0050] An optimal solution for importing the constructed 3D building mesh model into the digital twin platform specifically includes importing the triangular mesh model into the twin platform and docking it with the building sensor system. Let the collected physical state data be a sequence: , where: represents the value of the th sensor at time , and the types include: temperature, humidity, displacement, stress, etc. represents the number of sensors. If there exists a certain that satisfies the following conditions: , where, represents the data value of the th sensor in the th time sampling period. represents the historical mean value of this sensor under normal operation. represents the abnormal determination threshold of sensor , which is the tolerance floating range configured by the system. It is an indicator function that returns 1 if the expression in the parentheses holds, and 0 otherwise. The sum of accumulations divided by 3 represents whether the threshold has been exceeded at least once in the last three sampling periods. If the result is equal to 1, local reconstruction is triggered.

[0051] The sampling values of this sensor exceed its preset threshold for three consecutive periods , then the local update mechanism of the model is triggered: rescheduling the mobile device to collect images and point clouds of this area. Using the trained model for local structure recognition. Updating the corresponding grid area through local geometric stitching and mesh repair techniques.

[0052] Furthermore, the preset threshold of the present invention As shown in Table 1, it can be adjusted according to the actual situation.

[0053] Table 1 Preset Threshold Table

[0054] It should be noted that S3 constructs a registration and reconstruction process from multi-source point clouds to a unified world coordinate system, and through the fusion of Poisson surface reconstruction and multi-dimensional sensor threshold judgment mechanisms, realizes the high-precision synchronous update of the dynamic twin model. Compared with the problems of large data stitching errors, local distortion of model reconstruction, and inability to respond to structural changes in the prior art, this step realizes a dense, continuous, and topologically consistent three-dimensional reconstruction result through feature point-guided rigid registration, density and quality double-threshold filtering, and Gaussian field integral surface generation, and can trigger local model updates in real-time by linking building physical state monitoring data, effectively improving the integrity and timeliness of the model structure.

[0055] Embodiment 2 is an embodiment of the present invention, which provides a building digital twin three-dimensional reconstruction system based on a large model, including an acquisition and preprocessing module, a neural network model establishment module, and a point cloud stitching and unification module.

[0056] The acquisition and preprocessing module is used to deploy multi-source devices in the building, collect multi-angle data of the building, and perform type classification and structured preprocessing.

[0057] The acquisition and preprocessing module includes a device deployment and acquisition sub-module and a data preprocessing and standardization sub-module.

[0058] Furthermore, the device deployment and acquisition sub-module is used to deploy laser scanning devices and image acquisition devices at multiple perspectives in the building target area, collect point cloud data and image information at different angles, and record the timestamp, pose parameters, and sampling coordinates of each device during the acquisition process for constructing a spatio-temporal calibration reference frame for the data. The data preprocessing and standardization sub-module is used to perform outlier removal and voxelization filtering on the collected point cloud, and perform gray normalization and edge enhancement operations on the image data, and then uniformly convert them to the building world coordinate system, and mark the data type field for each type of data to ensure that the subsequent network model can process different modality data separately.

[0059] It should be noted that the device deployment and acquisition sub-module is the starting point of the acquisition and preprocessing module, ensuring the coverage integrity of the data in terms of time sequence and perspective. The data preprocessing and standardization sub-module is an important processing link for realizing data fusion. The output data format and quality directly affect the subsequent neural network training and feature extraction results. As the basic module of the system, the acquisition and preprocessing module's output results are the pre-input for the construction of the neural network model and the 3D stitching and fusion processing.

[0060] The neural network model building module is used to construct and train an adaptive deep network model to identify building features, and adopts multiple types of enhancement strategies to improve the model's robustness.

[0061] The neural network model building module includes a deep network structure construction sub-module and a data augmentation training sub-module.

[0062] Furthermore, the deep network structure construction sub-module is used to build a hybrid structure that integrates convolutional neural networks and graph neural networks, adaptively extract multi-layer spatial semantic features of images and point clouds, and introduce an attention mechanism module, an edge weight tensor adjustment mechanism, and a skip connection unit to enhance the model's feature retention ability in complex scenarios. The data augmentation training sub-module is used to generate affine transformation images, Gaussian perturbed point clouds, occlusion simulation samples, and structure missing samples for the original training samples, add them to the training set after keeping the label synchronization, and jointly optimize the model parameters through a combined loss function (such as cross-entropy and mean square error).

[0063] It should be noted that the deep network structure construction sub-module is used to define the model architecture and coding method, which is the structural basis for the subsequent learning process. The data augmentation training sub-module is the key to optimizing the model parameters and improving the robustness. The generated training samples and network behavior will significantly affect the generalization ability of the reconstruction system. As the core learning unit of the system, the neural network model building module's functional strength directly determines the accuracy and stability of the subsequent reconstruction tasks.

[0064] The point cloud stitching and unification module is used to implement point cloud stitching and coordinate unification using a fusion algorithm, and update and synchronize the twin models based on multi-dimensional thresholds.

[0065] The point cloud stitching and unification module includes a point cloud registration and fusion sub-module and a model synchronization and update sub-module.

[0066] Furthermore, the point cloud registration and fusion sub-module is used to apply the trained feature extraction model to the registered point cloud data. By extracting edge points, corner points, and local feature subsets, it constructs feature matching pairs, first performs feature-based consistency registration, and then conducts global fine registration through the ICP algorithm. At the same time, voxel division and quality assessment are carried out in the dense area, and a unified three-dimensional dense point cloud model is output. The model synchronization and update sub-module is used to import the reconstruction result into the twin platform, set the data reception threshold bound to the building sensor. If the sensor exceeds the standard continuously for three cycles, it triggers the local point cloud acquisition and grid area reconstruction operations to achieve the dynamic update of the twin model in the time dimension.

[0067] It should be noted that the point cloud registration and fusion sub-module is the key to point cloud alignment and modeling quality control, ensuring the accurate restoration of the three-dimensional structure. The model synchronization and update sub-module ensures the consistency between the reconstructed model and the actual building state, which is the core embodiment of "synchronization" in the twin concept. The point cloud stitching and unification module realizes a complete closed-loop from spatial alignment to twin linkage, which is the key link for this system to adapt to complex building scenarios and time-varying states.

Claims

1. A three-dimensional reconstruction method for building digital twins based on large models, characterized in that, Including: Deploy multi-source devices in the building, collect multi-angle data of the building, and conduct type classification and structured preprocessing; Construct and train an adaptive deep network model to identify building features, and adopt a multi-type enhancement strategy to improve the robustness of the model; Use a fusion algorithm to achieve point cloud stitching and coordinate unification, and update and synchronize the twin models based on multi-dimensional thresholds; Using the fusion algorithm to achieve point cloud stitching and coordinate unification includes transforming the point cloud data after the enhancement strategy to the global world coordinate system based on the building center point, and using an initial registration strategy based on feature point matching to roughly align different source point clouds; Use the Iterative Closest Point algorithm for fine registration. After all the point cloud data registration is completed, perform weighted merging processing according to the density and quality thresholds to generate a single high-density point cloud model; use the Poisson surface reconstruction algorithm to re-evaluate the global normal vector of the fused point cloud, and construct a continuously topologically structured triangular mesh model. A one-to-one mapping relationship is established between the triangular mesh and the original point cloud through a mapping table.

2. The method for three-dimensional reconstruction of building digital twins based on large models according to claim 1, wherein: The collection of multi-angle data of the building includes, Deploy laser scanners and ground image acquisition devices around the target building, and arrange scanners in different elevation and height areas; Each device records the timestamp and sampling coordinates during the acquisition process.

3. The method for three-dimensional reconstruction of building digital twins based on large models according to claim 1 or 2, characterized in that: The type classification and structured preprocessing include, For point cloud data, use a statistical outlier removal algorithm to process the original data stream, remove outliers whose distance exceeds 3 times the standard deviation from the average value, and compress the point cloud data through a voxel grid filter with a resolution of 0.1 meters; Organize the image data in chronological order of shooting, perform gray normalization and histogram equalization processing, and extract the texture of the key area; Organize different types of data according to a unified spatial coordinate system, and establish a type marking field.

4. The method for three-dimensional reconstruction of building digital twins based on large models according to claim 3, wherein: The construction and training of an adaptive deep network model to identify building features include, Based on the data after structured preprocessing, construct an image data set and a point cloud data set with labels. The image data labels include edges, corners, and structural connection parts, and the point cloud data annotates the building floors, window areas, and edge points; Construct an adaptive deep network model, embed an attention mechanism module and a multi-scale extraction structure on the basis of the convolutional neural network architecture, and introduce skip connections for feature compensation; For point cloud data, use a graph neural network based on a dynamic graph structure to construct an edge weight tensor to enhance the local structure preservation ability; During training, use a cross-entropy loss function combined with a mean square error combined loss, and the optimization algorithm adopts an Adam variable step size strategy.

5. The method for three-dimensional reconstruction of building digital twins based on large models according to any one of claims 1, 2 or 4, characterized in that: The adoption of a multi-type enhancement strategy to improve the robustness of the model includes, In the training stage, for image data, introduce affine transformation and brightness perturbation to simulate different shooting conditions, and construct a random occlusion area to simulate on-site occlusion, generating an additional sample set; Apply Gaussian noise to the point cloud data to simulate sensor errors, and at the same time introduce a random local point loss strategy to simulate measurement loss under occlusion; Retain the original labels for the enhanced image data samples and point cloud data samples and perform synchronization processing, and add them to the training set to participate in training.

6. The method for three-dimensional reconstruction of building digital twins based on large models according to claim 5, wherein: Using the fusion algorithm to achieve point cloud stitching and coordinate unification includes, The point cloud data after the enhancement strategy is set with the center point of the building structure as the global reference origin, and a unified world coordinate system is constructed based on the origin. The point cloud data from different devices and perspectives are mapped to the world coordinate system through coordinate transformation; The initial registration adopts a matching strategy based on 3D feature points. Geometrically significant feature points, including edge points, corner points, and curvature mutation points, are extracted from each point cloud. A matching algorithm based on distance and normal consistency is used to match the corresponding relationship between feature points between adjacent point clouds. The initial spatial alignment is completed by calculating the optimal rigid transformation matrix between corresponding points. After the initial registration is completed, the iterative closest point algorithm is used to perform precise registration on each group of point clouds. In each iteration, the closest point pair relationship is recalculated and the transformation matrix is continuously optimized according to the error minimization principle until the preset error threshold is met; After the precise registration is completed, all point cloud data are spatially divided into fixed-size voxel grids, and the number of points in each voxel and the mean square distance between points are counted as density and quality evaluation indicators. If the data density in the voxel meets the threshold and the distance variance is lower than the set standard, the regional point cloud is retained, otherwise it is eliminated, and finally a dense and evenly distributed single high-density point cloud model is generated; The Poisson surface reconstruction algorithm is used to estimate the global normal vector of the point cloud model. The direction of the normal vector of each point in its neighborhood is estimated and the direction is unified. Then, an implicit function field is constructed. The global integral field function is constructed through octree space partitioning to reconstruct a closed and continuous triangular mesh model. At the same time, the spatial mapping relationship between each mesh surface and the original point is recorded to form a one-to-one mapping table with a stable structure.

7. The method for three-dimensional reconstruction of building digital twins based on large models according to any one of claims 1, 2, 4 or 6, characterized in that: The twin model update and synchronization based on multi-dimensional thresholds includes: Import the constructed three-dimensional grid model of the building into the digital twin platform, execute the sensor data access step, collect the building physical status data through the sensor, and analyze the building physical status data. When the detection results of any type of sensor exceed the set threshold range within three consecutive sampling cycles, execute the local reconstruction step, including re-collecting the image data and point cloud data corresponding to the area, and calling the trained deep learning model to identify the structural features of the abnormal area and complete the local grid update.

8. A three-dimensional reconstruction system for building digital twins based on large models, characterized in that: It includes acquisition and preprocessing module, neural network model building module, and point cloud splicing unified module; The acquisition and preprocessing module is used to deploy multi-source equipment in the building, collect multi-angle data of the building, and perform type differentiation and structured preprocessing; The neural network model building module is used to build and train an adaptive deep network model to identify building features, and adopts a multi-type enhancement strategy to improve the model robustness; The point cloud stitching and unification module is used to realize point cloud stitching and coordinate unification by using a fusion algorithm, and to realize twin model updating and synchronization based on a multi-dimensional threshold.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the large-model-based building digital twin three-dimensional reconstruction method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the large-model-based building digital twin three-dimensional reconstruction method according to any one of claims 1 to 7 are implemented.

Images