A method and system for 3D reconstruction of building digital twins based on large models

Abstract

The present invention discloses a method and system for three-dimensional reconstruction of building digital twins based on large models, which relates to the technical field of fusion of building digital twins and three-dimensional modeling, including deploying multi-source equipment in buildings, collecting multi-angle data of buildings, and performing type differentiation and structured preprocessing. An adaptive deep network model is constructed and trained to identify building features, and a multi-type enhancement strategy is adopted to improve the robustness of the model. A fusion algorithm is used to achieve point cloud splicing and coordinate unification, and twin model updates and synchronization are achieved based on multi-dimensional thresholds. The method described in the present invention establishes high-quality input for the deep model; enables the recognition of building structural features to have robustness and accuracy advantages; and enables the twin model to have dynamic update capabilities. The three form a complete closed loop, ultimately breaking through the technical bottlenecks of traditional reconstruction methods in accuracy, stability and updateability, and realizing a new path for three-dimensional modeling of building digital twins for complex scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of the fusion of building digital twins and three-dimensional modeling, and specifically to a three-dimensional reconstruction method and system for building digital twins based on a large model. Background Art

[0002] With the integrated development of technologies such as Building Information Modeling (BIM), 3D Laser Scanning (LiDAR), and artificial intelligence, building digital twins have become an important means of supporting the perception and intelligent decision-making of buildings throughout their entire life cycle. Digital twin models require highly accurate, real-time, and synchronized 3D modeling capabilities to accurately map the physical state and structural form of buildings. Traditional 3D reconstruction methods often rely on static modeling processes, fixed topological structures, or regular point cloud matching algorithms, making it difficult to meet the dynamic update requirements of building deformation monitoring, operation and maintenance warnings, and other requirements. In recent years, with the development of deep learning, especially large-scale multimodal neural networks, the fusion and recognition capabilities of building images and point cloud data have been significantly improved, providing the possibility of higher accuracy and adaptive modeling for digital twin 3D reconstruction methods.

[0003] Existing methods for 3D reconstruction of buildings still have many technical bottlenecks in terms of multi-source data splicing, structural continuity expression, and dynamic update mechanisms. On the one hand, traditional point cloud splicing technology relies on fixed-rule feature point extraction and rigid registration algorithms. When faced with non-uniform sampling, occluded areas, or special-shaped building structures, it is difficult to ensure spatial alignment accuracy, and problems such as mismatches, splicing gaps, or local distortions often occur. On the other hand, many methods lack a twin model update mechanism based on real-time sensor feedback, and are unable to effectively map small changes in the state of the building to the 3D model, resulting in the digital twin system being "built but not updated" and unable to support key scenarios such as structural deformation warning and disaster response. In addition, current models mostly use static network structures, with limited collaborative modeling capabilities for images and point clouds, making it difficult to fully extract spatial semantic features and structural correlation information under different modalities. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is: the existing method of synchronous modeling of building three-dimensional reconstruction and digital twin has the problems of low point cloud splicing accuracy, discontinuous structural topology, and inability to respond to changes in physical state in real time, as well as how to achieve unified alignment of multi-source heterogeneous data, deep fusion expression of structural information, and dynamic update mechanism of digital twin three-dimensional model.

[0006] To address the above-mentioned technical problems, the present invention provides the following technical solutions: a large-scale model-based three-dimensional reconstruction method for digital twins of buildings, comprising deploying multi-source equipment in the building, collecting multi-angle building data, and performing type differentiation and structured preprocessing. An adaptive deep network model is constructed and trained to identify building features, and a multi-type enhancement strategy is employed to improve model robustness. A fusion algorithm is employed to achieve point cloud splicing and coordinate unification, and twin model updates and synchronization are achieved based on multi-dimensional thresholds. The fusion algorithm for achieving point cloud splicing and coordinate unification includes uniformly transforming the enhanced point cloud data to a global world coordinate system based on the building center point, and employing an initial registration strategy based on feature point matching to coarsely align different source point clouds. An iterative closest point algorithm is employed for fine registration. After all point cloud data are registered, they are weighted and merged based on density and quality thresholds to generate a single, high-density point cloud model. A Poisson surface reconstruction algorithm is employed to re-estimate the global normal vector of the fused point cloud, and a triangular mesh model with a continuous topological structure is constructed. A one-to-one mapping relationship is established between the triangular mesh and the original point cloud via a mapping table.

[0007] As a preferred embodiment of the large-model-based 3D reconstruction method for building digital twins described in the present invention, collecting multi-angle building data includes deploying laser scanners and ground-based image acquisition equipment around the target building, with scanners positioned at different elevations and heights. During the acquisition process, each device records timestamps and sampling coordinates.

[0008] As a preferred embodiment of the large-scale model-based 3D reconstruction method for building digital twins described in the present invention, the type differentiation and structured preprocessing includes processing the raw data stream of point cloud data using a statistical outlier removal algorithm to remove outliers exceeding three standard deviations from the mean, and compressing the point cloud data to a 0.1-meter resolution using a voxel grid filter. Image data is organized in chronological order, subjected to grayscale normalization and histogram equalization, and texture extraction in key areas. Different data types are organized according to a unified spatial coordinate system, and a type tag field is established.

[0009] As a preferred solution of the large-model-based building digital twin 3D reconstruction method described in the present invention, the construction and training of an adaptive deep network model to identify building features includes constructing a labeled image dataset and a point cloud dataset based on the structured preprocessed data, wherein the image data labels include edges, corners, and structural connection parts, and the point cloud data annotates the number of building floors, window areas, and edge points. An adaptive deep network model is constructed, and an attention mechanism module and a multi-scale extraction structure are embedded on the basis of a convolutional neural network architecture, and jump connections are introduced for feature compensation. The point cloud data adopts a graph neural network based on a dynamic graph structure, and an edge weight tensor is constructed to enhance the local structure retention capability. The cross entropy loss function is used in combination with the mean square error combined loss in training, and the optimization algorithm adopts the Adam variable step size strategy.

[0010] As a preferred solution of the large-model-based 3D reconstruction method for building digital twins described in the present invention, the multi-type enhancement strategy used to improve model robustness includes, during the training phase, introducing affine transformation and brightness perturbation to the image data to simulate different shooting conditions, and constructing random occlusion areas to simulate on-site occlusion to generate additional sample sets. Gaussian noise is applied to the point cloud data to simulate sensor errors, and a random local point loss strategy is introduced to simulate measurement loss under occlusion. The original labels of the enhanced image data samples and point cloud data samples are retained and processed synchronously, and then added to the training set for training.

[0011] As a preferred solution of the large-model-based building digital twin three-dimensional reconstruction method described in the present invention, the method includes: using a fusion algorithm to achieve point cloud splicing and coordinate unification, setting the point cloud data after the enhancement strategy with the center point of the building structure as the global reference origin, constructing a unified world coordinate system based on the origin, and mapping the point cloud data from different devices and perspectives to the world coordinate system through coordinate transformation. The initial alignment adopts a matching strategy based on three-dimensional feature points, extracts geometrically significant feature points for each group of point clouds, including edge points, corner points and curvature mutation points, uses a matching algorithm based on distance and normal consistency to match the feature point correspondence between adjacent point clouds, and completes the preliminary spatial alignment by calculating the optimal rigid transformation matrix between corresponding points. After completing the initial alignment, the iterative closest point algorithm is used to perform precise alignment on each group of point clouds. In each iteration, the closest point pair relationship is recalculated and the transformation matrix is ​​continuously optimized according to the error minimization principle until the preset error threshold is met. After precise registration, all point cloud data is spatially partitioned into a fixed-size voxel grid. The number of points in each voxel and the mean squared distance between points are counted as density and quality indicators. If the data density within the voxel meets the threshold and the distance variance is below the set standard, the regional point cloud is retained; otherwise, it is discarded, ultimately generating a dense and evenly distributed single high-density point cloud model. Using the Poisson surface reconstruction algorithm, the global normal vector of the point cloud model is estimated. For each point, the normal vector direction of the midpoints in its neighborhood is estimated and the orientation is unified. An implicit function field is then constructed. The global integral field function is constructed through octree spatial partitioning to reconstruct a closed and continuous triangular mesh model. The spatial mapping relationship between each mesh surface and the original point is recorded, forming a structurally stable one-to-one mapping table.

[0012] As a preferred solution of the large-model-based building digital twin three-dimensional reconstruction method described in the present invention, the method of updating and synchronizing the twin model based on multi-dimensional thresholds includes importing the constructed three-dimensional building grid model into the digital twin platform, executing the sensor data access step, collecting the building physical state data through the sensor, and analyzing the building physical state data. When the detection results of any type of sensor exceed the set threshold range within three consecutive sampling cycles, a local reconstruction step is executed, including re-collecting the image data and point cloud data corresponding to the area, and calling the trained deep learning model to identify the structural features of the abnormal area and then complete the local grid update.

[0013] Another object of the present invention is to provide a large-model-based building digital twin three-dimensional reconstruction system, which can realize point cloud splicing and coordinate unification by utilizing a fusion algorithm, and realize twin model update and synchronization based on a multi-dimensional threshold, thereby solving the problems of low point cloud splicing accuracy, discontinuous structural topology, and inability to respond to physical state changes in the current existing building three-dimensional reconstruction and digital twin synchronous modeling methods, as well as how to realize the unified alignment of multi-source heterogeneous data, deep fusion expression of structural information, and dynamic update mechanism of digital twin three-dimensional models.

[0014] As a preferred solution of the large-model-based building digital twin 3D reconstruction system described in the present invention, it includes: an acquisition and preprocessing module, a neural network model establishment module, and a point cloud splicing and unification module.

[0015] The acquisition and pre-processing module is used to deploy multi-source equipment in the building, collect multi-angle building data, and perform type differentiation and structured pre-processing.

[0016] The neural network model building module is used to construct and train an adaptive deep network model to identify architectural features, and a multi-type enhancement strategy is used to improve the robustness of the model.

[0017] The point cloud splicing and unification module is used to realize point cloud splicing and coordinate unification by using a fusion algorithm, and to realize twin model updating and synchronization based on a multi-dimensional threshold.

[0018] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a step of a method for three-dimensional reconstruction of a building digital twin based on a large model.

[0019] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for three-dimensional reconstruction of a building digital twin based on a large model.

[0020] Beneficial effects of the present invention: The large-model-based three-dimensional reconstruction method for building digital twins provided by the present invention realizes the comprehensive collection of multi-angle and multi-modal original data of buildings by deploying laser scanners and image acquisition equipment around the building, and combining spatial control strategies with timestamp synchronization mechanisms. Subsequently, anomaly removal and voxel compression are performed on the point cloud data, and grayscale normalization and histogram enhancement are performed on the image data to achieve noise suppression and structural enhancement of multi-source data. Finally, all data are uniformly mapped to the world coordinate system and type tags are added to achieve standardized fusion of heterogeneous data. The purpose of this step is to provide a high-quality, structurally consistent, and time-controllable data foundation for subsequent deep model input, avoid problems such as structural offset and feature loss caused by data clutter in traditional three-dimensional modeling, and ultimately achieve the effect of improving reconstruction accuracy and modeling stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 An overall flow chart of a large-model-based 3D reconstruction method for building digital twins provided in the first embodiment of the present invention. DETAILED DESCRIPTION

[0023] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0024] Example 1, with reference to Figure 1 , as one embodiment of the present invention, provides a method for 3D reconstruction of a building digital twin based on a large model, comprising:

[0025] S1: Deploy multi-source equipment in buildings to collect multi-angle building data for type differentiation and structured preprocessing.

[0026] Laser scanners and ground-based image acquisition equipment are deployed around the target building, with scanners positioned at different elevations and heights. During the acquisition process, each device records timestamps and sampling coordinates. A statistical outlier removal algorithm is used to process the raw data stream of point cloud data, removing outliers that exceed three standard deviations from the mean. The point cloud data is then compressed to a resolution of 0.1 meters using a voxel grid filter. Image data is organized in chronological order, undergoing grayscale normalization and histogram equalization to extract textures in key areas. Different types of data are organized according to a unified spatial coordinate system, and a type tag field is established.

[0027] A preferred solution for deploying laser scanners and ground image acquisition equipment around the target building and placing scanners in areas at different facades and heights specifically includes: before deploying equipment in the surrounding area of ​​the target building, first conduct an environmental assessment based on the building's facade distribution, number of floors, and unobstructed conditions, and select appropriate deployment points. The three-dimensional laser scanner is placed in the open areas on the front facade, back facade, and left and right sides of the building, and different height brackets or lifting platforms are set according to the building's floor height to cover the complete scanning area from the ground to the roof. The ground image acquisition equipment includes a high-resolution industrial camera and a mobile track device with a fixed shooting trajectory. It is deployed around the perimeter of the building and arranged at equal intervals to ensure that the shooting angles are staggered and cover all visible surfaces of the building. Each device initializes the synchronization module before starting the acquisition to ensure that the data timestamp and device spatial position information can be recorded in real time during the acquisition process for subsequent multi-source data alignment.

[0028] After data collection is complete, preprocessing operations are performed on different types of data. Point cloud data, the primary source of spatial geometric data, is first removed from noise points using a statistical outlier removal algorithm. Based on local neighborhood analysis, the mean distance between each point and its K nearest neighbors is calculated. Points with distances exceeding three standard deviations of the overall mean are identified as outliers and removed. After this removal, the remaining point cloud is thinned using a voxel grid filter. The point cloud is divided into voxel units with a side length of 0.1 meters, and only one representative point is retained within each voxel unit to compress the data volume and maintain the integrity of the geometric outline. Image data is sorted according to the chronological order of acquisition, and preprocessing operations are performed on each image. This includes image grayscale normalization to eliminate exposure differences and histogram equalization to enhance image contrast and enhance the recognizability of building edges and texture features. Texture information in key areas of the image is then extracted and its spatial acquisition location is marked. After preprocessing, both the point cloud and image data are converted to the building reference coordinate system using a unified spatial coordinate system. A type tag field is assigned to each data type to facilitate data diversion and structural integration during input into the deep learning model. Once these operations are complete, the next phase of the model training and feature recognition process begins.

[0029] It should be noted that the S1 design concept is to achieve comprehensive and multi-angle data collection of buildings through the deployment of multi-source heterogeneous equipment. This, combined with type differentiation and structured preprocessing, improves data quality and organizational efficiency. This step ensures the standardization and consistency of model inputs, providing a high-precision data foundation for subsequent depth recognition and 3D modeling.

[0030] S2: Build and train an adaptive deep network model to identify architectural features, and use multi-type enhancement strategies to improve model robustness.

[0031] Based on the structured preprocessed data, labeled image and point cloud datasets were constructed. The image data was labeled with edges, corners, and structural connections, while the point cloud data was annotated with the number of building floors, window areas, and edge points. An adaptive deep network model was constructed, embedding an attention mechanism module and a multi-scale extraction structure based on a convolutional neural network architecture, and introducing skip connections for feature compensation. A graph neural network based on a dynamic graph structure was used for the point cloud data, constructing edge weight tensors to enhance local structure preservation. Training used a cross-entropy loss function combined with a mean squared error loss, and the Adam variable step-size optimization algorithm was used.

[0032] Constructing an adaptive deep network model, embedding an attention mechanism module and a multi-scale extraction structure based on a convolutional neural network architecture, and introducing a preferred solution for feature compensation includes: after completing the structured preprocessing of building images and point clouds, first constructing a training data set containing spatial semantic labels. Among them, the image data is annotated according to edge lines, structural corners, and connector positions, and the point cloud data generates three-dimensional spatial labels according to the number of building floors, window areas, and boundary points. Each data sample is bound to the spatial positioning information with a timestamp to ensure that multimodal data can be uniformly loaded and processed synchronously during the training phase. In terms of image feature modeling, an adaptive deep convolutional neural network structure is constructed, an attention mechanism is embedded to highlight key structural responses, and a multi-scale convolution module is used to extract image texture features under different receptive fields. At each position , and calculate its weighted response tensor as follows:

[0033] ,

[0034] in, Represents the position in the feature map The attention response value. represents the main branch convolution activation output, Represents the auxiliary branch attention response. represents the reverse Euclidean kernel. Represents the fused texture response function. Indicates the number of attention dimensions. This expression is used to dynamically adjust the channel output intensity and highlight the key structural points. The value range is 0~1, and the larger the value, the more significant the structure. Expressed as The channel's own response strength in the self-attention module. and Represents the sampling coordinates of the horizontal (x-axis) and vertical (y-axis) directions of the image feature map respectively.

[0035] Expressed as:

[0036] ,

[0037] in, It is the value of the fused texture response function, which is used in the attention mechanism to extract the weighted features of each channel at the image coordinate position (u, v). is the number of channels of the input image. For the The weight coefficient of each channel in channel fusion. For image Channel in The pixel value at the location. For the image at this position, Axis and The second-order Laplace directional derivative of the axis is used to reflect the local texture change rate. For the Channel in The sum of squared gradients at , used to evaluate edge strength. The normalization parameter for this channel controls the edge response suppression range. is the total number of texture kernel sets, For the The fusion weights of a high-order texture filter. is the absolute value of the response of the sth high-order texture response kernel at (u, v), expressing the mid- and high-frequency structure. The term is used to compress the texture response enhancement term to prevent overfitting in strong texture areas, and the term 1 is used for numerical stability.

[0038] The value range is [0,+∞], the larger the value, the The higher the texture intensity after fusion, the greater the response weight the model should give in the attention mechanism, usually giving a larger value to high-edge and high-texture density areas, while the value approaches 0 in flat areas.

[0039] Point cloud data uses a graph neural network based on a dynamic graph structure to construct an edge weight tensor to enhance the local structure preservation ability. Specifically, in terms of point cloud data modeling, a graph neural network based on a dynamic graph structure is constructed. First, an edge weight tensor is constructed based on the local relative vector of the point cloud, and the edge weight tensor of any point pair is defined. The edge weight in the rth iteration is:

[0040] ,

[0041] in, Represents an edge In the The edge rights of the wheel, and Indicates a point with dot Construct a vector from the three-dimensional normal vector of . Indicates a point The spatial disturbance of its center point is Represents the number of neighboring points. This edge weight tensor is used to guide the neural network to adjust the adjacency matrix according to local structural changes, achieving adaptive modeling of point cloud topology. The value range is 0~ , the larger the , the greater the difference in direction. Represents the integral variable used to construct the spatial structure consistency kernel in the point cloud feature. The domain is the angle interval [0,π] and is used to approximate the directional consistency of the Gaussian kernel. and Represents two spatial sampling point indices in the point cloud, indicating the start and end points of the edge.

[0042] The cross entropy loss function is combined with the mean square error loss in training, and the optimization algorithm adopts an optimal solution of the Adam variable step size strategy. Specifically, during the network training phase, a combined loss function is designed to simultaneously evaluate the prediction accuracy of the image and point cloud, which is defined as follows:

[0043] ,

[0044] in, is the total training loss, and are the number of image and point cloud samples, respectively. is the sample confidence factor, is the image label value, It is the fusion of the model prediction values ​​of the image and point cloud, Quantify the complexity of image enhancement, is the occlusion strength, is the point cloud prediction error function, is the structure tensor of point j in dimension z. The model is optimized through back propagation using this loss function.

[0045] During the training phase, affine transformations and brightness perturbations are introduced to the image data to simulate different shooting conditions. Random occlusion regions are constructed to simulate on-site occlusions, generating additional sample sets. Gaussian noise is applied to the point cloud data to simulate sensor errors, and a random local point dropout strategy is introduced to simulate measurement losses due to occlusion. The enhanced image and point cloud data samples retain their original labels and are processed synchronously before being added to the training set for training.

[0046] Furthermore, to improve model robustness, multiple types of data augmentation are performed before training. Image data augmentation includes three types of operations: Affine transformations (rotation, scaling, and translation) are performed on the original image, followed by resampling to generate new samples. Random brightness perturbations are used to simulate different lighting conditions. A regional occlusion map is superimposed to simulate scene occlusion, while retaining a certain proportion of the original labels. The augmented image and its original labels are then fed into the model training simultaneously.

[0047] For point cloud data, a dual strategy of noise perturbation and sampling dropout was designed. First, a Gaussian random perturbation with zero mean and variance of 0.01 was added to some points to simulate laser scanning errors. Second, regional point dropout was performed by randomly selecting block-like regions in space. This directly removes points within the region to simulate missing structures after shadows or occlusions.

[0048] During training, all augmented samples are fed into the main network alongside the original samples. A dynamic batch update mechanism controls the sample ratio, ensuring that the model gradually learns to robustly discriminate against perturbations over time. Parameter updates utilize a momentum-based adaptive learning rate optimizer, dynamically adjusting the step size and enhancing training stability, ultimately outputting the trained model.

[0049] It should be noted that the design concept of S2 is to achieve accurate multimodal architectural feature extraction from images and point clouds through the joint modeling of adaptive deep network models and dynamic graph neural architectures. Structural perception is achieved by introducing an attention mechanism and edge weight tensors. Multi-scale extraction and skip connections enhance spatial hierarchical representation. A multi-type perturbation enhancement strategy is used to improve model robustness. Compared to existing technologies that only process a single modality or static topology, this step maintains high recognition accuracy and adaptive update capabilities in scenarios with complex occlusion, missing structures, and uneven lighting, significantly enhancing the generalization and engineering applicability of building information modeling.

[0050] S3: Use fusion algorithms to achieve point cloud stitching and coordinate unification, and realize twin model update and synchronization based on multi-dimensional thresholds.

[0051] The point cloud data after the enhancement strategy is set with the center point of the building structure as the global reference origin, and a unified world coordinate system is constructed based on the origin. The point cloud data from different devices and perspectives are mapped to the world coordinate system through coordinate transformation. The initial alignment adopts a matching strategy based on three-dimensional feature points. Geometrically significant feature points, including edge points, corner points, and curvature mutation points, are extracted for each group of point clouds. A matching algorithm based on distance and normal consistency is used to match the feature point correspondence between adjacent point clouds. The initial spatial alignment is completed by calculating the optimal rigid transformation matrix between the corresponding points. After the initial alignment is completed, the iterative closest point algorithm is used to perform precise alignment on each group of point clouds. In each iteration, the closest point pair relationship is recalculated and the transformation matrix is ​​continuously optimized according to the error minimization principle until the preset error threshold is met. After precise registration, all point cloud data is spatially partitioned into a fixed-size voxel grid. The number of points in each voxel and the mean squared distance between points are counted as density and quality indicators. If the data density within the voxel meets the threshold and the distance variance is below the set standard, the regional point cloud is retained; otherwise, it is discarded, ultimately generating a dense and evenly distributed single high-density point cloud model. A Poisson surface reconstruction algorithm is used to estimate the global normal vector of the point cloud model. For each point, the normal vector direction of the midpoints in its neighborhood is estimated and unified. An implicit function field is then constructed. The global integral field function is constructed through octree spatial partitioning to reconstruct a closed and continuous triangular mesh model. The spatial mapping relationship between each mesh surface and the original point is recorded, forming a structurally stable one-to-one mapping table.

[0052] The constructed three-dimensional grid model of the building is imported into the digital twin platform, and the sensor data access step is executed. The building physical state data is collected through the sensor and analyzed. When the detection results of any type of sensor exceed the set threshold range within three consecutive sampling cycles, the local reconstruction step is executed, including re-collecting the image data and point cloud data corresponding to the area, and calling the trained deep learning model to identify the structural features of the abnormal area and complete the local grid update.

[0053] A preferred solution for building a unified world coordinate system based on the origin and mapping point cloud data from different devices and perspectives to the world coordinate system through coordinate transformation specifically includes: transforming the enhanced multi-source point cloud data to the geometric center point of the building structure. As the global origin, set the three-dimensional unified world coordinate system, and collect the point cloud data of each set of equipment. Apply a coordinate transformation operation:

[0054] ,

[0055] in: Indicates the original points. Indicates the coordinates of the mapped point. is the rotation matrix. is the translation vector. Indicates the Group point cloud data The total number of points contained in . All point clouds enter the unified coordinate system through this transformation.

[0056] The initial registration adopts a preferred solution based on the matching strategy of 3D feature points. Specifically, for each pair of point clouds, edge points and curvature mutation points are extracted as a set of significant feature points, and a set of corresponding point pairs is constructed:

[0057] ,

[0058] in: and are the first 、 feature points. and is the normal vector of the corresponding point.

[0059] After the initial registration is completed, an optimal solution for fine registration of each group of point clouds using the iterative closest point algorithm is as follows: After the initial registration, the iterative closest point (ICP) algorithm is used for fine registration. The iterative process is as follows:

[0060] ,

[0061] in, and are the source and target points in the matching point pair respectively. and are the rotation and translation to be solved. is the number of matching point pairs. Represents the optimal rotation matrix obtained after the iterative closest point (ICP) algorithm is executed. The optimal translation vector is obtained after the Iterative Closest Point (ICP) algorithm is executed. It is updated iteratively until the error converges to a set threshold less than 0.001.

[0062] A preferred solution for dividing all point cloud data into fixed-size voxel grids according to space and counting the number of points in each voxel and the mean square distance between points as density and quality evaluation indicators specifically includes dividing the precisely registered point cloud into voxel grids, setting the voxel size to 0.1 meters, and counting the point density in each voxel. and mean square distance quality index between points , defined as follows:

[0063] ,

[0064] in: For the The number of points in a voxel. is the voxel volume. For the first points. is the center of the voxel.

[0065] Only voxel regions where the point density within the voxel is greater than the minimum density threshold and the mean square distance between points is less than the maximum quality index threshold are retained for generating a dense point cloud. In the present invention, the minimum density threshold is 400 points per cubic meter and the maximum quality index threshold is 0.005 square meters.

[0066] Using the Poisson surface reconstruction algorithm, a preferred solution for estimating the global normal vector of the point cloud model specifically includes performing Poisson surface reconstruction on the generated dense point cloud set, including normal estimation, implicit function construction and mesh extraction, calculating the normal direction of each point and unifying the direction. Constructing the integral field function , expressed as:

[0067] ,

[0068] in, Any position in space. for point The normal vector of . is the Dirac delta function. is the point cloud distribution domain. Perform layered discretization and extract The isosurface is generated to generate a closed triangular mesh model, and the spatial correspondence between each mesh surface and the original point is recorded to build a mapping table.

[0069] A preferred solution for importing the completed building 3D mesh model into the digital twin platform specifically includes importing the triangular mesh model into the twin platform and connecting it with the building sensor system. Assume that the collected physical state data is a sequence:

[0070] ,

[0071] in: Indicates the Sensors at the time The numerical values ​​include temperature, humidity, displacement, stress, etc. Indicates the number of sensors. The following conditions must be met:

[0072] ,

[0073] in, Indicates the The sensor in the The data value in a time sampling period. Indicates the historical average value of the sensor under normal operation. Indicates sensor The abnormal judgment threshold is the tolerance floating range of the system configuration. This is an indicator function that returns 1 if the expression in the brackets holds, otherwise 0. The accumulated sum divided by 3 indicates whether the threshold has been exceeded at least once in the last three sampling periods. If the result is 1, local reconstruction is triggered.

[0074] The sensor sampling value exceeds its preset threshold for three consecutive cycles , triggering a local model update mechanism: The mobile device is rescheduled to collect images and point clouds in that area. The trained model is used to identify local structures. The corresponding mesh area is updated using local geometry stitching and mesh repair techniques.

[0075] Furthermore, the present invention presets a threshold As shown in Table 1, it can be adjusted according to actual conditions.

[0076] Table 1 Preset threshold table

[0077]

[0078] It should be noted that S3 constructs a registration and reconstruction process from multi-source point clouds to a unified world coordinate system, and by integrating Poisson surface reconstruction with a multi-dimensional sensor threshold judgment mechanism, it achieves high-precision synchronous updates of dynamic twin models. Compared to existing technologies that suffer from large data splicing errors, local distortion in model reconstruction, and inability to respond to structural changes, this step achieves dense, continuous, and topologically consistent 3D reconstruction results through feature point-guided rigid registration, density and mass dual-threshold filtering, and Gaussian field integral surface generation. It can also trigger local model updates in real-time in conjunction with building physical status monitoring data, effectively improving the structural integrity and timeliness of the model.

[0079] Example 2 is an embodiment of the present invention, which provides a large-model-based building digital twin 3D reconstruction system, including an acquisition and preprocessing module, a neural network model building module, and a point cloud splicing unified module.

[0080] The acquisition and pre-processing module is used to deploy multi-source equipment in buildings, collect multi-angle building data, and perform type differentiation and structured pre-processing.

[0081] The acquisition and preprocessing module includes the equipment layout acquisition submodule and the data preprocessing standardization submodule.

[0082] Furthermore, the device deployment and acquisition submodule is used to deploy laser scanning equipment and image acquisition devices at multiple viewing angles within the building's target area, collecting point cloud data and image information from different angles. During the acquisition process, the timestamp, pose parameters, and sampling coordinates of each device are recorded to construct a spatiotemporal calibration reference frame for the data. The data preprocessing and standardization submodule is used to remove outliers and perform voxel filtering on the collected point cloud, as well as grayscale normalization and edge enhancement on the image data. The data is then uniformly converted to the building's world coordinate system and labeled with a data type field for each type of data to ensure that subsequent network models can process different modal data separately.

[0083] It should be noted that the equipment layout acquisition submodule is the starting point for the acquisition and preprocessing module, ensuring data coverage integrity in terms of time sequence and perspective. The data preprocessing and standardization submodule is a crucial step in achieving data fusion. The output data format and quality directly impact subsequent neural network training and feature extraction results. As the foundation of the system, the acquisition and preprocessing module's output serves as the input for neural network model construction and 3D splicing and fusion processing.

[0084] A neural network model module is established to build and train an adaptive deep network model to identify architectural features, and a multi-type enhancement strategy is adopted to improve the robustness of the model.

[0085] The neural network model building module includes a deep network structure building sub-module and a data enhancement training sub-module.

[0086] Furthermore, the deep network architecture submodule is used to build a hybrid structure that integrates convolutional neural networks and graph neural networks. It adaptively extracts multi-layer spatial semantic features from images and point clouds. It also introduces an attention mechanism, edge weight tensor adjustment, and skip connection units to enhance the model's feature preservation capabilities in complex scenarios. The data augmentation training submodule generates affine-transformed images, Gaussian-perturbed point clouds, occlusion-simulated samples, and structure-missing samples from the original training samples. These samples are then added to the training set after preserving label synchronization, and the model parameters are jointly optimized using a combination of loss functions (such as cross-entropy and mean squared error).

[0087] It should be noted that the deep network structure construction submodule defines the model architecture and encoding method, forming the structural foundation for the subsequent learning process. The data augmentation training submodule is key to optimizing model parameters and improving robustness. The resulting training samples and network behavior significantly impact the generalization capabilities of the reconstruction system. The neural network model module serves as the system's core learning unit, and its functionality directly determines the accuracy and stability of subsequent reconstruction tasks.

[0088] The point cloud stitching and unification module is used to realize point cloud stitching and coordinate unification by using fusion algorithm, and to realize twin model update and synchronization based on multi-dimensional threshold.

[0089] The unified point cloud stitching module includes a point cloud registration and fusion sub-module and a model synchronization update sub-module.

[0090] Furthermore, the point cloud registration fusion submodule applies the trained feature extraction model to the registered point cloud data. By extracting edge points, corner points, and local feature subsets, it constructs feature matching pairs, first performing feature-based consistency registration, then performing global fine registration using the ICP algorithm. Simultaneously, it performs voxel segmentation and quality assessment in dense areas, outputting a unified 3D dense point cloud model. The model synchronization update submodule imports the reconstruction results into the twin platform and sets the data reception threshold bound to the building sensors. If the sensor exceeds the threshold for three consecutive cycles, it triggers local point cloud acquisition and grid area reconstruction operations, enabling dynamic updating of the twin model in the temporal dimension.

[0091] It should be noted that the point cloud registration and fusion submodule is key to point cloud alignment and modeling quality control, ensuring accurate restoration of 3D structures. The model synchronization update submodule ensures consistency between the reconstructed model and the actual building state, embodying the core "synchronicity" of the twin concept. The point cloud stitching unification module completes the closed loop from spatial alignment to twin linkage, a key component in the system's ability to adapt to complex building scenarios and time-varying conditions.

Claims

1. A method for 3D reconstruction of building digital twins based on a large model, characterized in that: include: Deploy multi-source equipment in buildings to collect multi-angle building data for type differentiation and structured pre-processing; Build and train an adaptive deep network model to identify architectural features, and use a multi-type enhancement strategy to improve model robustness; Fusion algorithms are used to achieve point cloud splicing and coordinate unification, and twin model updates and synchronization are achieved based on multi-dimensional thresholds; The fusion algorithm is used to achieve point cloud stitching and coordinate unification. This includes transforming the enhanced point cloud data into a global world coordinate system based on the building center point, and using an initial registration strategy based on feature point matching to roughly align different source point clouds. The iterative closest point algorithm is used for precise registration. After all point cloud data are registered, they are weighted and merged according to the density and quality threshold to generate a single high-density point cloud model. The Poisson surface reconstruction algorithm is used to re-estimate the global normal vector of the fused point cloud and construct a triangular mesh model with a continuous topological structure. A one-to-one mapping relationship is established between the triangular mesh and the original point cloud through a mapping table. Building and training an adaptive deep network model to identify architectural features includes: Based on the structured preprocessed data, we construct labeled image datasets and point cloud datasets. The image data labels include edges, corners, and structural connection parts, and the point cloud data labels the number of building floors, window areas, and edge points. Build an adaptive deep network model, embed an attention mechanism module and a multi-scale extraction structure based on the convolutional neural network architecture, and introduce skip connections for feature compensation; Point cloud data uses a graph neural network based on a dynamic graph structure to construct edge weight tensors to enhance the ability to preserve local structure; The cross entropy loss function is combined with the mean square error loss in training, and the Adam variable step size strategy is used in the optimization algorithm. Using fusion algorithms to achieve point cloud stitching and coordinate unification includes: The enhanced point cloud data is set with the center point of the building structure as the global reference origin, and a unified world coordinate system is constructed based on the origin. The point cloud data from different devices and perspectives are mapped to the world coordinate system through coordinate transformation; Initial registration uses a matching strategy based on 3D feature points. Geometrically significant feature points, including edge points, corner points, and curvature mutation points, are extracted from each point cloud. A matching algorithm based on distance and normal consistency is used to match feature point correspondences between adjacent point clouds. Preliminary spatial alignment is achieved by calculating the optimal rigid transformation matrix between corresponding points. After the initial registration is completed, the iterative closest point algorithm is used to perform fine registration on each group of point clouds. In each iteration, the relationship between the closest point pairs is recalculated and the transformation matrix is ​​continuously optimized according to the error minimization principle until the preset error threshold is met. After the precise registration is completed, all point cloud data are spatially divided into fixed-size voxel grids. The number of points in each voxel and the mean square distance between points are counted as density and quality evaluation indicators. If the data density within the voxel meets the threshold and the distance variance is lower than the set standard, the regional point cloud is retained. Otherwise, it is discarded, and finally a dense and evenly distributed single high-density point cloud model is generated. The Poisson surface reconstruction algorithm is used to estimate the global normal vector of the point cloud model. The direction of the normal vector of each point in its neighborhood is estimated and the orientation is unified. Then, an implicit function field is constructed. The global integral field function is constructed through octree space partitioning to reconstruct a closed and continuous triangular mesh model. At the same time, the spatial mapping relationship between each mesh surface and the original point is recorded to form a one-to-one mapping table with a stable structure.

2. The method for 3D reconstruction of a building digital twin based on a large model according to claim 1, wherein: The collecting of multi-angle building data includes: Deploy laser scanners and ground image acquisition equipment around the target building, placing scanners at different facades and heights; During the acquisition process, each device records the timestamp and sampling coordinates.

3. The method for 3D reconstruction of a building digital twin based on a large model according to claim 1 or 2, characterized in that: The type differentiation and structured preprocessing include: The point cloud data is processed using a statistical outlier removal algorithm to process the raw data stream, removing outliers that are more than three times the standard deviation from the mean, and the point cloud data is compressed to a resolution of 0.1 meters using a voxel grid filter; The image data is organized in chronological order of shooting time, grayscale normalization and histogram equalization are performed, and the texture of key areas is extracted; Different types of data are organized according to a unified spatial coordinate system, and a type tag field is established.

4. The method for 3D reconstruction of a building digital twin based on a large model according to claim 3, wherein: The multi-type enhancement strategy used to improve the robustness of the model includes: During the training phase, affine transformation and brightness perturbation are introduced into the image data to simulate different shooting conditions, and random occlusion areas are constructed to simulate on-site occlusion to generate additional sample sets. Gaussian noise is applied to the point cloud data to simulate sensor errors, and a random local point loss strategy is introduced to simulate measurement loss under occlusion. The original labels of the enhanced image data samples and point cloud data samples are retained and processed synchronously, and added to the training set to participate in training.

5. The method for 3D reconstruction of a building digital twin based on a large model according to claim 4, wherein: The twin model update and synchronization based on multi-dimensional thresholds includes: The constructed three-dimensional grid model of the building is imported into the digital twin platform, and the sensor data access step is executed. The building physical state data is collected through the sensor and analyzed. When the detection results of any type of sensor exceed the set threshold range within three consecutive sampling cycles, the local reconstruction step is executed, including re-collecting the image data and point cloud data corresponding to the area, and calling the trained deep learning model to identify the structural features of the abnormal area and complete the local grid update.

6. A large-scale model-based building digital twin 3D reconstruction system, characterized by: Including acquisition and preprocessing module, neural network model building module, and point cloud splicing and unification module; The acquisition and pre-processing module is used to deploy multi-source equipment in the building, collect building data from multiple angles, and perform type differentiation and structured pre-processing; The neural network model module is used to build and train an adaptive deep network model to identify building features, and adopts a multi-type enhancement strategy to improve the robustness of the model; The point cloud splicing and unification module is used to realize point cloud splicing and coordinate unification by using a fusion algorithm, and to realize twin model updating and synchronization based on a multi-dimensional threshold.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the large-model-based building digital twin three-dimensional reconstruction method according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the large-model-based building digital twin three-dimensional reconstruction method according to any one of claims 1 to 6 are implemented.

Images