Deep learning based three-dimensional point cloud building automatic modeling system

Abstract

The application discloses a three-dimensional point cloud building automatic modeling system based on deep learning, comprising a point cloud acquisition and input module, a point cloud preprocessing module, a deep feature extraction module, a semantic segmentation and component recognition module, a structure parameter analysis module and a parameterized model generation module. The application realizes the full-process automation of building three-dimensional modeling by deeply fusing the point cloud deep learning network and the parameterized modeling engine. The use of the deep feature extraction module can effectively improve the accuracy of the identification of the edges of doors, windows and roofs and the like, guarantee the accuracy of the building model, and as the system is continuously used, the deep feature extraction module can continuously improve the accuracy of the identification. The use of the semantic segmentation and component recognition module can identify and distinguish the irregularity of the building point cloud, the complexity of the structure and the shielding noise, and solve the problem of manual semantic labeling and topological repair in the prior art.

Description

Technical Field

[0001] This application relates to the field of automated modeling system technology, specifically to an automated modeling system for 3D point cloud buildings based on deep learning. Background Technology

[0002] In existing technologies, the following two techniques are mostly used when creating 3D models of buildings:

[0003] Firstly, modeling is done manually, which is not only inefficient but also costly.

[0004] Secondly, modeling can be performed using laser scanning and photogrammetry techniques. For example, Chinese invention patent application number CN202310783999.0 discloses an automated building modeling method and system based on point cloud data (application date: 2023-06-29, publication date: 2023-12-26). While its implementation has achieved the technical effect of improving the efficiency of building modeling based on point cloud data, ensuring the fidelity of the point cloud model to the building entity, and reducing the degree of human intervention in the modeling process, it still has problems such as difficulty in effectively distinguishing various complex structures in the point cloud (such as windows, roofs, and ancillary facilities) and the need for manual intervention for semantic annotation and topology repair.

[0005] Therefore, there is an urgent need for an automated modeling system for 3D point cloud buildings based on deep learning. Summary of the Invention

[0006] To address this, this application provides an automated modeling system for 3D point cloud buildings based on deep learning, which solves the problems of existing technologies that make it difficult to effectively distinguish complex structures in point clouds, are sensitive to noise and occlusion, and require manual intervention for semantic annotation and topology repair.

[0007] To achieve the above objectives, this application provides the following technical solution:

[0008] The deep learning-based automated modeling system for 3D point cloud buildings includes a point cloud acquisition and input module, a point cloud preprocessing module, a depth feature extraction module, a semantic segmentation and component recognition module, a structural parameter parsing module, a parametric model generation module, and a model optimization and output module.

[0009] The point cloud acquisition and input module is used to receive 3D point cloud data of buildings from lidar scanning equipment, photogrammetry system or multi-source fusion sensor, and supports the import of standard format point cloud files and real-time data stream access.

[0010] The point cloud preprocessing module is connected to the point cloud acquisition and input module and is used for point cloud denoising, downsampling, coordinate normalization and scene coarse segmentation operations. It can also remove non-building point clouds and extract the main area of ​​the building to be modeled, and output the preprocessed 3D point cloud data.

[0011] The deep feature extraction module is built on a multi-layer graph convolutional neural network architecture and includes an iterative local feature aggregation unit and a global context awareness unit. The preprocessed 3D point cloud data is input into the deep feature extraction module. Through the iterative local feature aggregation unit and the global context awareness unit, high-dimensional semantic feature vectors are learned point by point, and the processed 3D point cloud data is output.

[0012] The semantic segmentation and component recognition module receives the processed 3D point cloud data, optimizes the point-level classification results using a fully connected conditional random field, achieves accurate segmentation and semantic annotation of core building components such as walls, roofs, doors, windows, and columns, and outputs labeled point cloud partitions.

[0013] The structural parameter parsing module is connected to the semantic segmentation and component recognition module. Based on the geometric distribution characteristics of the labeled point cloud, it automatically fits the plane equation, calculates the component size parameters and spatial pose, identifies the topological connection relationship of the building components, and outputs point cloud data with connection relationship.

[0014] The parametric model generation module uses the geometric constraints and topological relationships of point cloud data with connectivity to call a predefined building component template library for instantiation matching, and automatically generates a three-dimensional building entity model with parametric attributes through boundary representation.

[0015] Preferably, the point cloud acquisition and input module includes a multi-source data fusion subunit, used to align timestamps and spatial coordinate systems to achieve dynamic registration and complementary enhancement of laser point clouds and oblique photography point clouds.

[0016] Preferably, the point cloud preprocessing module integrates a noise filtering algorithm based on statistical outlier removal and a voxel grid downsampling algorithm, and uses Euclidean clustering to separate building and ground point clouds in scene segmentation.

[0017] Preferably, the deep feature extraction module adopts a multi-layer graph neural network architecture, specifically including:

[0018] Local feature encoding layer: Constructs a dynamic graph structure through K-nearest neighbors and extracts local geometric features using edge convolution operators;

[0019] Hierarchical feature propagation layer: uses skip connections to fuse multi-scale feature maps to preserve detailed information;

[0020] Global attention pooling layer: aggregates scene-level semantic features and enhances long-distance dependency modeling capabilities.

[0021] Preferably, the multi-layer graph neural network architecture performs cloud feature learning through the following steps:

[0022] (a) Construct a local spatial graph structure for the input point cloud, and generate edge feature vectors representing local geometric relationships by calculating the relative coordinate difference and Euclidean distance between each point and its K nearest neighbors;

[0023] (b) An attention weight coefficient is used to adaptively weight and fuse the edge feature vectors of neighboring points, where the weights are jointly determined by the geometric relationship between points and feature similarity, thereby enhancing the features of key local structures.

[0024] (c) The hierarchical feature map is downsampled by max pooling to gradually compress the point cloud resolution and expand the receptive field, while preserving significant structural features at different scales;

[0025] (d) In the decoding stage, skip connections are used to fuse the high-resolution detail features of the encoder with the deep semantic features of the decoder to achieve sub-pixel level accuracy reconstruction of point cloud segmentation boundaries.

[0026] Preferably, the semantic segmentation and component recognition module introduces an iterative conditional random field optimization algorithm after point-level classification. By defining spatial continuity constraints and semantic compatibility constraints, it corrects isolated erroneous labeled points and improves the accuracy of segmentation boundaries.

[0027] Preferably, the structural parameter parsing module includes:

[0028] Plane Fitting Submodule: Extracts principal plane equations from semantic point clouds based on the random sampling consensus algorithm;

[0029] Topology reasoning submodule: Constructs a connection diagram of building components by analyzing the angles, distances, and bounding box overlaps between components;

[0030] Parameter calculation submodule: Automatically calculates component dimensions, angles, and spatial coordinates and generates a parameterized description file.

[0031] Preferably, the parametric model generation module has a built-in extensible building component template library. The templates contain parameter-driven geometric generation rules and dynamically instantiate standardized building component models that conform to actual dimensions by binding the output data of the structural parameter parsing module.

[0032] Preferably, it also includes a model optimization and output module, which is used to perform hole repair, edge smoothing and lightweighting processing on the generated 3D model, supports output as BIM format, OBJ mesh model or point cloud augmented model, and provides a visual interactive interface for model verification and editing.

[0033] Compared with the prior art, this application has at least the following beneficial effects:

[0034] This invention achieves full automation of the 3D modeling process of buildings by deeply integrating point cloud deep learning networks with a parametric modeling engine;

[0035] The use of the deep feature extraction module can effectively improve the accuracy of the recognition of structures such as the edges of doors, windows, and roofs, ensuring the accuracy of the building model. As the system is used continuously, the deep feature extraction module can continuously improve the recognition accuracy.

[0036] The use of the semantic segmentation and component recognition module can identify and distinguish the irregularity, structural complexity and occlusion noise of building point clouds, solving the problem that existing technologies require manual semantic annotation and topology repair. Attached Figure Description

[0037] Figure 1 This is a block diagram of the automated modeling system for 3D point cloud buildings based on deep learning proposed in this application. Detailed Implementation

[0038] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] like Figure 1 As shown, this application discloses an automated modeling system for 3D point cloud buildings based on deep learning, including a point cloud acquisition and input module, a point cloud preprocessing module, a depth feature extraction module, a semantic segmentation and component recognition module, a structural parameter parsing module, a parameterized model generation module, and a model optimization and output module;

[0040] The point cloud acquisition and input module is used to receive 3D point cloud data of buildings from lidar scanning equipment, photogrammetry system or multi-source fusion sensor, and supports the import of standard format point cloud files and real-time data stream access.

[0041] The point cloud preprocessing module is connected to the point cloud acquisition and input module and is used for point cloud denoising, downsampling, coordinate normalization and scene coarse segmentation operations. It can also remove non-building point clouds and extract the main area of ​​the building to be modeled, and output the preprocessed 3D point cloud data.

[0042] The deep feature extraction module is built on a multi-layer graph convolutional neural network architecture and includes an iterative local feature aggregation unit and a global context awareness unit. The preprocessed 3D point cloud data is input into the deep feature extraction module. Through the iterative local feature aggregation unit and the global context awareness unit, high-dimensional semantic feature vectors are learned point by point, and the processed 3D point cloud data is output.

[0043] The semantic segmentation and component recognition module receives the processed 3D point cloud data, optimizes the point-level classification results using a fully connected conditional random field, achieves accurate segmentation and semantic annotation of core building components such as walls, roofs, doors, windows, and columns, and outputs labeled point cloud partitions.

[0044] The structural parameter parsing module is connected to the semantic segmentation and component recognition module. Based on the geometric distribution characteristics of the labeled point cloud, it automatically fits the plane equation, calculates the component size parameters and spatial pose, identifies the topological connection relationship of the building components, and outputs point cloud data with connection relationship.

[0045] The parametric model generation module uses the geometric constraints and topological relationships of point cloud data with connectivity to call a predefined building component template library for instantiation matching, and automatically generates a three-dimensional building entity model with parametric attributes through boundary representation.

[0046] This system achieves fully automated conversion from raw point clouds to structured 3D building models using a deep learning architecture. First, the point cloud acquisition and input module receives raw point cloud data from multiple sensors (such as LiDAR and oblique photography), ensuring data access and format compatibility. The point cloud preprocessing unit then performs key data cleaning functions: filtering out environmental noise points, compressing data volume, and separating the main building point cloud. A deep feature extraction network, based on a graph neural network, intelligently encodes the cleaned point cloud features, generating high-dimensional feature vectors through local geometric perception and global semantic association. The semantic segmentation and component recognition module utilizes these features to achieve core functions: accurately labeling point cloud partitions for building components such as walls, roofs, doors, and windows. The structural parameter parsing engine analyzes the geometric distribution and spatial relationships of the labeled point cloud, automatically calculating component dimensions, poses, and topological connection rules. The parametric model generation unit, based on the parsing results, calls a prefabricated template library to generate parametrically driven 3D solid components and automatically assembles them into a complete building model. Finally, the model optimization and output interface performs model lightweighting and format conversion functions, outputting a BIM or mesh model that can be directly applied to engineering scenarios.

[0047] The point cloud acquisition and input module includes a multi-source data fusion subunit, which is used to align timestamps and spatial coordinate systems to achieve dynamic registration and complementary enhancement of laser point clouds and oblique photogrammetric point clouds. It dynamically aligns the spatiotemporal coordinate systems of laser point clouds and photogrammetric point clouds, and uses a registration algorithm to fuse the geometric accuracy and texture detail advantages of the two types of data to generate an enhanced point cloud dataset to improve the accuracy of subsequent modeling.

[0048] The point cloud preprocessing module integrates a noise filtering algorithm based on statistical outlier removal and a voxel grid downsampling algorithm. It uses voxel grid downsampling to balance data density and processing efficiency. In scene segmentation, it uses Euclidean clustering to separate building and ground point clouds and automatically separates building point clouds from ground / vegetation and other interfering point clouds by using distance thresholds.

[0049] The deep feature extraction module adopts a multi-layer graph neural network architecture, specifically including:

[0050] Local feature encoding layer: Constructs a dynamic graph structure through K-nearest neighbors and extracts local geometric features using edge convolution operators;

[0051] Hierarchical feature propagation layer: uses skip connections to fuse multi-scale feature maps to preserve detailed information;

[0052] Global attention pooling layer: aggregates scene-level semantic features and enhances long-distance dependency modeling capabilities.

[0053] The local feature encoding layer captures detailed geometric features of components through dynamic neighborhood mapping; the hierarchical feature propagation layer integrates semantic information at different scales to preserve details; and the global attention pooling layer enhances the function of modeling long-range dependencies across components, ensuring the contextual consistency of complex structures (such as corridors and domes).

[0054] The multi-layer graph neural network architecture performs cloud feature learning through the following steps:

[0055] (a) Construct a local spatial graph structure for the input point cloud. By calculating the relative coordinate difference and Euclidean distance between each point and its K nearest neighbors, generate edge feature vectors that represent local geometric relationships. This operation can record the features of each unit. For example, the size and orientation of doors and windows can be measured and recorded.

[0056] (b) An attention weight coefficient is used to adaptively weight and fuse the edge feature vectors of neighboring points. The weights are jointly determined by the geometric relationship between points and feature similarity, thereby enhancing the features of key local structures. Through the attention weight coefficient, key parts of the building (such as the edges of doors, windows, and roofs) and irrelevant parts (such as the surrounding environment of the building) can be screened and distinguished.

[0057] (c) The hierarchical feature map is downsampled by max pooling operation to gradually compress the point cloud resolution and expand the receptive field, thus creating a blueprint framework for the building while preserving significant structural features at different scales.

[0058] (d) In the decoding stage, skip connections are used to fuse the high-resolution detail features of the encoder with the deep semantic features of the decoder to achieve sub-pixel level accuracy reconstruction of point cloud segmentation boundaries. That is, the building is modeled by the blueprint framework in c and the unit features in a, thereby realizing the reconstruction of the building.

[0059] The semantic segmentation and component recognition module introduces an iterative conditional random field optimization algorithm after point-level classification. By defining spatial continuity constraints and semantic compatibility constraints, it corrects isolated erroneous labels and improves the accuracy of segmentation boundaries. The above scheme uses conditional random fields to define label compatibility constraints between spatially adjacent points, and improves the continuity and accuracy of component segmentation by iteratively correcting segmentation boundary errors and isolated erroneous labels.

[0060] The structural parameter parsing module includes:

[0061] Plane Fitting Submodule: Extracts principal plane equations from semantic point clouds based on the random sampling consensus algorithm;

[0062] Topology reasoning submodule: Constructs a connection diagram of building components by analyzing the angles, distances, and bounding box overlaps between components;

[0063] Parameter calculation submodule: Automatically calculates component dimensions, angles, and spatial coordinates and generates a parameterized description file.

[0064] The plane fitting submodule can extract principal plane parameters from semantic point clouds; the topology reasoning submodule can analyze the angle / distance relationship between components and then construct a connection graph to describe key information such as walls and roofs; the parameter calculation submodule automatically outputs the component size, position and orientation parameter set to provide geometric constraints for parametric modeling.

[0065] The parametric model generation module has a built-in extensible building component template library. The templates contain parameter-driven geometric generation rules. By binding the output data of the structural parameter parsing module, it can dynamically instantiate standardized building component models that conform to actual dimensions. By binding the size / pose data output by the structural parameter parsing module to predefined parametric templates (such as window frames and roof ridges), it can dynamically instantiate standard components that conform to actual geometric constraints, and then assemble them into an overall building model.

[0066] It also includes a model optimization and output module, which is used to perform hole repair, edge smoothing and lightweighting on the generated 3D model, so that the produced model is more in line with actual requirements. It supports outputting in BIM format, OBJ mesh model or point cloud augmented model, and provides a visual interactive interface for model verification and editing.

[0067] The technical features of the above embodiments can be combined in any way (as long as there is no contradiction in the combination of these technical features). For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; these embodiments not explicitly written should also be considered to be within the scope of this specification.

Claims

1. A deep learning-based automated modeling system for 3D point cloud buildings, characterized in that, It includes a point cloud acquisition and input module, a point cloud preprocessing module, a deep feature extraction module, a semantic segmentation and component recognition module, a structural parameter parsing module, and a parameterized model generation module; The point cloud acquisition and input module is used to receive 3D point cloud data of buildings from lidar scanning equipment, photogrammetry system or multi-source fusion sensor, and supports the import of standard format point cloud files and real-time data stream access. The point cloud preprocessing module is connected to the point cloud acquisition and input module and is used for point cloud denoising, downsampling, coordinate normalization and scene coarse segmentation operations. It can also remove non-building point clouds and extract the main area of ​​the building to be modeled, and output the preprocessed 3D point cloud data. The deep feature extraction module is built on a multi-layer graph convolutional neural network architecture and includes an iterative local feature aggregation unit and a global context awareness unit. The preprocessed 3D point cloud data is input into the deep feature extraction module. Through the iterative local feature aggregation unit and the global context awareness unit, high-dimensional semantic feature vectors are learned point by point, and the processed 3D point cloud data is output. The semantic segmentation and component recognition module receives the processed 3D point cloud data, optimizes the point-level classification results using a fully connected conditional random field, achieves accurate segmentation and semantic annotation of core building components such as walls, roofs, doors, windows, and columns, and outputs labeled point cloud partitions. The structural parameter parsing module is connected to the semantic segmentation and component recognition module. Based on the geometric distribution characteristics of the labeled point cloud, it automatically fits the plane equation, calculates the component size parameters and spatial pose, identifies the topological connection relationship of the building components, and outputs point cloud data with connection relationship. The parametric model generation module uses the geometric constraints and topological relationships of point cloud data with connectivity to call a predefined building component template library for instantiation matching, and automatically generates a three-dimensional building entity model with parametric attributes through boundary representation. The deep feature extraction module adopts a multi-layer graph neural network architecture, specifically including: Local feature encoding layer: Constructs a dynamic graph structure through K-nearest neighbors and extracts local geometric features using edge convolution operators; Hierarchical feature propagation layer: uses skip connections to fuse multi-scale feature maps to preserve detailed information; Global attention pooling layer: aggregates scene-level semantic features and enhances long-distance dependency modeling capabilities; The multi-layer graph neural network architecture performs cloud feature learning through the following steps: (a) Construct a local spatial graph structure for the input point cloud, and generate edge feature vectors representing local geometric relationships by calculating the relative coordinate difference and Euclidean distance between each point and its K nearest neighbors; (b) An attention weight coefficient is used to adaptively weight and fuse the edge feature vectors of neighboring points, where the weights are jointly determined by the geometric relationship between points and the feature similarity. (c) The hierarchical feature map is downsampled by max pooling to gradually compress the point cloud resolution and expand the receptive field, while preserving the structural features at different scales; (d) In the decoding stage, skip connections are used to fuse the high-resolution detail features of the encoder with the deep semantic features of the decoder to achieve sub-pixel level accuracy reconstruction of point cloud segmentation boundaries.

2. The automated modeling system for 3D point cloud buildings based on deep learning according to claim 1, characterized in that, The point cloud acquisition and input module includes a multi-source data fusion subunit, which is used to align timestamps and spatial coordinate systems to achieve dynamic registration and complementary enhancement of laser point clouds and oblique photography point clouds.

3. The automated modeling system for 3D point cloud buildings based on deep learning according to claim 1, characterized in that, The point cloud preprocessing module integrates a noise filtering algorithm based on statistical outlier removal and a voxel grid downsampling algorithm, and uses Euclidean clustering to separate building and ground point clouds in scene segmentation.

4. The automated modeling system for 3D point cloud buildings based on deep learning according to claim 1, characterized in that, The semantic segmentation and component recognition module introduces an iterative conditional random field optimization algorithm after point-level classification. By defining spatial continuity constraints and semantic compatibility constraints, it corrects isolated erroneous labeled points and improves the accuracy of segmentation boundaries.

5. The automated modeling system for 3D point cloud buildings based on deep learning according to claim 1, characterized in that, The structural parameter parsing module includes: Plane Fitting Submodule: Extracts principal plane equations from semantic point clouds based on the random sampling consensus algorithm; Topology reasoning submodule: Constructs a connection diagram of building components by analyzing the angles, distances, and bounding box overlaps between components; Parameter calculation submodule: Automatically calculates component dimensions, angles, and spatial coordinates and generates a parameterized description file.

6. The automated modeling system for 3D point cloud buildings based on deep learning according to claim 1, characterized in that, The parametric model generation module has a built-in extensible building component template library. The templates contain parameter-driven geometric generation rules and dynamically instantiate standardized building component models that conform to actual dimensions by binding the output data of the structural parameter parsing module.

7. The automated modeling system for 3D point cloud buildings based on deep learning according to claim 1, characterized in that, It also includes a model optimization and output module, which is used to perform hole repair, edge smoothing and lightweighting processing on the generated 3D model. It supports outputting in BIM format, OBJ mesh model or point cloud augmented model, and provides a visual interactive interface for model verification and editing.

Images