Three-dimensional point cloud building automatic modeling system based on deep learning

Abstract

The invention discloses a three-dimensional point cloud building automatic modeling system based on deep learning. The system comprises a point cloud acquisition and input module, a point cloud preprocessing module, a depth feature extraction module, a semantic segmentation and component identification module, a structure parameter analysis module and a parameterized model generation module. According to the invention, through deep fusion of the point cloud deep learning network and the parametric modeling engine, full-process automation of building three-dimensional modeling is realized; the depth feature extraction module is used, so that the recognition accuracy of structures such as doors, windows and roof edges can be effectively improved, the precision of a building model is ensured, and the depth feature extraction module can continuously improve the recognition accuracy along with continuous use of the system; and by using the semantic segmentation and component identification module, the irregularity of the building point cloud, the complexity of the structure and the shielding noise can be identified and distinguished, and the problem that semantic annotation and topological repair need to be manually performed in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the technical field of automated modeling systems, and specifically to a three-dimensional point cloud building automated modeling system based on deep learning. Background Art

[0002] In the prior art, when performing three-dimensional modeling of buildings, the following two technical means are mostly used:

[0003] First, modeling is done manually, which is not only inefficient but also costly.

[0004] Secondly, modeling is carried out through laser scanning and photogrammetry technology. For example, in the Chinese invention patent with application number CN202310783999.0, a method and system for automatic building modeling based on point cloud data is disclosed (application date is 2023-06-29, publication date is 2023-12-26). When it is implemented, although it achieves the technical effect of improving the efficiency of building modeling based on point cloud data while ensuring the restoration of the point cloud model to the building entity and providing a method for reducing the manual participation in the modeling process, it still has the problem of being difficult to effectively distinguish the various complex structures in the point cloud (such as windows, roofs, and ancillary facilities), and requires manual intervention for semantic labeling and topological repair.

[0005] Therefore, there is an urgent need for a deep learning-based 3D point cloud building automated modeling system. Summary of the Invention

[0006] To this end, this application provides a deep learning-based three-dimensional point cloud building automatic modeling system to solve the problems existing in the existing technology, such as difficulty in effectively distinguishing complex structures in point clouds, sensitivity to noise and occlusion, and the need for manual intervention for semantic labeling and topological repair.

[0007] In order to achieve the above objectives, this application provides the following technical solutions:

[0008] A deep learning-based 3D point cloud building automated modeling system, including a point cloud acquisition and input module, a point cloud preprocessing module, a deep feature extraction module, a semantic segmentation and component identification module, a structural parameter analysis module, a parametric model generation module, and a model optimization and output module;

[0009] The point cloud acquisition and input module is used to receive three-dimensional point cloud data of buildings from laser radar scanning equipment, photogrammetry systems or multi-source fusion sensors, and supports the import of standard format point cloud files and real-time data stream access;

[0010] The point cloud preprocessing module is connected to the point cloud acquisition and input module and is used for point cloud denoising, downsampling, coordinate normalization and scene coarse segmentation operations. It can also remove non-building object point clouds and extract the main area of ​​the building to be modeled, and output the preprocessed three-dimensional point cloud data;

[0011] The deep feature extraction module is built on a multi-layer graph convolutional neural network architecture and includes an iterative local feature aggregation unit and a global context perception unit. The pre-processed 3D point cloud data is input into the deep feature extraction module. Through the iterative local feature aggregation unit and the global context perception unit, the high-dimensional semantic feature vector is learned point by point, and the processed 3D point cloud data is output.

[0012] The semantic segmentation and component identification module receives the processed 3D point cloud data and uses a fully connected conditional random field to optimize the point-level classification results to achieve accurate segmentation and semantic labeling of core building components such as walls, roofs, doors, windows, and columns, and outputs labeled point cloud partitions.

[0013] The structural parameter analysis module is connected to the semantic segmentation and component identification module to automatically fit the plane equation, calculate the component size parameters and spatial posture according to the geometric distribution characteristics of the annotated point cloud, identify the topological connection relationship of the building components, and output the point cloud data with the connection relationship;

[0014] The parametric model generation module calls a predefined building component template library for instantiation matching based on the geometric constraints and topological relationships of the point cloud data with connection relationships, and automatically generates a three-dimensional building entity model with parametric properties through a boundary representation method.

[0015] Preferably, the point cloud acquisition and input module includes a multi-source data fusion subunit for aligning timestamps with spatial coordinate systems to achieve dynamic registration and complementary enhancement of laser point clouds and oblique photography point clouds.

[0016] Preferably, the point cloud preprocessing module integrates a noise filtering algorithm based on statistical outlier removal and a voxel grid downsampling algorithm, and adopts Euclidean clustering method to separate building and ground point clouds in scene segmentation.

[0017] Preferably, the deep feature extraction module adopts a layered graph neural network architecture, specifically including:

[0018] Local feature encoding layer: constructs a dynamic graph structure through K-nearest neighbors and uses edge convolution operators to extract local geometric features;

[0019] Hierarchical feature propagation layer: uses skip connections to fuse multi-scale feature maps to preserve detail information;

[0020] Global attention pooling layer: Aggregates scene-level semantic features and enhances the ability to model long-range dependencies.

[0021] Preferably, the hierarchical graph neural network architecture performs cloud feature learning through the following steps:

[0022] (a) Construct a local spatial graph structure for the input point cloud. By calculating the relative coordinate difference and Euclidean distance between each point and its K nearest neighbors, an edge feature vector representing the local geometric relationship is generated.

[0023] (b) Adopting attention weight coefficients, the edge feature vectors of neighboring points are adaptively weighted and fused, where the weights are determined jointly by the geometric relationship between points and feature similarity, thus enhancing the features of key local structures.

[0024] (c) Downsampling the hierarchical feature maps through the maximum pooling operation, gradually compressing the point cloud resolution and expanding the receptive field while retaining salient structural features at different scales;

[0025] (d) In the decoding stage, skip connections are used to fuse the high-resolution detail features of the encoder with the deep semantic features of the decoder to achieve sub-pixel accuracy reconstruction of point cloud segmentation boundaries.

[0026] Preferably, the semantic segmentation and component identification module introduces an iterative conditional random field optimization algorithm after point-level classification, and corrects isolated incorrectly labeled points and improves segmentation boundary accuracy by defining spatial continuity constraints and semantic compatibility constraints.

[0027] Preferably, the structural parameter analysis module includes:

[0028] Plane fitting submodule: extracts the principal plane equation from the semantic point cloud based on the random sampling consistency algorithm;

[0029] Topology reasoning submodule: Constructs a building component connection diagram by analyzing the angles, distances, and bounding box overlaps between components;

[0030] Parameter calculation submodule: automatically calculates component dimensions, angles and spatial coordinates and generates parametric description files.

[0031] Preferably, the parametric model generation module has a built-in expandable building component template library, and the template contains parameter-driven geometry generation rules. By binding the output data of the structural parameter parsing module, the standardized building component model that conforms to the actual size is dynamically instantiated.

[0032] Preferably, it also includes a model optimization and output module, which is used to perform hole repair, edge smoothing and lightweight processing on the generated three-dimensional model, supports output in BIM format, OBJ mesh model or point cloud enhanced model, and provides a visual interactive interface for model verification and editing.

[0033] Compared with the prior art, this application has at least the following beneficial effects:

[0034] This invention realizes the full process automation of building 3D modeling by deeply integrating point cloud deep learning network and parametric modeling engine;

[0035] The use of the deep feature extraction module can effectively improve the recognition accuracy of structures such as doors, windows, and roof edges, ensuring the accuracy of the building model. As the system is used continuously, the deep feature extraction module can continuously improve the recognition accuracy.

[0036] The use of semantic segmentation and component recognition modules can identify and distinguish the irregularities, structural complexity and occlusion noise of building point clouds, solving the problem of manual semantic labeling and topology repair in existing technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a module diagram of the deep learning-based 3D point cloud building automated modeling system of this application. DETAILED DESCRIPTION

[0038] The present application will be further described below in detail through specific embodiments in conjunction with the accompanying drawings.

[0039] like Figure 1 As shown, the present application discloses a three-dimensional point cloud building automatic modeling system based on deep learning, including a point cloud acquisition and input module, a point cloud preprocessing module, a deep feature extraction module, a semantic segmentation and component recognition module, a structural parameter analysis module, a parametric model generation module, and a model optimization and output module;

[0040] The point cloud acquisition and input module is used to receive three-dimensional point cloud data of buildings from laser radar scanning equipment, photogrammetry systems or multi-source fusion sensors, and supports the import of standard format point cloud files and real-time data stream access;

[0041] The point cloud preprocessing module is connected to the point cloud acquisition and input module and is used for point cloud denoising, downsampling, coordinate normalization and scene coarse segmentation operations. It can also remove non-building object point clouds and extract the main area of ​​the building to be modeled, and output the preprocessed three-dimensional point cloud data;

[0042] The deep feature extraction module is built on a multi-layer graph convolutional neural network architecture and includes an iterative local feature aggregation unit and a global context perception unit. The pre-processed 3D point cloud data is input into the deep feature extraction module. Through the iterative local feature aggregation unit and the global context perception unit, the high-dimensional semantic feature vector is learned point by point, and the processed 3D point cloud data is output.

[0043] The semantic segmentation and component identification module receives the processed 3D point cloud data and uses a fully connected conditional random field to optimize the point-level classification results to achieve accurate segmentation and semantic labeling of core building components such as walls, roofs, doors, windows, and columns, and outputs labeled point cloud partitions.

[0044] The structural parameter analysis module is connected to the semantic segmentation and component identification module to automatically fit the plane equation, calculate the component size parameters and spatial posture according to the geometric distribution characteristics of the annotated point cloud, identify the topological connection relationship of the building components, and output the point cloud data with the connection relationship;

[0045] The parametric model generation module calls a predefined building component template library for instantiation matching based on the geometric constraints and topological relationships of the point cloud data with connection relationships, and automatically generates a three-dimensional building entity model with parametric properties through a boundary representation method.

[0046] This system utilizes a deep learning architecture to fully automatically convert raw point clouds into structured 3D building models. The point cloud acquisition and input module first receives raw point cloud data from multiple sensors (such as LiDAR and oblique photography), ensuring data access and format compatibility. The point cloud preprocessing unit then performs key data cleaning functions: filtering out environmental noise points, compressing the data, and isolating the building's main point cloud. A deep feature extraction network, based on a graph neural network, intelligently encodes features from the cleaned point cloud, generating high-dimensional feature vectors through local geometric perception and global semantic association. The semantic segmentation and component identification module leverages these features to implement core functions: accurately annotating point cloud partitions for building components such as walls, roofs, doors, and windows. The structural parameter parsing engine analyzes the geometric distribution and spatial relationships of the annotated point cloud, automatically calculating component dimensions, poses, and topological connection rules. Based on the parsing results, the parametric model generation unit invokes a library of prefabricated templates to generate parameterized 3D solid components and automatically assemble them into a complete building model. Finally, the model optimization and output interface performs model lightweighting and format conversion, outputting a BIM or mesh model that can be directly applied in engineering scenarios.

[0047] The point cloud acquisition and input module includes a multi-source data fusion subunit, which is used to align the timestamp and spatial coordinate system, realize the dynamic registration and complementary enhancement of the laser point cloud and the oblique photography point cloud, dynamically align the spatiotemporal coordinate system of the laser point cloud and the photography point cloud, and fuse the geometric accuracy and texture detail advantages of the two types of data through the registration algorithm to generate an enhanced point cloud dataset to improve the subsequent modeling accuracy.

[0048] The point cloud preprocessing module integrates a noise filtering algorithm based on statistical outlier removal and a voxel grid downsampling algorithm. It balances data density and processing efficiency through voxel grid downsampling. In scene segmentation, it uses the Euclidean clustering method to separate building and ground point clouds, and automatically separates building point clouds from interfering point clouds such as ground / vegetation through distance thresholds.

[0049] The deep feature extraction module adopts a layered graph neural network architecture, specifically including:

[0050] Local feature encoding layer: constructs a dynamic graph structure through K-nearest neighbors and uses edge convolution operators to extract local geometric features;

[0051] Hierarchical feature propagation layer: uses skip connections to fuse multi-scale feature maps to preserve detail information;

[0052] Global attention pooling layer: Aggregates scene-level semantic features and enhances the ability to model long-range dependencies.

[0053] The local feature encoding layer captures the detailed geometric features of components through dynamic neighborhood composition; the hierarchical feature propagation layer fuses semantic information of different scales to preserve details; and the global attention pooling layer enhances the modeling function of long-range dependencies across components to ensure the contextual consistency of complex structures (such as corridors and domes).

[0054] The hierarchical graph neural network architecture performs cloud feature learning through the following steps:

[0055] (a) Construct a local spatial graph structure for the input point cloud. By calculating the relative coordinate difference and Euclidean distance between each point and its K nearest neighbors, an edge feature vector representing the local geometric relationship is generated. This operation can record the characteristics of each unit, for example, the size and orientation of doors and windows can be measured and recorded.

[0056] (b) Adopting attention weight coefficients, the edge feature vectors of neighboring points are adaptively weighted and fused. The weights are determined by the geometric relationship between the points and the feature similarity. This enhances the features of key local structures. The attention weight coefficients can be used to screen and distinguish between key parts of a building (such as doors, windows, and roof edges) and irrelevant parts (such as the surrounding environment).

[0057] (c) Downsampling the hierarchical feature maps through a max-pooling operation gradually compresses the point cloud resolution and expands the receptive field to produce a blueprint framework of the building while preserving salient structural features at different scales;

[0058] (d) In the decoding stage, skip connections are used to fuse the high-resolution detail features of the encoder with the deep semantic features of the decoder to achieve sub-pixel accuracy reconstruction of the point cloud segmentation boundaries. That is, the building is modeled through the blueprint framework in c and the unit features in a, and then the building is restored.

[0059] The semantic segmentation and component identification module introduces an iterative conditional random field optimization algorithm after point-level classification. By defining spatial continuity constraints and semantic compatibility constraints, it corrects isolated incorrectly labeled points and improves segmentation boundary accuracy. The above scheme uses conditional random fields to define label compatibility constraints between spatially adjacent points, and iteratively corrects segmentation boundary errors and isolated incorrectly labeled points to improve the continuity and accuracy of component segmentation.

[0060] The structural parameter analysis module includes:

[0061] Plane fitting submodule: extracts the principal plane equation from the semantic point cloud based on the random sampling consistency algorithm;

[0062] Topology reasoning submodule: Constructs a building component connection diagram by analyzing the angles, distances, and bounding box overlaps between components;

[0063] Parameter calculation submodule: automatically calculates component dimensions, angles and spatial coordinates and generates parametric description files.

[0064] The plane fitting submodule can extract the main plane parameters from the semantic point cloud; the topology reasoning submodule can analyze the angle / distance relationship between components and then construct a connection diagram to describe key information such as walls and roofs; the parameter calculation submodule automatically outputs the component size, position and orientation parameter sets, providing geometric constraints for parametric modeling.

[0065] The parametric model generation module has a built-in expandable building component template library. The template contains parameter-driven geometric generation rules. By binding the output data of the structural parameter parsing module, standardized building component models that conform to actual dimensions are dynamically instantiated. By binding the size / pose data output by the structural parameter parsing module to predefined parametric templates (such as window frames and roof ridges), standard components that conform to actual geometric constraints can be dynamically instantiated and then assembled into an overall building model.

[0066] It also includes a model optimization and output module, which is used to perform hole repair, edge smoothing and lightweight processing on the generated three-dimensional model, so that the produced model is more in line with actual requirements. It supports output in BIM format, OBJ mesh model or point cloud enhanced model, and provides a visual interactive interface for model verification and editing.

[0067] The technical features of the above embodiments can be combined arbitrarily (as long as there is no contradiction in the combination of these technical features). In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described; these embodiments that are not explicitly written should also be considered to be within the scope of this specification.

Claims

1. A 3D point cloud building automated modeling system based on deep learning, characterized by: It includes point cloud acquisition and input module, point cloud preprocessing module, deep feature extraction module, semantic segmentation and component identification module, structural parameter analysis module and parametric model generation module; The point cloud acquisition and input module is used to receive three-dimensional point cloud data of buildings from laser radar scanning equipment, photogrammetry systems or multi-source fusion sensors, and supports the import of standard format point cloud files and real-time data stream access; The point cloud preprocessing module is connected to the point cloud acquisition and input module and is used for point cloud denoising, downsampling, coordinate normalization and scene coarse segmentation operations. It can also remove non-building object point clouds and extract the main area of ​​the building to be modeled, and output the preprocessed three-dimensional point cloud data; The deep feature extraction module is built on a multi-layer graph convolutional neural network architecture and includes an iterative local feature aggregation unit and a global context perception unit. The pre-processed 3D point cloud data is input into the deep feature extraction module. Through the iterative local feature aggregation unit and the global context perception unit, the high-dimensional semantic feature vector is learned point by point, and the processed 3D point cloud data is output. The semantic segmentation and component identification module receives the processed 3D point cloud data and uses a fully connected conditional random field to optimize the point-level classification results to achieve accurate segmentation and semantic labeling of core building components such as walls, roofs, doors, windows, and columns, and outputs labeled point cloud partitions. The structural parameter analysis module is connected to the semantic segmentation and component identification module to automatically fit the plane equation, calculate the component size parameters and spatial posture according to the geometric distribution characteristics of the annotated point cloud, identify the topological connection relationship of the building components, and output the point cloud data with the connection relationship; The parametric model generation module calls a predefined building component template library for instantiation matching based on the geometric constraints and topological relationships of the point cloud data with connection relationships, and automatically generates a three-dimensional building entity model with parametric properties through a boundary representation method.

2. The deep learning-based 3D point cloud building automated modeling system according to claim 1, characterized in that: The point cloud acquisition and input module includes a multi-source data fusion subunit, which is used to align the timestamp and the spatial coordinate system to achieve dynamic registration and complementary enhancement of the laser point cloud and the oblique photography point cloud.

3. The deep learning-based 3D point cloud building automated modeling system according to claim 1, characterized in that: The point cloud preprocessing module integrates a noise filtering algorithm based on statistical outlier removal and a voxel grid downsampling algorithm, and uses the Euclidean clustering method to separate building and ground point clouds in scene segmentation.

4. The deep learning-based 3D point cloud building automated modeling system according to claim 1, characterized in that: The deep feature extraction module adopts a layered graph neural network architecture, specifically including: Local feature encoding layer: constructs a dynamic graph structure through K-nearest neighbors and uses edge convolution operators to extract local geometric features; Hierarchical feature propagation layer: uses skip connections to fuse multi-scale feature maps to preserve detail information; Global attention pooling layer: Aggregates scene-level semantic features and enhances the ability to model long-range dependencies.

5. The deep learning-based 3D point cloud building automated modeling system according to claim 4, characterized in that: The hierarchical graph neural network architecture performs cloud feature learning through the following steps: (a) Construct a local spatial graph structure for the input point cloud. By calculating the relative coordinate difference and Euclidean distance between each point and its K nearest neighbors, an edge feature vector representing the local geometric relationship is generated. (b) Adopting the attention weight coefficient, the edge feature vectors of the neighboring points are adaptively weighted and fused, where the weight is determined by the geometric relationship between the points and the feature similarity; (c) Downsampling the hierarchical feature maps through the maximum pooling operation, gradually compressing the point cloud resolution and expanding the receptive field while preserving structural features at different scales; (d) In the decoding stage, skip connections are used to fuse the high-resolution detail features of the encoder with the deep semantic features of the decoder to achieve sub-pixel accuracy reconstruction of point cloud segmentation boundaries.

6. The deep learning-based 3D point cloud building automated modeling system according to claim 1, characterized in that: The semantic segmentation and component identification module introduces an iterative conditional random field optimization algorithm after point-level classification. By defining spatial continuity constraints and semantic compatibility constraints, it corrects isolated incorrectly labeled points and improves the segmentation boundary accuracy.

7. The deep learning-based 3D point cloud building automated modeling system according to claim 1, characterized in that: The structural parameter analysis module includes: Plane fitting submodule: extracts the principal plane equation from the semantic point cloud based on the random sampling consistency algorithm; Topology reasoning submodule: Constructs a building component connection diagram by analyzing the angles, distances, and bounding box overlaps between components; Parameter calculation submodule: automatically calculates component dimensions, angles and spatial coordinates and generates parametric description files.

8. The deep learning-based 3D point cloud building automated modeling system according to claim 1, characterized in that: The parametric model generation module has a built-in expandable building component template library. The template contains parameter-driven geometric generation rules. By binding the output data of the structural parameter parsing module, it dynamically instantiates standardized building component models that conform to actual sizes.

9. The deep learning-based 3D point cloud building automated modeling system according to claim 1, characterized in that: It also includes a model optimization and output module, which is used to perform hole repair, edge smoothing and lightweight processing on the generated three-dimensional model, supports output in BIM format, OBJ mesh model or point cloud enhanced model, and provides a visual interactive interface for model verification and editing.

Images