An underground garage model reconstruction method and system based on point cloud deep learning

Abstract

The application discloses a kind of underground garage model reconstruction method and system based on point cloud deep learning, the method of the application includes: first, the point cloud dataset of underground garage is constructed, including underground garage point cloud data and the semantic category corresponding to each point.Then construct underground garage point cloud semantic segmentation deep learning neural network, on the basis of PointNet++, introduce RES-SE, multi-head attention feature aggregation MAFA and DMAFA module two modules, to realize the effective coding and decoding of point cloud data.Data set is used to train the neural network, and obtain the underground garage point cloud semantic segmentation model.The model can identify the components in the garage, such as pipe, wall, fire hydrant, etc.The method of the application has high efficiency and accuracy when processing complex underground garage point cloud data, can automatically complete the semantic segmentation of underground garage scene, and carry out three-dimensional model reconstruction, suitable for underground garage intelligent operation and building information model system.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of civil engineering and artificial intelligence interaction, and particularly relates to an underground garage model reconstruction method and system based on point cloud deep learning. BACKGROUND

[0002] The underground garage is an important part of the building, which undertakes the functions of parking and accommodating mechanical, electrical and pipeline systems, and needs to be maintained regularly. The three-dimensional model reconstruction based on point cloud can obtain the BIM model of the underground garage, thereby effectively promoting the operation and maintenance of the underground garage. The first step of three-dimensional model reconstruction is semantic segmentation. The existing point cloud semantic segmentation method usually relies on prior knowledge, and the accuracy is low when dealing with complex underground garage scenes. In order to improve the efficiency and accuracy of point cloud semantic segmentation, a new automatic processing method is urgently needed. SUMMARY

[0003] The application provides an underground garage model reconstruction method and system based on point cloud deep learning, which can process complex underground garage scenes, not only generate three-dimensional digital twin models of the garage, but also facilitate subsequent facility management, monitoring and maintenance work of the underground garage.

[0004] To solve the above technical problems, the application provides the following technical scheme: an underground garage model reconstruction method based on point cloud deep learning, comprising the following steps:

[0005] S1, preprocessing, data labeling and enhancing the point cloud data of the underground garage, constructing the point cloud data set of the underground garage, including the underground garage point cloud data and the semantic category corresponding to each point; the point cloud data is the XYZ coordinates and RGB color information of each point;

[0006] S2, constructing an underground garage point cloud semantic segmentation neural network, training the neural network with the point cloud data set of the underground garage in step S1, and obtaining an underground garage point cloud semantic segmentation model;

[0007] The underground garage point cloud semantic segmentation neural network is constructed based on the PointNet++ network, and two modules of RES-SE, multi-head attention feature aggregation MAFA and DMAFA are introduced;

[0008] The training process includes an encoding stage and a decoding stage. In the encoding stage, a plurality of RES-SE layers, multi-head attention feature aggregation MAFA and DMAFA modules are used to extract point-by-point features and capture local geometric structures, and the number of points in the point cloud is reduced layer by layer while the number of point-by-point feature channels is increased. In the decoding stage, the spatial information is restored through the KNN algorithm and the RES-Attention module, and the semantic category of each point is predicted;

[0009] S3, collecting the underground garage point cloud data, inputting to the underground garage point cloud semantic segmentation model, obtaining the semantic category of each point, and further establishing an underground garage three-dimensional model.

[0010] Further, in the aforementioned step S1, the preprocessing of the point cloud data of the underground garage includes removing noise points and abnormal points.

[0011] Further, in the aforementioned step S1, the data labeling and enhancement include color random discarding, rotation, noise addition, scaling, and labeling of different objects of the underground garage using a three-dimensional labeling tool, and the semantic category corresponding to each point includes walls, floors, vehicles, columns, and pipes.

[0012] Further, in the aforementioned encoding stage, first, FC layers and RES-SE layers are used to learn point-by-point features and extract local context features, and the shape of the input point cloud data is transformed from (N, din) to (N, 64), and then four encoding layers are used to reduce the number of points in the point cloud data while increasing the number of point-by-point feature channels to obtain local geometric structures.

[0013] Each encoding layer includes two RES-SE layers with the same structure, and the point cloud data is downsampled by four times the segmentation ratio, and the point-by-point feature channel is increased by two times the multiplication ratio layer by layer.

[0014] Further, the aforementioned decoding stage uses four decoding layers to restore spatial information, each decoding layer uses a KNN algorithm to find the nearest neighbor point for each query point, then uses a nearest neighbor interpolation method to upsample the point feature set, and each decoding layer combines a RES-Attention module, and the interpolated features are connected with the jump connection point features from the encoding layer and the output features of the previous decoding layer.

[0015] Further, the aforementioned each encoding layer includes two RES-SE layers with the same structure, one of which inputs a MAFA module as the core, and the other inputs a DMAFA module as the core.

[0016] Further, the aforementioned underground garage point cloud semantic segmentation neural network outputs the predicted semantic category of all points, represented as (N, dout), where dout is the number of semantic categories, and the output layer uses two shared FC layers to transform the features from (N, 128) to (N, 64), and then from (N, 64) to (N, dout), while adding a dropout layer in the conversion process to prevent overfitting.

[0017] Another aspect of the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any one of the present application when executing the computer program.

[0018] The present application also provides a computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the steps of the method of any one of the present application.

[0019] Compared with the prior art, the present application has the following beneficial technical effects by using the above technical solutions: by using the deep learning method, the present application greatly improves the semantic segmentation accuracy and efficiency of underground garage point cloud data, and significantly reduces manual intervention and human operation time. Using the trained model, large-scale point cloud data can be quickly and efficiently processed to generate accurate three-dimensional models, supporting the construction, monitoring and maintenance of underground garages. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a method flowchart of the present application.

[0021] Figure 2 is a deep learning model GarageNet network architecture of the present application.

[0022] Figure 3 is a semantic segmentation result schematic diagram of the present application, in which (a) is a three-dimensional view of the semantic segmentation result, and (b) is a semantic segmentation top view. DETAILED DESCRIPTION

[0023] In order to better understand the technical content of the present application, specific embodiments are described below with reference to the accompanying drawings.

[0024] Aspects of the present application are described in the present application with reference to the accompanying drawings, which show many illustrative embodiments. The embodiments of the present application are not limited to the drawings described. It should be understood that the present application is implemented by any one of the above-described various concepts and embodiments, and the concepts and embodiments described in detail below, since the disclosed concepts and embodiments of the present application are not limited to any implementation. In addition, some aspects of the present application can be used alone or in any suitable combination with other aspects of the present application.

[0025] Reference Figure 1 The present application provides a point cloud deep learning-based underground garage model reconstruction method, comprising the following steps:

[0026] S1, the point cloud data of the underground garage is preprocessed, data labeled and enhanced, and the point cloud data set of the underground garage is constructed, including underground garage point cloud data and the semantic category corresponding to each point; the point cloud data is the XYZ coordinate of each point, RGB color information;

[0027] As a preferred embodiment of the present application, the point cloud data of the underground garage can be obtained using a laser scanner such as Leica RTC360. The collected point cloud data is preprocessed, including noise point removal and outlier rejection. In addition, the point cloud data is down-sampled, and the point cloud data is simplified to a reasonable size using a voxel down-sampling method while retaining key geometric features, facilitating subsequent processing. The preprocessed point cloud data is stored in a standard format such as.pcd or.las for subsequent processing.

[0028] The preprocessed point cloud data is manually labeled, and different objects in the underground garage, such as walls, floors, vehicles, columns, pipes, etc. are semantically classified and given corresponding labels. In the data labeling process, a three-dimensional labeling tool (such as CloudCompare) is used to ensure the accuracy of the labeling. In order to improve the generalization ability of the model, data augmentation techniques are used to process the labeled data set, including rotation, translation, scaling, noise addition, etc. to generate diverse training samples. The data-enhanced point cloud data is used for model training and verification.

[0029] S2, construct an underground garage point cloud semantic segmentation neural network, train the neural network with the point cloud data set of the underground garage of step S1, and obtain an underground garage point cloud semantic segmentation model;

[0030] Reference Figure 2 The underground garage point cloud semantic segmentation neural network is based on the PointNet++ network and introduces two modules of RES-SE, multi-head attention feature aggregation MAFA and DMAFA; residual connection (RES) enhances the connection between different layers, and compression and excitation (SE) mechanism strengthens the connection between channels.

[0031] The training process includes an encoding stage and a decoding stage. In the encoding stage, a plurality of RES-SE layers, a multi-head attention feature aggregation (MAFA) module and a DMAFA module are used to extract point-by-point features and capture local geometric structures, and the number of points in the point cloud is reduced layer by layer while the number of point-by-point feature channels is increased. In the decoding stage, spatial information is recovered through a KNN algorithm (k-nearest neighbor classification) and a RES-Attention module, and the semantic category of each point is predicted. The network input is point cloud data with a size of (N, din), where N represents the number of points, and din represents the feature dimension of each point. The MAFA module enhances the ability to capture various aspects of feature representation of point cloud data through multiple attention heads by multi-layer feature extraction and aggregation, combined with multiple input feature types and aggregation methods. The DMAFA module not only extracts features, but also reduces the number of points in the input point cloud through downsampling, resulting in lower output resolution (for example, by doubling the neighbor search radius and subsampling grid size).

[0032] S3, collecting the point cloud data of the underground garage, inputting the point cloud data into the underground garage point cloud semantic segmentation model, obtaining the semantic category of each point, and further establishing a three-dimensional model of the underground garage.

[0033] As shown in Figure 2 As a preferred embodiment of the present application, in the encoder stage, FC layers and RES-SE layers (with MAFA modules as the core) are first used to learn point-by-point features and extract local context features, and the data shape is transformed from (N, din) to (N, 64). Subsequently, four encoding layers are used to reduce the number of points in the point cloud while increasing the number of point-by-point feature channels to capture local geometric structures. Each encoding layer includes two RES-SE layers, one of which inputs with a MAFA module as the core and the other inputs with a DMAFA module as the core. The point cloud is downsampled at a four-fold segmentation ratio, and 25% of the point features are retained at each layer (i.e., N to N / 4, N / 16, N / 64, N / 256). At the same time, the number of point-by-point feature channels is increased layer by layer with a doubling ratio of two (i.e., 128 to 256, 512 to 1024, 2048). RES-SE is a residual connection and squeeze-and-excitation layer, MAFA is a multi-head attention feature aggregation module, DFAMA is a downsampled multi-head attention feature aggregation module, FC layer is a fully connected layer, US is an up-sampling layer, and MLP is a multi-layer perceptron.

[0034] At the decoder stage, four decoding layers are used to recover the spatial information. Each decoding layer uses the KNN algorithm to find the nearest neighbor for each query point, and then uses the nearest neighbor interpolation method to up-sample the point feature set. To enhance the decoding process, each decoding layer incorporates a RES-Attention module. The interpolated features are concatenated with the skip-connection point features from the encoding layers and the output features of the previous decoding layer. Figure 2 where RES-Attention module is the residual attention module, and skip connection is the skip connection.

[0035] The output of the network is the predicted semantic class for all points, represented as (N, d out ), where d out is the number of semantic classes. In the output layer, two shared FC layers are used to transform the features from (N, 128) to (N, 64), and then from (N, 64) to (N, d out ). To prevent overfitting, a dropout layer is added during the transformation. Finally, the network generates a semantic segmentation class for each point.

[0036] During model construction, appropriate network layers and hyperparameters (such as learning rate, batch size) are selected, and the labeled dataset is divided into a training set and a test set in a ratio of 3:2. The training set is used to train the model, and the model parameters are adjusted in real time during training. During model training, the weighted cross-entropy loss function and the focal loss function are used to address the class imbalance problem of different categories of point clouds. The trained model and parameters are saved in the system for subsequent use.

[0037] Model optimization and evaluation. After training, the model is fine-tuned using the validation set, adjusting the network structure and hyperparameters to further optimize the model performance. Through cross-validation and multiple experiments, the best model configuration is selected. The evaluation process includes calculating the mean intersection over union (mIoU) and overall accuracy (OA) of the model on the test set to ensure that the accuracy and robustness of the model meet the requirements of practical applications. The optimized model is used for point cloud semantic segmentation tasks in actual scenarios. Figure 3 The trained and optimized deep learning model is applied to the actual point cloud data processing of an underground garage. First, the point cloud data of the new scene is collected using a laser scanner, and the same preprocessing as step S1 is performed. Then, the preprocessed point cloud data is input into the trained model for automatic semantic segmentation, identifying and classifying various objects within the garage, such as cars, walls, and floors. As shown in FIG. 6, (a) shows the three-dimensional effect of the semantic segmentation result, and (b) shows the semantic segmentation result from the top view. The segmentation result is used to reconstruct a three-dimensional model in BIM software such as Revit, and a three-dimensional digital twin model of the garage is generated, which can be used for subsequent facility management, monitoring, and maintenance work.

[0038] Laboratory conditions required for training include: high-performance GPU (such as NVIDIA RTX 4090), running Linux system, using Python programming language and Pytorch deep learning framework.

[0039] Although the present application has been described in connection with the preferred embodiment thereof with reference to the drawings, it is not intended to be limited to the described preferences and embodiments. It will be apparent to those skilled in the art that various modifications and changes can be made thereto without departing from the scope of the present application. Therefore, the scope of the present application should be defined by the appended claims.

Claims

1. A method for reconstructing an underground garage model based on point cloud deep learning, characterized in that, The method comprises the following steps: S1, preprocessing, data labeling and enhancing the point cloud data of the underground garage, constructing a point cloud data set of the underground garage, including underground garage point cloud data and the semantic category corresponding to each point; the point cloud data is the XYZ coordinates and RGB color information of each point; S2, constructing an underground garage point cloud semantic segmentation neural network, training the neural network with the point cloud data set of the underground garage in step S1, and obtaining an underground garage point cloud semantic segmentation model; The underground garage point cloud semantic segmentation neural network is constructed based on a PointNet++ network, and residual connection, squeeze and excitation layer RES-SE, multi-head attention feature aggregation module MAFA and down-sampling multi-head attention feature aggregation module DMAFA are introduced; The training process includes an encoding stage and a decoding stage. In the encoding stage, first, a full connection layer FC and a plurality of residual connection and squeeze and excitation layer RES-SE, multi-head attention feature aggregation module MAFA and down-sampling multi-head attention feature aggregation module DMAFA are used to extract point-by-point features, the shape of the input point cloud data is changed from (N, din) to (N, 64), the local geometric structure is captured, the number of points in the point cloud is reduced layer by layer, and the number of point-by-point feature channels is increased; each encoding layer includes two residual connection and squeeze and excitation layers RES-SE with the same structure, one of which inputs a multi-head attention feature aggregation module MAFA, and the other of which inputs a down-sampling multi-head attention feature aggregation module DMAFA; The point cloud data is down-sampled at a four-fold segmentation ratio, and the feature channel of each point is increased by a factor of two layer by layer, wherein N represents the number of points, and din represents the feature dimension of each point. In the decoding stage, spatial information is restored through a KNN algorithm and a residual attention module RES-Attention, and the semantic category of each point is predicted. S3, collecting underground garage point cloud data, inputting into the underground garage point cloud semantic segmentation model, obtaining the semantic category of each point, and further establishing an underground garage three-dimensional model.

2. The underground garage model reconstruction method based on point cloud deep learning according to claim 1, characterized in that, In step S1, preprocessing the point cloud data of the underground garage includes removing noise points and abnormal points.

3. The underground garage model reconstruction method based on point cloud deep learning according to claim 1, characterized in that, In step S1, data labeling and enhancement includes color random discard, rotation, noise addition, scaling, and labeling different objects of the underground garage using a three-dimensional labeling tool, and the semantic category corresponding to each point includes walls, floors, vehicles, columns and pipes.

4. The underground garage model reconstruction method based on point cloud deep learning according to claim 3, characterized in that, Four decoding layers are used in the decoding stage to restore spatial information. Each decoding layer uses a KNN algorithm to find the nearest neighbor points for each query point, and then uses a nearest neighbor interpolation method to up-sample the point feature set. Each decoding layer combines a residual attention module RES-Attention, and the interpolated features are connected with the skip connection point features from the encoding layer and the output features of the previous decoding layer.

5. The underground garage model reconstruction method based on point cloud deep learning according to claim 4, characterized in that, The underground garage point cloud semantic segmentation neural network outputs the predicted semantic class of all points, represented as (N, dout), wherein dout is the number of semantic classes, and the output layer uses two shared fully connected layers FC to transform the features from (N, 128) to (N, 64), and then from (N, 64) to (N, dout), while adding a dropout layer in the conversion process to prevent overfitting.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 5.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 5.

Images