Underground garage model reconstruction method and system based on point cloud deep learning

Abstract

The invention discloses an underground garage model reconstruction method and system based on point cloud deep learning, and the method comprises the steps: firstly constructing a point cloud data set of an underground garage, including underground garage point cloud data and a semantic category corresponding to each point; and then constructing an underground garage point cloud semantic segmentation deep learning neural network, and introducing two modules, namely an RES-SE module, a multi-head attention feature aggregation MAFA module and a DMAFA module, on the basis of PointNet + + so as to realize effective coding and decoding of point cloud data. And training the neural network by using the data set to obtain an underground garage point cloud semantic segmentation model. The model can identify components such as pipelines, wall surfaces and fire hydrants in the garage, the method has high efficiency and accuracy when processing complex underground garage point cloud data, semantic segmentation of underground garage scenes can be automatically completed, three-dimensional model reconstruction is carried out, and the method is suitable for underground garage intelligent operation and maintenance and building information model systems.

Description

Technical Field

[0001] The present invention belongs to the technical field of civil engineering and artificial intelligence interaction, and specifically relates to a method and system for reconstructing an underground garage model based on point cloud deep learning. Background Art

[0002] The underground garage is an important part of the building. It is responsible for parking and accommodating mechanical, electrical, and plumbing systems, and needs to be maintained regularly. Point cloud-based 3D model reconstruction can obtain the BIM model of the underground garage, thereby effectively promoting the operation and maintenance of the underground garage. The first step in 3D model reconstruction is semantic segmentation. Existing point cloud semantic segmentation methods usually rely on prior knowledge and have low accuracy when dealing with complex underground garage scenes. In order to improve the efficiency and accuracy of point cloud semantic segmentation, a new automated processing method is urgently needed. Summary of the invention

[0003] In response to the problems existing in the prior art, the present invention provides a method and system for reconstructing an underground garage model based on point cloud deep learning, which can handle complex underground garage scenes. It can not only generate a three-dimensional digital twin model of the garage, but also facilitate the subsequent facility management, monitoring and maintenance of the underground garage.

[0004] In order to solve the above technical problems, the present invention provides the following technical solution: a method for reconstructing an underground garage model based on point cloud deep learning, comprising the following steps:

[0005] S1. Preprocess, annotate and enhance the point cloud data of the underground garage to construct a point cloud dataset of the underground garage, including the point cloud data of the underground garage and the semantic category corresponding to each point; the point cloud data is the XYZ coordinates and RGB color information of each point;

[0006] S2, constructing a point cloud semantic segmentation neural network for underground garages, training the neural network with the point cloud data set of the underground garage in step S1, and obtaining a point cloud semantic segmentation model for underground garages;

[0007] The underground garage point cloud semantic segmentation neural network is built based on the PointNet++ network, and introduces two modules: RES-SE, multi-head attention feature aggregation MAFA and DMAFA;

[0008] The training process includes encoding and decoding stages. In the encoding stage, several RES-SE layers, multi-head attention feature aggregation MAFA and DMAFA modules are used to extract point-by-point features and capture local geometric structures, reducing the number of points in the point cloud layer by layer while increasing the number of point-by-point feature channels. In the decoding stage, the KNN algorithm and RES-Attention module are used to restore spatial information and predict the semantic category of each point.

[0009] S3. Collect the point cloud data of the underground garage, input it into the underground garage point cloud semantic segmentation model, obtain the semantic category of each point, and further establish the three-dimensional model of the underground garage.

[0010] Furthermore, in the aforementioned step S1, preprocessing the point cloud data of the underground garage includes removing noise points and abnormal points.

[0011] Furthermore, in the aforementioned step S1, data annotation and enhancement include random color discarding, rotation, noise addition, and scaling, and a three-dimensional annotation tool is used to annotate different objects in the underground garage. The semantic categories corresponding to each point include walls, ground, vehicles, columns, and pipes.

[0012] Furthermore, in the aforementioned encoding stage, the FC layer and RES-SE layer are first used to learn point-by-point features and extract local context features, transforming the input point cloud data shape from (N, din) to (N, 64), and then four encoding layers are used to reduce the number of points in the point cloud data while increasing the number of point-by-point feature channels to obtain local geometric structures;

[0013] Each encoding layer includes two RES-SE layers with the same structure. The point cloud data is downsampled at a four-fold segmentation ratio, and the feature channel of each point is increased by a two-fold multiplication ratio layer by layer.

[0014] Furthermore, the aforementioned decoding stage uses four decoding layers to recover spatial information. Each decoding layer uses the KNN algorithm to find the nearest neighbor point for each query point, and then uses the nearest neighbor interpolation method to upsample the point feature set. Each decoding layer is combined with the RES-Attention module, and the interpolated features are connected with the jump connection point features from the encoding layer and the output features of the previous decoding layer.

[0015] Furthermore, each of the aforementioned encoding layers includes two RES-SE layers with the same structure, one of which takes the MAFA module as the core input, and the other takes the DMAFA module as the core input.

[0016] Furthermore, the output of the aforementioned underground garage point cloud semantic segmentation neural network is the predicted semantic category of all points, expressed as (N, dout), where dout is the number of semantic categories. The output layer uses two shared FC layers to transform the features from (N, 128) to (N, 64), and then from (N, 64) to (N, dout). At the same time, a dropout layer is added in the conversion process to prevent overfitting.

[0017] Another aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the present invention when executing the computer program.

[0018] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any one of the methods described in the present invention when executed by a processor.

[0019] Compared with the prior art, the beneficial technical effects of the above technical solutions adopted by the present invention are as follows: By using deep learning methods, the present invention greatly improves the semantic segmentation accuracy and efficiency of underground garage point cloud data, and significantly reduces manual intervention and manual operation time. Using the trained model, large-scale point cloud data can be processed quickly and efficiently to generate accurate three-dimensional models to support the construction, monitoring and maintenance of underground garages. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the method of the present invention.

[0021] Figure 2 This is the GarageNet network architecture of the deep learning model of the present invention.

[0022] Figure 3 Schematic diagram of the semantic segmentation result of the present invention, in which (a) is a three-dimensional view of the semantic segmentation result, and (b) is a top view of the semantic segmentation result. DETAILED DESCRIPTION

[0023] In order to better understand the technical content of the present invention, specific embodiments are given and described as follows in conjunction with the accompanying drawings.

[0024] Various aspects of the invention are described herein with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the invention are not limited to those described in the accompanying drawings. It should be understood that the invention is implemented by any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed in the invention are not limited to any implementation. In addition, some aspects disclosed in the invention may be used alone or in any appropriate combination with other aspects disclosed in the invention.

[0025] refer to Figure 1 The present invention provides a method for reconstructing an underground garage model based on point cloud deep learning, comprising the following steps:

[0026] S1. Preprocess, annotate and enhance the point cloud data of the underground garage to construct a point cloud dataset of the underground garage, including the point cloud data of the underground garage and the semantic category corresponding to each point; the point cloud data is the XYZ coordinates and RGB color information of each point;

[0027] As a preferred embodiment of the present invention, a laser scanner such as Leica RTC360 can be used to obtain point cloud data of the underground garage. The collected point cloud data is preprocessed, including noise point removal and abnormal point removal. In addition, the point cloud data is downsampled, and the voxel downsampling method is used to simplify the point cloud data to a reasonable scale while retaining key geometric features for subsequent processing. The preprocessed point cloud data is stored in a standard format such as .pcd or .las for subsequent processing.

[0028] The pre-processed point cloud data is manually annotated, and different objects in the underground garage, such as walls, ground, vehicles, columns, pipes, etc., are semantically classified and assigned corresponding labels. In the data annotation process, 3D annotation tools (such as CloudCompare) are used to ensure the accuracy of annotation. In order to improve the generalization ability of the model, data enhancement technology is used to process the annotated data set, including rotation, translation, scaling, noise addition and other operations to generate diversified training samples. The point cloud data after data enhancement is used for model training and verification.

[0029] S2, constructing a point cloud semantic segmentation neural network for underground garages, training the neural network with the point cloud data set of the underground garage in step S1, and obtaining a point cloud semantic segmentation model for underground garages;

[0030] refer to Figure 2 The underground garage point cloud semantic segmentation neural network is built based on the PointNet++ network, and two modules, RES-SE, multi-head attention feature aggregation MAFA and DMAFA, are introduced; the residual connection (RES) enhances the links between different layers, while the compression and excitation (SE) mechanism strengthens the connections between channels.

[0031] The training process includes an encoding phase and a decoding phase. In the encoding phase, several RES-SE layers, multi-head attention feature aggregation MAFA and DMAFA modules are used to extract point-by-point features and capture local geometric structures, reducing the number of points in the point cloud layer by layer while increasing the number of point-by-point feature channels. In the decoding phase, the KNN algorithm (k-nearest neighbor classification) and the RES-Attention module are used to restore spatial information and predict the semantic category of each point. The network input is point cloud data of size (N, din), where N represents the number of each point and din represents the feature dimension of each point. The MAFA module enhances the feature representation capability of capturing various aspects of point cloud data through multiple attention heads through multi-layer feature extraction and aggregation, combined with multiple input feature types and aggregation methods. The DMAFA module not only extracts features, but also reduces the number of points in the input point cloud by downsampling, resulting in lower output resolution (for example, by doubling the neighbor search radius and subsampling grid size).

[0032] S3. Collect the point cloud data of the underground garage, input it into the underground garage point cloud semantic segmentation model, obtain the semantic category of each point, and further establish the three-dimensional model of the underground garage.

[0033] like Figure 2 As shown, as a preferred embodiment of the present invention, in the encoder stage, the FC layer and the RES-SE layer (with the MAFA module as the core) are first used to learn point-by-point features and extract local context features, transforming the data shape from (N, din) to (N, 64). Subsequently, four encoding layers are used to reduce the number of points in the point cloud while increasing the number of point-by-point feature channels to capture local geometric structures. Each encoding layer includes two RES-SE layers, one of which uses the MAFA module as the core input and the other uses the DMAFA module as the core input. The point cloud is downsampled at a four-fold segmentation ratio, and each layer retains 25% of the point features (i.e., N to N / 4, N / 16, N / 64, N / 256). At the same time, the feature channel of each point increases by a doubling ratio layer by layer (i.e., 128 to 256, 512 to 1024, 2048). RES-SE is a residual connection and squeeze and excitation layer, MAFA is a multi-head attention feature aggregation module, DFAMA is a down-sampling multi-head attention feature aggregation module, FC layer is a fully connected layer, US is an upsampling layer, and MLP is a multi-layer perceptron.

[0034] In the decoder stage, four decoding layers are used to recover spatial information. Each decoding layer uses the KNN algorithm to find the nearest neighbor for each query point, and then upsamples the point feature set using the nearest neighbor interpolation method. To enhance the decoding process, each decoding layer is combined with the RES-Attention module. The interpolated features are concatenated with the jump connection point features from the encoding layer and the output features of the previous decoding layer. Figure 2 In , the RES-Attention module is the residual attention module, and the skipconnection is the skip connection.

[0035] The output of the network is the predicted semantic category of all points, expressed as (N,d out ), where d out is the number of semantic categories. In the output layer, two shared FC layers are used to transform the features from (N, 128) to (N, 64) and then from (N, 64) to (N, d out ). To prevent overfitting, a dropout layer is added during the conversion process. Finally, the network generates a semantic segmentation category for each point.

[0036] During the model building process, the appropriate number of network layers and hyperparameters (such as learning rate and batch size) are selected, and the labeled data set is divided into a training set and a test set with a ratio of 3:2. The model is trained using the training set, and the model parameters are adjusted in real time during the training process. When training the model, the weighted cross entropy loss function and the focal loss function are used to deal with the category imbalance problem of point clouds of different categories. The trained model and parameters are saved in the system for subsequent use.

[0037] Model optimization and evaluation. After training is completed, the validation set is used to tune the model, adjust the network structure and hyperparameters, and further optimize the model performance. Through cross-validation and multiple experiments, the best model configuration is selected. The evaluation process includes calculating the mean intersection over union (mIoU) and overall accuracy (OA) of the model on the test set to ensure that the accuracy and robustness of the model meet the needs of practical applications. The optimized model is used for point cloud semantic segmentation tasks in actual scenes. Actual scene application. The trained and optimized deep learning model is applied to the actual point cloud data processing of the underground garage. First, a laser scanner is used to collect point cloud data of the new scene, and the same preprocessing as step S1 is performed. Next, the preprocessed point cloud data is input into the trained model for automatic semantic segmentation to identify and classify various objects in the garage, such as Figure 3 As shown in the figure, (a) shows the 3D effect of the semantic segmentation result, and (b) shows the top-down effect of the semantic segmentation. The segmentation result is used to reconstruct the 3D model in BIM software such as Revit and generate a 3D digital twin model of the garage, which can be used for subsequent facility management, monitoring and maintenance work.

[0038] The laboratory conditions required for training include: high-performance GPU (such as NVIDIA RTX 4090), running Linux system, using Python programming language and Pytorch deep learning framework.

[0039] Although the present invention has been described above with preferred embodiments, it is not intended to limit the present invention. A person skilled in the art of the present invention may make various modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the definition of the claims.

Claims

1. A method for reconstructing an underground garage model based on point cloud deep learning, characterized in that: The following steps are involved: S1. Preprocess, annotate and enhance the point cloud data of the underground garage to construct a point cloud dataset of the underground garage, including the point cloud data of the underground garage and the semantic category corresponding to each point; the point cloud data is the XYZ coordinates and RGB color information of each point; S2, constructing a point cloud semantic segmentation neural network for underground garages, training the neural network with the point cloud data set of the underground garage in step S1, and obtaining a point cloud semantic segmentation model for underground garages; The underground garage point cloud semantic segmentation neural network is built based on the PointNet++ network, and introduces two modules: RES-SE, multi-head attention feature aggregation MAFA and DMAFA; The training process includes encoding and decoding stages. In the encoding stage, several RES-SE layers, multi-head attention feature aggregation MAFA and DMAFA modules are used to extract point-by-point features and capture local geometric structures, reducing the number of points in the point cloud layer by layer while increasing the number of point-by-point feature channels. In the decoding stage, the KNN algorithm and RES-Attention module are used to restore spatial information and predict the semantic category of each point. S3. Collect the point cloud data of the underground garage, input it into the underground garage point cloud semantic segmentation model, obtain the semantic category of each point, and further establish the three-dimensional model of the underground garage.

2. The underground garage model reconstruction method based on point cloud deep learning according to claim 1 is characterized in that: In step S1, the point cloud data of the underground garage is preprocessed, including removing noise points and abnormal points.

3. The underground garage model reconstruction method based on point cloud deep learning according to claim 1 is characterized in that: In step S1, data annotation and enhancement include random color discarding, rotation, noise addition, and scaling. At the same time, a three-dimensional annotation tool is used to annotate different objects in the underground garage. The semantic categories corresponding to each point include wall, ground, vehicle, column, and pipe.

4. The underground garage model reconstruction method based on point cloud deep learning according to claim 1 is characterized in that: In the encoding stage, the FC layer and RES-SE layer are first used to learn point-by-point features and extract local context features, transforming the input point cloud data shape from (N, din) to (N, 64). Then, four encoding layers are used to reduce the number of points in the point cloud data and increase the number of point-by-point feature channels to obtain local geometric structures. Each encoding layer includes two RES-SE layers with the same structure. The point cloud data is downsampled at a four-fold segmentation ratio, and the feature channel of each point is increased by a two-fold multiplication ratio layer by layer.

5. The underground garage model reconstruction method based on point cloud deep learning according to claim 4 is characterized in that: Four decoding layers are used in the decoding stage to restore spatial information. Each decoding layer uses the KNN algorithm to find the nearest neighbor point for each query point, and then uses the nearest neighbor interpolation method to upsample the point feature set. Each decoding layer is combined with the RES-Attention module, and the interpolated features are connected with the jump connection point features from the encoding layer and the output features of the previous decoding layer.

6. The underground garage model reconstruction method based on point cloud deep learning according to claim 4 is characterized in that: Each encoding layer consists of two RES-SE layers with the same structure, one of which takes the MAFA module as the core input and the other takes the DMAFA module as the core input.

7. The underground garage model reconstruction method based on point cloud deep learning according to claim 6 is characterized in that: The output of the underground garage point cloud semantic segmentation neural network is the predicted semantic category of all points, expressed as (N, dout), where dout is the number of semantic categories. The output layer uses two shared FC layers to transform the features from (N, 128) to (N, 64), and then from (N, 64) to (N, dout). At the same time, a dropout layer is added during the conversion process to prevent overfitting.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images