MEP scene point cloud completion and identification method and system based on deep learning

Abstract

The invention discloses a deep learning-based MEP scene point cloud completion and identification method and system. The method comprises the following steps of firstly, creating a BIM model of an MEP component, and performing parametric modeling to generate a component instance; then, a uniform sampling method and an angle shielding simulation method are adopted to respectively generate a complete MEP assembly point cloud and a shielded partial point cloud, so that a rich training data set is constructed; and then, based on the pre-training model and the joint-trained improved deep learning point cloud completion neural network, improving the precision of MEP scene point cloud completion. And then identifying and segmenting the complemented point cloud data to realize accurate classification of MEP components such as pipelines, air pipes and the like. And finally, an identification result is imported into BIM software to construct a three-dimensional BIM model, and full-life-cycle management of the MEP component is realized. The MEP scene point cloud data processing method has high efficiency and accuracy when processing the MEP scene point cloud data, and is suitable for intelligent operation and maintenance of an MEP system and a building information model system.

Description

Technical Field

[0001] The present invention belongs to the field of civil engineering and artificial intelligence technology, and specifically relates to a method and system for MEP scene point cloud completion and recognition based on deep learning. Background Art

[0002] Mechanical, electrical, and plumbing (MEP) systems play an important role in achieving functionality and efficient management throughout the life cycle of a building. Point cloud-based 3D reconstruction technology can generate a building information model (BIM) of the MEP system, thereby effectively promoting the operation and maintenance of the MEP system. However, due to complex occlusion phenomena, existing technologies often encounter difficulties in identifying and segmenting MEP components, resulting in information loss. In order to improve the efficiency and accuracy of semantic segmentation of point clouds, an innovative method that can realize automatic completion and recognition of point clouds is urgently needed. Summary of the invention

[0003] In view of the problems existing in the prior art, the present invention provides a method and system for MEP scene point cloud completion and recognition based on deep learning, which can support the modeling and maintenance of mechanical, electrical and piping (MEP) systems in building information modeling (BIM).

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for MEP scene point cloud completion and recognition based on deep learning, comprising the following steps:

[0005] S1. Perform parametric modeling for MEP components in the MEP scenario and generate an initial BIM model for each MEP component. The initial BIM model includes MEP component instances of various types and sizes.

[0006] S2, based on the initial BIM model, generate a synthetic point cloud dataset of MEP components, including the complete point cloud and partially occluded point cloud of the MEP components;

[0007] S3. Build a point cloud deep learning completion network, use the MEP component to synthesize the point cloud data set to train the point cloud completion network, obtain the point cloud completion model, and use the pre-training model and joint training method to improve the model accuracy;

[0008] S4. Scene-level point cloud completion of MEP components: Divide the complex scene point cloud into separate component-level point clouds, use the point cloud completion model to complete the separate component-level point clouds, and then integrate them into the scene-level point cloud of MEP components; S5. Identify the scene-level point clouds of MEP components in the MEP scene to obtain the identification results of the MEP components, and then input the identification results into the initial BIM model of the corresponding MEP components to obtain the MEP component BIM model, including the three-dimensional node position and spatial distribution of the MEP components, as well as the specific position.

[0009] Furthermore, the aforementioned step S1 uses Dynamo parametric programming to generate multiple different types of MEP component instances.

[0010] Furthermore, in the aforementioned step S2, a partially occluded point cloud with different occlusion rates is generated by an angle occlusion simulation method, and the MEP component synthetic point cloud data set includes air ducts, T-connections, elbows A, elbows B, I-beams, lamps, pipes, pumps, tanks, fans, and valve components.

[0011] Furthermore, in the aforementioned step S3, a point cloud completion network is constructed based on the improved PoinTr algorithm, and the point cloud completion network is trained using the MEP component synthesized point cloud data set to obtain a point cloud completion model. Specifically, the point cloud completion network is trained using a pre-trained model and a joint training method, and the component types of the MEP component synthesized point cloud data set are increased to expand data diversity.

[0012] Furthermore, in the aforementioned step S3, when the MEP component is used to synthesize the point cloud data set to train the point cloud completion network, the average slice distance is used as a key indicator to quantify the error between the predicted point cloud and the real point cloud. The calculation formula is as follows:

[0013]

[0014] Among them, P and G represent the predicted point set and the true point set respectively.

[0015] Furthermore, in the aforementioned step S3, when the MEP component is used to synthesize the point cloud dataset to train the point cloud completion network, F-Score is used as a supplementary evaluation indicator. F-Score calculates the matching of the predicted point and the real point through the threshold distance dd, and calculates the precision and recall rate. The F-Score is calculated according to the following formula:

[0016]

[0017] Among them, precision refers to the ratio of the number of correctly matched points in the predicted points within the distance d to the total number of predicted points; recall rate is the ratio of the number of correctly matched points in the actual point cloud within the distance d to the number of real point cloud points.

[0018] Furthermore, in the aforementioned step S3, when the point cloud completion network is trained using the MEP component synthesized point cloud dataset, the point cloud of each component is normalized to the range of (-1,1), and the farthest point sampling FPS is used to ensure that the input point cloud contains a preset number of key structural points. The direction of the point cloud is consistent with the training model, the Z axis is vertically upward, the rotation in the XY plane is variable, and the generalization ability of the model to different orientations is enhanced through random XY rotation during training. At the same time, all transformation matrices applied before completion are recorded, and after the completion process is completed, the XYZ coordinates of the point cloud are restored to the original coordinate system through the inverse transformation matrix.

[0019] Furthermore, in the aforementioned step S4, when completing the scene-level point cloud, the noise density clustering method DBSCAN is used to segment the complex scene point cloud into independent component-level point clouds.

[0020] Another aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the present invention when executing the computer program.

[0021] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any one of the methods described in the present invention when executed by a processor.

[0022] Compared with the prior art, the beneficial technical effects of the above technical solutions of the present invention are as follows: By using the deep learning point cloud completion method, the present invention greatly improves the completion efficiency of MEP point cloud data and improves the completion accuracy. Using the trained model, large-scale MEP point cloud data can be quickly and efficiently identified, and accurate three-dimensional models can be generated to support the construction, monitoring and maintenance of underground garages. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of the method of the present invention.

[0024] Figure 2 These are different types of BIM model components generated by Dynamo parametric modeling. In the figure, (a) is a schematic diagram of different types of valves, (b) is a schematic diagram of different parameters of valves, and (c) is an example diagram of different valves generated using Dynamo.

[0025] Figure 3 are examples of the different types of BIM models generated.

[0026] Figure 4 is an example of the complete point cloud generated.

[0027] Figure 5 It is a schematic diagram of the angle occlusion simulation method. DETAILED DESCRIPTION

[0028] In order to better understand the technical content of the present invention, specific embodiments are given and described as follows in conjunction with the accompanying drawings.

[0029] Various aspects of the invention are described herein with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the invention are not limited to those described in the accompanying drawings. It should be understood that the invention is implemented by any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed in the invention are not limited to any implementation. In addition, some aspects disclosed in the invention may be used alone or in any appropriate combination with other aspects disclosed in the invention.

[0030] refer to Figure 1 The present invention provides a method for MEP scene point cloud completion and recognition based on deep learning, comprising the following steps:

[0031] S1. Perform parametric modeling for MEP components in the MEP scenario and generate an initial BIM model for each MEP component. The initial BIM model includes MEP component instances of various types and sizes.

[0032] S2, based on the initial BIM model, generate a synthetic point cloud dataset of MEP components, including the complete point cloud and partially occluded point cloud of the MEP components;

[0033] S3. Build a point cloud deep learning completion network, use the MEP component to synthesize the point cloud data set to train the point cloud completion network, obtain the point cloud completion model, and use the pre-training model and joint training method to improve the model accuracy;

[0034] S4, scene-level point cloud completion of MEP components: Segment the complex scene point cloud into separate component-level point clouds, use the point cloud completion model to complete the separate component-level point clouds, and then integrate them into the scene-level point cloud of MEP components;

[0035] S5. Identify the scene-level point cloud of the MEP components in the MEP scene to obtain the identification results of the MEP components, and then input the identification results into the initial BIM model of the corresponding MEP components to obtain the MEP component BIM model, including the three-dimensional node position and spatial distribution of the MEP components, as well as the specific position.

[0036] As a preferred embodiment of the present invention, in step S1, in order to generate a point cloud dataset of an MEP component, it is first necessary to collect BIM models. There are two ways to collect BIM models: one is to directly use existing BIM models, and the other is to create BIM models of MEP components through parametric modeling methods. For example, taking valves as an example, the generated BIM models include multiple types, such as lifting rod elastic wedge gate valves (model I), lifting rod wedge gate valves (model II), lifting rod parallel gate valves (model III) and non-lifting rod wedge gate valves (model IV). By controlling the size parameters (such as R1, H1, H2, H3 and R2), different instances are generated in each type of valve, and Dynamo is used to automatically generate valve instances of different types and sizes in Autodesk Revit, such as Figure 2 As shown, (a) is a schematic diagram of different types of valves, (b) is a schematic diagram of different valve parameters, and (c) is an example diagram of different valves generated using Dynamo.

[0037] The present invention creates BIM models of 543 MEP components, covering 11 categories of MEP components, including ducts, T-connections, elbows A, elbows B, I-beams, lamps, pipes, pumps, tanks, fans and valves, such as Figure 3 shown.

[0038] In S2, as a preferred embodiment of the present invention, in step S2, by exporting the BIM model of the MEP component into the OBJ format and uniformly sampling the surface of the OBJ model using CloudCompare, the complete point cloud data of 543 MEP components are generated, such as Figure 4 In order to simulate the occlusion in the real scene, the present invention adopts the viewpoint occlusion simulation method, randomly selects a viewpoint around the centroid of the component point cloud, and occludes the farthest 25% to 75% of the points of the viewpoint, so as to obtain partial point clouds with different occlusion rates (25% to 75%), as shown in FIG. Figure 5 This method effectively simulates the point cloud occlusion at different angles.

[0039] In S3, as a preferred embodiment of the present invention, the present invention adopts an improved PoinTr algorithm to complete the point cloud in the MEP scene. First, the generated synthetic data set is used to train the deep learning model, and the real MEP components are completed. In the experiment, two improvement methods were adopted: one is to use the pre-trained model to fine-tune the target completion task, and the other is to expand the data diversity by increasing the component categories in the training data set. The ShapeNet data set was used in the experiment, in which ShapeNet34 contains 34 categories and ShapeNet55 is expanded to 55 categories, both of which can be used as data sources for pre-trained models or joint training. Joint training is to train the data of ShapeNet34 or ShapeNet55 together with the data of this article. The pre-trained model is a model trained with ShapeNet34 and ShapeNet55, using various weights of the previous feature extraction layer, and then training based on the previous feature extraction weights when training the synthetic point cloud data set of the MEP component.

[0040] As a preferred embodiment of the present invention, in step S3, a deep learning point cloud completion network is trained using a synthetic point cloud dataset of MEP components, which is divided into two stages: training and testing. Each epoch in the training stage includes 400 training iterations and 100 validation iterations. In each iteration, the system randomly selects a complete point cloud sample from each category and applies 25% to 75% random occlusion to generate a partial point cloud to complete the point cloud completion task. In the testing phase, the point cloud completion task is set to three difficulty scenarios: simple (25% occlusion), medium (50% occlusion), and difficult (75% occlusion). At each difficulty, the point cloud of each MEP component category is tested repeatedly 100 times to ensure the reliability of the evaluation.

[0041] 300 epochs were set for training. The entire training process generated a total of about 120,000 different occlusion scenes for training, and an additional 30,000 unique occlusion scenes for verification. All experiments were run on a single NVIDIA L40 GPU, and the training parameters used the default configuration of the model. To ensure the standardized processing of the input point cloud, the center of mass of each input point cloud was first aligned to the coordinate origin (0,0,0) and scaled to the XYZ range of (-1,1). In addition, the number of input point clouds was controlled to 2048 points by the farthest point sampling (FPS) to meet the processing requirements of the deep learning model.

[0042] In terms of completion effect evaluation, the present invention uses the average slice distance (Chamfer Distance, CD) as a key indicator to quantify the error between the predicted point cloud and the real point cloud. The calculation formula is as follows:

[0043]

[0044] Among them, P and G represent the predicted point set and the true point set respectively. Two versions of CD, CD-L1 and CD-L2, are used to evaluate the model performance. CD-L1 is based on Manhattan distance calculation, which mainly sums the absolute values ​​of the coordinate differences of the points, while CD-L2 is based on Euclidean distance and calculates the square root of the sum of the squares of the coordinate differences.

[0045] In addition to CD, the present invention also uses F-Score as a supplementary evaluation indicator. F-Score calculates the matching of the predicted point and the real point through the threshold distance dd, thereby calculating the precision and recall, and the F-Score is obtained according to the following formula:

[0046]

[0047] Among them, precision refers to the ratio of the number of correctly matched points in the predicted points within the distance d to the total number of predicted points; recall rate is the ratio of the number of correctly matched points in the actual point cloud within the distance d to the number of real point cloud points.

[0048] In S4, scene-level point cloud completion of MEP components is performed. Most existing algorithms are independent component-level completion algorithms, which are not suitable for the overall scene. As a preferred embodiment of the present invention, in order to meet the needs of scene-level completion, step S4 is specifically: the scene point cloud is first segmented into separate component-level point clouds by clustering or segmentation algorithms (such as noise-based density clustering method DBSCAN). The segmented point clouds of each component are used as the input of the completion model.

[0049] Secondly, the completion model has specific requirements for the input point cloud. Each component point cloud must be normalized to the range of (-1,1), and the farthest point sampling (FPS) is applied to ensure that the input point cloud contains a fixed number (such as 2048) of key structural points. In addition, to improve the completion accuracy, the direction of the point cloud must be consistent with the training model, usually with the Z axis vertically upward. Rotation in the XY plane is allowed, and the model's generalization ability to different orientations is enhanced by random XY transformations during training.

[0050] In order to restore the point cloud to the original scene after completion, it is necessary to record all the transformation matrices of the point cloud before completion. After the completion process is completed, the XYZ coordinates of the point cloud are restored through the inverse transformation matrix, so that the completed point cloud returns to the original coordinate system. Subsequently, each completed component-level point cloud is reintegrated back into the original scene to achieve scene-level point cloud completion. The steps of the scene-level MEP component point cloud completion algorithm are as follows:

[0051] Initialize P_scene_completed←[]

[0052] Apply DBSCAN or other clustering algorithms to P_scene to segment it into component-level point clouds

[0053] For each component-level point cloud P_component:

[0054] Normalize P_component to the range (-1,1).

[0055] Sample 2048 points from P_component using the farthest point sampling (FPS).

[0056] Adjust the P_component so the Z axis is vertical.

[0057] Record the transformation matrix M_transform.

[0058] Input P_component into the deep learning model for completion.

[0059] After completion, P_component_completed is restored to the original coordinate system through the inverse transformation of M_transform.

[0060] Append P_component_completed to P_scene_completed.

[0061] Returns P_scene_completed.

[0062] As a preferred embodiment of the present invention, step S5 is specifically: based on the completed components, the point cloud is input into the PointNet deep learning network for further classification tasks to achieve accurate recognition of MEP components. Since the point cloud completion process restores the missing key geometric information, the shape of the component is more complete, which significantly improves the recognition accuracy. Compared with the point cloud before completion, the completed point cloud not only contains more feature details, but also can more accurately express the overall outline and local details of the component. The experimental results show that the recognition accuracy of MEP components after the point cloud completion operation is significantly improved, which helps to improve the robustness and recognition effect of the model in complex scenes, and lays a more solid foundation for subsequent building information modeling (BIM) modeling.

[0063] Based on the precise results of point cloud recognition, the component information is imported into the Revit software to automatically generate a BIM model. This model can not only effectively reproduce the three-dimensional structure of the MEP component, but also accurately represent its spatial distribution and installation location at the scene level. The reconstructed BIM model can be further applied to the full life cycle management of MEP components, including later maintenance management, facility replacement, risk prediction, etc. At the same time, combined with digital twin technology, the reconstructed BIM model can realize dynamic monitoring and data synchronization of the MEP system, providing high-precision data support for smart buildings and digital facility management. This process from point cloud to BIM model optimizes data processing efficiency and greatly expands the application potential of MEP systems in smart cities and digital buildings. The laboratory conditions required for training include: high-performance GPU (such as NVIDIA RTX 4090), running Linux system, using Python programming language and Pytorch deep learning framework.

[0064] Although the present invention has been described above with preferred embodiments, it is not intended to limit the present invention. A person skilled in the art of the present invention may make various modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the definition of the claims.

Claims

1. A method for MEP scene point cloud completion and recognition based on deep learning, characterized in that: The following steps are involved: S1. Perform parametric modeling for MEP components in the MEP scenario and generate an initial BIM model for each MEP component. The initial BIM model includes MEP component instances of various types and sizes. S2, based on the initial BIM model, generate a synthetic point cloud dataset of MEP components, including the complete point cloud and partially occluded point cloud of the MEP components; S3. Build a point cloud deep learning completion network, use the MEP component to synthesize the point cloud data set to train the point cloud completion network, obtain the point cloud completion model, and use the pre-training model and joint training method to improve the model accuracy; S4, scene-level point cloud completion of MEP components: Segment the complex scene point cloud into separate component-level point clouds, use the point cloud completion model to complete the separate component-level point clouds, and then integrate them into the scene-level point cloud of MEP components; S5. Identify the scene-level point cloud of the MEP components in the MEP scene to obtain the identification results of the MEP components, and then input the identification results into the initial BIM model of the corresponding MEP components to obtain the MEP component BIM model, including the three-dimensional node position and spatial distribution of the MEP components, as well as the specific position.

2. According to the method of MEP scene point cloud completion and recognition based on deep learning in claim 1, it is characterized in that: Step S1 uses Dynamo parametric programming to generate multiple different types of MEP component instances.

3. The method for MEP scene point cloud completion and recognition based on deep learning according to claim 1 is characterized in that: In step S2, the partially occluded point clouds with different occlusion rates are generated by the angle occlusion simulation method, and the synthetic point cloud dataset of MEP components includes air ducts, T-connections, elbows A, elbows B, I-beams, lamps, pipes, pumps, tanks, fans, and valve components.

4. The method for MEP scene point cloud completion and recognition based on deep learning according to claim 1, characterized in that: In step S3, a point cloud completion network is constructed based on the improved PoinTr algorithm, and the point cloud completion network is trained using the MEP component synthesized point cloud data set to obtain a point cloud completion model. Specifically, the point cloud completion network is trained using a pre-trained model and a joint training method, and the component types of the MEP component synthesized point cloud data set are increased to expand data diversity.

5. The method for MEP scene point cloud completion and recognition based on deep learning according to claim 1, characterized in that: In step S3, when the MEP component is used to synthesize the point cloud data set to train the point cloud completion network, the average slice distance is used as the key indicator to quantify the error between the predicted point cloud and the real point cloud. The calculation formula is as follows: Among them, P and G represent the predicted point set and the true point set respectively.

6. The method for MEP scene point cloud completion and recognition based on deep learning according to claim 1, characterized in that: In step S3, when the MEP component is used to synthesize the point cloud dataset to train the point cloud completion network, F-Score is used as a supplementary evaluation indicator. F-Score calculates the matching of the predicted point and the real point through the threshold distance dd, and calculates the precision and recall rate. The F-Score is calculated according to the following formula: Among them, precision refers to the ratio of the number of correctly matched points in the predicted points within the distance d to the total number of predicted points; recall rate is the ratio of the number of correctly matched points in the actual point cloud within the distance d to the number of real point cloud points.

7. The method for MEP scene point cloud completion and recognition based on deep learning according to claim 1, characterized in that: In step S3, when the point cloud completion network is trained using the MEP component synthesized point cloud dataset, the point cloud of each component is normalized to the range of (-1, 1), and the farthest point sampling FPS is used to ensure that the input point cloud contains a preset number of key structural points. The direction of the point cloud is consistent with the training model, the Z axis is vertically upward, and the rotation in the XY plane is variable. The model's generalization ability to different orientations is enhanced through random XY rotation during training. At the same time, all transformation matrices applied before completion are recorded, and after the completion process is completed, the XYZ coordinates of the point cloud are restored to the original coordinate system through the inverse transformation matrix.

8. The method for MEP scene point cloud completion and recognition based on deep learning according to claim 1, characterized in that: In step S4, when completing the scene-level point cloud, the noise density clustering method DBSCAN is used to segment the complex scene point cloud into independent component-level point clouds.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images