A method and system for mechanical, electrical and piping scene deep learning reconstruction based on synthetic point cloud enhancement
By generating synthetic point clouds and training neural networks, the problem of deep learning training in MEP scenes relying on real point cloud datasets is solved, achieving high-precision semantic segmentation and 3D model reconstruction, and improving the processing efficiency and accuracy of MEP scenes.
Patent Information
- Application Number
- CN202411661545.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-20
- Publication Date
- 2025-11-11
- Estimated Expiration
- 2044-11-20
AI Technical Summary
In existing technologies, deep learning training for mechanical, electrical, and piping (MEP) scenarios relies on a large amount of real and labeled point cloud datasets, which makes the process time-consuming and labor-intensive, and results in insufficient semantic segmentation accuracy.
By generating synthetic point clouds to simulate the real laser scanning process, synthetic point clouds of mechanical, electrical, and pipeline scenes are generated using the Ray Laser Scanning and Intersection Algorithm (RBLSIA). The synthetic point clouds are then used to train neural networks to improve the reconstruction accuracy of MEP scene models.
It significantly improves the semantic segmentation accuracy of MEP scenes, reduces the time for manual point cloud scanning and annotation, and can quickly and efficiently process large-scale point cloud data to generate accurate 3D models, supporting the construction and maintenance of underground parking garages.
Smart Images
Figure CN119888212B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of civil engineering and artificial intelligence interaction technology, specifically relating to a deep learning reconstruction method and system for mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement. Background Technology
[0002] Mechanical, electrical, and plumbing (MEP) scenarios are crucial components of buildings. Reconstructing Building Information Models (BIMs) for MEP components using point cloud-based deep learning methods can effectively facilitate the operation and maintenance of these components. However, deep learning training typically relies on large datasets of real-world, labeled point clouds, a process that is both time-consuming and labor-intensive. To address the shortage of real-world point cloud data, there is an urgent need for a method to automatically generate synthetic point clouds for MEP scenarios. Training with generated synthetic point clouds can improve the accuracy of semantic segmentation in MEP scenarios, thereby supporting high-quality model reconstruction. Summary of the Invention
[0003] This invention addresses the problems of existing technologies by providing a deep learning reconstruction method and system for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement. This method improves the accuracy of semantic segmentation in MEP scenes, thereby achieving high-precision MEP model reconstruction. The method generates synthetic point clouds from a Building Information Model (BIM) to simulate a real laser scanning process, producing point cloud data that reflects occlusion in the real scene. The accuracy of MEP scene model reconstruction is further improved through training with the synthetic point clouds.
[0004] To address the above technical problems, this invention provides the following technical solution: a deep learning reconstruction method for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement, comprising the following steps:
[0005] S1. Preprocess the building information model for mechanical, electrical, and piping scenarios to obtain the geometric information of the scene, including: vertices, faces, and normals, with vertices represented by XYZ coordinates;
[0006] S2. Based on ray laser scanning and the intersection algorithm RBLSIA, synthetic point clouds of mechanical, electrical, and pipeline scenes are generated, which have...
[0007] The body is:
[0008] S2.1 Place a virtual laser scanner at a pre-selected scanning site, create a simulated laser beam, and introduce high...
[0009] The error is used to simulate the parameters of a real laser scanner; then the position of the virtual laser scanner in the scene is set.
[0010] S2.2 Calculate the number of laser beams in the vertical and horizontal directions according to the specified angular resolution. and Calculation based on vertical field of view and horizontal field of view Generate arrays of vertical and horizontal scan angles. and ,
[0011] S2.3 Calculate and adjust the direction vector with error, convert the angle to radians, and use trigonometric functions to calculate the components of the direction vector. , and This ultimately forms matrix D;
[0012] S2.4. Use the RBLSIA method to calculate the nearest intersection point between each laser beam and the scene triangular mesh, and record the relevant information of the intersection point and the points in the generated synthetic point cloud;
[0013] S3. Divide the mechanical, electrical, and pipeline scenes into passable areas, high-alert areas, and impassable areas. Evenly distribute scanning points in the passable areas to generate a synthetic point cloud of the mechanical, electrical, and pipeline scenes, and assign a semantic category label to each point.
[0014] S4. Construct point cloud datasets for mechanical, electrical, and pipeline scenes, including synthetic point clouds of mechanical, electrical, and pipeline scenes, and semantic category labels corresponding to each point in the point cloud. Simultaneously, construct a reconstruction neural network for mechanical, electrical, and pipeline scenes, and train the neural network using the point cloud dataset to obtain reconstruction models for mechanical, electrical, and pipeline scenes.
[0015] S5. Deploy the mechanical, electrical, and piping scene reconstruction model in the building information model, input the geometric information of the scene to be tested, and obtain the reconstruction results of the mechanical, electrical, and piping scenes.
[0016] Furthermore, step S2.1 above sets the position of the virtual laser scanner in the scene, as follows:
[0017] , for The XYZ coordinates.
[0018] Furthermore, in step S2.2 above, the number of laser beams in the vertical and horizontal directions is calculated according to the specified angular resolution:
[0019] in, and These are the number of laser beams in the vertical and horizontal directions, respectively. and These are the field of view angles in the vertical and horizontal directions, respectively. It refers to angular resolution.
[0020] Furthermore, step S2.2 above generates scan angle arrays in the vertical and horizontal directions, as follows:
[0021] ,
[0022] linspace indicates that values are generated at regular intervals.
[0023] Furthermore, the aforementioned deep learning reconstruction method for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement...
[0024] Calculate the direction vector in the error simulation: for each vertical angle and horizontal angle Gaussian error is introduced to simulate the angular error between two directions in the real world:
[0025]
[0026] in, It is the introduced vertical angle error. It is the introduced horizontal angle error. It is the standard deviation of the vertical angle error. It is the standard deviation of the horizontal angle error.
[0027] Furthermore, in the aforementioned step S2.3, the adjusted horizontal and vertical angles are converted into horizontal radians. and vertical radii And apply the corresponding rotational offset, as follows:
[0028]
[0029] Use trigonometric functions to calculate the components of the direction vector. , and ,as follows:
[0030] , ,
[0031] Combine the x, y, and z direction components into a matrix D, as follows:
[0032] .
[0033] Furthermore, the aforementioned step S2.4 specifically involves checking whether each ray intersects with the triangular mesh in the scene. If an intersection occurs, the distance from the intersection point to the origin of the ray is calculated, and the nearest intersection point and its related information are recorded. The nearest intersection point is a point in the generated synthetic point cloud.
[0034] Furthermore, in step S4 above, the construction of the reconstructed neural network for mechanical, electrical, and pipeline scenes specifically utilizes one of the following: PointNet network, PointNet++ network, or ResPointNet++ network.
[0035] Another aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described in the present invention.
[0036] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the methods described in the present invention.
[0037] Compared with the prior art, the beneficial technical effects of the present invention using the above technical solution are as follows:
[0038] By using a synthetic point cloud generation and enhancement method, this invention significantly improves the semantic segmentation accuracy and efficiency of MEP scenes, and substantially reduces the time spent on manual scanning and annotation of real point clouds. Using the trained model, large-scale point cloud data can be processed quickly and efficiently to generate accurate 3D models, supporting the construction, monitoring, and maintenance of underground parking garages. Attached Figure Description
[0039] Figure 1 This is a flowchart of the method of the present invention.
[0040] Figure 2 This is an example diagram of the sampling point cloud of the simulated laser scanner of the present invention.
[0041] Figure 3 This is a walkable map of the present invention. In the figure, (a) is the initial BIM model and (b) is the 2D walkable map.
[0042] Figure 4 This is an example diagram of the virtual laser scanner scanning process of the present invention. In the diagram, (a) is a virtual laser scanning position diagram, (b) is an emitted laser ray diagram, (c) is a three-dimensional view of the generated synthetic point cloud, and (d) is a planar view of the generated synthetic point cloud.
[0043] Figure 5 It is a point cloud scene diagram with semantic information. In the diagram, (a) is the actual semantic situation diagram and (b) is the inferred semantic situation diagram. Detailed Implementation
[0044] To better understand the technical content of the present invention, specific embodiments are described below in conjunction with the accompanying drawings.
[0045] In this invention, various aspects of the invention are described with reference to the accompanying drawings, in which numerous illustrative embodiments are shown. Embodiments of the invention are not limited to those depicted in the drawings. It should be understood that the invention is implemented through any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed herein are not limited to any particular implementation. Furthermore, some aspects of the invention disclosed may be used alone or in any suitable combination with other aspects of the invention disclosed.
[0046] refer to Figure 1 This invention provides a deep learning reconstruction method for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement, comprising the following steps:
[0047] S1. Preprocess the Building Information Model (BIM) for mechanical, electrical, and piping scenarios to obtain the geometric information of the scene, including vertices, faces, and normals, with vertices represented by XYZ coordinates;
[0048] The BIM model includes typical Mechanical, Electrical, and Plumbing (MEP) components, such as pipes, I-beams, pumps, R-beams, and tanks, with clear and explicit semantic annotations. The BIM model is then converted to OBJ format files and read using Trimesh in Python. OBJ format is a widely used 3D model file format that supports various modeling software (such as Revit, Blender, and Maya) and can be parsed using the Trimesh library. The structure of an OBJ file includes vertices, faces, and normals. Vertices are represented by XYZ coordinates, and each vertex provides geometric information about the model.
[0049] S2. Based on the Ray-Laser Laser Scanning and Intersection Algorithm (RBLSIA), a composite point cloud is generated for mechanical, electrical, and piping scenes. First, a virtual laser scanner is placed at a pre-selected scanning site. A simulated laser beam is created based on the parameters of an actual laser scanner, and Gaussian errors are introduced into the laser beam direction. Then, RBLSIA is used to calculate the nearest intersection point between each laser beam and the scene's triangular mesh, and relevant information about this intersection point (such as material, color, and category) is recorded. These intersection points will become points in the generated composite point cloud. A schematic diagram of this algorithm simulating the laser scanner emitting lasers to generate the composite point cloud is shown below. Figure 2 As shown. Specifically:
[0050] S2.1 Place the virtual laser scanner at the pre-selected scanning site, create a simulated laser beam, and introduce Gaussian error to simulate the parameters of a real laser scanner; then set the position of the virtual laser scanner in the scene.
[0051] S2.2 Calculate the number of laser beams in the vertical and horizontal directions according to the specified angular resolution. and Calculation based on vertical field of view and horizontal field of view Generate arrays of vertical and horizontal scan angles. and ,
[0052] S2.3 Calculate and adjust the direction vector with error, convert the angle to radians, and use trigonometric functions to calculate the components of the direction vector. , and This ultimately forms matrix D;
[0053] S2.4. Use the RBLSIA method to calculate the nearest intersection point between each laser beam and the scene triangular mesh, and record the relevant information of the intersection point and the points in the generated synthetic point cloud;
[0054] S3. Based on the actual situation of the scene, the scanning site planning follows these principles: During scene initialization, the boundary information of the scene and the position and height of all components are obtained, and the scene is divided into passable areas, height warning areas, and impassable areas. Impassable areas (such as areas containing MEP components) are marked as obstacles to ensure the passability of real-world scanning. Scanning sites are evenly distributed within the passable area to ensure full coverage of the scene.
[0055] S4. Construct point cloud datasets for mechanical, electrical, and piping scenes, including synthetic point clouds of these scenes and semantic category labels for each point, such as pipe, support, and valve. These areas are marked as obstacles and impassable; areas with significant height are marked as passable, but attention should be paid to the height of I-beams and R-beams to avoid limited viewpoints or occlusion. Other areas are marked as walkable areas. Figure 3 As shown, a walkable map was generated. Next, scanning stations were evenly selected within the walkable area to ensure full coverage of the entire scene. The selection of stations followed these conditions: first, a safe distance of at least 0.5 meters should be maintained between each station and the nearest pipe to avoid an excessively narrow scanning area; second, the distance between stations should be reasonably distributed to ensure effective laser scanning coverage and reduce blind spots. In the simulated environment, because the virtual laser scanner can move freely without incurring time costs, a wider coverage area than real point cloud scanning can be achieved by increasing the number of scanning stations.
[0056] like Figure 4 As shown, this invention uses virtual laser scanning to generate synthetic point clouds. Figure 4 The location of the scanner is marked in (a), and it is shown in (b). Figure 4(b) The intersection process of the laser ray and the BIM model. The final generated synthetic point cloud of the MEP scene, as shown... Figure 4 Figures 4(c) and 4(d) show the 3D and 2D views, respectively. The simulation parameters for the laser scanner can be set based on the actual parameters of a real laser scanner.
[0057] Simultaneously, a neural network for reconstructing mechanical, electrical, and pipeline scenarios is constructed. The neural network is trained using a point cloud dataset to obtain a reconstruction model for the mechanical, electrical, and pipeline scenarios. This model can be used in scenarios such as digital twins and digital operation and maintenance to achieve intelligent management and maintenance.
[0058] S5. Deploy the mechanical, electrical, and piping scene reconstruction model in the Building Information Model (BIM). Input the geometric information of the scene to be tested to obtain the reconstruction results of the mechanical, electrical, and piping scenes. The synthetic point cloud generated in this embodiment is fed into a deep learning model (such as PointNet, PointNet++, ResPointNet++) for training, and inference is performed on real MEP scenes to obtain point cloud scenes with semantic information, such as... Figure 5 As shown in the figure, (a) illustrates the actual semantic situation, and (b) illustrates the inferred semantic situation. For point cloud scenarios with semantic information, BIM models can be constructed in Revit for subsequent digital twin and intelligent maintenance processes.
[0059] The lab requirements for training include: a high-performance GPU (such as an NVIDIA RTX 4090), a Linux operating system, and the Python programming language, the trimesh library, and the PyTorch deep learning framework.
[0060] Step S2.1 Set the position of the virtual laser scanner in the scene, as follows:
[0061] , for The XYZ coordinates.
[0062] In step S2.2, the number of laser beams in the vertical and horizontal directions is calculated according to the specified angular resolution:
[0063] in, and These are the number of laser beams in the vertical and horizontal directions, respectively. and These are the field of view angles in the vertical and horizontal directions, respectively. It refers to angular resolution.
[0064] The parameters for virtual laser scanning in this embodiment are set as follows: horizontal field of view 360°, vertical field of view 300°, angular resolution 0.25°, and angular error conforming to a Gaussian distribution with a mean of 0 and a variance of 0.01° to simulate the positioning error of a laser scanner. Although actual laser scanners can achieve higher angular resolution, excessively high resolution will lead to resampling during deep learning model training. Therefore, this invention balances point cloud density, coverage, and processing efficiency by increasing the number of scanning stations.
[0065] Step S2.2 generates scan angle arrays in the vertical and horizontal directions, as follows:
[0066] ,
[0067] linspace indicates that values are generated at regular intervals.
[0068] Calculate the direction vector in the error simulation: for each vertical angle and horizontal angle Gaussian error is introduced to simulate the angular error between two directions in the real world:
[0069]
[0070] in, It is the introduced vertical angle error. It is the introduced horizontal angle error. It is the standard deviation of the vertical angle error. It is the standard deviation of the horizontal angle error.
[0071] In step S2.3, the adjusted horizontal and vertical angles are converted into horizontal radians. and vertical radii And apply the corresponding rotational offset, as follows:
[0072]
[0073] Use trigonometric functions to calculate the components of the direction vector. , and ,as follows:
[0074] , ,
[0075] Combine the x, y, and z direction components into a matrix D, as follows:
[0076] .
[0077] Step S2.4 specifically involves checking whether each ray intersects with a triangular mesh in the scene. If an intersection occurs, the distance from the intersection point to the ray's origin is calculated, and the nearest intersection point and its related information are recorded. This nearest intersection point becomes a point in the generated composite point cloud. The pseudocode for this method is shown below. The steps of the above RBLSIA method are implemented using the Trimesh library in Python.
[0078] Another aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described in the present invention.
[0079] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the methods described in the present invention.
[0080] While the present invention has been described above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.
Claims
1. A deep learning reconstruction method for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement, characterized in that, Includes the following steps: S1. Preprocess the building information model for mechanical, electrical, and piping scenarios to obtain the geometric information of the scene, including: vertices, faces, and normals, with vertices represented by XYZ coordinates; S2. Generate synthetic point clouds of mechanical, electrical, and piping scenes based on the RBLSIA (Ray Laser Scanning and Intersection Algorithm). Specifically: S2.
1. Place a virtual laser scanner at pre-selected scanning sites, create a simulated laser beam, and introduce Gaussian errors to simulate the parameters of a real laser scanner; then set the position of the virtual laser scanner in the scene. S2.2 Calculate the number of laser beams in the vertical and horizontal directions according to the specified angular resolution. and Calculation based on vertical field of view and horizontal field of view Generate arrays of vertical and horizontal scan angles. and , S2.3 Calculate and adjust the direction vector with error, convert the angle to radians, and use trigonometric functions to calculate the components of the direction vector. , and This ultimately forms matrix D; S2.
4. Use the RBLSIA method to calculate the nearest intersection point between each laser beam and the scene triangular mesh, and record the relevant information of the intersection point and the points in the generated synthetic point cloud; S3. Divide the mechanical, electrical, and pipeline scenes into passable areas, high-alert areas, and impassable areas. Evenly distribute scanning points in the passable areas to generate a synthetic point cloud of the mechanical, electrical, and pipeline scenes, and assign a semantic category label to each point. S4. Construct point cloud datasets for mechanical, electrical, and pipeline scenes, including synthetic point clouds of mechanical, electrical, and pipeline scenes, and semantic category labels corresponding to each point in the point cloud. Simultaneously, construct a reconstruction neural network for mechanical, electrical, and pipeline scenes, and train the neural network using the point cloud dataset to obtain reconstruction models for mechanical, electrical, and pipeline scenes. S5. Deploy the mechanical, electrical, and piping scene reconstruction model in the building information model, input the geometric information of the scene to be tested, and obtain the reconstruction results of the mechanical, electrical, and piping scenes.
2. The deep learning reconstruction method for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement according to claim 1, characterized in that, Step S2.1 Set the position of the virtual laser scanner in the scene, as follows: , for The XYZ coordinates.
3. The deep learning reconstruction method for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement according to claim 2, characterized in that, Calculate the number of laser beams in the vertical and horizontal directions based on the specified angular resolution: , in, and These are the number of laser beams in the vertical and horizontal directions, respectively. and These are the field of view angles in the vertical and horizontal directions, respectively. It refers to angular resolution.
4. The deep learning reconstruction method for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement according to claim 3, characterized in that, Step S2.2 generates scan angle arrays in the vertical and horizontal directions, as follows: ， linspace indicates that values are generated at regular intervals.
5. The deep learning reconstruction method for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement according to claim 4, characterized in that, Calculate the direction vector in the error simulation: for each vertical angle and horizontal angle Gaussian error is introduced to simulate the angular error between two directions in the real world: ， in, It is the introduced vertical angle error. It is the introduced horizontal angle error. It is the standard deviation of the vertical angle error. It is the standard deviation of the horizontal angle error.
6. The deep learning reconstruction method for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement according to claim 5, characterized in that, Convert the adjusted horizontal and vertical angles into horizontal radians. and vertical radii And apply the corresponding rotational offset, as follows: ， Use trigonometric functions to calculate the components of the direction vector. , and ,as follows: ， , ， Combine the x, y, and z direction components into a matrix D, as follows: 。 7. The deep learning reconstruction method for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement according to claim 1, characterized in that, Step S2.4 specifically involves checking whether each ray intersects with the triangular mesh in the scene. If an intersection occurs, the distance from the intersection point to the origin of the ray is calculated, and the nearest intersection point and its related information are recorded. The nearest intersection point is a point in the generated synthetic point cloud.
8. The deep learning reconstruction method for mechanical, electrical, and pipeline scenes based on synthetic point cloud enhancement according to claim 1, characterized in that, In step S4, the construction of the reconstruction neural network for mechanical, electrical, and pipeline scenes specifically utilizes one of the following: PointNet network, PointNet++ network, or ResPointNet++ network.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.