Mechanical, electrical and pipeline scene deep learning reconstruction method and system based on synthetic point cloud enhancement

Abstract

The invention discloses a mechanical, electrical and pipeline scene deep learning reconstruction method and system based on synthetic point cloud enhancement, and aims to solve the problem that a current deep learning model depends on a large amount of real marked point cloud data. According to the method, the synthetic point cloud is generated from the building information model BIM, and the real laser scanning process is simulated, so that the point cloud data capable of reflecting the shielding characteristics of the real scene is generated. Specifically, the method comprises the following steps: firstly, initializing the position, scanning resolution, scanning error and passable area of a laser scanner, and then sampling a BIM model by using a laser ray intersection algorithm to generate a synthetic point cloud; and then, improving semantic segmentation accuracy of an MEP scene by training a deep learning model, thereby realizing high-precision MEP model reconstruction. Operation and maintenance of mechanical, electrical and pipeline components in buildings can be effectively promoted, and the working efficiency and quality in related fields are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of civil engineering and artificial intelligence interaction, and specifically relates to a deep learning reconstruction method and system for mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement. Background Art

[0002] Mechanical, electrical, and plumbing (MEP) scenes are important components of buildings. Reconstructing the Building Information Model (BIM) of MEP components through point cloud-based deep learning methods can effectively promote the operation and maintenance of these components. However, deep learning training usually relies on a large number of real and annotated point cloud datasets, which is time-consuming and labor-intensive. In order to solve the problem of insufficient point cloud data of real scenes, a method for automatically generating synthetic point clouds for MEP scenes is urgently needed. By generating synthetic point clouds for training, the accuracy of semantic segmentation of MEP scenes can be improved, thereby supporting high-quality reconstruction of the model. Summary of the invention

[0003] In view of the problems existing in the prior art, the present invention provides a method and system for deep learning reconstruction of mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement, which can improve the accuracy of semantic segmentation in MEP scenes and thus achieve high-precision MEP model reconstruction. The method generates synthetic point clouds from building information models, simulates the real laser scanning process, and generates point cloud data that can reflect the occlusion of real scenes. And the accuracy of MEP scene model reconstruction is improved through synthetic point cloud training.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: a deep learning reconstruction method for mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement, comprising the following steps:

[0005] S1. Preprocess the building information model of mechanical, electrical and pipeline scenes to obtain the geometric information of the scene, including: vertices, faces and normals, where vertices are represented by XYZ coordinates;

[0006] S2, based on ray laser scanning and intersection algorithm RBLSIA to generate synthetic point clouds for mechanical, electrical and pipeline scenes, specifically:

[0007] S2.1, place a virtual laser scanner at a pre-selected scanning site, create a simulated laser beam, and introduce Gaussian errors to simulate the parameters of a real laser scanner; then set the position of the virtual laser scanner in the scene,

[0008] S2.2. Calculate the number of laser beams n in the vertical and horizontal directions according to the specified angular resolution v and n h , calculated based on the vertical field of view angle θ v_range and horizontal field of view angle θh_range ; Generate vertical and horizontal scanning angle array Θ v and θ h ,

[0009] S2.3. Calculate the direction vector with error and make adjustments, convert the angle to radians, and use trigonometric functions to calculate the component d of the direction vector x ,d y and d z , and finally form the matrix D;

[0010] S2.4, using the RBLSIA method to calculate the closest intersection point between each laser beam and the scene triangular mesh, and record the relevant information of the intersection point and the point in the generated synthetic point cloud;

[0011] S3, divide the mechanical, electrical and pipeline scenes into passable areas, high warning areas and impassable areas, evenly arrange scanning points in the walkable areas for scanning, generate synthetic point clouds of mechanical, electrical and pipeline scenes, and assign semantic category labels to each point;

[0012] S4, constructing point cloud datasets of mechanical, electrical and pipeline scenes, including synthetic point clouds of mechanical, electrical and pipeline scenes, and semantic category labels corresponding to each point in the point clouds, and constructing mechanical, electrical and pipeline scene reconstruction neural networks, training the neural networks using the point cloud datasets, and obtaining mechanical, electrical and pipeline scene reconstruction models;

[0013] S5. Deploy the mechanical, electrical and pipeline scene reconstruction models in the building information model, input the geometric information of the scene to be tested to obtain the reconstruction results of the mechanical, electrical and pipeline scenes.

[0014] Furthermore, the aforementioned step S2.1 sets the position of the virtual laser scanner in the scene as follows:

[0015] P s =[x s ,y s , z s ], where x s ,y s , z s P s The XYZ coordinates of the

[0016] Furthermore, the aforementioned step S2.2 of creating a simulated laser beam based on the parameters of the actual laser scanner is: calculating the number of laser beams in the vertical and horizontal directions according to the specified angular resolution:

[0017]

[0018] Among them, n v and n hare the number of laser beams in the vertical and horizontal directions, θ v_range and θ h_range are the vertical and horizontal field of view, respectively, and Δθ is the angular resolution.

[0019] Furthermore, the aforementioned step S2.2 generates scanning angle arrays in the vertical and horizontal directions as follows:

[0020] Θ h =linspace(0,θ h_range , n h ),

[0021] linspace means generating values ​​at certain intervals.

[0022] Furthermore, the aforementioned deep learning reconstruction method for mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement calculates the direction vector with error simulation: for each vertical angle θ v and horizontal angle θ h , introduce Gaussian error to simulate the angular error in two directions in the real world:

[0023] θ ve ~N(0,σ v ), θ he ~N(0,σ h )

[0024] Among them, θ ve is the vertical angle error introduced, θ he is the horizontal angle error introduced, σ v is the standard deviation of the vertical angle error, σ h is the standard deviation of the horizontal angle error.

[0025] Furthermore, in the aforementioned step S2.3, the direction vector with error is calculated and adjusted, and the adjusted horizontal and vertical angles are converted into horizontal radians. and vertical arc And apply the corresponding rotation offset as follows:

[0026]

[0027] Use trigonometric functions to calculate the components of the direction vector d x ,d y and d z ,as follows:

[0028]

[0029] The x, y, and z direction components are combined into a matrix D as follows:

[0030]

[0031] Furthermore, the aforementioned step S2.4 is specifically: checking whether each ray intersects with the triangular mesh in the scene; if an intersection occurs, calculating the distance from the intersection point to the origin of the ray, and recording the nearest intersection point and its related information, wherein the nearest intersection point is a point in the generated synthetic point cloud.

[0032] Furthermore, in the aforementioned step S4, the mechanical, electrical and pipeline scene reconstruction neural network is constructed by specifically utilizing: one of the PointNet network, the PointNet++ network and the Res PointNet++ network.

[0033] Another aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the present invention when executing the computer program.

[0034] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any one of the methods described in the present invention when executed by a processor.

[0035] Compared with the prior art, the beneficial technical effects of the present invention using the above technical solution are as follows:

[0036] By using synthetic point cloud generation and enhancement methods, the present invention greatly improves the semantic segmentation accuracy and efficiency of MEP scenes, and significantly reduces the time for manual scanning and annotation of real point clouds. Using the trained model, large-scale point cloud data can be processed quickly and efficiently to generate accurate three-dimensional models to support the construction, monitoring and maintenance of underground garages. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the method of the present invention.

[0038] Figure 2 This is an example diagram of the sampling point cloud of the simulated laser scanner of the present invention.

[0039] Figure 3 It is a walkable map of the present invention, in which (a) is the initial BIM model diagram and (b) is the 2D walking map.

[0040] Figure 4 This is an example diagram of the scanning process of the virtual laser scanner of the present invention. In the figure, (a) is a virtual laser scanning position diagram, (b) is an emitted laser ray diagram, (c) is a generated three-dimensional view of the synthetic point cloud, and (d) is a generated planar view of the synthetic point cloud.

[0041] Figure 5It is a point cloud scene graph with semantic information. (a) is the real semantic situation graph, and (b) is the reasoning semantic situation graph. DETAILED DESCRIPTION

[0042] In order to better understand the technical content of the present invention, specific embodiments are given and described as follows in conjunction with the accompanying drawings.

[0043] Various aspects of the invention are described herein with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the invention are not limited to those described in the accompanying drawings. It should be understood that the invention is implemented by any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed in the invention are not limited to any implementation. In addition, some aspects disclosed in the invention may be used alone or in any appropriate combination with other aspects disclosed in the invention.

[0044] refer to Figure 1 The present invention provides a deep learning reconstruction method for mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement, comprising the following steps:

[0045] S1. Preprocess the building information model (BIM) of mechanical, electrical and pipeline scenes to obtain the geometric information of the scene, including vertices, faces and normals. Vertices are represented by XYZ coordinates.

[0046] The BIM model includes typical mechanical, electrical, and plumbing (MEP) components, such as pipes, I-beams, pumps, R-beams, tanks, and other typical MEP components, with clear semantic annotations. Subsequently, the BIM model is converted into an OBJ format file, and the OBJ format file is read in python through trimesh. The OBJ format is a widely used 3D model file format that supports a variety of modeling software (such as Revit, Blender, and Maya), and can be parsed through the Trimesh library. The structure of the OBJ file includes vertices, faces, and normals. Vertices are represented by XYZ coordinates, and each vertex provides geometric information of the model;

[0047] S2. Generate synthetic point clouds for mechanical, electrical, and piping scenes based on ray laser scanning and intersection algorithm RBLSIA. First, place a virtual laser scanner at a pre-selected scanning site, create a simulated laser beam based on the parameters of the actual laser scanner, and introduce a Gaussian error in the direction of the laser beam. Then, use RBLSIA to calculate the nearest intersection of each laser beam with the scene triangular mesh, and record relevant information about the intersection (such as material, color, and category). These intersections will become points in the generated synthetic point cloud. The algorithm simulates the laser scanner emitting lasers to generate a synthetic point cloud. Figure 2 Specifically:

[0048] S2.1, place a virtual laser scanner at a pre-selected scanning site, create a simulated laser beam, and introduce Gaussian errors to simulate the parameters of a real laser scanner; then set the position of the virtual laser scanner in the scene,

[0049] S2.2. Calculate the number of laser beams n in the vertical and horizontal directions according to the specified angular resolution v and n h , calculated based on the vertical field of view angle θ v_range and horizontal field of view angle θ h_range ; Generate vertical and horizontal scanning angle array Θ v and θ h ,

[0050] S2.3. Calculate the direction vector with error and make adjustments, convert the angle to radians, and use trigonometric functions to calculate the component d of the direction vector x ,d y and d z , and finally form the matrix D;

[0051] S2.4, using the RBLSIA method to calculate the closest intersection point between each laser beam and the scene triangular mesh, and record the relevant information of the intersection point and the point in the generated synthetic point cloud;

[0052] S3. Based on the actual situation of the scene, the scanning site planning follows the following principles: When the scene is initialized, the boundary information of the scene and the position and height of all components are obtained, and the scene is divided into a passable area, a high warning area, and an inaccessible area. Inaccessible areas (such as areas containing MEP components) are marked as obstacles to ensure the accessibility of real-world scanning. Scanning sites are evenly distributed in the walkable area to ensure full coverage of the scene.

[0053] S4, build point cloud datasets for mechanical, electrical and pipeline scenes, including synthetic point clouds of mechanical, electrical and pipeline scenes, and semantic category labels corresponding to each point in the point cloud, such as pipes, brackets, valves, etc. These areas are marked as obstacles and cannot be passed; the height area indicates that it is passable, but attention should be paid to the height of the I-shaped beam and R-shaped beam to avoid limited or blocked viewing angles. Other areas are marked as walkable areas. Figure 3As shown in the figure, a walkability map is generated. Next, scanning sites are evenly selected in the walkable area to ensure full coverage of the entire scene. The selection of sites follows the following conditions: first, a safety distance of at least 0.5 meters should be maintained between the site and the nearest pipeline to avoid a too narrow scanning range; second, the distances between sites should be reasonably distributed to ensure effective coverage of laser scanning and reduce blind spots. In the simulation environment, since the virtual laser scanner can move freely without incurring time costs, a wider coverage range than the real point cloud scan is generated by increasing the number of scanning sites.

[0054] like Figure 4 As shown, the present invention uses a virtual laser scanning method to generate a synthetic point cloud. Figure 4 (a) shows the location of the scanner and Figure 4 The intersection process of the laser ray and the intersection point of the BIM model in (b) is shown in Figure 1. Finally, the synthetic point cloud of the MEP scene is generated, as shown in Figure 1. Figure 4 4(c) and 4(d) show the three-dimensional view and plane view respectively. For the laser scanner simulation parameter setting, it can be set according to the real laser scanner parameters.

[0055] At the same time, a neural network for mechanical, electrical and pipeline scene reconstruction is constructed, and the neural network is trained using a point cloud data set to obtain a mechanical, electrical and pipeline scene reconstruction model; this model can be used in scenarios such as digital twins and digital operation and maintenance to achieve intelligent management and maintenance.

[0056] S5. Deploy the mechanical, electrical and pipeline scene reconstruction models in the building information model, input the geometric information of the scene to be tested to obtain the reconstruction results of the mechanical, electrical and pipeline scenes. The synthetic point cloud generated in this embodiment is brought into a deep learning model (such as PointNet, PointNet++, Res PointNet++) for training, and inference is performed on the real MEP scene to obtain a point cloud scene with semantic information, such as Figure 5 As shown in the figure, (a) shows the real semantic situation diagram, and (b) shows the inference semantic situation diagram. For point cloud scenes with semantic information, BIM models can be constructed in Revit for subsequent digital twin and intelligent maintenance processes.

[0057] The laboratory conditions required for training include: a high-performance GPU (such as NVIDIA RTX 4090), running a Linux system, using the Python programming language, the trimesh library, and the Pytorch deep learning framework.

[0058] Step S2.1 sets the position of the virtual laser scanner in the scene as follows:

[0059] P s=[x s ,y s ,z s ], where x s ,y s ,z s P s The XYZ coordinates of the

[0060] Step S2.2 creates a simulated laser beam based on the parameters of the actual laser scanner: Calculate the number of laser beams in the vertical and horizontal directions according to the specified angular resolution:

[0061]

[0062] Among them, n v and n h are the number of laser beams in the vertical and horizontal directions, θ v_range and θ h_range are the vertical and horizontal field of view, respectively, and Δθ is the angular resolution.

[0063] The parameters of the virtual laser scanning in this embodiment are set as follows: horizontal field of view 360°, vertical field of view 300°, angular resolution 0.25°, and angular error conforms to a Gaussian distribution with a mean of 0 and a variance of 0.01° to simulate the positioning error of the laser scanner. Although actual laser scanners can achieve higher angular resolution, excessively high resolution will be resampled during deep learning model training. Therefore, the present invention balances point cloud density, coverage, and processing efficiency by increasing the number of scanning sites.

[0064] Step S2.2 generates scan angle arrays in the vertical and horizontal directions as follows:

[0065] Θ h =linspace(0,θ h_range ,n h ),

[0066] linspace means generating values ​​at certain intervals.

[0067] Calculate the direction vector with error simulation: For each vertical angle θ v and horizontal angle θ h , introduce Gaussian error to simulate the angular error in two directions in the real world:

[0068] θ ve ~N(0,σ v ),θ he ~N(0,σ h )

[0069] Among them, θ veis the vertical angle error introduced, θ he is the horizontal angle error introduced, σ v is the standard deviation of the vertical angle error, σ h is the standard deviation of the horizontal angle error.

[0070] In step S2.3, the direction vector with error is calculated and adjusted, and the adjusted horizontal and vertical angles are converted into horizontal radians. and vertical arc And apply the corresponding rotation offset as follows:

[0071]

[0072] Use trigonometric functions to calculate the components of the direction vector d x ,d y and d z ,as follows:

[0073]

[0074] The x, y, and z direction components are combined into a matrix D as follows:

[0075]

[0076] Step S2.4 is specifically: check whether each ray intersects with the triangular mesh in the scene. If an intersection occurs, calculate the distance from the intersection point to the origin of the ray, and record the nearest intersection point and its related information. The nearest intersection point is a point in the generated synthetic point cloud. The pseudo code of the method is as follows. The steps of the above RBLSIA method are implemented based on the Trimesh library in Python.

[0077]

[0078] Another aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the present invention when executing the computer program.

[0079] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any one of the methods described in the present invention when executed by a processor.

[0080] Although the present invention has been described above with preferred embodiments, it is not intended to limit the present invention. A person skilled in the art of the present invention may make various modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the definition of the claims.

Claims

1. A deep learning reconstruction method for mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement, characterized in that: The following steps are involved: S1. Preprocess the building information model of mechanical, electrical and pipeline scenes to obtain the geometric information of the scene, including: vertices, faces and normals, where vertices are represented by XYZ coordinates; S2, based on ray laser scanning and intersection algorithm RBLSIA to generate synthetic point clouds for mechanical, electrical and pipeline scenes, specifically: S2.1, place a virtual laser scanner at a pre-selected scanning site, create a simulated laser beam, and introduce Gaussian errors to simulate the parameters of a real laser scanner; then set the position of the virtual laser scanner in the scene, S2.

2. Calculate the number of laser beams n in the vertical and horizontal directions according to the specified angular resolution v and n h , calculated based on the vertical field of view angle θ v_range and horizontal field of view angle θ h_range ; Generate vertical and horizontal scanning angle array Θ v and θ h , S2.

3. Calculate the direction vector with error and make adjustments, convert the angle to radians, and use trigonometric functions to calculate the component d of the direction vector x d y and d z , and finally form the matrix D; S2.4, using the RBLSIA method to calculate the closest intersection point between each laser beam and the scene triangular mesh, and record the relevant information of the intersection point and the point in the generated synthetic point cloud; S3, divide the mechanical, electrical and pipeline scenes into passable areas, high warning areas and impassable areas, evenly arrange scanning points in the walkable areas for scanning, generate synthetic point clouds of mechanical, electrical and pipeline scenes, and assign semantic category labels to each point; S4, constructing point cloud datasets of mechanical, electrical and pipeline scenes, including synthetic point clouds of mechanical, electrical and pipeline scenes, and semantic category labels corresponding to each point in the point clouds, and constructing mechanical, electrical and pipeline scene reconstruction neural networks, training the neural networks using the point cloud datasets, and obtaining mechanical, electrical and pipeline scene reconstruction models; S5. Deploy the mechanical, electrical and pipeline scene reconstruction models in the building information model, input the geometric information of the scene to be tested to obtain the reconstruction results of the mechanical, electrical and pipeline scenes.

2. According to claim 1, a method for deep learning reconstruction of mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement is characterized in that: Step S2.1 sets the position of the virtual laser scanner in the scene as follows: P s =[x s ,y s ,z s ], where x s ,y s ,z s P s The XYZ coordinates of the 3. The method for deep learning reconstruction of mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement according to claim 2 is characterized in that: Step S2.2 creates a simulated laser beam based on the parameters of the actual laser scanner: Calculate the number of laser beams in the vertical and horizontal directions according to the specified angular resolution: Among them, n v and n h are the number of laser beams in the vertical and horizontal directions, θ v_range and θ h_range are the vertical and horizontal field of view, respectively, and Δθ is the angular resolution.

4. The method for deep learning reconstruction of mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement according to claim 3 is characterized in that: Step S2.2 generates scan angle arrays in the vertical and horizontal directions as follows: linspace means generating values ​​at certain intervals.

5. The method for deep learning reconstruction of mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement according to claim 4 is characterized in that: Calculate the direction vector with error simulation: For each vertical angle θ v and the horizontal angle θ h , introduce Gaussian error to simulate the angular error in two directions in the real world: i ve ~N(0,σ v ),θ he ~N(0,σ h ) Among them, θ ve is the vertical angle error introduced, θ he is the horizontal angle error introduced, σ v is the standard deviation of the vertical angle error, σ h is the standard deviation of the horizontal angle error.

6. The method for deep learning reconstruction of mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement according to claim 5, characterized in that: In step S2.3, the direction vector with error is calculated and adjusted, and the adjusted horizontal and vertical angles are converted into horizontal radians. and vertical arc And apply the corresponding rotation offset as follows: Use trigonometric functions to calculate the components of the direction vector d x d y and d z ,as follows: The x, y, and z direction components are combined into a matrix D as follows:

7. The method for deep learning reconstruction of mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement according to claim 1, characterized in that: Step S2.4 is specifically: check whether each ray intersects with the triangular mesh in the scene. If an intersection occurs, calculate the distance from the intersection point to the origin of the ray, and record the nearest intersection point and its related information. The nearest intersection point is a point in the generated synthetic point cloud.

8. The method for deep learning reconstruction of mechanical, electrical and pipeline scenes based on synthetic point cloud enhancement according to claim 1, characterized in that: In step S4, the mechanical, electrical and pipeline scene reconstruction neural network is constructed by using one of the following: PointNet network, PointNet++ network and Res PointNet++ network.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images