Three-dimensional point cloud generation and semantic segmentation model training method and system based on BIM

Abstract

The invention discloses a BIM-based three-dimensional point cloud generation and semantic segmentation model training method and system. The method comprises the steps of generating synthetic point cloud data with labels based on a tunnel BIM model; collecting actual measurement point cloud data of the tunnel, and marking the actual measurement point cloud data; and training a semantic segmentation model through the synthetic point cloud data and the actually measured point cloud data. According to the BIM-based three-dimensional point cloud generation and semantic segmentation model training method and system provided by the invention, the synthetic point cloud data and the actually measured point cloud data are fused to train the semantic segmentation model, so that the recognition precision of the model is improved, and the recognition effect of the three-dimensional point cloud data of the tunnel is improved.

Description

Technical Field

[0001] The present invention belongs to the field of tunnel model construction, and in particular relates to a BIM-based three-dimensional point cloud generation and semantic segmentation model training method and system. Background Art

[0002] To construct a tunnel building information model (BIM) during the subway operation period, a laser scanner is usually manually operated to collect the tunnel's 3D point cloud data on site, thereby realizing a digital twin for tunnel disease remediation analysis and maintenance decision-making during the operation period. To construct a tunnel BIM model using 3D point cloud data, the 3D point cloud data needs to be labeled, which greatly increases labor costs.

[0003] In the prior art, in order to improve the efficiency of marking 3D point cloud data, a neural network model is used to mark 3D point cloud data. However, the existing model has weak classification and recognition capabilities due to the small amount of training data and single data conditions, which affects the accuracy of marking. Summary of the invention

[0004] The present invention provides a BIM-based 3D point cloud generation and semantic segmentation model training method and system to solve the above-mentioned technical problems, and specifically adopts the following technical solutions:

[0005] A BIM-based 3D point cloud generation and semantic segmentation model training method, comprising:

[0006] Generate synthetic point cloud data with labels based on the BIM model of the tunnel;

[0007] Collecting measured point cloud data of the tunnel and marking the measured point cloud data;

[0008] A semantic segmentation model is trained using the synthetic point cloud data and the measured point cloud data.

[0009] Furthermore, the specific method for generating synthetic point cloud data with labels based on the BIM model of the tunnel is:

[0010] Converting the BIM model into a three-dimensional mesh model containing only spatial information;

[0011] The three-dimensional grid model is scanned by simulating a multi-point scanning method to generate the synthetic point cloud data.

[0012] Furthermore, the specific method of converting the BIM model into a three-dimensional grid model containing only spatial information is:

[0013] Converting the BIM model into a CAD model;

[0014] The CAD model is simplified and converted to obtain the three-dimensional mesh model.

[0015] Furthermore, the specific method of simplifying and converting the CAD model to obtain the three-dimensional mesh model is:

[0016] Importing the CAD model into three-dimensional modeling software for shell extraction;

[0017] Export the shelled model as a preliminary three-dimensional mesh model;

[0018] The preliminary three-dimensional mesh model is then three-dimensionally segmented to convert it into the three-dimensional mesh model containing only triangular meshes.

[0019] Furthermore, the ratio of the data volume of the synthetic point cloud data to the measured point cloud data is 8:1.

[0020] A BIM-based 3D point cloud generation and semantic segmentation model training system, comprising:

[0021] A generation module for generating labeled synthetic point cloud data based on the BIM model of the tunnel;

[0022] A collection module, used for collecting measured point cloud data of the tunnel and marking the measured point cloud data;

[0023] The semantic segmentation module includes a semantic segmentation model, and the semantic segmentation model is trained by the synthetic point cloud data and the measured point cloud data.

[0024] Furthermore, the generation module comprises:

[0025] A conversion unit, used to convert the BIM model into a three-dimensional grid model containing only spatial information;

[0026] The scanning unit is used to simulate a multi-point scanning method to scan the three-dimensional grid model to generate the synthetic point cloud data.

[0027] Furthermore, the specific method of the conversion unit converting the BIM model into a three-dimensional grid model containing only spatial information is:

[0028] Converting the BIM model into a CAD model;

[0029] The CAD model is simplified and converted to obtain the three-dimensional mesh model.

[0030] Furthermore, the specific method by which the conversion unit simplifies and converts the CAD model to obtain the three-dimensional mesh model is:

[0031] Importing the CAD model into three-dimensional modeling software for shell extraction;

[0032] Export the shelled model as a preliminary three-dimensional mesh model;

[0033] The preliminary three-dimensional mesh model is then three-dimensionally segmented to convert it into the three-dimensional mesh model containing only triangular meshes.

[0034] Furthermore, the ratio of the amount of synthetic point cloud data generated by the generation module to the amount of measured point cloud data collected by the collection module is 8:1.

[0035] The benefit of the present invention lies in the BIM-based 3D point cloud generation and semantic segmentation model training method and system provided, which integrates synthetic point cloud data and measured point cloud data to train the semantic segmentation model, improves the recognition accuracy of the model, and improves the recognition effect of the 3D point cloud data of the tunnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 It is a schematic diagram of a BIM-based 3D point cloud generation and semantic segmentation model training method of the present invention;

[0038] Figure 2 is a schematic diagram of generating synthetic point cloud data based on a BIM model according to the present invention;

[0039] Figure 3 It is a schematic diagram of a BIM-based three-dimensional point cloud generation and semantic segmentation model training system of the present invention. DETAILED DESCRIPTION

[0040] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0041] like Figure 1The present application shows a method for generating three-dimensional point cloud and training a semantic segmentation model based on BIM, comprising: S1: generating synthetic point cloud data with labels based on the BIM model of the tunnel. S2: collecting the measured point cloud data of the tunnel and marking the measured point cloud data. S3: training a semantic segmentation model through synthetic point cloud data and measured point cloud data. The present application generates synthetic point cloud data with labels through a BIM model, and trains a semantic segmentation model by fusing the synthetic point cloud data and the measured point cloud data, thereby improving the accuracy of semantic segmentation. The above steps are described in detail below.

[0042] For step S1: generate synthetic point cloud data with labels based on the BIM model of the tunnel.

[0043] During the construction period, an accurate BIM model will be built for the subway tunnel. In the BIM model, all kinds of facilities have been segmented and labeled. The BIM model during the construction period can generate synthetic point cloud data with labels. The surface of the BIM model is scanned to simulate the collection principle of actual point cloud data to ensure that the generated point cloud data is closer to the real point cloud data, which is more conducive to training the 3D point cloud semantic segmentation deep learning model.

[0044] refer to Figure 2 As shown, in the implementation of the present application, the specific method for generating synthetic point cloud data with labels based on the BIM model of the tunnel is:

[0045] S11: Convert the BIM model into a three-dimensional mesh model containing only spatial information.

[0046] Only the spatial information of the tunnel model is needed to construct the synthetic point cloud data. Therefore, preferably, the BIM model containing complex information is first converted into a three-dimensional mesh model containing only spatial information.

[0047] In the implementation of the present application, the specific method of converting the BIM model into a three-dimensional grid model containing only spatial information is:

[0048] Convert BIM models into CAD models. Use mainstream BIM software such as Revit or Bentley MicroStation to convert BIM models (rvt or dgn format) into CAD models (dwg format).

[0049] The CAD model is simplified and converted to obtain a three-dimensional mesh model. Specifically, the CAD model is imported into the three-dimensional modeling software SketchUp, and the shell is extracted to further simplify the model to reduce its complexity. SketchUp will thin the model in the normal direction of the model to a specified thickness, and then generate a simplified shell consisting of surface points and faces.

[0050] SketchUp exports the shelled model as a preliminary 3D mesh model (stl format). The preliminary 3D mesh model is then divided into three dimensions and further converted into a 3D mesh model containing only triangular meshes. The final 3D mesh model only contains surface points and triangular faces, which greatly reduces the amount of calculation for subsequent processing.

[0051] S12: Simulate a multi-point scanning method to scan the three-dimensional mesh model to generate synthetic point cloud data.

[0052] To simulate 3D laser scanning, the raycasting function of Open3D is used in this application to simulate multi-point scanning. Specifically, a virtual point light source is placed at the center of the tunnel model, and lasers are randomly emitted in multiple directions. The intersection of the laser and the tunnel model can be used as point cloud data. The intersection is calculated by the intersection of the spatial geometric midline and the triangular face, as shown in the following formula:

[0053] r(t)=P 0 +td (1-1)

[0054] n·(rA)=0 (1-2)

[0055]

[0056] Among them, r(t) in formula (1-1) is the parameter equation of the virtual laser, P 0 (x 0 ,y 0 ,z 0 ) is the laser source point, d(d x ,d y ,d z ) is the laser direction vector, and t is the parameter to be determined.

[0057] Formula (1-2) is the plane equation of the triangular mesh surface, which is defined by A(x 1 ,y 1 ,z 1 ),B(x 2 ,y 2 ,z 2 ),C(x 3 ,y 3 ,z 3 ) is composed of three points, n = (BA) × (CA) is the normal vector of the plane, and r is the coordinate vector of any point on the plane.

[0058] The parameter A in the above formulas (1-2) and (1-3) represents the point A(x 1 ,y 1 ,z 1 ) this vector.

[0059] Formula (1-3) is the desired parameter t of the intersection of the laser and the plane, which is obtained by substituting formula (1-1) into formula (1-2). Finally, substitute t obtained by formula (1-3) into formula (1-1) to obtain the coordinates of the intersection.

[0060] Based on the above-mentioned simulated single-point scanning, the present application further simulates the actual three-dimensional laser scanning technology by simulating multi-point scanning. That is, multiple virtual point light sources are placed equidistantly along the tunnel, and the three-dimensional point cloud data obtained by scanning each virtual point light source are spliced ​​to obtain complete and evenly distributed synthetic three-dimensional point cloud data. It can be understood that all tunnel components exist and are labeled during the virtual scanning process. Therefore, the virtual scanning can not only simulate the laser blocking phenomenon caused by the overlapping of tunnel components during the real scanning process, but also automatically label the point cloud data according to the scanned tunnel components (segments, tracks, track plates, sidewalks, signal facilities and ring networks), and obtain a large amount of point cloud data that is close to the real and labeled. This approach takes into account the mutual occlusion between components, making the obtained data closer to the real thing.

[0061] Regarding step S2: collecting the measured point cloud data of the tunnel, and marking the measured point cloud data.

[0062] Static or dynamic laser scanners are used to manually collect measured point cloud data during the tunnel maintenance window period during the operation period, and the collected data are manually marked.

[0063] For step S3: train a semantic segmentation model using synthetic point cloud data and measured point cloud data.

[0064] This application selects PointNet++ as the semantic segmentation model and uses multi-source fusion tunnel point cloud data to train the model. Specifically, the training data set consists of two parts: (a) synthetic point cloud data obtained based on virtual multi-point scanning of the BIM model; (b) measured point cloud data obtained based on three-dimensional laser scanning and manual calibration. The ratio of synthetic point cloud data to measured point cloud data is 8:1, which greatly reduces the demand for manually labeled data. The model parameters are trained using the stochastic gradient descent algorithm. After testing, the segmentation effect of the model trained with multi-source fusion data (MIoU: 0.71) is better than that of using synthetic data (MIoU: 0.30) or measured data (MIoU: 0.67) alone. The test results are shown in Figure 3 .

[0065] The trained semantic segmentation model can be used to classify the 3D point cloud data of the measured tunnel. During the tunnel operation period, the 3D point cloud data is collected. After being processed by the trained semantic segmentation model, the collected 3D point cloud data can be further used to build the tunnel BIM model.

[0066] like Figure 3 As shown, the present application also discloses a BIM-based 3D point cloud generation and semantic segmentation model training system, which is used to implement the aforementioned BIM-based 3D point cloud generation and semantic segmentation model training system, and comprises:

[0067] A generation module is used to generate labeled synthetic point cloud data based on the BIM model of the tunnel.

[0068] The acquisition module is used to collect the measured point cloud data of the tunnel and mark the measured point cloud data.

[0069] The semantic segmentation module includes a semantic segmentation model, which is trained using synthetic point cloud data and measured point cloud data.

[0070] In an embodiment of the present application, the generation module includes:

[0071] A conversion unit is used to convert the BIM model into a three-dimensional mesh model containing only spatial information.

[0072] The scanning unit is used to simulate the multi-point scanning method to scan the three-dimensional grid model and generate synthetic point cloud data.

[0073] In the implementation manner of the present application, the specific method of the conversion unit converting the BIM model into a three-dimensional grid model containing only spatial information is:

[0074] Convert BIM models into CAD models.

[0075] The CAD model is simplified and converted to obtain a three-dimensional mesh model.

[0076] In the implementation manner of the present application, the specific method by which the conversion unit simplifies and converts the CAD model to obtain the three-dimensional mesh model is:

[0077] Import the CAD model into 3D modeling software for shell extraction.

[0078] Export the shelled model as a preliminary 3D mesh model.

[0079] The preliminary three-dimensional mesh model is then divided into three dimensions to convert it into a three-dimensional mesh model containing only triangular meshes.

[0080] In an embodiment of the present application, the ratio of the amount of synthetic point cloud data generated by the generation module to the amount of measured point cloud data collected by the collection module is 8:1.

[0081] The specific content of the BIM-based 3D point cloud generation and semantic segmentation model training system corresponds to the aforementioned BIM-based 3D point cloud generation and semantic segmentation model training method, which will not be repeated here.

[0082] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A method for generating 3D point cloud and training semantic segmentation model based on BIM, characterized in that: Include: Generate synthetic point cloud data with labels based on the BIM model of the tunnel; Collecting measured point cloud data of the tunnel and marking the measured point cloud data; A semantic segmentation model is trained using the synthetic point cloud data and the measured point cloud data.

2. The method for generating 3D point cloud and training semantic segmentation model based on BIM according to claim 1, characterized in that: The specific method for generating synthetic point cloud data with labels based on the BIM model of the tunnel is: Converting the BIM model into a three-dimensional mesh model containing only spatial information; The three-dimensional grid model is scanned by simulating a multi-point scanning method to generate the synthetic point cloud data.

3. The method for generating 3D point cloud and training semantic segmentation model based on BIM according to claim 2, characterized in that: The specific method of converting the BIM model into a three-dimensional grid model containing only spatial information is: Converting the BIM model into a CAD model; The CAD model is simplified and converted to obtain the three-dimensional mesh model.

4. The method for generating 3D point cloud and training semantic segmentation model based on BIM according to claim 3, characterized in that: The specific method of simplifying and converting the CAD model to obtain the three-dimensional mesh model is: Importing the CAD model into three-dimensional modeling software for shell extraction; Export the shelled model as a preliminary three-dimensional mesh model; The preliminary three-dimensional mesh model is then three-dimensionally segmented to convert it into the three-dimensional mesh model containing only triangular meshes.

5. The method for generating 3D point cloud and training semantic segmentation model based on BIM according to claim 1, characterized in that: The ratio of the data volume of the synthetic point cloud data to the measured point cloud data is 8:

1.

6. A BIM-based 3D point cloud generation and semantic segmentation model training system, characterized in that: Include: A generation module for generating labeled synthetic point cloud data based on the BIM model of the tunnel; A collection module, used for collecting measured point cloud data of the tunnel and marking the measured point cloud data; The semantic segmentation module includes a semantic segmentation model, and the semantic segmentation model is trained by the synthetic point cloud data and the measured point cloud data.

7. The BIM-based 3D point cloud generation and semantic segmentation model training system according to claim 6, characterized in that: The generation module comprises: A conversion unit, used to convert the BIM model into a three-dimensional grid model containing only spatial information; The scanning unit is used to simulate a multi-point scanning method to scan the three-dimensional grid model to generate the synthetic point cloud data.

8. The BIM-based 3D point cloud generation and semantic segmentation model training system according to claim 7, characterized in that: The specific method for the conversion unit to convert the BIM model into a three-dimensional grid model containing only spatial information is: Converting the BIM model into a CAD model; The CAD model is simplified and converted to obtain the three-dimensional mesh model.

9. The BIM-based 3D point cloud generation and semantic segmentation model training system according to claim 8, characterized in that: The specific method of the conversion unit simplifying and converting the CAD model to obtain the three-dimensional mesh model is: Importing the CAD model into three-dimensional modeling software for shell extraction; Export the shelled model as a preliminary three-dimensional mesh model; The preliminary three-dimensional mesh model is then three-dimensionally segmented to convert it into the three-dimensional mesh model containing only triangular meshes.

10. The BIM-based 3D point cloud generation and semantic segmentation model training system according to claim 6, characterized in that: The ratio of the amount of synthetic point cloud data generated by the generation module to the amount of measured point cloud data collected by the collection module is 8:1.

Images