A method for generating high-precision 3D models from building point cloud data

Abstract

The present invention discloses a method for generating a high-precision three-dimensional model of building point cloud data, which belongs to the technical field of point cloud deep learning modeling, and includes the following steps: S1: multi-source building point cloud data acquisition; S2: deep learning point cloud registration; S3: deep learning point cloud completion; S4: deep learning point cloud segmentation; S5: point cloud slicing and extracting building parameters; S6: establishing a BIM model according to the extracted parameters. The present invention optimizes the point cloud registration, completion, segmentation and parameter extraction steps through a deep learning method, and provides an efficient method for automatically generating a BIM model from point cloud data, which has the advantages of high precision, high robustness and adaptability to multi-source data.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud deep learning modeling, and in particular to a method for generating a high-precision three-dimensional model of building point cloud data. Background Art

[0002] With the in-depth development of building informatization, point cloud data has become one of the core technologies of building digitization. However, traditional point cloud data processing methods face problems such as large data volume, low precision, difficulty in integrating multi-source data, and missing information, which makes it difficult to meet the needs of building information modeling (BIM) and engineering analysis.

[0003] Traditional methods for reconstructing 3D building models rely on a single data source, such as airborne LiDAR point cloud data or oblique photogrammetry data. These methods have difficulty balancing local accuracy, overall integrity, and data size. The parameter setting of the traditional ICP point cloud registration algorithm relies on empirical values, and it is difficult to guarantee rigid transformation. Deep learning can directly estimate the rigid transformation matrix, which is conducive to ensuring the integrity and accuracy of the point cloud. Deep learning also performs well in point cloud completion and segmentation.

[0004] The above problems need to be solved urgently. To this end, the present invention proposes a method for generating a high-precision three-dimensional model of building point cloud data. Summary of the invention

[0005] The technical problem to be solved by the present invention is: how to solve the problem that the parameter setting of the traditional ICP point cloud registration algorithm depends on empirical values ​​and it is difficult to ensure rigid transformation, and by applying deep learning technology to point cloud completion and segmentation, a method for generating high-precision three-dimensional models of building point cloud data is provided.

[0006] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps:

[0007] Step S1: Multi-source building point cloud data collection

[0008] Use a stand-mounted 3D laser scanner and an airborne 3D laser scanner to scan and obtain point cloud data of each part of the building;

[0009] Step S2: Deep learning point cloud registration

[0010] Use the improved CoFi-Net point cloud registration network to register the various parts of the point cloud data obtained in step S1 to obtain the registered building point cloud data; merge all the registered building point cloud data into an overall building point cloud data;

[0011] Step S3: Deep learning point cloud completion

[0012] The improved PF-Net point cloud completion network is used to complete the point cloud data of the entire building to obtain the completed point cloud data of the building;

[0013] Step S4: Deep learning point cloud segmentation

[0014] The PointCNN point cloud segmentation network is trained to obtain a segmentation network adapted to indoor scenes. The segmentation network is used to segment the building body and internal objects from the completed building point cloud data, and the surrounding environment and internal objects are removed to obtain a pure building point cloud model.

[0015] Step S5: Building parameter extraction

[0016] Perform point cloud slicing and extract building parameters;

[0017] Step S6: Create BIM model

[0018] Using the building parameters extracted in step S5, a BIM rough model is generated according to the IFC standardized modeling format, and then the BIM model is refined with reference to the plan, elevation and section drawings, and finally a textured building BIM model is established for model mapping.

[0019] Furthermore, in step S1, the specific processing process is as follows:

[0020] Step S11: Conduct a site survey of the building's surroundings and interior, plan station-type scanning sites inside and outside the building, and use a drone oblique photography method to establish a rough oblique photography model of the building;

[0021] Step S12: Use a stand-type 3D laser scanner to scan the building facade to obtain point cloud data of the building facade, and then scan the interior of the building to obtain point cloud data of the interior of the building. The two scanning stations need to have line of sight, and the scanning areas have an overlap of more than 30%;

[0022] Step S13: Plan a 3D scanning route using route planning software based on the oblique photography rough model. After importing the planned route into the drone remote controller, the drone is equipped with a 3D laser scanner to perform scanning to obtain point cloud data of the building top and facade.

[0023] Furthermore, in the step S2, in the step S2, an overlapping attention module is introduced into the context aggregation process in the rough registration stage of the original CoFi-Net point cloud registration network to obtain an improved CoFi-Net point cloud registration network and train it. The processing process of using the improved CoFi-Net point cloud registration network for registration is as follows:

[0024] Step S21: Input the two point clouds with overlapping areas collected in step S1 , , where n and m are the number of points, and 3 represents the three-dimensional coordinates; a shared KPConv network is used to extract local geometric features and output the downsampled superpoints , , and features , ,in, and is the number of points after downsampling, b=256;

[0025] Step S22: , and its characteristics , The self-attention mechanism, cross-attention mechanism, and self-attention mechanism are used in sequence to expand the receptive field of the feature on its own point cloud, and then the overlapping attention module is added for processing to extract the contextual information between the features of the two point clouds; then the overlapping area is determined, that is, by comparing the super points from the two point clouds in pairs, it is determined whether they are in the overlapping area, and features with overlapping information are output. The overlapping information includes feature representation, overlap score, and matching score;

[0026] Step S23: Input the features with overlapping information into the optimal transmission module and the corresponding point refinement module to obtain a rough super-point level matching relationship. , and then get the point-level matching relationship ;

[0027] Step S24: Matching the point-level The point cloud is precisely aligned by using the RANSAC algorithm to perform rotation and translation transformations, solve the pose transformation matrix, and achieve precise point cloud registration.

[0028] Furthermore, in step S3, a network branch structure for extracting semantic and geometric features from RGB-D data is added in parallel to the feature extraction stage of the original PF-Net point cloud completion network to obtain an improved PF-Net point cloud completion network, and global geometric semantic consistency loss and local geometric semantic consistency loss are introduced to train it. The processing process of completion using the improved PF-Net point cloud completion network is as follows:

[0029] Step S31: Input the point cloud data of the entire building, the shape is , N is the number of points, 3 represents the three-dimensional coordinates, after two IFPS , obtain point cloud input data of three scales; then pass through the CMLP fully connected network to obtain the overall and local features of point clouds with different downsampling multiples, among which the overall building point cloud data is incomplete point cloud data;

[0030] Step S32: Input RGB-D image data in parallel, first generate a point cloud using the depth map, and then use the KPConv network to extract point cloud geometric features at different levels through the generated point cloud;

[0031] Step S33: using the deeplabv3+ network in parallel to directly generate features with semantic information from the RGB-D image data, i.e., obtaining semantic features;

[0032] Step S34: The point cloud features in step S31, the geometric features in step S32, and the semantic features in step S33 are spliced ​​together through a splicing operation and input into the decoder as input features to generate the completed building point cloud data.

[0033] Furthermore, in step S3, when training the improved PF-Net point cloud completion network, the following comprehensive loss function is designed for training:

[0034] ;

[0035] in, is the global geometric semantic consistency loss, is the local geometric semantic consistency loss, and is the corresponding weight hyperparameter;

[0036] The calculation formula is as follows:

[0037] ;

[0038] in, is the geometric weighting coefficient, N is the number of points, KL stands for Kullback-Leibler divergence, is the target semantic distribution of the i-th sample, is the predicted semantic distribution of the i-th sample;

[0039] The calculation formula is as follows:

[0040] ;

[0041] in, is the neighborhood of point i, and represent the feature vectors of point i and point j respectively, Represents the square of the Euclidean distance between the feature vectors of point i and point j.

[0042] Furthermore, in the step S5, the minimum rectangular bounding box of the building point cloud is calculated, and the point cloud posture of the obtained building point cloud model is adjusted so that the long side of the minimum rectangular bounding box of the building point cloud is parallel to the X-axis, and then the building point cloud is sliced ​​by setting the slice thickness along the X-axis, Y-axis, and Z-axis directions respectively, and then the relevant parameters of the building structure in the sliced ​​point cloud are extracted, and the sliced ​​point cloud is converted into a format and imported into CAD to draw the building plan, elevation and section drawings, and the shape and size of the remaining complex structure are completed, wherein the building parameters include the relevant parameters of the building structure and the shape and size of the remaining complex structure.

[0043] Furthermore, in step S5, the specific processing process is as follows:

[0044] Step S51: Calculate the minimum rectangular bounding box of the processed building point cloud

[0045] After inputting the point cloud, the main direction is estimated. The principal component analysis method is used to calculate the main direction vector of the point cloud. The point cloud is first centralized, and then the covariance matrix is ​​calculated. Then the eigenvalues ​​and eigenvectors of the covariance matrix are solved. The main direction is determined by the eigenvector corresponding to the maximum eigenvalue. The three eigenvectors obtained represent the three main axis directions of the building point cloud: ;

[0046] Construct the minimum rectangular bounding box in the main direction of the building point cloud:

[0047] ;

[0048] Among them, x min 、x max ,y min ,y max 、z min 、z max It is the coordinates of the six boundary points of the minimum rectangular bounding box, indicating the minimum and maximum values ​​of the minimum rectangular bounding box on the X, Y, and Z axes, and is used to calculate the size of the bounding box;

[0049] Step S52: Point cloud posture adjustment

[0050] Calculate the rotation matrix , the spindle Align to the specified coordinate axis, where the rotation matrix The angle between the main direction of the point cloud and the target direction is calculated using the rotation matrix Rotate the point cloud to the target orientation:

[0051] ;

[0052] Check whether the adjusted point cloud satisfies the requirement that the long side of the OBB is parallel to the target coordinate axis;

[0053] Step S53: Point cloud slicing

[0054] Slice direction Slice along the adjusted main axis direction, according to the specified slice thickness Segmentation of point cloud data:

[0055] ;

[0056] Among them, z min 、z max They represent the minimum and maximum heights of the slice area, respectively, and are used to specify the upper and lower boundaries of the slice. p represents a specific point in the point cloud, and z p Represents the Z-axis coordinate of point p, that is, the position of the point on the Z-axis, Indicates that each slice of the i-th slice is stored as a point set , including the (x, y, z) coordinates of the point cloud within the slice;

[0057] Step S54: Slice parameter extraction

[0058] The slice point cloud is fitted with a plane using the least squares method, and the fitting results are used to extract the inclination angle and offset. The convex hull algorithm is used to extract the outer contour of the slice point cloud and calculate the plane contour area and perimeter. The α-shape edge detection algorithm is used to extract the straight line boundary to identify the size of the wall or column. The highest and lowest points of each floor are extracted from the slice point cloud along the Z axis to calculate the floor height, and the average height difference between adjacent floor slices is calculated as the inter-floor distance.

[0059] Step S55: Completing the remaining complex structures

[0060] At the same time, the sliced ​​point cloud is converted into .rcp or .rcs format through Recap software, imported into CAD, and a two-dimensional line drawing is drawn using the relevant parameters extracted in step S54. The shape and size of the remaining complex structure are completed based on the point cloud to obtain multiple plan, elevation and section drawings of the building. The remaining complex structures include stairs, stained glass windows, and complex roof ridges.

[0061] Furthermore, in step S51, the point cloud centralization processing formula is as follows:

[0062] ;

[0063] ;

[0064] in, is the average position of the point cloud, N is the number of points, and P represents the three-dimensional position of the point;

[0065] The formula for calculating the covariance matrix is ​​as follows:

[0066] .

[0067] Compared with the prior art, the present invention has the following advantages:

[0068] The present invention can utilize the collected multi-source point cloud data, use the deep learning method to register the point cloud, avoid the traditional algorithm parameter dependence on empirical values, use deep learning to complete the missing point cloud, use the deep learning instance segmentation method to segment various instances in the building point cloud, cut out objects that do not belong to the building, obtain the pure building body, obtain the size parameters of the building construction through point cloud slicing, and model according to the parameters; optimize the point cloud registration, completion, segmentation and parameter extraction steps through the deep learning method, and provide an efficient method for automatically generating from point cloud data to BIM model, which has the advantages of high precision, high robustness and adaptability to multi-source data. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a schematic diagram of the process of the method for generating a high-precision three-dimensional model of building point cloud data in an embodiment of the present invention;

[0070] Figure 2 is a schematic diagram of the structure of the improved CoFi-Net point cloud registration network in an embodiment of the present invention;

[0071] Figure 3(a) is Figure 2 A partial enlarged view of the middle A part;

[0072] Figure 3(b) is Figure 2 A partial enlarged view of the middle B part;

[0073] Figure 4 is a schematic diagram of the structure of the improved PF-Net point cloud completion network in an embodiment of the present invention;

[0074] Figure 5 is an example image of a building point cloud after point cloud registration and completion in an embodiment of the present invention;

[0075] Figure 6 is a schematic diagram of a point cloud slicing method in an embodiment of the present invention;

[0076] FIG7 (a) is a CAD plan view of a point cloud slice of a building 1 m above the ground drawn after parameter extraction in an embodiment of the present invention;

[0077] FIG7( b ) is a CAD front elevation view of a building drawn according to the extracted point cloud parameters in an embodiment of the present invention;

[0078] FIG7 (c) is a CAD rear elevation view of a building drawn based on the extracted point cloud parameters in an embodiment of the present invention;

[0079] FIG7( d ) is a left elevation view of a building CAD drawn according to the extracted point cloud parameters in an embodiment of the present invention;

[0080] FIG7 (e) is a CAD right elevation view of a building drawn according to the extracted point cloud parameters in an embodiment of the present invention;

[0081] FIG7( f ) is a CAD cross-sectional view of a building point cloud slice drawn by extracting parameters from one of the cross-sections in the point cloud slice in an embodiment of the present invention;

[0082] Figure 8 It is the building BIM model finally established in the embodiment of the present invention. DETAILED DESCRIPTION

[0083] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0084] like Figure 1 As shown, this embodiment provides a technical solution: a method for generating a high-precision three-dimensional model of building point cloud data, comprising the following steps:

[0085] Step S1: Multi-source building point cloud data collection

[0086] Use a stand-mounted 3D laser scanner to scan and obtain as much point cloud data of the building's facade as possible, then enter the building to obtain point cloud data. The two stations must have overlapping scanning areas to facilitate subsequent alignment, and then use an airborne 3D laser scanner to obtain point cloud data on the top and facade of the building.

[0087] Step S2: Deep learning point cloud registration

[0088] The overlapping attention module is introduced into the original CoFi-Net point cloud registration network to obtain the improved CoFi-Net point cloud registration network (see Figure 2 ), using the improved CoFi-Net point cloud registration network to register the point cloud data obtained in step S1 to obtain the registered building point cloud data; merging all the registered building point cloud data into an overall building point cloud data;

[0089] Specifically, in the context aggregation process in the rough registration stage of the CoFiNet point cloud registration network, an overlap-attention block is added after the self-attention mechanism, cross-attention mechanism, and self-attention mechanism are processed to predict which points are not only feature-significant but also located in the overlapping area, so as to filter out super-points in the non-overlapping area, thereby improving the accuracy of subsequent point matching, and obtaining a rough registration point correspondence relationship. Then, the original CoFi-Net point cloud registration network is used to optimize the matching. After obtaining the optimized registration point correspondence relationship, the RANSAC algorithm is used to solve the pose transformation matrix, and the RANSAC algorithm is used to perform rotation and translation transformation on the optimized registration point pair relationship to solve the pose transformation matrix and perform precise registration of the point cloud.

[0090] Step S3: Deep learning point cloud completion

[0091] In the feature extraction stage of the original PF-Net point cloud completion network, a network branch structure for extracting semantic and geometric features from RGB-D data is added in parallel to obtain an improved PF-Net point cloud completion network (see Figure 4 ), and introduce semantic consistency loss and geometric consistency loss to train it, and use the improved PF-Net point cloud completion network to complete the point cloud data of the entire building to obtain the completed point cloud data of the building, see Figure 5 ;

[0092] Specifically, in the feature extraction stage of the PF-Net point cloud completion network, the network branch structure is added in parallel to extract semantic and geometric features from RGB-D data. The multi-level features extracted from the point cloud and the scene semantic and geometric features extracted from RGB-D are spliced ​​together, and then input into the multi-scale feature fusion FPN feature pyramid decoder for subsequent operations. The global geometric semantic consistency loss and the local geometric semantic consistency loss are added to the discriminator for training, so as to complete the missing parts of the building point cloud data caused by occlusion and other reasons.

[0093] Step S4: Deep learning point cloud segmentation

[0094] The S3DIS dataset is input into the PointCNN point cloud segmentation network for training to obtain a segmentation network that adapts to indoor scenes. The segmentation network is used to segment the building body (such as walls, columns, beams, floor slabs, etc.) and internal objects (such as furniture, decorations, etc.), and the surrounding environment and internal objects are cropped to remove them to obtain a pure building point cloud model.

[0095] Step S5: Slice the point cloud and extract building parameters

[0096] Calculate the minimum rectangular bounding box of the building point cloud, and adjust the point cloud posture of the obtained building point cloud model so that the long side of the minimum rectangular bounding box of the building point cloud is parallel to the X axis, and then set the slice thickness along the X axis, Y axis, and Z axis to slice the building point cloud (see Figure 6 ), then extract the relevant parameters of the building structure in the slice point cloud, convert the slice point cloud into a format and import it into CAD to draw the building plan, elevation and section drawings, and complete the shape and size of the remaining complex structure. The building parameters include the relevant parameters of the building structure and the shape and size of the remaining complex structure.

[0097] Step S6: Establish a BIM model according to the extracted building parameters

[0098] Using the building parameters extracted in step S5, a BIM rough model is generated according to the IFC standardized modeling format, and then the BIM model is refined with reference to the plan, elevation and section drawings, and finally a textured building BIM model is established for model mapping.

[0099] In this embodiment, in step S1, the multi-source building point cloud data collection process is as follows:

[0100] Step S11: Conduct a site survey of the building's surroundings and interior, plan station-type scanning sites inside and outside the building, and use a drone oblique photography method to establish a rough oblique photography model of the building;

[0101] Step S12: Use a stand-type scanner to scan the building facade to obtain point cloud data of the building facade, and then scan the interior of the building to obtain point cloud data of the interior of the building. The two scanning stations need to have line of sight to ensure that the scanning areas have an overlap of more than 30%;

[0102] Step S13: Plan a 3D scanning route using route planning software based on the oblique photography rough model. After importing the planned route into the drone remote controller, the drone is equipped with a 3D laser scanner to perform scanning and obtain point cloud data of the building top and facade.

[0103] In this embodiment, in step S2, Figure 2 As shown in Figure 1, the process of deep learning point cloud registration is as follows:

[0104] Step S21: Input the two-site cloud with overlapping areas collected by the scanner in step S1 , , where n and m are the number of points, and 3 represents the three-dimensional coordinates; the shared KPConv network is used to extract local geometric features and the output is The super point after , , and features , , where n′ and m′ are the number of downsampled points and b=256.

[0105] Step S22: , and its characteristics , Use self-attention mechanism, cross-attention mechanism, and self-attention mechanism to expand the receptive field of the feature on its own point cloud, and then add an overlap-attention block to extract the contextual information between the features of the two point clouds, and then determine the overlapping area, that is, by comparing the super points from the two point clouds in pairs, to determine whether they are in the overlapping area. This process is implemented using Transformer, and two overlap scores are assigned to each super point (to measure the possibility that this super point and its matching point are in the overlapping area), and non-overlapping area super points are filtered out. The output is decoded into feature representation, overlap score, and matching score, thereby improving the accuracy of subsequent point matching;

[0106] Step S23: Input the features with overlapping information, including feature representation, overlap score and matching score, into the remaining structure of the CoFi-Net network to obtain a rough super-point level matching relationship. , and then get the point-level matching relationship ,The remaining structure of the CoFi-Net network refers to the optimal transmission module (Correspondence Proposal Block) and the corresponding point refinement module (Correspondence Refinement Block);

[0107] Step S24: Matching the point-level The point cloud is precisely aligned by using the RANSAC algorithm to perform rotation and translation transformations, solve the pose transformation matrix, and achieve precise point cloud registration.

[0108] In this embodiment, after registering each part of the point cloud data, they are combined and output as an overall building point cloud.

[0109] In this embodiment, in step S2, the overlapping attention module is used to find two point clouds , The specific process is as follows:

[0110] S221: First use DGCNN (Dynamic Graph CNN) to further expand the super point , DGCNN expands the super point The method is:

[0111] ;

[0112] in, Indicates super point of ; Indicates super point Feature encoding of neighboring points, Represents instance normalization ; cat represents the splicing operation;

[0113] Here will After two updates, the output is:

[0114] .

[0115] S222: Determine the overlapping area, if the superpoint exist A similar superpoint is found in May be located in overlapping areas.

[0116] This process is implemented using Transformer, where k, v (from point cloud ), s (i.e., query in Transformer, from point cloud ) are respectively expressed as:

[0117] ;

[0118] ;

[0119] ;

[0120] ;

[0121] in, represents the output of transformer; k j is the key vector of the jth element; W k The weight matrix used to generate the key vector; f j GNN is the feature vector of the jth element after being processed by the graph neural network (DGGNN); v j is the value vector of the jth element; W v is the weight matrix used to generate the value vector; q i is the query vector of the i-th element; W s is the weight matrix used to generate the query vector; m i is the weighted value vector of the i-th element; is the attention weight between the i-th element and the j-th element; A set that indicates that there is a connection between elements i and j; is the transpose of the query vector; b is the scaling factor, which is the square root of the feature dimension and is used to stabilize the gradient; f i CA is the original DGGNN feature vector of the i-th element; f i GNN is the original DGGNN feature vector of the i-th element; MLP is a multi-layer perceptron (Multi-LayerPerceptron), which is used to further process the features; cat is used to convert the query vector q i and weighted value vector m i Put it together.

[0122] S223: Finally, the features with overlapping information are output, including feature representation, overlapping score and matching score.

[0123] In this embodiment, in step S3, Figure 4 As shown in Figure 1, the process of deep learning point cloud completion is as follows:

[0124] Step S31: Point cloud feature extraction

[0125] Input the incomplete point cloud (whole building point cloud data) obtained by point cloud registration network registration and merging, the shape is , N is the number of points, after two IFPS , obtain point cloud input data of three scales, and then pass through the CMLP fully connected network to obtain the overall and local features of point clouds with different downsampling multiples;

[0126] Step S32: Geometric feature extraction

[0127] Input RGB-D image data in parallel. For geometric feature extraction, first generate point cloud using depth map, and then use KPConv network to extract point cloud geometric features at different levels.

[0128] Step S33: semantic feature extraction

[0129] The deeplabv3+ network is used in parallel to generate features with semantic information directly from RGB-D image data.

[0130] Step S34: Point cloud completion

[0131] The point cloud features, geometric features and semantic features are spliced ​​together and given as input features to the decoding structure of the PF-Net network to generate the completed building point cloud data.

[0132] In this embodiment, in order to improve semantic consistency, a comprehensive loss function is designed in combination with geometric constraints, in which the spatial weighted loss assigns different weights to different positions of the point cloud according to geometric characteristics, that is, the global geometric semantic consistency loss function:

[0133] ;

[0134] in, is the geometric weighting coefficient, which is calculated based on the normal vector, curvature, and point density in the depth map. It gives higher weights to more important geometric areas (such as edge points). N is the number of points. KL stands for Kullback-Leibler divergence, an asymmetric measure used to measure the difference between two probability distributions. is the target semantic distribution of the i-th sample, which is the ideal distribution. is the predicted semantic distribution of the i-th sample. The purpose of the entire loss function is to minimize the predicted semantic distribution With target semantic distribution The KL divergence between , taking into account the weight of each sample .

[0135] The semantic prediction of local neighborhood points must be consistent, and the local geometric semantic consistency loss is as follows:

[0136] ;

[0137] in, Yes i Neighborhood of and represent the feature vectors of point i and point j respectively, Represents the square of the Euclidean distance between the feature vectors of point i and point j, which is used to measure the similarity of the features of two points. The smaller the distance, the more similar the features are, thereby ensuring that the semantic prediction is smooth and consistent in the local area.

[0138] Combining global geometric semantic consistency loss and local geometric semantic consistency loss:

[0139] ;

[0140] in, and is the corresponding weight parameter used to balance global and local consistency.

[0141] Based on the above, the following complete semantic consistency loss function can be designed:

[0142] ;

[0143] Among them, the first loss is used to ensure global geometric semantic consistency, and the second loss is used to ensure local geometric semantic smoothness.

[0144] In this embodiment, in step S4, the process of deep learning point cloud segmentation is as follows:

[0145] Step S41: Model training

[0146] The S3DIS dataset is directly used to input the PointCNN point cloud segmentation network to train a segmentation network that adapts to indoor scenes.

[0147] Step S42: The completed building point cloud data is used to segment the building body, building exterior environment, internal structure, furniture and decoration, etc., and all point clouds except the building body are removed to retain a pure building point cloud model.

[0148] In this embodiment, in step S5, the process of slicing the point cloud and extracting building parameters is as follows:

[0149] Step S51: Calculate the minimum rectangular bounding box (OBB) of the processed building point cloud: The minimum bounding box is a compact outer bounding structure of the point cloud data, which is used to estimate the main direction of the building. After inputting the point cloud, the main direction estimation is performed, and the principal component analysis (PCA) is used to calculate the main direction vector of the point cloud. The point cloud is first centralized: , ,in, is the average position of the point cloud, N is the number of points, and P represents the three-dimensional position of the point; then calculate the covariance matrix: , solve the eigenvalues ​​and eigenvectors of the covariance matrix. The main direction is determined by the eigenvector corresponding to the maximum eigenvalue. The three eigenvectors obtained represent the three main axis directions of the building point cloud: ; Construct the minimum rectangular bounding box in the main direction of the building point cloud: , where x min 、x max ,y min ,y max 、z min 、z max It is the coordinates of the six boundary points of the minimum rectangular bounding box, indicating the minimum and maximum values ​​of the minimum rectangular bounding box on the X, Y, and Z axes, and is used to calculate the size of the bounding box;

[0150] Step S52: Point cloud posture adjustment: calculate the rotation matrix and change the main axis Align to the specified coordinate axis (such as the X axis). Rotation matrix The angle between the main direction of the point cloud and the target direction is calculated, and the point cloud is rotated to the target direction using a rotation matrix: , check whether the adjusted point cloud satisfies the long side of the OBB and is parallel to the target coordinate axis (such as the X axis);

[0151] Step S53: Point cloud slicing: Slicing direction is along the adjusted principal axis direction (such as Z-axis stratification, or X / Y-axis segmentation), according to the specified slice thickness (For example, each layer is 50cm) to segment the point cloud data:

[0152] ;

[0153] Among them, z min 、z max They represent the minimum and maximum heights of the slice area, respectively, and are used to specify the upper and lower boundaries of the slice. p represents a specific point in the point cloud, and z p Represents the Z-axis coordinate of point p, that is, the position of the point on the Z-axis. Indicates that each slice of the i-th slice is stored as a point set , including the (x, y, z) coordinates of the point cloud within the slice;

[0154] Step S54: Slice parameter extraction: Extract key building parameters from the point cloud data of each slice, such as size, angle, shape features, etc. Perform plane fitting on the slice point cloud, extract the geometric parameters of the building elements, and use the least squares method to fit the plane equation: The fitting results can be used to extract parameters such as inclination and offset. The convex hull algorithm is used to extract the outer contour of the sliced ​​point cloud and calculate the plane contour area and perimeter. The α-shape edge detection algorithm is used to extract the straight line boundary to identify the size of the wall or column. The highest and lowest points of each floor are extracted from the sliced ​​point cloud along the Z axis to calculate the floor height: , the average height difference between adjacent floor slices is calculated as the inter-floor distance.

[0155] Step S55: Completing the remaining complex structures

[0156] At the same time, the sliced ​​point cloud is converted into .rcp or .rcs format through Recap software, imported into CAD, and the extracted relevant parameters are used to draw a two-dimensional line drawing. Based on the point cloud, the shape and size of other complex structures such as stairs, stained glass windows, complex roof ridges, etc. are completed to obtain multiple plan, elevation and section drawings of the building, as shown in Figure 7 (a) to Figure 7 (f).

[0157] In this embodiment, in step S6, the process of establishing the BIM model according to the extracted parameters is as follows:

[0158] Step S61: Organize the IFC standard modeling format file according to the extracted building parameters, the core components of which are entity classes, such as: IfcWall (wall), IfcSlab (slab), IfcColumn (column); geometric representation IfcShapeRepresentation is used to define the shape of the object (such as volume-based geometry or contour curve); property sets (Property Sets) IfcPropertySet is used to define the parameter properties of the object (such as width, height, material); hierarchical structure, such as: IfcBuilding (building) -> IfcBuildingStorey (floor) -> IfcElement (element). The geometric parameters are organized as IFC geometry, and the property parameters are mapped to the corresponding property sets (Psets). Use the open source IFC tool IfcOpenShell to process IFC data in a Python environment, define projects (IfcProject), sites (IfcSite), buildings (IfcBuilding), floors (IfcBuildingStorey), and parametrically create geometries including walls (IfcWallStandardCase), define the start and end coordinates, and set the width and height of the wall; for slabs (IfcSlab), define the contour boundary (IfcPolyline) and set the thickness. Use IfcExtrudedAreaSolid or IfcSweptSolid to generate complex shapes and process arcs and inclined surfaces. Create a property set (IfcPropertySet) to attach properties to components, then associate objects with hierarchies, assign geometric objects (such as walls and slabs) to floors or building objects, use IfcRelAggregates and IfcRelContainedInSpatialStructure to define relationships, use the tool IfcOpenShell Viewer to check the correctness of the IFC file, and finally export the IFC file;

[0159] Step S62: First, the IFC file is imported into the Revit software to automatically create a BIM rough model, and then the rough model is modified and edited with reference to the CAD plan, elevation and section two-dimensional line drawing to obtain a model with higher precision. The degree of detail is divided into LOD 1-4, and finally a textured building BIM model is created for the model mapping, such as Figure 8 shown.

[0160] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for generating a high-precision three-dimensional model of building point cloud data, characterized in that: The following steps are involved: Step S1: Multi-source building point cloud data collection Use a stand-mounted 3D laser scanner and an airborne 3D laser scanner to scan and obtain point cloud data of each part of the building; Step S2: Deep learning point cloud registration Use the improved CoFi-Net point cloud registration network to register the various parts of the point cloud data obtained in step S1 to obtain the registered building point cloud data; merge all the registered building point cloud data into an overall building point cloud data; Step S3: Deep learning point cloud completion The improved PF-Net point cloud completion network is used to complete the point cloud data of the entire building to obtain the completed point cloud data of the building; Step S4: Deep learning point cloud segmentation The PointCNN point cloud segmentation network is trained to obtain a segmentation network adapted to indoor scenes. The segmentation network is used to segment the building body and internal objects from the completed building point cloud data, and the surrounding environment and internal objects are removed to obtain a pure building point cloud model. Step S5: Building parameter extraction Perform point cloud slicing and extract building parameters; Step S6: Create BIM model Using the building parameters extracted in step S5, a BIM rough model is generated according to the IFC standardized modeling format, and then the BIM model is refined with reference to the plan, elevation and section drawings, and finally a textured building BIM model is established for model mapping.

2. The method for generating a high-precision three-dimensional model of building point cloud data according to claim 1, characterized in that: In step S1, the specific processing process is as follows: Step S11: Conduct a site survey of the building's surroundings and interior, plan station-type scanning sites inside and outside the building, and use a drone oblique photography method to establish a rough oblique photography model of the building; Step S12: Use a stand-type 3D laser scanner to scan the building facade to obtain point cloud data of the building facade, and then scan the interior of the building to obtain point cloud data of the interior of the building. The two scanning stations need to have line of sight, and the scanning areas have an overlap of more than 30%; Step S13: Plan a 3D scanning route using route planning software based on the oblique photography rough model. After importing the planned route into the drone remote controller, the drone is equipped with a 3D laser scanner to perform scanning to obtain point cloud data of the building top and facade.

3. The method for generating a high-precision three-dimensional model of building point cloud data according to claim 1, characterized in that: In the step S2, in the step S2, an overlapping attention module is introduced into the context aggregation process in the rough registration stage of the original CoFi-Net point cloud registration network to obtain an improved CoFi-Net point cloud registration network and train it. The processing process of using the improved CoFi-Net point cloud registration network for registration is as follows: Step S21: Input the two point clouds with overlapping areas collected in step S1 , , where n and m are the number of points, and 3 represents the three-dimensional coordinates; the shared KPConv network is used to extract local geometric features and the output is The super point after , , and features , ,in, and is the number of points after downsampling, b=256; Step S22: , and its characteristics , The self-attention mechanism, cross-attention mechanism, and self-attention mechanism are used in sequence to expand the receptive field of the feature on its own point cloud, and then the overlapping attention module is added for processing to extract the contextual information between the features of the two point clouds; then the overlapping area is determined, that is, by comparing the super points from the two point clouds in pairs, it is determined whether they are in the overlapping area, and the features with overlapping information are output. The overlapping information includes feature representation, overlap score, and matching score; Step S23: Input the features with overlapping information into the optimal transmission module and the corresponding point refinement module to obtain a rough super-point level matching relationship. , and then get the point-level matching relationship ; Step S24: Matching the point-level The point cloud is precisely aligned by using the RANSAC algorithm to perform rotation and translation transformations, solve the pose transformation matrix, and achieve precise point cloud registration.

4. The method for generating a high-precision three-dimensional model of building point cloud data according to claim 3, characterized in that: In step S3, a network branch structure for extracting semantic and geometric features from RGB-D data is added in parallel to the feature extraction stage of the original PF-Net point cloud completion network to obtain an improved PF-Net point cloud completion network, and global geometric semantic consistency loss and local geometric semantic consistency loss are introduced to train it. The processing process of completion using the improved PF-Net point cloud completion network is as follows: Step S31: Input the point cloud data of the entire building, the shape is , N is the number of points, 3 represents the three-dimensional coordinates, after two IFPS , obtain point cloud input data of three scales; then pass through the CMLP fully connected network to obtain the overall and local features of point clouds with different downsampling multiples, among which the overall building point cloud data is incomplete point cloud data; Step S32: Input RGB-D image data in parallel, first generate a point cloud using the depth map, and then use the KPConv network to extract point cloud geometric features at different levels through the generated point cloud; Step S33: using the deeplabv3+ network in parallel to directly generate features with semantic information from the RGB-D image data, i.e., obtaining semantic features; Step S34: The point cloud features in step S31, the geometric features in step S32, and the semantic features in step S33 are spliced ​​together through a splicing operation and input into the decoder as input features to generate the completed building point cloud data.

5. The method for generating a high-precision three-dimensional model of building point cloud data according to claim 4, characterized in that: In step S3, when training the improved PF-Net point cloud completion network, the following comprehensive loss function is designed for training: ； in, is the global geometric semantic consistency loss, is the local geometric semantic consistency loss, and is the corresponding weight hyperparameter; The calculation formula is as follows: ； in, is the geometric weighting coefficient, N is the number of points, KL stands for Kullback-Leibler divergence, is the target semantic distribution of the i-th sample, is the predicted semantic distribution of the i-th sample; The calculation formula is as follows: ； in, is the neighborhood of point i, and represent the feature vectors of point i and point j respectively, Represents the square of the Euclidean distance between the feature vectors of point i and point j.

6. The method for generating a high-precision three-dimensional model of building point cloud data according to claim 1 or 5, characterized in that: In the step S5, the minimum rectangular bounding box of the building point cloud is calculated, and the point cloud posture of the obtained building point cloud model is adjusted so that the long side of the minimum rectangular bounding box of the building point cloud is parallel to the X-axis, and then the building point cloud is sliced ​​by setting the slice thickness along the X-axis, Y-axis, and Z-axis directions, and then the relevant parameters of the building structure in the sliced ​​point cloud are extracted, and the sliced ​​point cloud is converted into a format and imported into CAD to draw the building plan, elevation and section drawings, and the shape and size of the remaining complex structure are completed, wherein the building parameters include the relevant parameters of the building structure and the shape and size of the remaining complex structure.

7. The method for generating a high-precision three-dimensional model of building point cloud data according to claim 6, characterized in that: In step S5, the specific processing process is as follows: Step S51: Calculate the minimum rectangular bounding box of the processed building point cloud After inputting the point cloud, the main direction is estimated. The principal component analysis method is used to calculate the main direction vector of the point cloud. The point cloud is first centralized, and then the covariance matrix is ​​calculated. Then the eigenvalues ​​and eigenvectors of the covariance matrix are solved. The main direction is determined by the eigenvector corresponding to the maximum eigenvalue. The three eigenvectors obtained represent the three main axis directions of the building point cloud: ; Construct the minimum rectangular bounding box in the main direction of the building point cloud: ； Among them, x min 、x max ,y min ,y max 、z min 、z max It is the coordinates of the six boundary points of the minimum rectangular bounding box, indicating the minimum and maximum values ​​of the minimum rectangular bounding box on the X, Y, and Z axes, and is used to calculate the size of the bounding box; Step S52: Point cloud posture adjustment Calculate the rotation matrix , the spindle Align to the specified coordinate axis, where the rotation matrix The angle between the main direction of the point cloud and the target direction is calculated using the rotation matrix Rotate the point cloud to the target orientation: ； Check whether the adjusted point cloud satisfies the requirement that the long side of the OBB is parallel to the target coordinate axis; Step S53: Point cloud slicing Slice direction Slice along the adjusted main axis direction, according to the specified slice thickness Segmentation of point cloud data: ； Among them, z min 、z max They represent the minimum and maximum heights of the slice area, respectively, and are used to specify the upper and lower boundaries of the slice. p represents a specific point in the point cloud, and z p Represents the Z-axis coordinate of point p, that is, the position of the point on the Z-axis, Indicates that each slice of the i-th slice is stored as a point set , including the (x, y, z) coordinates of the point cloud within the slice; Step S54: Slice parameter extraction The slice point cloud is fitted with a plane using the least squares method, and the fitting results are used to extract the inclination angle and offset. The convex hull algorithm is used to extract the outer contour of the slice point cloud and calculate the plane contour area and perimeter. The α-shape edge detection algorithm is used to extract the straight line boundary to identify the size of the wall or column. The highest and lowest points of each floor are extracted from the slice point cloud along the Z axis to calculate the floor height, and the average height difference between adjacent floor slices is calculated as the inter-floor distance. Step S55: Completing the remaining complex structures At the same time, the sliced ​​point cloud is converted into .rcp or .rcs format through Recap software, imported into CAD, and a two-dimensional line drawing is drawn using the relevant parameters extracted in step S54. The shape and size of the remaining complex structure are completed based on the point cloud to obtain multiple plan, elevation and section drawings of the building. The remaining complex structures include stairs, stained glass windows, and complex roof ridges.

8. The method for generating a high-precision three-dimensional model of building point cloud data according to claim 7, characterized in that: In step S51, the point cloud centralization processing formula is as follows: ； ； in, is the average position of the point cloud, N is the number of points, and P represents the three-dimensional position of the point; The formula for calculating the covariance matrix is ​​as follows: 。

Images