Building refined three-dimensional reconstruction method based on airborne LiDAR point cloud

Abstract

The invention relates to a building refined three-dimensional reconstruction method based on airborne LiDAR point cloud, and belongs to the technical field of three-dimensional modeling. According to the method, high-quality building point cloud data is separated from original airborne LiDAR point cloud data, features and bottom contour lines related to a building are extracted, a roof surface is fitted, and the features and the bottom contour lines are projected to the same horizontal reference surface, so that correlation correction of the features and the bottom contour lines is realized; and completing preliminary reconstruction of a building model by using the corrected building bottom contour line and roof surface, constructing constraint conditions by using priori knowledge, and completing expression of detail components on the model and correction of the reconstructed building roof surface. According to the method, the problems of irregularity, distortion, cavities, difficult facade detail structure expression and the like of the reconstruction model caused by the problems of sparse, missing, cavities and the like of the building facade point cloud data in the prior art are effectively solved, and the problems of irregular edges, distortion, cavities and the like of the constructed building model are solved while efficient, automatic and refined modeling is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional modeling, and in particular relates to a method for fine-grained three-dimensional reconstruction of buildings based on airborne LiDAR point clouds. Background Art

[0002] As the foundation of the urban virtual environment, three-dimensional building models can realize the management and planning of urban buildings. Reconstructing three-dimensional building models from point cloud data is the main method for reconstructing three-dimensional city models. In recent years, with the development of airborne laser ranging and detection (LiDAR) technology, airborne LiDAR point clouds with high resolution and fast acquisition speed have been widely used in three-dimensional reconstruction projects in a large number of cities around the world, providing sufficient data sources and technical means for the construction of smart cities. However, due to the discreteness of the point cloud data acquired by LiDAR, the characteristics of close-to-dense and far-to-sparse, the small angle between the laser emission direction of the airborne LiDAR and the direction of the building facade, the reflection of the building facade and many other factors, the acquired airborne LiDAR point cloud data has problems such as sparse density, missing points, and voids in the building facade point cloud, resulting in distortion of the building facade point cloud data, which cannot intuitively and accurately reflect the building facade structure, and thus affects the reconstruction of the refined building model.

[0003] Existing reconstruction methods generally fall into three categories: point topology-based, voxel-based, and a priori hypothesis-based. Point topology-based reconstruction methods utilize the topological relationships of points to reconstruct building surfaces, but they cannot reconstruct missing portions of point clouds. Voxel-based reconstruction methods can reconstruct missing portions, but they may suffer from reconstruction errors when dealing with noisy areas. A priori hypothesis-based methods reduce the complexity of the reconstruction process by assuming regular building features and are widely used for large-scale urban building reconstruction. However, existing research has mostly focused on the rapid reconstruction of simple body-frame models and lacks support for detailed structures such as building facades and roofs. Furthermore, some studies have proposed extracting detailed facade features from point clouds and reconstructing detailed facade structures through template matching, but these studies do not consider the reconstruction of complete, detailed models. Furthermore, existing methods often suffer from irregular edges, distortions, and voids in building models constructed from point clouds, resulting in insufficient model accuracy and detail, low modeling efficiency, and high production costs.

[0004] In view of this, the existing technology needs to be further improved. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a method for fine-grained three-dimensional reconstruction of buildings based on airborne LiDAR point clouds, the purpose of which is to overcome at least one of the above-mentioned problems and solve the problem of fine-grained three-dimensional reconstruction of complete buildings based on airborne LiDAR building point clouds.

[0006] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0007] A method for fine-grained three-dimensional reconstruction of a building based on airborne LiDAR point cloud, characterized in that the method comprises the following steps:

[0008] Step S1, obtaining original airborne LiDAR point cloud data;

[0009] Step S2: pre-processing the original airborne LiDAR point cloud data to obtain high-quality building point cloud data;

[0010] Step S3: extracting features from the point cloud data based on the high-quality building point cloud data, extracting basic geometric feature information, and extracting feature points, feature lines, and the bottom contour of the building based on the basic geometric feature information. Subsequently, the extracted features are used to segment the building roof surface and facade, and plane fitting is performed on the roof surface.

[0011] Step S4: Based on the extracted building bottom contour line and the plane-fitted roof surface, the roof surface is associated with the building bottom contour to obtain a corrected building bottom contour line and a corrected fitted roof surface;

[0012] Step S5: The corrected building bottom contour line from step S4 is directly extruded in a direction perpendicular to the ground, intersecting it with the corrected fitted roof surface to form a watertight building frame model, thereby completing the preliminary reconstruction of the building model. Subsequently, constraint conditions are constructed using prior knowledge to complete the expression of detailed components on the preliminary reconstructed building model and the correction of the reconstructed building roof surface. Finally, constraint conditions are constructed to optimize the model and output the final refined 3D reconstruction result.

[0013] Compared with the prior art, the present invention has at least the following beneficial effects:

[0014] 1. This paper proposes a method for fine-grained 3D reconstruction of buildings based on airborne LiDAR point clouds. This method addresses the common problems of irregularity, distortion, holes, and difficulty expressing detailed facade structures in reconstructed building models caused by the sparse, missing, and void-filled facade point clouds of airborne LiDAR building point cloud data. By extracting and analyzing features and optimizing the 3D reconstruction framework based on prior knowledge constraints such as geometry and pose, this method accurately, regularly, and completely reconstructs fine-grained building facades. This method, combined with the roof reconstruction results, forms a fine-grained building model. This method effectively addresses the challenges of fine-grained building modeling caused by the sparse, missing, and void-filled facade point clouds of airborne LiDAR point cloud data.

[0015] 2. Problems such as irregular edges, distortions, and voids are common in existing point cloud modeling results, resulting in insufficient model accuracy and detail expression, as well as low modeling efficiency and high production costs. Before model reconstruction, the present invention introduces point cloud data completion and simplification steps to highlight building point cloud features, obtain high-quality building point clouds, and balance detail accuracy with computational efficiency; then, feature analysis is used to extract feature points, feature lines, and feature surfaces from building point cloud data, and the point cloud features and prior constraints are combined to participate in the automatic reconstruction of the three-dimensional building model, which greatly improves the accuracy, refinement, regularity, and completeness of the generated model, significantly reduces the number of model facets, improves modeling efficiency, reduces modeling costs, and provides an efficient solution for city-level real-scene three-dimensional modeling;

[0016] 3. In the modeling process, the existing technology lacks mutual verification and correction of the ground feature contours and roof surfaces of the building model, resulting in large differences between the constructed model and the actual building or the generation of local loopholes, and insufficient model accuracy. The present invention establishes a relationship between the extracted building bottom contour line and the plane-fitted roof surface by orthographic projection onto the same reference horizontal plane, and realizes mutual correction based on their characteristics, ensuring that a watertight building frame model can be formed subsequently, thereby further improving the accuracy of the final generated model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0018] Figure 1 This is the main flow chart of the three-dimensional modeling of the present invention;

[0019] Figure 2 Schematic diagram of the effect of ground point separation performed by the present invention;

[0020] Figure 3 Schematic diagram of the effect of building semantic segmentation performed by the present invention;

[0021] Figure 4 Schematic diagram of the effect of thinning the line features of the edge of the point cloud in the present invention;

[0022] Figure 5 2. It is a schematic diagram showing the effect of the present invention on fitting the roof plane of an example building point cloud data;

[0023] Figure 6 Schematic diagram of the effect of constraining the posture features of doors and windows in the present invention;

[0024] Figure 7 Schematic diagram of the effect of constraining roof surface features according to the present invention;

[0025] Figure 8This is a schematic diagram of the effects of buildings constructed using different reconstruction methods. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with examples. The illustrative embodiments of the present invention and their descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0027] like Figures 1 to 8 As shown, the present invention provides a method for fine-grained 3D reconstruction of buildings based on airborne LiDAR point cloud, the method comprising the following steps:

[0028] Step S1: Acquisition of airborne LiDAR point cloud data

[0029] Airborne LiDAR is used to scan a predetermined range of a selected scene, acquiring raw airborne LiDAR point cloud data for that range. It should be understood that the scene can be a large urban scene, or a variety of scenes such as a rural area, suburbs, or industrial area. For example, the predetermined range can be a large area within the scene. Of course, since the goal is to construct a 3D building model, a large urban scene is the most suitable, but other scenes are not excluded. Furthermore, acquiring point cloud data is a prior art and will not be discussed in detail here.

[0030] Step S2: Point cloud data preprocessing

[0031] Specifically, it uses a non-ground point segmentation algorithm such as the CSF (Cloth Simulation Filtering) cloth simulation filtering algorithm to segment and extract the point cloud data corresponding to the non-ground points from the original point cloud data (separation of ground points and non-ground points). Then, a deep learning method is used to separate the building point cloud data from the point cloud data corresponding to the non-ground points (accurate extraction of building point clouds); then, to address the problem of missing points on the building point cloud facade, a point cloud completion method (such as the PointTr model) is used to fill the holes (supplement the missing point clouds of the building facade in the airborne laser point cloud except for detailed structures such as doors and windows) to generate a complete building point cloud; finally, the point cloud data is simplified to reduce the data volume and retain key geometric features to obtain high-quality building point cloud data. Through non-ground point segmentation, building semantic segmentation, point cloud completion and simplified pre-processing steps, high-quality building point clouds are extracted from the original airborne LiDAR point cloud data with facade distortion, which significantly improves the quality of the building point cloud and provides a reliable data foundation for subsequent feature extraction and three-dimensional reconstruction. Further details are given here as follows:

[0032] First of all, non-ground point segmentation is the first step in large-scale point cloud processing. Objects such as buildings, trees, vehicles, and pedestrians belong to the category of non-ground points. They differ from ground point data such as roads in terms of height and surface base. Therefore, to segment buildings from the original point cloud data, it is first necessary to identify non-ground points. In the present invention, the CSF filtering algorithm is used to realize the geometric segmentation of point cloud data and thus realize the process of extracting non-ground points. Specifically, the "cloth" here can be regarded as composed of particles within a certain continuous range in the point cloud data. These particles are connected together by physical constraints. By determining the position and change of these particles, the shape characteristics of the cloth at a specific time can be simulated. The position and velocity of each particle are determined by the force acting on it. According to Newton's second law, the relationship between position and force is as follows:

[0033]

[0034] Where X is the position of the particle at time t, F ext Indicates the external force on the particle (including gravity and collision force, etc.), F int represents the internal force generated by the connections between particles. By analyzing the interaction between the distribution nodes and the corresponding lidar points, the location of the distribution nodes can be determined, thus generating an approximation of the ground surface. In addition, the displacement vector of the particle can be expressed by the following formula:

[0035]

[0036] Among them, b is 1 when the particle can move, p0 is the position the particle is about to move to, pi is the particle connected to the current particle, and n is the normalized vector in the vertical direction. Based on the particle displacement vector, by comparing the original point cloud data and the generated surface, the ground points can be initially separated from the building data through filtering extraction, thereby obtaining the point cloud data corresponding to the non-ground points. The effect diagram of ground point separation can be seen in Figure 2 .

[0037] After the non-ground points are separated, the next step is to use deep learning methods to perform semantic segmentation on the non-ground point cloud data (the point cloud data corresponding to the non-ground points) to accurately extract the building point cloud data. Preferably, the PointCNN deep learning framework is used. In this framework, each point in the point cloud can dynamically select the neighborhood size, and can be adaptively applied to point cloud data sets with different density sampling distances. At the same time, through the effect of convolution, a large number of sparse point clouds are effectively processed, and the local structure and association of the point cloud data are captured. The algorithm first converts the input point cloud into a more streamlined form through two x-dimensional convolution operators, but the information expressed by each point is richer and more representative, and then the number of point clouds is increased through two layers of transformation. After the above processing, the model fully takes into account the local information of the point cloud, so that a relatively accurate segmentation effect can be achieved. Figure 3 The following figure shows the effect of building semantic segmentation. It should be noted that the CSF filtering algorithm itself has the advantages of high automation, high computational efficiency, and wide applicability. However, the present invention combines deep learning semantic segmentation with the CSF filtering algorithm to further accurately segment areas outside of buildings when segmenting non-ground points. The advantage of this combination is that it not only improves the overall segmentation efficiency but also effectively saves the time cost of deep learning model training.

[0038] After extracting the building point cloud data, the next step is to complete the building point cloud data. Due to interference factors such as occlusion and environmental influences in the process of data collection using lidar, the acquired point cloud data building facades are partially missing, which affects the point cloud feature extraction, so the missing parts need to be completed. In the present invention, the PointTr model is preferably used to solve the problem of completing the building facade point cloud, which will not be introduced in detail here. It should be noted that the parameter setting has an important influence on the training effect and final performance of the model. In order to ensure the completion effect, we set the following training parameters for the PoinTr model: learning rate, weight decay rate, learning rate decay factor, batch size (Batch Size), training cycle (Epochs) and optimizer; the specific settings are as follows:

[0039] parameter Numerical Initial learning rate 0.0001 Weight decay rate 0.0005 Learning rate decay factor 0.9 Batch Size 96 Training cycles (Epochs) 500 Optimizer AdamW

[0040] To quantify the model performance, we introduced the following evaluation metrics:

[0041] (1) Precision (P): reflects the proportion of predicted point clouds that are correctly matched to the real point clouds. The calculation formula is:

[0042]

[0043] Where n is the number of correctly completed points, and N is the total number of completed points. The higher the accuracy, the fewer mismatches the model makes when predicting point clouds, and the more accurately it can complete point cloud information.

[0044] (2) Recall (R): reflects the proportion of points in the real point cloud that correspond to the predicted point cloud. The calculation formula is:

[0045]

[0046] Where m is the number of original missing points. A higher recall rate indicates that the model is better able to cover the missing parts of the real point cloud when completing the point cloud, and retrieve more original information.

[0047] (3) F1 score: The harmonic mean of precision and recall, used to comprehensively evaluate the performance of the model. The calculation formula is:

[0048]

[0049] The F1 score takes into account both precision and recall, and can more comprehensively reflect the performance of the model in the point cloud completion task. The F1 score ranges from 0 to 1. The higher the value, the better the performance of the model, indicating that the model can effectively recover the missing parts in the real point cloud while ensuring the accuracy of the completed point cloud, achieving a good balance between precision and recall. Taking precision as an example, the matching threshold is set to 0.01. When the distance between the predicted point cloud and the corresponding point in the real point cloud is less than 0.01, the point is considered to be correctly completed. By setting the above training parameters and evaluation indicators, the deep learning-based airborne lidar building facade point cloud completion method can be systematically optimized to ensure that the model achieves the best balance between precision, recall and computational efficiency.

[0050] Next, the building point cloud simplification step is performed. Preferably, in this embodiment of the present invention, simplification of the building point cloud data is performed using an octree index downsampling method. This simplification technique is known in the art and will not be described in detail here. Data simplification can effectively reduce redundant point cloud data, highlight building point cloud features, and balance detail accuracy and computational efficiency in subsequent modeling.

[0051] Step S3: Feature extraction of point cloud data, edge feature refinement, and plane fitting of the roof surface

[0052] Based on the high-quality building point cloud data obtained in step S2, basic geometric feature information (such as normal vectors and curvature, etc.) is extracted. On this basis, edge point detection is further performed to extract building edges, edges of detailed structures such as doors and windows, and the bottom contour of the building, and edge features are refined to achieve segmentation of the point cloud data corresponding to the building roof surface and the point cloud data corresponding to the facade, and further achieve plane fitting of the roof surface. Specifically, it includes the following steps:

[0053] S31. Use deep learning and other methods to calculate the high-quality building point cloud data obtained in step S2, and obtain basic geometric feature information such as the normal vector and curvature of each point cloud. The curvature information includes the maximum and minimum principal curvatures, the principal curvature direction, the normal curvature, the Gaussian curvature, etc. Preferably, the acquisition of the normal vector and curvature and other information is achieved through the PCPNET deep learning method. PCPNET can effectively handle strong noise and multi-scale point cloud data by directly learning the standard normal vector and curvature values ​​in the local point neighborhood. Its core idea is to normalize the local neighborhood of the point cloud and then estimate the normal vector and curvature of the center point through a deep learning network. Experiments show that the normal vector estimation of PCPNET in flat areas is consistent with the standard value, and the curvature change can also be clearly expressed in the edge area, providing high-precision geometric features for subsequent segmentation and reconstruction;

[0054] S32: Based on the basic geometric feature information extracted in step S31, extract feature points, contour lines, roof surfaces, building facades, etc. Specifically, it includes:

[0055] S321. Use an angle-based edge detection method to extract building feature points (such as corner points, roof points, door and window points, etc.) from the high-quality building point cloud data. Specifically, this edge detection method identifies edge points by calculating the angle between neighboring points. The core idea is that the angle difference between an edge point and two consecutive adjacent projection points is much larger than that of other internal points. The probability of determining a boundary point is:

[0056]

[0057] Where Π∠(P) represents the probability that point P is a boundary point, and its output range is between 0 and 1; N P represents the domain point set with the current point P as the center point; g represents the neighborhood point set N P The maximum distance between two consecutive adjacent projected points on the tangent plane is calculated. This method can effectively reduce the influence of small errors in the normal direction.

[0058] S322. After the feature points are extracted, the curvature values ​​estimated by PCPNET are used to conduct an in-depth analysis of the extracted feature points and extract feature lines, thereby identifying the edges and contours of detailed structures such as buildings, doors and windows. In the specific operation, we first conduct a detailed analysis of the curvature values ​​and classify the feature points according to the set curvature threshold. In particular, those points whose curvature values ​​exceed the threshold are marked as edge feature points. These edge feature points are the key to identifying the edges of structures because they are usually located in areas where the surface shape changes significantly. After identifying the edge feature points, they are connected to form contour lines. This step is achieved through geometric methods or graph theory algorithms, and aims to construct continuous contour lines that reflect the details of the building. In this process, since the contours of structures such as doors and windows usually have obvious geometric features, such as straight lines and rectangles, they can be distinguished from the main contours of the building through methods such as cluster analysis.

[0059] S323. Next, the bottom contour of the building is extracted.

[0060] Specifically, by analyzing the distribution of point cloud data in the height direction, the junction of the building bottom and the ground is identified; then, the extracted edge feature points and feature lines are used to accurately define the outline of the building bottom and obtain the building bottom contour line.

[0061] S324. After the feature lines are extracted and the building bottom contour line is obtained, the feature is refined. Preferably, a clustering algorithm such as Mean-Shift is used for this purpose. The main idea of ​​this algorithm is to calculate the gradient, continuously update the center point of each cluster, and continuously move the points in the space along the gradient direction until the gradient is zero, thereby achieving the purpose of clustering or refining the point cloud. Experimental results show that the refined point cloud not only reduces the number of points, but also maintains the original geometric shape, providing high-quality input data for subsequent reconstruction. For details, please refer to Figure 4 The effect before and after feature refinement is shown.

[0062] S325. Based on the refined feature points and feature lines, combined with the basic geometric feature information obtained in step S31, the high-quality building point cloud data obtained in step S2 is segmented, thereby distinguishing the point cloud data corresponding to the roof surface from the point cloud data corresponding to the facade. Then, the point cloud data corresponding to the roof surface is used to perform roof surface plane fitting.

[0063] In this process, first, the feature points and feature lines are used as the basis for segmentation to perform preliminary regional division of the point cloud data; then, the basic geometric feature information obtained in step S31, such as normal vectors and curvature, can be used to further identify each point in the point cloud, thereby further refining and refining the area to which these point clouds belong. Finally, the roof surface plane is fitted based on the point cloud data corresponding to the roof surface to achieve reconstruction of the roof surface of each individual building; preferably, when performing roof surface plane fitting, the RANSAC (random sampling consensus) algorithm is preferably used. The RANSAC algorithm is widely used in technologies such as shape detection and point cloud plane fitting. It can effectively fit the roof plane of the building, has good segmentation effect and high computational efficiency, and its plane fitting effect can be seen in [1]. Figure 4 .

[0064] S4. Association and correction between roof surface and building bottom contour

[0065] S41. Select a reference horizontal plane, and then orthographically project the building bottom contour line extracted in step S3 and the fitted roof surface onto the reference horizontal plane to obtain the building bottom contour projection and roof surface projection (the above projections are all two-dimensional geometric figures, and the projected outline is a two-dimensional polygonal outline);

[0066] S42. Calculate the geometric center coordinates of the bottom contour projection and the roof surface projection respectively (this is a prior art and will not be described in detail). Mark the geometric center coordinates of the roof surface projection as (x1, y1, z1), and the geometric center coordinates of the building bottom contour projection as (x2, y2, z2), where z1 = z2. Then, determine the translation parameters based on the two calculated geometric center coordinates. The translation parameters include the difference between the horizontal and vertical coordinates on the horizontal plane, that is, Δx = x1-x2, Δy = y1-y2, where Δx represents the translation parameter on the horizontal coordinate and Δy represents the translation parameter on the vertical coordinate. Afterwards, translate the roof surface projection according to the translation parameters. The geometric center of the translated roof surface projection coincides with the geometric center of the building bottom contour projection. It should be noted that in actual airborne LiDAR scanning or aerial photogrammetry, due to various factors such as scanning angle, flight altitude, equipment accuracy, etc., there may be positional deviations between the roof surface and the ground contour. In order to correct this deviation and facilitate subsequent modeling work, the roof surface projection needs to be translated so that its geometric center coincides with the geometric center of the building bottom outline projection. The reason for not translating the bottom outline here is that the bottom outline is usually in direct contact with the ground. Its extraction process can refer to the geometric features of the ground, so it has a higher position accuracy, which is not available for the roof surface.

[0067] S43. Identify the contour edges of the bottom contour projection and the roof surface projection on the reference horizontal plane, respectively, to form two edge sets, denoted as Ebase (corresponding to the bottom contour edge) and Eroof (corresponding to the roof surface contour edge). Since the bottom contour projection and the roof surface projection are essentially two-dimensional polygonal geometric figures on the reference horizontal plane, it is preferred to use image processing techniques such as the Canny algorithm to extract the contour edges, and then use a feature matching algorithm such as SIFT, SURF, or ORB to identify corresponding feature points on the contour edges of the bottom contour projection and the roof surface projection. Based on the feature point matching results, a one-to-one correspondence between the contour edges of the bottom contour projection and the roof surface projection is established;

[0068] S44. Perform similarity comparison on each pair of matching contour edges, and selectively correct the contour edge of the bottom contour projection according to the similarity comparison result, and finally obtain the corrected bottom contour projection. Specifically, the similarity comparison can be achieved, for example, by the Hausdorff distance algorithm, the Frechet algorithm, the shape context matching algorithm, etc. (this is the prior art and will not be introduced in detail). For the calculation results of each algorithm, a similarity scoring mechanism can be established, and a corresponding contour similarity scoring threshold Q can be set. For example, when the obtained similarity score is less than Q, it is determined that the pair of matching contour edges currently analyzed are not similar, otherwise, it is determined that the two are similar. When it is determined that a pair of matching contour edges are not similar, the contour edge of the roof surface projection in the pair of contour edges currently analyzed is used as a reference template, and the contour edge of the corresponding bottom contour projection is refitted. In the fitting process, the contour edge obtained by refitting remains unchanged from the two end points of the contour edge before fitting. There are already many algorithms for contour edge fitting in the prior art. For example, convolutional neural networks (CNN) or other deep learning models are used to learn the mapping relationship between the top projection contour edge and the bottom projection contour edge, and then the corrected bottom contour edge is directly output through the trained model. This will not be introduced in detail here. According to the above process, similarity comparison and correction operations are carried out on all matching contour edges in turn until all processing is completed, and finally an accurately corrected bottom contour projection is obtained. It should be noted that although the bottom contour line of the building can refer to the geometric features of the ground when extracting it, and has a high position accuracy, there is often local occlusion on the ground, which causes the extracted ground contour line to be inaccurate at the local position. For most buildings, the bottom contour line of the building is relatively similar to the contour line of its roof surface. Therefore, the present invention uses the roof surface projection contour edge to correct the bottom contour projection contour edge, which can reduce the adverse effects caused by local occlusion, etc., and effectively improve the accuracy of the bottom contour projection.

[0069] S45. First, orthographically project the corrected bottom contour from the reference horizontal plane back onto the ground, and intersect the ground to obtain a corrected building bottom contour line. Specifically, the orthographic projection onto the ground requires combining ground elevation data or a terrain model, and reversely mapping the two-dimensional projection points to the actual ground surface through a three-dimensional spatial coordinate transformation to ensure that the corrected bottom contour line conforms to the terrain (for example, by matching ground point cloud data through an interpolation algorithm).

[0070] Next, the roof surface fitted in step S3 is directly translated according to the translation parameter (after translation, its orthographic projection on the reference horizontal plane is the translated roof surface projection, which can be used for verification) to obtain a translated roof surface; then, the corrected bottom contour projection is overlapped with the translated roof surface projection on the reference horizontal plane, and the corrected bottom contour projection located outside the translated roof surface projection is identified and recorded as a compensation projection; identifying the compensation projection can clarify the difference area between the building bottom contour and the roof surface projection, providing a basis for further adjusting the shape and size of the roof surface, and promoting the formation of a watertight body frame model in subsequent modeling to avoid local disconnection and mismatch;

[0071] The translated roof surface is topologically extended so that it precisely overlaps the compensated projection, resulting in a corrected fitted roof surface. Specifically, the extended roof surface includes: first, determining the required extension direction and distance based on the shape and position of the compensated projection. This can be achieved by calculating the distance between the boundary points of the compensated projection and the nearest point of the translated roof surface projection. Next, a suitable extension algorithm is selected, such as a composite algorithm based on the region growing method and the random sampling consensus (RANSAC) algorithm. This composite algorithm combines the sensitivity of the region growing method to local features with the robustness of the RANSAC algorithm to noise, effectively addressing the problem of extracting geometric primitives in complex scenes. Next, the translated roof surface is extended according to the determined extension direction and distance, using the selected extension algorithm. During the extension process, care must be taken to maintain the geometric continuity and topological consistency of the roof surface, ensuring that the extended roof surface perfectly aligns with the compensated projection area. Finally, the corrected fitted roof surface is verified and adjusted. By comparing it with the compensated projection, it is checked for any local mismatches or over-extension. If necessary, the extended roof surface can be fine-tuned to ensure that it fully overlaps the compensated projection area. Preferably, during the compensation process, when the compensated projection area exceeds a preset threshold, a manual verification process is triggered to prevent the algorithm from over-correcting.

[0072] Through the operation of step S4, the roof surface and the bottom contour of the building are mutually correlated and corrected, which is conducive to the subsequent formation of a watertight model, avoiding local disconnection and mismatch, and thus improving the accuracy of model generation.

[0073] Step S5: Refined 3D reconstruction of the building

[0074] S51, directly extrude the corrected building bottom contour line obtained in step S4 in a direction perpendicular to the ground, and intersect it with the corrected fitted roof surface to form a watertight building frame model, thereby completing the preliminary reconstruction of the building model;

[0075] S52. Based on the outlines of the doors, windows, and other detailed components generated in step S3, a set of all identifiable door and window facets in the building is constructed (note: this set does not only include doors and windows, but may also include other detailed components). By assuming their depths and extruding them onto the preliminary reconstructed building model, the detailed components on the preliminary reconstructed building model are expressed.

[0076] To better achieve the objectives of the present invention, when completing the representation of detailed components on the preliminary reconstructed building model, facade feature constraints are established for detailed components such as doors and windows, including geometric constraints and position constraints. Taking doors and windows as an example, the geometric constraints for doors and windows mainly include depth ratio constraints and spacing ratio constraints. Among them, the depth ratio refers to the degree to which the doors and windows are indented or protruded outward. The depth usually has a certain proportional relationship with the length and width of the doors and windows. The proportional relationship between the degree to which the window is indented or protruded outward and the height of the window is called the depth ratio, or depth ratio. For doors and windows, common depth ratios range from 1:4 to 1:2. A smaller depth ratio means that the window is relatively less inward or outward, and the window is closer to a plane. Conversely, a larger depth ratio means that the window depth is more pronounced, and the window sill or window edge may protrude more significantly. In addition to the depth ratio, the spacing ratio between each window also needs to be considered during the reconstruction process. If there are multiple windows arranged together in a building, it is necessary to maintain a certain gap between the windows to maintain the balance of the overall facade. Typically, the horizontal and vertical spacing between windows can be adjusted according to the overall pattern and proportion of the facade to ensure visual harmony. Further, the pose constraints are implemented using the following steps:

[0077] Assume that the normal vector of the door and window plane is n=[n x ,n y ,n z ], the direction of the building facade is axis, the edge line direction vector is edge, to ensure that the door and window planes are parallel to the building facade, that is, n is parallel to axis, use the following dot product formula to calculate:

[0078] n·axis=n x axis x +n y axis y +n z axis z

[0079] Among them, n x 、n y 、n z They are the components of the normal vector n on the x-axis, y-axis, and z-axis, respectively. x 、axis y 、axis z The components of the building facade’s orientation axis on the x-axis, y-axis, and z-axis respectively. If the dot product is close to 0, it can be considered that the door and window planes are parallel to the building facade. Usually, a threshold can be set to determine the orientation close to 0, for example, a radian value of 1·10 -3 As a standard. In addition, we must ensure that the edge line of the door and window plane is parallel or perpendicular to the corresponding coordinate axes (x-axis, y-axis and z-axis). Assuming that the direction vector of the edge line of the door and window plane is edge, we can check whether it is parallel to the x, y and z axes by calculating the cross product of edge and axis. If the result is a zero vector, it means that edge is parallel to axis. Otherwise, if the result is not a zero vector, it means that they are not parallel and a posture rotation is required. The calculation process is as follows:

[0080] cross=edge※axis, that is, if the three components of cross are close to 0, the edge line is parallel or perpendicular to the corresponding coordinate axis; the effect of the pose constraint is as follows Figure 6 As shown;

[0081] S53. Modify the roof surface of the preliminary building model through roof surface feature constraints

[0082] The roof of a building is usually a single-layer structure, which is conducive to the processing of roof surface segmentation and reconstruction. However, in some cases, the point cloud of a building may contain multi-layer nodes (such as semi-transparent or rebuilt roofs), which may cause the problem of multi-layer roofs in the modeling process, and then the facade and roof surfaces in the model cannot be correctly closed. To avoid this situation, we assume that the higher roof surface is better than the lower surface and introduce a roof preference energy term to ensure that the higher roof surface is preferred in the modeling process. The roof preference energy term is:

[0083]

[0084] Among them, z i represents the z coordinate of the centroid of the hypothetical surface, z max and z min are the maximum and minimum coordinate height values ​​of the current single-family building, |F| represents the total number of hypothetical faces in the model, and x i∈{0, 1}, which is a binary variable. The size of Er reflects the preference for higher-positioned roof surfaces during the optimization process. A larger Er value indicates that higher-positioned roof surfaces account for a larger proportion of the selected surfaces, which helps to generate a reconstruction model that is more consistent with the actual building structure.

[0085] In addition, the correction of the roof surface also includes constructing a single-layer roof constraint for the roof. This constraint is used to ensure that the reconstructed 3D building model has a single-layer roof. The constraint conditions are defined as follows:

[0086]

[0087] Among them, V(f i ) represents the set of hypothetical surfaces that intersect the roof surface in the vertical direction, x k is a binary variable that indicates whether the face that intersects with the roof face is selected. It is usually assumed that the roof face at the top layer has a higher confidence than the face at the bottom layer. The facade in the final model also uses the height to the top layer as the extrusion distance to achieve the purpose of generating a closed model. The effect of the roof face feature constraint is as follows: Figure 7 As shown;

[0088] S54, Model optimization and output

[0089] Specifically, a binary linear function is constructed to process the reconstructed building model obtained by the detailed component expression in step S42 and the roof surface correction in step S43, and extreme value calculations are performed on all its facets. The constructed binary linear function is:

[0090]

[0091] Among them, f i Indicates the extreme value calculated for the i-th patch, E d 、E c and E r They refer to the degree of data fitting, model complexity and roof preference energy term, and assign different weight values ​​λ to each of them. d ,λ c and λ r ; Among them, the data fitting degree E d The energy term calculation formula is:

[0092]

[0093] The degree of fit between the point cloud and the patch is measured by calculating the confidence of the point neighborhood. P is the total number of points fitted to the patch. Each point in the neighborhood is traversed to calculate the support function value of its fit to the plane and solve the energy term E. dThe value of is between 0 and 1. The closer it is to 1, the better the point cloud data can fit the model plane in the neighborhood of the point.

[0094] In addition, E c is the complexity of the model result, and its calculation formula is as follows:

[0095]

[0096] Where |E| represents the total number of pairwise intersections in the hypothetical face set, e i The function represents the i-th edge between two faces in the hypothetical face set. By calculating the proportion of sharp edges in the pairwise intersection set |E| of the reconstructed model, the likelihood of irregular or distorted shapes is estimated. The corner function detects the shape of the intersection edge between two candidate planes. If the intersection of the two model faces is a sharp edge, the function returns 1; otherwise, it returns 0, indicating that the two planes are coplanar. By selecting the optimal data complexity, the resulting model is smoother and avoids sharp corners as much as possible.

[0097] Determine the weight values ​​λ of the three options through multiple experiments d ,λ c and λ r The optimal values ​​are 0.34, 0.62, and 0.04, respectively. By solving the extreme value of the function corresponding to each facet, the triangle with the smallest extreme value is added to the candidate model face set. Finally, the selected model faces are stitched together to ensure that the faces in the generated model are tightly connected and meet the inferred constraints. The final reconstruction result is then output.

[0098] See also Figure 8 Compared with existing automatic reconstruction methods (ArcGIS Pro and Geomagic Studio), the method of the present invention has obvious advantages in processing airborne LiDAR point cloud data to generate refined building models. Compared with the existing method of manually inputting point clouds, the method of the present invention can generate more complete and clear building models. Compared with the models generated by ArcGIS Pro and Geomagic Studio, the method of the present invention not only retains the overall structure of the building, but also can more accurately restore the detailed features of the building, such as the shape of windows and roofs. This shows that the method of the present invention has higher accuracy and better results in point cloud data processing and model reconstruction.

[0099] We also used the same accuracy evaluation index to quantitatively evaluate the automatic reconstruction results of different software, and the results are as follows:

[0100]

[0101] In the table, RMSE is the root mean square error, MPPE is the average point-to-plane distance error, and Hausdorff is the Hausdorff distance, and their units are all m (meters). As can be seen from the above table, ArcGIS Pro software is more efficient than other software when it comes to reconstructing large-scale scenes, but its resulting model still has problems with irregular and broken shapes; although the reconstruction results of GeomaticStuidio software have a high degree of model fitting and rich details of model expression, it cannot effectively handle the missing data. In addition, there are many holes in the resulting model, and a closed building model cannot be formed; although the reconstruction results of the method of the present invention require verification of assumptions and complex calculations, and the time consumption is slightly increased among several methods, the number of facets in its model is much less than that of the existing technology methods, and the model it generates is more regular and has higher reconstruction accuracy.

[0102] Finally, it should be noted that, in the present invention, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include," "comprise," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. Portions not described in detail in this specification belong to the prior art known to those skilled in the art.

[0103] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to the embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to be embodied in the widest possible manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for fine-grained 3D reconstruction of buildings based on airborne LiDAR point clouds, characterized in that: The method comprises the following steps: Step S1, obtaining original airborne LiDAR point cloud data; Step S2: pre-processing the original airborne LiDAR point cloud data to obtain high-quality building point cloud data; Step S3: extracting features from the point cloud data based on the high-quality building point cloud data, extracting basic geometric feature information, and extracting feature points, feature lines, and the bottom contour of the building based on the basic geometric feature information. Subsequently, the extracted features are used to segment the building roof surface and facade, and plane fitting is performed on the roof surface. Step S4: Based on the extracted building bottom contour line and the plane-fitted roof surface, the roof surface and the building bottom contour are associated and corrected to obtain a corrected building bottom contour line and a corrected fitted roof surface; Step S5: The corrected building bottom contour line from step S4 is directly extruded in a direction perpendicular to the ground, intersecting it with the corrected fitted roof surface to form a watertight building frame model, thereby completing the preliminary reconstruction of the building model. Subsequently, constraint conditions are constructed using prior knowledge to complete the expression of detailed components on the preliminary reconstructed building model and the correction of the reconstructed building roof surface. Finally, constraint conditions are constructed to optimize the model and output the final refined 3D reconstruction result.

2. The method for fine-grained 3D reconstruction of a building based on airborne LiDAR point cloud according to claim 1, wherein: In step S2, the original airborne LiDAR point cloud data is preprocessed, including: segmenting and extracting point cloud data of non-ground points from the original airborne LiDAR point cloud data; segmenting building point cloud data from the point cloud data of non-ground points; and processing the separated building point cloud data using point cloud data completion and simplification operations to obtain high-quality building point cloud data.

3. The method for fine-grained 3D reconstruction of a building based on airborne LiDAR point cloud according to claim 1 or 2, characterized in that: In step S3, the basic geometric feature information includes a normal vector and a curvature.

4. The method for fine-grained 3D reconstruction of a building based on airborne LiDAR point cloud according to claim 3, wherein: In step S5, when the expression of the detailed components on the preliminary reconstructed building model is completed, facade feature constraints are established for the detailed components, including geometric constraints and posture constraints.

5. The method for fine-grained 3D reconstruction of buildings based on airborne LiDAR point cloud according to claim 4, characterized in that: The detailed components are doors and windows, and the pose constraint is calculated based on the following formula to ensure that the door and window planes are parallel to the building facade: n·axis=n x ·axis x +n y ·axis y +n z ·axis z Among them, the normal vector of the door and window plane is n=[n x ,n y ,n z ], the building facade is oriented to axis, n x 、n y 、n z They are the components of the normal vector n on the x-axis, y-axis, and z-axis, respectively. x 、axis y 、axis z They are the components of the axis of the building facade on the x-axis, y-axis, and z-axis respectively. If the result of the dot product calculation is close to or equal to 0, it is considered that the door and window plane is parallel to the building facade.

6. The method for fine-grained 3D reconstruction of a building based on airborne LiDAR point cloud according to claim 5, wherein: The pose constraint is calculated based on the following formula to ensure that the edge lines of the door and window planes are parallel or perpendicular to the corresponding coordinate axes: Where edge is the edge line direction vector. If the components of cross in the x, y, and z directions are all close to 0, the edge line is parallel or perpendicular to the corresponding coordinate axis.

7. The method for fine-grained 3D reconstruction of buildings based on airborne LiDAR point cloud according to claim 5, wherein: In step S5, when correcting the reconstructed roof surface, the following steps are performed to solve the problem that the facade model and the roof surface cannot be closed due to multi-layer roofs: First, define the roof preference energy term as: Among them, z i represents the z coordinate of the centroid of the hypothetical surface, z max and z min are the maximum and minimum coordinate height values ​​of the current single-family building, |F| represents the total number of hypothetical faces in the model, and x i ∈{0, 1}, which is a binary variable; Then construct a single-layer roof constraint for the roof, and its constraint conditions are defined as follows: Among them, V(f i ) represents the set of hypothetical surfaces that intersect the roof surface in the vertical direction, x k is a binary variable indicating whether the face intersecting with the roof face is selected.

8. The method for fine-grained 3D reconstruction of a building based on airborne LiDAR point cloud according to claim 7, wherein: In step S5, the construction of constraint conditions for model optimization includes: A binary linear function is constructed to process the reconstructed building model obtained after detailed component expression and roof surface correction, and extreme value calculations are performed on all its facets. The constructed binary linear function is: Among them, E d 、E c and E r They refer to the degree of data fitting, model complexity and roof preference energy term, λ d ,λ c and λ r Assigned to E d 、E c and E r The weight value, and the data fitting degree E d The energy term calculation formula is: The degree of fit between the point cloud and the patch is measured by calculating the confidence of the point neighborhood. P is the total number of points fitted to the patch. Each point in the neighborhood is traversed to calculate the support function value of its fit to the plane and solve the energy term E. d The value of is between 0 and 1. The closer it is to 1, the better the point cloud data can fit the model plane in the neighborhood of the point.

9. The method for fine-grained 3D reconstruction of a building based on airborne LiDAR point cloud according to claim 8, wherein: The complexity of the model results E c The calculation formula is as follows: Among them, the proportion of sharp edges in |E| is used to estimate the possibility of irregular or distorted shapes, e i Represents the i-th edge between two faces in the hypothetical face set. The function of the corner function is to detect the shape of the intersection edge between two candidate planes. If the intersection of the two model faces is a sharp edge line, the function value returns 1, otherwise it is 0, that is, the two planes are coplanar.

Images