A method for reconstructing a multi-level-of-detail model of a building based on multi-source data fusion

Abstract

A method for reconstructing a building multi-level of detail model based on multi-source data fusion provided by the present invention includes: performing adaptive filtering on ground point clouds based on the three-dimensional geometric information of airborne laser point clouds, and constructing a DEM for the ground points; based on remote sensing image data, automatically extracting two-dimensional building patches by using object-oriented and convolutional neural networks, and regularizing the initial building patches by using graph optimization theory to obtain building patches; reconstructing a building LoD1 model by combining the three-dimensional elevation of the building roof and the DEM elevation value; based on the building monomerized point clouds, inferring and complementing the building facade structure by using a structure-constrained triangular network to reconstruct a building LoD2 model; and performing automatic texture mapping on the reconstructed building LoD2 model and the building LoD1 model based on oblique images. It can quickly solve the problem of quickly reconstructing a three-dimensional structured model of buildings in a large-scale urban scene, and promote urban intelligent and refined management.

Description

Technical Field

[0001] The present invention relates to the field of large-scale urban building models, and in particular, to a method for reconstructing a multi-level detail model of a building based on multi-source data fusion. Background Art

[0002] The reconstruction of large-scale urban building models has always been a hot issue in real-scene 3D modeling. The building point cloud elevation data obtained by a single airborne LiDAR measurement system is missing and uneven in density, resulting in problems such as difficult building monomerization and low automation. With the rapid development of surveying and mapping equipment, multi-platform and multi-type ground measurement systems in the air, space, and on the ground can obtain 3D laser point clouds and image texture data all-weather, at high speed, and with high density, having significant advantages in large-scale urban spatial information collection and providing a data basis for automated data processing. However, due to complex real-world environments, rich targets, and ground object occlusion, etc., the development level of 3D data automated and intelligent processing software is lower than that of hardware. Specifically, it is manifested as problems such as difficult feature extraction, low classification accuracy, low automation, and low reconstruction accuracy, and it is difficult to meet the requirements of 3D entity expression in urban scenes. Summary of the Invention

[0003] In view of the above problems, the present invention proposes a method for reconstructing a multi-level detail model of a building based on multi-source data fusion to overcome or at least partially solve the above problems.

[0004] According to one aspect of the present invention, there is provided a method for reconstructing a multi-level detail model of a building based on multi-source data fusion, and the model reconstruction method includes:

[0005] Performing adaptive filtering on ground point clouds based on the 3D geometric information of airborne laser point clouds and constructing a DEM for the ground points;

[0006] Based on remote sensing image data, automatically extracting 2D building patches by using object-oriented and convolutional neural networks, and regularizing the initial building patches by using graph optimization theory to obtain building patches;

[0007] Obtaining building monomerization point clouds based on the building patches, and reconstructing a LoD1 model of the building by combining the 3D elevation of the building roof and the DEM elevation value;

[0008] Based on the building monomerization point clouds, performing building facade structure reasoning and complementation by using a structure-constrained triangular mesh to reconstruct a LoD2 model of the building;

[0009] Performing automatic texture mapping on the reconstructed LoD2 model of the building and the LoD1 model of the building based on oblique images.

[0010] Optionally, the step of performing adaptive filtering of ground point clouds based on the three-dimensional geometric information of airborne laser point clouds and constructing DEM for ground points specifically includes:

[0011] The original point cloud is evenly divided based on the virtual gridding method. A virtual point is interpolated for the grid without point cloud. The lowest point in the grid is used as the grid point, and the remaining point clouds in the grid are marked as other point clouds.

[0012] Corresponding primitives are used to express different types of areas of grid points.

[0013] Optionally, the expressing using corresponding primitives according to different types of regions of the grid points specifically includes:

[0014] Areas with gentle elevation are represented by patches, and areas with drastic changes are represented by points. For the patch set, the point cloud segmentation and filtering method is used. There is an elevation difference between the boundary point cloud of the non-ground patch and the ground. The lowest point in a certain area is set as the ground seed point to build a temporary DEM. The elevation difference between each patch boundary point and the temporary DEM is calculated and compared with the elevation difference threshold to divide the patch into ground patches and non-ground patches.

[0015] For discrete independent point cloud sets, a multi-scale morphological filtering method is used to process them and remove non-ground point clouds;

[0016] For other point clouds, the distance from the unclassified point cloud to the temporary DEM and the corresponding terrain slope are calculated, and compared with the threshold to distinguish the unclassified point cloud into ground point cloud and non-ground point cloud;

[0017] For the separated ground point cloud, inverse distance interpolation is performed on the holes, and then the DEM is generated using limited Delaunay triangulation.

[0018] Optionally, based on the remote sensing image data, the object-oriented and convolutional neural network are used to automatically extract the two-dimensional building spots, and the initial building spots are regularized by using graph optimization theory to obtain the building spots, which specifically includes:

[0019] The multi-scale segmentation method is used to segment the same image at different scales to express the multi-scale building coverage characteristics and spatial structure information of the image;

[0020] Based on the multi-scale segmentation results of the image, the optimal feature combination is established for the color, texture, geometry and other features of each feature.

[0021] Construct a multi-dimensional convolutional neural network model and create a large number of sample sets to extract the two-dimensional image information of buildings and improve the accuracy and precision of building recognition;

[0022] Based on the problems of uneven edges, jagged boundaries, and irregular shapes in the extracted building patches, the angular and distance deviations of the building contours are globally corrected using graph optimization theory and the G2o solver;

[0023] During the optimization process, the specified line segments are used as vertices and the adjacent line segments are used as edges to form a graph; by adjusting the vertices to satisfy the edge constraints, the overall error is minimized to regularize the building patches;

[0024] The error correction formula is (1), and the weight parameter λ is used to balance the data term and the smooth term;

[0025] E(x) = (1 - λ)·D(x) + λ·B(x) (1)

[0026] For angle correction, the data term D(x) is used to correct the angular deviation from the initial direction, and the expression is:

[0027]

[0028] Angle correction value x i ∈[-θ max , θ max , is added to the initial direction of line segment i, with the clockwise direction being positive and the counterclockwise direction being negative; θ max is the angle correction threshold, which is adjusted according to the quality of the point cloud, and n is the total number of extracted line segments;

[0029] The smooth term B(x) is used to correct the geometric relationship between adjacent line segments, and the expression is:

[0030]

[0031]

[0032]

[0033] θ ij is the angle between adjacent line segments s i , s j (θ ij ∈[-2π, 2π]), and after correction, the adjacent line segments are close to being parallel, perpendicular, or collinear, and the angle θ ij is adjusted to be closer to the coordinate axes, and the expression is (4);

[0034] If |θ ij | < 2*θ max , the parameter u ij = 1, otherwise u ij = 0. If the angle between adjacent line segments is within the threshold range, optimization is performed; otherwise, no optimization is performed. At the same time, the angle threshold θ maxAfter correcting the angle, it cannot deviate too much from its own direction;

[0035] where k is related to the line segment s i the number of adjacent line segments, with the line segment s i as the center, find its k nearest neighbors;

[0036] After correcting the angle of the line segment, distance correction needs to be performed on it, and the correction principle is the same as that of angle correction;

[0037] In the optimization process of regularization processing, the boundaries of all component straight line segments are optimized as a whole to ensure the inherent collinear, parallel, and orthogonal relationships between adjacent straight line segments.

[0038] Optionally, obtaining the building monomerized point cloud based on the building patch, and reconstructing the building LoD1 model by combining the three-dimensional elevation of the building roof and the DEM elevation value specifically includes:

[0039] Taking the regularized building patch as a semantic constraint, extract the building monomerized point cloud;

[0040] In the urban area, the amount of airborne lidar point cloud data is large. If each point is traversed and judged whether it is in the building patch, it will be very time-consuming. Only traverse the non-ground points after point cloud filtering. The non-ground points include the building point cloud and the point clouds of other ground objects;

[0041] Build a kd-tree for the non-ground points and calculate the circumcircle of the building patch;

[0042] Taking the center of the building patch bounding box as the center of the circle, according to the radius, obtain the point cloud in the circumcircle area of the building patch;

[0043] Judge whether the points in this area are in the building patch polygon to obtain the building monomerized point cloud;

[0044] According to the extracted building monomerized point cloud, use the RANSAC plane segmentation algorithm to fit the building roof plane to obtain the average roof height;

[0045] Project the boundary points of the regularized building patch onto the terrain model DEM, calculate the intersection point coordinates, and take the average Z value of the intersection point coordinates as the lowest elevation value of the building;

[0046] Stretch the building patch to reconstruct the building LOD1 model, the reconstructed single building LoD1 model.

[0047] Optionally, based on the building monomerized point cloud, using the structure-constrained triangular mesh to perform building facade structure inference and completion, and reconstructing the building LoD2 model specifically includes:

[0048] The roof plane information is obtained by using the RANSAC plane segmentation algorithm, and the contour boundary of the plane is extracted;

[0049] Intersecting line segments are obtained by the intersection of adjacent plane polygons; the roof polygon, the intersecting line segments and the elevation are projected onto the horizontal plane to construct a two-dimensional constrained Delaunay triangulation network, and the structural feature constrained triangulation network is obtained;

[0050] A multi-label graph cut function model is constructed, and by solving the minimum energy, the plane label corresponding to each triangle and the roof surface to which the triangular patch should belong are obtained;

[0051] The optimized polygon boundary has no overlap, no intersection and no gap, and the two-dimensional footprint polygon is divided into multiple roof surfaces; based on the elevation information of the three-dimensional plane, the elevation is stretched from the roof to the ground to reconstruct the three-dimensional roof model.

[0052] Optionally, the automatic texture mapping of the reconstructed building LoD2 model and the building LoD1 model based on the oblique image specifically includes:

[0053] Straight lines are extracted from the projection range on the image based on the LSD algorithm, and the straight lines are back-projected onto the plane and added to the network construction to obtain a triangulation network constrained by image straight lines, and the vertices or edges with smooth texture are folded and simplified;

[0054] Since the positions of the shadows on different oblique images are different and affect the texture color, taking advantage of the fact that the positions of the image boundary straight lines on different temporal images are different, the shadow boundaries are extracted from the multi-view images to infer the shadow areas;

[0055] Based on the triangular mesh texture mapping algorithm, the texture mapping of the oblique image is realized.

[0056] A method for reconstructing a building multi - level of detail model based on multi - source data fusion provided by the present invention, the model reconstruction method includes: performing adaptive filtering on ground point clouds based on the three - dimensional geometric information of airborne laser point clouds, and constructing a DEM for the ground points; based on remote sensing image data, automatically extracting building two - dimensional patches using object - oriented and convolutional neural networks, and regularizing the initial building patches using graph optimization theory to obtain building patches; obtaining building - monomerized point clouds based on the building patches, and reconstructing a building LoD1 model by combining the three - dimensional elevation of the building roof and the DEM elevation value; based on the building - monomerized point clouds, using a structure - constrained triangular network to infer and complete the building facade structure, and reconstructing a building LoD2 model; performing automatic texture mapping on the reconstructed building LoD2 model and the building LoD1 model based on oblique images. It can quickly solve the problem of rapid reconstruction of three - dimensional structured models of buildings in large - scale urban scenes, provide an entity - based three - dimensional data foundation for urban governance and urban renewal, create a three - dimensional visual and analyzable virtual digital twin city, and promote intelligent and refined urban management.

[0057] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable, the following specifically gives the specific embodiments of the present invention. Brief Description of the Drawings

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0059] Figure 1 It is a flowchart of a method for reconstructing a building multi - level of detail model based on multi - source data fusion provided by an embodiment of the present invention. Detailed Description of the Embodiments

[0060] The exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0061] In the embodiments of the specification, claims, and drawings of the present invention, the terms "comprising", "having", and any variations thereof are intended to cover non-exclusive inclusion. For example, a series of steps or units are included.

[0062] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments.

[0063] As Figure 1 shown, an automated reconstruction method for a building multi-level of detail model based on multi-source data fusion specifically includes:

[0064] Adaptive partition point cloud filtering based on airborne laser point cloud to achieve automatic separation of ground points and non-ground points of the original point cloud, and construct a DEM based on the ground points;

[0065] Based on remote sensing image data, fuse object-oriented and convolutional neural networks for building patch extraction, and global consistency correction of error lines based on graph optimization theory;

[0066] Based on the building patches, combine the three-dimensional elevation of the building roof and the DEM elevation value to reconstruct the LoD1 model of the building. Based on the building monomerized point cloud, use the structure-constrained triangular network for building facade structure reasoning and completion, and thus reconstruct the LoD2 model of the building;

[0067] Automatically texture map the reconstructed model based on the oblique image.

[0068] The point cloud adaptive filtering specifically includes:

[0069] The present invention applies an adaptive partitioning point cloud filtering method, integrates point cloud segmentation filtering methods of different primitives and multi-scale morphological filtering methods, and realizes the automatic separation of ground points and non-ground points of the original point cloud. In the case of fractured terrain, the filtering effect is better than most filtering algorithms, has good robustness and adaptability, and can process large-scale point cloud data. First, the present invention evenly divides the original point cloud based on the virtual gridding method to make the point cloud distribution as uniform as possible. For the grid without point cloud, a virtual point is interpolated, the lowest point in the grid is used as the grid point, and the remaining point cloud in the grid is marked as other point clouds. Then, different primitives are used to express different types of areas in the grid points, that is, flat areas are represented by patches, and areas with drastic changes are represented by points. For the patch set, a point cloud segmentation filtering method is adopted, that is, there is a certain elevation difference between the boundary point cloud of the non-ground patch and the ground. Therefore, the lowest point in a certain area is set as the ground seed point to construct a temporary DEM, and the elevation difference between each patch boundary point and the temporary DEM is calculated, and compared with the elevation difference threshold to divide the patch into ground patch and non-ground patch. For the discrete independent point cloud set, a multi-scale morphological filtering method is used to process it to eliminate non-ground point clouds. Finally, for other point clouds, the distance from the unclassified point cloud to the temporary DEM and the corresponding terrain slope are calculated and compared with the threshold to distinguish the unclassified point cloud into ground point cloud and non-ground point cloud. For the separated ground point cloud, inverse distance interpolation is performed on the holes, and then the DEM is generated using the restricted Delaunay triangulation.

[0070] Extract buildings based on remote sensing images, including:

[0071] The present invention studies the object-oriented and convolutional neural network building extraction based on high-resolution remote sensing images. In high-resolution remote sensing images, a single object is composed of multiple pixels and there are obvious differences between them.

[0072] First, the present invention adopts a multi-scale segmentation method to segment the same image at different scales to express the multi-scale building coverage characteristics and spatial structure information of the image and better describe the target shape of the target building.

[0073] Then, based on the multi-scale segmentation results of the image, the optimal feature combination is established for the color, texture, geometry and other characteristics of each object patch, a multi-dimensional convolutional neural network model is constructed, and a large number of sample sets are produced to extract the two-dimensional patch information of the building, thereby improving the accuracy and precision of building recognition.

[0074] For the problems of uneven edges, jagged boundaries, and irregular shapes existing in the extracted building patches, the present invention globally corrects the angular and distance deviations of the building contours based on the graph optimization theory and the G2o solver. During the optimization process, the specified line segments are used as vertices and their adjacent line segments are used as edges to form a graph; by adjusting the vertices to satisfy the constraints of the edges, the overall error is minimized, and finally the regularization of the building patches is completed; the error correction formula is (1), and the weight parameter λ is used to balance the data term and the smooth term.

[0075] E(x) = (1 - λ)·D(x) + λ·B(x) (1)

[0076] For angle correction, the data term D(x) is used to correct the angular deviation from the initial direction, and the expression is:

[0077]

[0078] Angle correction value x i ∈[-θ max , θ max , is added to the initial direction of line segment i, with the clockwise direction being positive and the counterclockwise direction being negative; θ max is the angle correction threshold, which is adjusted according to the quality of the point cloud, and n is the total number of extracted line segments.

[0079] The smooth term B(x) is used to correct the geometric relationship between adjacent line segments, and the expression is:

[0080]

[0081]

[0082]

[0083] θ ij is the included angle between adjacent line segments s i , s j (θ ij ∈[-2π, 2π]), and the corrected adjacent line segments are as close as possible to being parallel, perpendicular, or collinear, and the included angle θ ij is adjusted to be closer to the coordinate axes, and the expression is (4).

[0084] If |θ ij | < 2*θ max , the parameter u ij = 1, otherwise u ij = 0. The purpose is that if the included angle between adjacent line segments is within the threshold range, optimization is performed, otherwise not; at the same time, the angle threshold θ max ensures that the corrected angle does not deviate too much from its own direction. Where k is related to line segment s iThe number of adjacent line segments, with line segment s i as the center, find its k nearest neighbors. After the line segment angle correction, distance correction needs to be carried out, and the correction principle is the same as that of the angle correction.

[0085] In the optimization process of regularization, the boundaries of all component straight line segments are optimized as a whole to ensure the inherent collinear, parallel, and orthogonal relationships between adjacent straight line segments.

[0086] Reconstruction of the building LOD1 model, specifically including:

[0087] Taking the regularized building patches as semantic constraints, extracting the building monomerized point cloud. The airborne lidar point cloud data within the urban area is large in quantity. If each point is traversed to determine whether it is within the building patch, it will be very time-consuming. In the present invention, only the non-ground points after point cloud filtering are traversed. The non-ground points include the building point cloud and the point clouds of other ground objects, greatly reducing the number of traversed point clouds.

[0088] Construct a kd-tree for the non-ground points, calculate the circumcircle of the building patch; with the center of the building patch bounding box as the center of the circle, according to the radius, obtain the point cloud in the circumcircle area of the building patch. Furthermore, determine whether the points in this area are within the building patch polygon to obtain the building monomerized point cloud.

[0089] For the extracted building monomerized point cloud, use the RANSAC plane segmentation algorithm to fit the building roof plane to obtain the average roof height; at the same time, project the boundary points of the regularized building patch onto the DEM (terrain model), calculate the intersection point coordinates, and take the average Z value of the intersection point coordinates as the lowest elevation value of the building; stretch the building patch to reconstruct the building LOD1 model.

[0090] Texture mapping based on oblique images, specifically including:

[0091] Based on the reconstructed building structured model, use the oblique image for automatic texture mapping to enhance the 3D visualization of the reconstructed model.

[0092] First, extract straight lines based on the LSD algorithm from the projection range on the image, and back-project the straight lines onto the plane and add them to the network construction to obtain a triangular network constrained by the image straight lines, and fold and simplify the vertices or edges with smooth textures to reduce the number of triangular mesh patches.

[0093] Then, since the positions of shadows on different oblique images are different, affecting the texture color, the present invention uses the characteristic that the positions of the image boundary straight lines on different temporal images are different to extract the shadow boundaries from multi-view images, so as to infer the shadow areas and make the texture image avoid the shadow areas.

[0094] Beneficial effects: The method of the present invention can quickly reconstruct the problem of quickly reconstructing the three-dimensional structured model of buildings in a large-scale urban scene, provide an entity-based three-dimensional data foundation for urban governance and urban renewal, create a three-dimensional visual and analyzable virtual digital twin city, and promote intelligent and refined urban management.

[0095] The above specific embodiments have further detailed the purpose, technical solution and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for reconstructing a building multi-level detail model based on multi-source data fusion, characterized in that: The model reconstruction method comprises: Based on the 3D geometric information of airborne laser point cloud, adaptive filtering of ground point cloud is performed and DEM is constructed for ground points; Based on remote sensing image data, object-oriented and convolutional neural networks are used to automatically extract two-dimensional building patches, and graph optimization theory is used to regularize the initial building patches to obtain building patches. Obtaining a building monomer point cloud based on the building patch, and reconstructing the building LoD1 model by combining the building roof three-dimensional elevation and DEM elevation value; Based on the building's monomer point cloud, the building's facade structure is inferred and completed using the structural constraint triangulation network to reconstruct the building's LoD2 model; Automatic texture mapping is performed on the reconstructed building LoD2 model and the building LoD1 model based on the oblique image.

2. The method for reconstructing a building multi-level detail model based on multi-source data fusion according to claim 1, characterized in that: The method of adaptively filtering the ground point cloud based on the three-dimensional geometric information of the airborne laser point cloud and constructing the DEM for the ground points specifically includes: The original point cloud is evenly divided based on the virtual gridding method. A virtual point is interpolated for the grid without point cloud. The lowest point in the grid is used as the grid point, and the remaining point clouds in the grid are marked as other point clouds. Corresponding primitives are used to express different types of areas of grid points.

3. The method for reconstructing a building multi-level detail model based on multi-source data fusion according to claim 2, characterized in that: The expression of using corresponding primitives according to different types of regions of grid points specifically includes: Areas with gentle elevation are represented by patches, and areas with drastic changes are represented by points. For patch sets, a point cloud segmentation and filtering method is used. There is an elevation difference between the boundary point cloud of non-ground patches and the ground. The lowest point in a certain area is set as the ground seed point to construct a temporary DEM. The elevation difference between each patch boundary point and the temporary DEM is calculated and compared with the elevation difference threshold to divide the patches into ground patches and non-ground patches. For discrete independent point cloud sets, a multi-scale morphological filtering method is used to process them and remove non-ground point clouds; For other point clouds, the distance from the unclassified point cloud to the temporary DEM and the corresponding terrain slope are calculated, and compared with the threshold to distinguish the unclassified point cloud into ground point cloud and non-ground point cloud; For the separated ground point cloud, inverse distance interpolation is performed on the holes, and then the DEM is generated using limited Delaunay triangulation.

4. The method for reconstructing a building multi-level detail model based on multi-source data fusion according to claim 1, characterized in that: The method of automatically extracting two-dimensional building patches based on remote sensing image data by using object-oriented and convolutional neural networks, and regularizing the initial building patches by using graph optimization theory to obtain the building patches specifically includes: The multi-scale segmentation method is used to segment the same image at different scales to express the multi-scale building coverage characteristics and spatial structure information of the image; Based on the multi-scale segmentation results of the image, the optimal feature combination is established for the color, texture, geometry and other features of each feature. Construct a multi-dimensional convolutional neural network model and create a large number of sample sets to extract the two-dimensional image information of buildings and improve the accuracy and precision of building recognition; According to the problems of uneven edges, jagged boundaries and irregular shapes of the extracted building patches, the angle and distance deviations of the building outlines are corrected globally based on graph optimization theory and the G2o solver. In the optimization process, the specified line segments are used as vertices and the adjacent line segments are used as edges to form a graph. The overall error is minimized by adjusting the vertices to meet the edge constraints and complete the regularization of the building pattern. The error correction formula is (1), and the weight parameter λ is used to balance the data term and the smooth term; E(x)=(1-λ)·D(x)+λ·B(x) (1) For angle correction, the data item D(x) is used to correct the angle deviation from the initial direction, and the expression is: Angle correction value x i ∈[-θ max ,θ max ], is added to the initial direction of line segment i, the clockwise direction is positive and the counterclockwise direction is negative; θ max is the angle correction threshold, which is adjusted according to the quality of the point cloud, and n is the total number of extracted line segments; The smoothing term B(x) is used to correct the geometric relationship between adjacent line segments and is expressed as: θ ij is the adjacent line segment s i ,s j The angle (θ ij ∈[-2π,2π]), the corrected adjacent line segments are close to parallel, vertical or collinear, and the angle θ ij Adjust to be closer to the coordinate axis, the expression is (4); If |θ ij |<2*θ max , parameter u ij =1, otherwise u ij = 0, if the angle between adjacent line segments is within the threshold range, then optimization is performed, otherwise no optimization is performed; at the same time, the angle threshold θ max Make sure that the angle does not deviate too much from its own direction after correction; where k is the line segment s i The number of adjacent line segments, expressed as line segment s i As the center, find its k nearest neighbors; After the line segment angle is corrected, the distance needs to be corrected. The correction principle is the same as that of the angle correction. In the optimization process of regularization, the boundaries of all component straight line segments are optimized as a whole to ensure the inherent collinearity, parallelism and orthogonality between adjacent straight line segments.

5. The method for reconstructing a building multi-level detail model based on multi-source data fusion according to claim 1, characterized in that: Obtaining a building monomer point cloud based on the building patch and reconstructing the building LoD1 model in combination with the building roof three-dimensional elevation and DEM elevation value specifically includes: The regularized building patches are used as semantic constraints to extract the individual point clouds of the buildings. The amount of airborne laser point cloud data in the city is large. It would be very time-consuming to traverse each point and determine whether it is in the building patch. Only non-ground points after point cloud filtering are traversed. Non-ground points include building point clouds and point clouds of other objects. Establish a kd-tree for non-ground points and calculate the circumscribed circle of the building patch; Taking the center of the bounding box of the building pattern as the center of the circle, according to the radius, obtain the point cloud in the circumscribed circle area of ​​the building pattern; Determine whether the area point is in the building patch polygon and obtain the building monomer point cloud; According to the extracted building monomer point cloud, the RANSAC plane segmentation algorithm is used to fit the building roof plane to obtain the mean roof height; Project the regularized building patch boundary points onto the terrain model DEM, calculate the intersection coordinates, and use the average Z value of the intersection coordinates as the lowest elevation value of the building; The building pattern is stretched to reconstruct the building LOD1 model and the single building LoD1 model is reconstructed.

6. The method for reconstructing a building multi-level detail model based on multi-source data fusion according to claim 1, characterized in that: The method of reconstructing the building LoD2 model by using the structural constraint triangulation network to infer and complete the building facade structure based on the building monomer point cloud specifically includes: Use the RANSAC plane segmentation algorithm to obtain the roof plane information and extract the contour boundary of the plane; Adjacent plane polygons intersect to obtain intersecting line segments; roof polygons, intersecting line segments and facades are projected onto a horizontal plane to construct a two-dimensional constrained Delaunay triangulation to obtain a structural feature constrained triangulation; Construct a multi-labeled graph cut function model, and obtain the plane label corresponding to each triangle and the roof surface to which the triangle should belong by solving the minimum energy. The optimized polygonal boundaries have no overlap, no intersection, and no gaps, and the two-dimensional footprint polygon is divided into multiple roof surfaces; based on the elevation information of the three-dimensional plane, the facade is stretched from the roof down to the ground to reconstruct the three-dimensional roof model.

7. The method for reconstructing a building multi-level detail model based on multi-source data fusion according to claim 1, characterized in that: The automatic texture mapping of the reconstructed building LoD2 model and the building LoD1 model based on the oblique image specifically includes: Extract straight lines from the projection range on the image based on the LSD algorithm, and back-project the straight lines to the plane, add them to the network, obtain the triangulated network constrained by the image straight lines, and fold and simplify the vertices or edges with smooth textures; In view of the different positions of shadows on different oblique images, which affect the texture color, the shadow boundary is extracted from the multi-view images and the shadow area is inferred by taking advantage of the different positions of the image boundary lines on images of different phases. Based on the triangulated network texture mapping algorithm, texture mapping of oblique images is realized.

Images