A method for constructing building models based on dense matching point clouds of UAV low-altitude images

Abstract

The present invention belongs to the technical field of three-dimensional building modeling, and specifically is a method for constructing a building model based on dense matching of point clouds of unmanned aerial vehicle low-altitude images. The method uses the original point cloud data of the building as input, classifies it according to the geometric features of the physical building, obtains point cloud data representing each plane of the building, and completes the missing planes to construct a candidate plane set. The candidate plane set is classified according to the affiliation of a single building to obtain point cloud data representing a single building, thereby realizing building modeling. During the modeling process, the point cloud representing a single building is divided into a roof and a wall, and the top of the wall and the bottom of the roof are completed. The Manhattan model is used to model the wall that completes the top, and the intersecting plane selection method is used to model the roof that completes the bottom. The wall model and the roof model of the building are spliced ​​to obtain a model of a single building with a sloping roof. In summary, the building model construction method of the present invention improves the expression precision and accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional building modeling, and in particular relates to a method for constructing a building model based on dense matching point clouds of unmanned aerial vehicle low-altitude images. Background Art

[0002] At present, the methods for constructing three-dimensional building models mainly include traditional methods based on manual drawing, surface reconstruction based on mesh models, and building model construction based on point clouds. Traditional manual modeling methods rely on a lot of manpower and material resources to collect and measure building parameters or vector maps, and have problems such as high modeling costs and low efficiency. Although surface reconstruction based on mesh models can build mesh models of large-scale scenes at a lower cost, the constructed models will appear jagged, not smooth enough, and cannot be updated in areas. The building model construction method based on point clouds has the advantages of high accuracy, small model data volume, and high degree of construction automation, and is gradually becoming the mainstream.

[0003] In the field of building modeling, point cloud-based building modeling methods can be divided into four categories: boundary expression modeling, dimensionality reduction modeling, divide-and-conquer strategy modeling, and prior hypothesis modeling. Among them:

[0004] The key to the boundary expression modeling idea is to segment the facade and roof of the building and extract the boundary lines of the facade and roof of the building. On this basis, according to the topological relationship between these facets and boundary lines, the key points of the model are generated, and then the geometric model of the building is drawn. However, this method has high requirements on the quality of the point cloud used and is extremely sensitive to noise and missing.

[0005] The idea of ​​dimensionality reduction modeling is to transform the modeling problem of 3D point cloud into the problem of face segmentation and topology optimization in 2D space. However, this method weakens the integrity and comprehensiveness of the 3D information of the point cloud, making it difficult to construct a detailed 3D model.

[0006] The key point of the divide-and-conquer strategy modeling is to divide the complex building point cloud into multiple sub-units, then reconstruct each sub-unit, and finally obtain the complete building model by splicing these sub-unit models. However, this method usually divides the building point cloud at a smaller scale, so it cannot accurately express the global structure of the building.

[0007] The core idea of ​​prior hypothesis modeling is to presuppose that the created model is a certain shape, such as a multi-level flat-top prism model, a Manhattan model, a polyhedron combination model, or a model in a predefined model primitive library, and then construct a three-dimensional building model that meets the prior hypothesis according to the characteristics of the building point cloud itself; but this method can only model buildings that meet the prior hypothesis. In addition, a major defect of point cloud data is its incompleteness and clutter. It is challenging to quickly extract building point clouds from huge point cloud data. In addition, factors such as vegetation in the environment will affect modeling. For example, the point cloud of tall vegetation next to the building is easily mistaken for the building point cloud, which will cause the building point cloud to be missing due to occlusion, which will eventually affect the modeling effect; although the vast majority of urban buildings are multi-level flat-top buildings that meet the Manhattan hypothesis, there are still some sloping-roof buildings that do not meet the Manhattan hypothesis. For these buildings, the method based on the Manhattan hypothesis is not applicable. Summary of the invention

[0008] In view of this, the present invention proposes a building model construction method based on dense matching point cloud of UAV low-altitude images to solve the problem of insufficient expression accuracy of the model obtained by existing group building modeling.

[0009] To solve the above problems, the present invention adopts the following technical solutions:

[0010] A method for constructing a building model based on dense matching point clouds of UAV low-altitude images, comprising the following steps:

[0011] Step 1: Obtain and process the unordered low-altitude drone images of the target scene to obtain the original point cloud data of the target scene;

[0012] Step 2: According to the geometric features of the physical building, the original point cloud data is classified to obtain the point cloud data corresponding to each building plane in the target scene; based on these data, the plane fitting method is used to fit the plane of the building, and then a candidate plane set is constructed; at the same time, for the blocked facade, it is determined with the help of adjacent walls, that is, it is determined by the roof and the two facades adjacent to the missing wall and the ground, and the completed plane is added to the candidate plane set;

[0013] Step 3: According to the ownership relationship of a single building, the candidate plane set is classified to obtain multiple sets of point cloud data representing a single building, and a large number of initial planes in the candidate plane set are processed to obtain the effective plane of a single building;

[0014] Step 4: Based on the point cloud data of individual buildings obtained in step 3, model each individual building as follows:

[0015] Step 4.1, use the semantic dual classification network based on PointNet to segment each group of point cloud data representing a single building into roof and wall, and obtain the intersection line between the two; perform geometric calculations based on the intersection line between the roof and the wall to determine the interface point cloud to complete the top of the wall and the bottom of the roof, so as to form two completely independent closed spaces;

[0016] Step 4.2: Based on the wall after the top is completed, the Manhattan model is used to model the wall; based on the roof after the bottom is completed, the intersecting plane selection method is used to model the roof; the wall model and the roof model are spliced ​​together to obtain a single building model;

[0017] Step 5: Put all the individual building models together to get the model of the building complex.

[0018] Furthermore, the implementation method of step 1 comprises the following steps:

[0019] 1.1. Obtain unordered low-altitude drone images of the target scene. Using the unordered low-altitude drone images as input, use the motion recovery structure algorithm to calculate the position and posture of the onboard camera and generate a sparse point cloud of the scene;

[0020] 1.2. Using the original image, camera pose and sparse point cloud as input, a multi-view stereo vision algorithm is used to generate a dense point cloud of the scene, which is used as the original point cloud data of the target scene.

[0021] Furthermore, the implementation method of step 2 includes the following steps:

[0022] 2.1. Construct a fast semantic segmentation method for urban scenes that incorporates geometric metric information;

[0023] 2.2. Use the urban scene fast semantic segmentation method that incorporates geometric metric information to classify the original point cloud data and obtain the point cloud data corresponding to each building plane in the target scene;

[0024] 2.3. Missing wall completion: First, orthogonally project the point cloud of the building onto the ground plane to generate a two-dimensional grid map; generate a height map based on the height values ​​of the projection points in the grid; filter and smooth the height map, and then perform line element detection on the height map; use the detected line elements to fit the missing facades, and add the completed planes to the candidate planes.

[0025] Furthermore, the fast semantic segmentation method of urban scenes incorporating geometric metric information is specifically described as follows:

[0026] Common building features in urban scenes are combined with similar features of point clouds. Common building features include straight lines and planes that constitute the building outline. The geometric prior of the building's two-dimensional footprint is introduced to convert the unordered point cloud data into a geometric representation, so as to transform the point cloud semantic segmentation into a geometric two-class labeling problem. The energy function is used to constrain the classification quality and improve the accuracy and stability of the segmentation results.

[0027] Furthermore, the implementation method of step 3 comprises the following steps:

[0028] 3.1. Find the minimum and maximum values ​​of each point cloud in the x direction, and the minimum and maximum values ​​in the y direction, and use them as four vertices. The quadrilateral area surrounded by these four vertices is used as the bounding box.

[0029] 3.2. Divide the bounding box into multiple grids according to the set resolution, and establish the association index between the point cloud and the grid in the grid;

[0030] 3.3. For each grid, count the point cloud density and average height in the grid, where the point cloud density is the number of point clouds in the grid, and the average height is the average value of the Z coordinates of the point clouds in the grid;

[0031] 3.4. Select any network as the starting grid. Starting from the starting grid, search for the cutting line from the outer layer based on the principle of optimal cutting cost. In each search process, select the path with the minimum cost for expansion. The cost function is defined as:

[0032] ;

[0033] ;

[0034] in, represents the cost value, is the distance cost constant passing through a single grid, Represents the sum of the clipping costs from the starting point to the current point, Represents the sum of the clipping costs from the current point to the end point, is the weight of point cloud density in cost calculation, is the weight of the average height of the point cloud in the cost calculation, Representation Grid The density of Representation Grid The average height of represents the maximum value of the point cloud density in all grids, Indicates the maximum value of the average height of the point cloud in all grids;

[0035] 3.5. Crop the point cloud according to the cropping line searched out in step 3.4 to obtain multiple sets of point cloud data representing a single building;

[0036] 3.6. Setting the Angle Threshold 𝜑 and support point threshold n , according to the angle threshold 𝜑 and support point threshold n Screen the initial planes in a single building, when the angles between the plane and the three main directions are greater than the angle threshold 𝜑 Or the number of points on the plane is less than the support point threshold n When , the plane is regarded as an invalid plane and discarded; finally, multiple valid planes that can represent a single building are obtained.

[0037] Furthermore, the implementation method of step 4.1 comprises the following steps:

[0038] a1. Determine the interface and fill the point cloud:

[0039] Randomly select three points on the obtained intersection line of the roof and the wall , , And determine the minimum value of the point cloud X coordinate direction on the intersection line and maximum value , the minimum value in the Y coordinate direction of the point cloud and maximum value ;

[0040] a2. Assume that the plane equation of the interface between the roof and the wall is:

[0041] ;

[0042] in, A, B, C, D All represent constant coefficients, x, y, z is the coordinate axis;

[0043] a3. Three points randomly selected on the intersection line , , Substituting the plane equation into the interface equation, we get:

[0044] ;

[0045] in, Indicates generating a three-dimensional point and assigning its coordinates to ;

[0046] a4. Fill the point cloud on the interface according to the horizontal and vertical coordinate boundaries determined by the plane equation to generate the point cloud required for the top of the wall and the bottom of the roof; continuously generate three-dimensional points by looping through the three-dimensional plane space to form a plane point set, and finally cut the generated plane point cloud with the intersection line as the boundary to obtain the interface point cloud composed of multiple planes to complete the bottom of the roof and the top of the wall.

[0047] Furthermore, the implementation method of step 4.2 comprises the following steps:

[0048] Model the building walls using the Manhattan model:

[0049] Extract point clouds in three main directions that satisfy the Manhattan hypothesis from the building wall after completing the top;

[0050] The RANSAC algorithm is used to perform plane fitting on the extracted point cloud, and the fitted plane is divided into cube frames of several sizes;

[0051] A set of planes aligned with three axes are obtained from the candidate planes, and the point cloud space is divided into a set of axis-aligned cubic boxes by using the planes, and the optimal cubic boxes are selected as a subset to represent the three-dimensional model of the building; the selection conditions of the optimal cubic boxes are: first, the face of the constructed model is close to the input point cloud, and second, the selected cubic boxes should meet the compactness requirements set after the combination;

[0052] Model the roof using the intersecting plane selection method:

[0053] b1. For the multiple planes in the interface point cloud obtained in step 4.1, first calculate the angle set {a1, a2, ..., an} between the two planes;

[0054] b2. Select a pair of planes with the smallest angle and determine whether they meet the merging condition. If they meet the merging condition, the principal component analysis technology is used to merge the two planes into a new plane; the merging condition is that the plane angle is less than the preset minimum angle threshold a. min , and the number of support points distributed on each plane is greater than the set minimum number threshold; iterate the process until there are no planes that can be merged;

[0055] b3. Using the bounding box of the building roof point cloud, clip the plane obtained in step b2, and only keep the part within the bounding box; calculate the intersection points between the clipped planes to obtain a series of candidate intersecting planes, and select the closed space formed by the best set of intersecting planes from the candidate intersecting planes as the model representing the roof;

[0056] b4. Connect the building wall model and roof model to obtain a complete single building model.

[0057] The building model construction method of the present invention takes the original point cloud data of the building in the target scene as input, classifies according to the geometric features of the physical building, and obtains point cloud data representing each plane of the building. The group building scene is complex, and the building facade point cloud is missing due to occlusion. For the building point cloud with incomplete facade, the intersection line of the height map is used to determine the missing plane from the adjacent plane. A candidate plane set is constructed with the plane composed of the completed plane and the converged point cloud data. According to the ownership relationship of a single building, the candidate plane set is classified to obtain multiple groups of point cloud data representing a single building, and the building is modeled according to the point cloud data of the single building. In the modeling process, in view of the problem that the existing Manhattan model does not accurately express the sloping roof building, the point cloud representing the single building is first divided into two parts, the roof and the wall, and the top of the wall and the bottom of the roof are completed, and then the divide-and-conquer strategy is adopted, that is, the wall modeling of the completed top and the roof modeling of the completed bottom are respectively modeled, wherein the building wall modeling is implemented by the Manhattan model, and the roof modeling is implemented by the intersection plane selection method. In this way, the accuracy of the modeling expression of the sloping roof building is improved. Finally, the building wall model and roof model are spliced ​​together to obtain a model of a single building with a sloping roof.

[0058] Compared with the prior art, the building model construction method of the present invention improves the expression precision and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a flowchart of the building model construction based on the dense matching point cloud of the drone low-altitude image in the embodiment;

[0060] Figure 2 This is the missing wall completion process in the embodiment. DETAILED DESCRIPTION

[0061] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0062] like Figure 1 As shown, the present embodiment provides a method for constructing a building model based on dense matching point clouds of drone low-altitude images, comprising the following steps:

[0063] Step 1: Obtain and process the unordered low-altitude drone image of the target scene to obtain the original point cloud data of the target scene. The implementation method includes:

[0064] 1.1. Obtain unordered low-altitude drone images of the target scene. Using the unordered low-altitude drone images as input, use the Structure from Motion (SFM) algorithm to calculate the pose of the airborne camera and generate a sparse point cloud of the scene.

[0065] 1.2. Taking the original image, camera pose and sparse point cloud as input, the Multi-View Stereo (MVS) algorithm is used to generate a dense matching point cloud of the scene, which is used as the original point cloud data of the target scene.

[0066] During implementation, the drone is equipped with a depth camera and a monocular camera with a gimbal. The depth camera is Realsened453i, which is installed in front of the drone with the view facing forward. The monocular camera and the camera gimbal are installed at the bottom of the drone. The drone positioning uses the open source VIO framework VINS-Fusion. The computing platform transmits the visual odometer obtained by VINS-Fusion to the flight control firmware Pixhawk4 via serial communication; the gimbal motor communicates with the computing platform through the serial port, and the computing platform obtains information such as the motor's speed and angle, and uses PID control to servo control the gimbal motor.

[0067] Step 2: Classify the original point cloud data according to the geometric features of the physical building to obtain the point cloud data corresponding to each building plane in the target scene, and complete the missing building facades; gather the classified and completed data to construct a candidate plane set; each plane in the set is outlined by a three-dimensional discrete point cloud to represent the plane structure of the building; its implementation method includes the following steps:

[0068] 2.1. In order to efficiently and accurately process point cloud data to distinguish semantic categories, this embodiment constructs a fast semantic segmentation method for urban scenes that incorporates geometric metric information. The method is described in detail as follows:

[0069] The common building features in urban scenes are combined with similar features of point clouds. Common building features include straight lines and planes that constitute the building outline. The geometric prior of the building's two-dimensional footprint is introduced to convert the unordered point cloud data into geometric representations such as straight line distance and plane information, so as to convert the point cloud semantic segmentation into a geometric two-class labeling problem. The energy function is used to constrain the classification quality and improve the classification stability.

[0070] 2.2. Use the urban scene fast semantic segmentation method that incorporates geometric metric information to classify the original point cloud data and obtain the point cloud data corresponding to each building plane in the target scene.

[0071] 2.3. Due to the overhead shooting of drones and the occlusion between buildings, the building facade point cloud may have holes or even missing entire planes, which will affect the construction of the building model. This embodiment uses the plane fitting and reconstruction method based on point cloud data to complete the missing planes. Figure 2 As shown:

[0072] The point cloud data of the building is orthogonally projected onto the ground plane to generate a two-dimensional grid map; a height map is generated according to the height values ​​corresponding to the projection points in the grid map; the generated height map is filtered and smoothed, and line element detection is performed on the height map after filtering and smoothing; the missing plane is fitted using the detected line elements to obtain the missing plane point cloud.

[0073] Step 3: Classify the point cloud data in the candidate plane set according to the ownership of the individual building to obtain multiple sets of point cloud data representing individual buildings. This embodiment uses the A* algorithm to search and crop for classification based on the integrity of the building. The specific description is:

[0074] 3.1. Find the minimum value X_min and maximum value X_max in the x direction, the minimum value Y_min and maximum value Y_max in the y direction of each point cloud, and use them as four vertices. The four vertices are (X_min, Y_min), (X_max, Y_max), (X_min, Y_max), and (X_max, Y_min). The quadrilateral area enclosed by these four vertices is used as the bounding box.

[0075] 3.2. Divide the bounding box into multiple grids according to the set resolution, and establish the association index between the point cloud and the grid within the grid.

[0076] 3.3. For each grid, count the point cloud density and average height in the grid, where the point cloud density is the number of point clouds in the grid, and the average height is the average value of the point clouds in the grid at the Z coordinate.

[0077] 3.4. Select any network as the starting grid. Starting from the starting grid, search for the cutting line from the outer layer based on the principle of optimal cutting cost. In each search process, select the path with the minimum cost for expansion. The cost function is defined as:

[0078] ;

[0079] ;

[0080] in, represents the cost value, is the distance cost constant passing through a single grid, Represents the sum of the clipping costs from the starting point to the current point, Represents the sum of the clipping costs from the current point to the end point, is the weight of point cloud density in cost calculation, is the weight of the average height of the point cloud in the cost calculation, Representation Grid The density of Representation Grid The average height of represents the maximum value of the point cloud density in all grids, Indicates the maximum average height of the point cloud in all grids.

[0081] 3.5. The point cloud is cropped according to the cropping line searched in step 3.4. Each cropped block is represented as point cloud data of a building.

[0082] 3.6. Further refine the planes of the candidate plane set. The specific steps are as follows:

[0083] Step 1: Set a score value s for each plane, where s is the number of support points on the plane;

[0084] Step2: Set an angle threshold 𝜃 and a distance threshold d;

[0085] Step 3: Sort by score s, starting from the pair of planes with the lowest score. If the angle between the two planes is less than 𝜃 and the distance between the centroids of the two planes is less than d, merge the two planes through least squares fitting to form a new plane.

[0086] Step 4: Repeat the above steps until there are no planes to merge.

[0087] Step 4: Based on the point cloud data of individual buildings obtained in step 3 and the refined candidate plane set, model all individual buildings as follows:

[0088] Step 4.1: Use the semantic dual classification network based on PointNet to segment each set of point cloud data representing a single building into roof and wall, and obtain the intersection line between the two. Perform geometric calculations based on the intersection line between the roof and the wall to determine the interface point cloud to complete the top of the wall and the bottom of the roof to form two completely independent closed spaces. The implementation process is as follows:

[0089] a1. Determine the interface and fill the point cloud:

[0090] Randomly select three points on the obtained intersection line of the roof and the wall , , And determine the minimum value of the point cloud X coordinate direction on the intersection line and maximum value , the minimum value in the Y coordinate direction of the point cloud and maximum value ;

[0091] a2. Assume that the plane equation of the interface between the roof and the wall is: ;

[0092] a3. Three points randomly selected on the intersection line , , Substituting the plane equation into the interface equation, we get:

[0093] ;

[0094] in, Indicates generating a three-dimensional point and assigning its coordinates to ;

[0095] a4. Fill the point cloud on the interface according to the horizontal and vertical coordinate boundaries to generate the point cloud required for the top of the wall and the bottom of the roof; continuously generate three-dimensional points by looping through the three-dimensional plane space to form a plane point set, and finally cut the generated plane point cloud with the intersection line as the boundary to obtain the interface point cloud composed of multiple planes to complete the bottom of the roof and the top of the wall.

[0096] Step 4.2: Use the Manhattan model to complete the wall modeling based on the wall after the top is completed, and use the intersecting plane selection method to complete the roof modeling based on the roof after the bottom is completed; splice the wall model and the roof model together to obtain a single building model.

[0097] The method of modeling a building wall using the Manhattan model in this embodiment includes:

[0098] Extract point clouds in three main directions that satisfy the Manhattan hypothesis from the building wall after completing the top;

[0099] The RANSAC algorithm is used to perform plane fitting on the extracted point cloud, and the fitted plane is divided into cube frames of several sizes;

[0100] The selection of cube boxes mainly considers two factors: the face of the constructed model should be as close to the input point cloud as possible, and the combination of the selected boxes should be as compact as possible, and the holes and protrusions on the model surface should be minimized. The selection process of the box is regarded as a function optimization process of data fitting and model compactness, and the building model is represented by the optimal subset of the cube boxes.

[0101] The method for modeling a roof using the intersecting plane selection method in this embodiment includes:

[0102] b1. For the multiple planes in the interface point cloud obtained in step 4.1, first calculate the angle set {a1, a2, ..., an} between the two planes;

[0103] b2. Select a pair of planes with the smallest angle and determine whether they meet the merging condition. If they meet the merging condition, use the principal component analysis (PCA) technique to merge the two planes into a new plane; the merging condition is that the plane angle is less than the preset minimum angle threshold a. min And the number of support points distributed on each plane is greater than the set minimum number threshold; iterate the process until there are no planes that can be merged.

[0104] b3. Using the bounding box of the building roof point cloud, the plane obtained in step b2 is clipped, and only the part within the bounding box is retained; the intersection points between the clipped planes are calculated to obtain a series of candidate intersecting planes. There are still invalid planes introduced by noise in the candidate intersecting planes, so it is necessary to select the candidate planes and use the optimal plane subset to represent the building model. Define an objective function, and obtain the final optimal plane subset by optimizing the objective function. The objective function consists of two energy terms: data fitting term and model complexity term: data fitting aims to evaluate the quality of the plane fitting to the point cloud; incomplete point clouds may cause gaps in the final model, and noise and outliers will also introduce gaps and protrusions in the reconstructed model. In order to avoid these defects, a model complexity energy term is introduced to fit simple structures. Using the above energy terms, under certain constraints, the optimal subset of candidate faces is selected by minimizing the weighted sum of energy terms, so that the final model is closed and smooth.

[0105] It should be noted that when processing intersecting planes, the association information between the faces and the edges is retained, so that each edge of the intersecting planes has only two situations: one is connecting two adjacent candidate faces; the other is representing a boundary. In this embodiment, the plane intersection is divided into three situations. The first is that two planes intersect to form four parts. The second is that the two planes are coplanar after being connected by adjacent edges. This situation can be directly merged into one plane. The third intersection is the intersection of any boundary of the two planes to form an angle. Only this intersection situation will form a roof.

[0106] Based on the intersecting planes, a set of planes aligned parallel to the X, Y, and Z coordinate axes of the three-dimensional space are generated, and these planes are used to divide the three-dimensional space containing the point cloud into a set of axis-aligned cubic boxes. Assuming that the number of planes in the three directions of X, Y, and Z in the three-dimensional space is 𝑁𝑥, 𝑁𝑦, and 𝑁𝑧 respectively, the total number of cubic boxes is N = (𝑁𝑥 − 1) × (𝑁𝑦 −1) × (𝑁𝑧 − 1).

[0107] b4. Select the optimal subset from the cube boxes generated in step b3 to represent the 3D model of the building. The selection process is expressed as a labeling problem, and the goal is to select boxes that can both fit the point cloud data with high fidelity and maintain the compactness of the model to construct the final building model.

[0108] When selecting a cube frame, two key conditions should be met: first, the generated model surface should be as close as possible to the input original point cloud data, so as to restore the shape of the scanned object to the greatest extent; second, the selected cube frame should be as compact as possible to prevent holes and protrusions on the model surface and ensure the integrity and smoothness of the model. In view of this, this embodiment sets the selection process of the cube frame as a function optimization problem centered on data fitting and model compactness, that is, by optimizing the corresponding objective function, these two key factors are balanced, so as to accurately select a cube frame that meets the requirements.

[0109] In terms of data fitting, the fitting score F(𝑏 𝑖 ) is used to measure the support of the point cloud for the candidate box 𝑏𝑖, and the fitting score F(𝑏 𝑖 ) is calculated based on the consistency of the normal vector direction and the distance weight between the frame surface f and its support point set, and is defined as:

[0110] ;

[0111] ;

[0112] in n f Denotes the cubic box 𝑏 𝑖 The normal of the mid-surface f, n j Represents the normal of the support point. Calculate the normal of the surface f relative to its support point set. The fitting quality of the support point set The distance between the point projected on the surface f and the surface f is less than d min composition.

[0113] Fit score F(𝑏 𝑖 )The result of the calculation may be a positive value, a negative value, a zero value, or a value close to zero. A positive value indicates that the cube box is located within the building point cloud space and is a box that should be selected; a negative value indicates that the box is located outside the building point cloud space and does not contribute to model fitting and needs to be removed; both zero and near-zero values ​​indicate that the cube box is a blank box, which does not contribute to model fitting and should also be removed. Since the cube boxes on the surfaces of some buildings contain fewer point clouds, they are easily misidentified as blank boxes and removed, resulting in holes in the final model. In order to solve this problem, this embodiment smoothes the fitting scores of all blank boxes, and uses the weighted value of the fitting scores of the blank box's adjacent cube boxes as the fitting score F of the blank box. The specific calculation formula is as follows:

[0114] ;

[0115] ;

[0116] in Cb i is with b i Directly adjacent cube boxes, F ( b j ) is the box obtained by the previous calculation b j The fitting score, It is based on the contact area of ​​the two cube frames A(f ij ) The distance between the geometric center d ij Defined weights.

[0117] In terms of model compactness, in order to avoid holes or protrusions caused by incorrect frame marking, this embodiment introduces a compactness index of adjacent frames. The compactness is defined inversely proportional to the minimum value of the thickness of adjacent frames. When the thickness of adjacent frames is large (for example, greater than 1 meter), the highest compactness value is directly assigned.

[0118] The box labeling is optimized by optimizing an energy function that includes a data term and a smoothing term. The data term selects boxes with higher fitting scores, while the smoothing term ensures the consistency of adjacent box labels to avoid holes and protrusions. Energy function The form is:

[0119] ;

[0120] ;

[0121] ;

[0122] in, Db i Represents a data item, S ( b i , b j ) represents the smoothing term, is the balance coefficient.

[0123] Finally, the energy function is minimized by the graph segmentation algorithm to generate labels for all cube boxes. The candidate box set marked as positive boxes is used to represent the building model of the input point cloud, thereby achieving high-fidelity fitting and compactness optimization of the building 3D model.

[0124] b5. Splice the building wall model and the roof model together to obtain a complete single building model. The splicing process is the same as step 5 and will not be repeated here.

[0125] Step 5: All individual building models are spliced ​​together to form a global model to obtain a model of the building complex.

[0126] The data format for building a building model consists of two parts: vertices and faces. Vertices are marked with the letter "V", and the following three numbers correspond to the x, y, and z coordinates of the vertex in three-dimensional space. All vertices are numbered from top to bottom, starting from 1. In the data array, each line starting with "V" represents a vertex, and the written form is ,in i represents the vertex number, ( x,y,z ) is its three-dimensional coordinate. The face is represented by the letter f, followed by the number of the vertex. Connecting the vertices corresponding to these numbers from left to right will form a plane. In the data array, each row starting with "f" represents a plane, and the presentation format is ,in," j " is the face number, and The model obtained in this embodiment consists of 8 vertices and 6 planes.

[0127] In the target scene obtained in this embodiment, there are a total of k models ,in have Vertices , n i Planes In the data merging process, all vertices are placed in front and all planes are placed in the back. Except for the vertex number corresponding to the first model, the vertex numbers of other models will increase. i Model For example, the numbers of all vertices are increased based on the original numbers. , then its corresponding j The plane becomes .

[0128] The vertices of all block models are renumbered, and then the vertex numbers corresponding to the planes are adjusted according to the aforementioned method to obtain the fused global data, that is, the complete model of the building group scene.

[0129] It should be noted that, in addition to this method, other splicing methods in the prior art may also be used for model splicing.

Claims

1. A method for constructing a building model based on dense matching point cloud of UAV low-altitude images, characterized in that: The following steps are involved: Step 1: Obtain and process the unordered low-altitude drone images of the target scene to obtain the original point cloud data of the target scene; Step 2: According to the geometric features of the physical building, the original point cloud data is classified to obtain the point cloud data corresponding to each building plane in the target scene; Based on these data, the plane fitting method is used to fit the plane of the building, and then a candidate plane set is constructed; at the same time, the plane corresponding to the wall is completed by using the method of determining the blocked facade with the help of adjacent walls, and the completed plane is added to the candidate plane set; Step 3: According to the ownership relationship of a single building, the candidate plane set is classified to obtain multiple sets of point cloud data representing a single building, and a large number of initial planes in the candidate plane set are processed to obtain the effective plane of a single building; Step 4: Based on the point cloud data of individual buildings obtained in step 3, model each individual building as follows: Step 4.1, use the semantic dual classification network based on PointNet to segment each group of point cloud data representing a single building into roof and wall, and obtain the intersection line between the two; perform geometric calculations based on the intersection line between the roof and the wall to determine the interface point cloud to complete the top of the wall and the bottom of the roof to form two completely independent closed spaces; Step 4.2: Based on the wall after the top is completed, the Manhattan model is used to model the wall; based on the roof after the bottom is completed, the intersecting plane selection method is used to model the roof; the wall model and the roof model are spliced ​​together to obtain a single building model; Step 5: Put all the individual building models together to get the model of the building complex.

2. The method according to claim 1, characterized in that The implementation method of step 1 comprises the following steps: 1.

1. Obtain unordered low-altitude drone images of the target scene. Using the unordered low-altitude drone images as input, use the motion recovery structure algorithm to calculate the position and posture of the onboard camera and generate a sparse point cloud of the scene; 1.

2. Using the original image, camera pose and sparse point cloud as input, a multi-view stereo vision algorithm is used to generate a dense point cloud of the scene, which is used as the original point cloud data of the target scene.

3. The method according to claim 1, characterized in that The implementation method of step 2 comprises the following steps: 2.

1. Construct a fast semantic segmentation method for urban scenes that incorporates geometric metric information; 2.

2. Use the urban scene fast semantic segmentation method that incorporates geometric metric information to classify the original point cloud data and obtain the point cloud data corresponding to each building plane in the target scene; 2.

3. Missing wall completion: First, orthogonally project the point cloud of the building onto the ground plane to generate a two-dimensional grid map; generate a height map based on the height values ​​of the projection points in the grid; filter and smooth the height map, and then perform line element detection on the height map; use the detected line elements to fit the missing facades, and add the completed planes to the candidate planes.

4. The method according to claim 3, characterized in that The method for constructing a fast semantic segmentation method for urban scenes that incorporates geometric metric information is specifically described as follows: Common building features in urban scenes are combined with similar features of point clouds. Common building features include straight lines and planes that constitute the building outline. The geometric prior of the building's two-dimensional footprint is introduced to convert the unordered point cloud data into a geometric representation, so as to transform the point cloud semantic segmentation into a geometric two-class labeling problem. The energy function is used to constrain the classification quality and improve the accuracy and stability of the segmentation results.

5. The method according to claim 1, characterized in that The implementation method of step 3 comprises the following steps: 3.

1. Find the minimum and maximum values ​​of each point cloud in the x direction, and the minimum and maximum values ​​in the y direction, and use them as four vertices. The quadrilateral area surrounded by these four vertices is used as the bounding box. 3.

2. Divide the bounding box into multiple grids according to the set resolution, and establish the association index between the point cloud and the grid in the grid; 3.

3. For each grid, count the point cloud density and average height in the grid, where the point cloud density is the number of point clouds in the grid, and the average height is the average value of the Z coordinates of the point clouds in the grid; 3.

4. Select any network as the starting grid. Starting from the starting grid, search for the cutting line from the outer layer based on the principle of optimal cutting cost. In each search process, select the path with the minimum cost for expansion. The cost function is defined as: Among them, f(i, j) represents the cost value, c dis is the distance cost constant passing through a single grid, Represents the sum of the clipping costs from the starting point to the current point, represents the sum of the clipping costs from the current point to the end point, ω1 is the weight of the point cloud density in the cost calculation, ω2 is the weight of the average height of the point cloud in the cost calculation, is the weight of the average height of the point cloud in the cost calculation, d(i, j) represents the density of the grid (i, j), h(i, j) represents the average height of the grid (i, j), d max Indicates the maximum value of the point cloud density in all grids, h max Indicates the maximum value of the average height of the point cloud in all grids; 3.

5. Crop the point cloud according to the cropping line searched out in step 3.4 to obtain multiple sets of point cloud data representing a single building; 3.

6. Setting the Angle Threshold and support point threshold, according to the angle threshold The initial planes in a single building are screened by the support point threshold. When the angles between the plane and the three main directions are greater than the angle threshold Or when the number of points on the plane is less than the support point threshold, the plane is regarded as an invalid plane and is discarded; ultimately, multiple valid planes that can represent a single building are obtained.

6. The method according to claim 1, characterized in that The implementation method of step 4.1 comprises the following steps: a1. Determine the interface and fill the point cloud: Randomly select three points (x1, y1, z1), (x2, y2, z2), (x3, y3, z3) on the obtained roof and wall intersection line and determine the minimum value x in the X coordinate direction of the point cloud on the intersection line min and the maximum value x max , the minimum value y in the Y coordinate direction of the point cloud min and the maximum value y max ; a2. Assume that the plane equation of the interface between the roof and the wall is: Ax+By+Cz+D=0; Among them, A, B, C, and D all represent constant coefficients, and x, y, and z are coordinate axes; a3. Three points (x1, y1, z1), (x2, y2, z2), and (x3, y3, z3) randomly selected on the intersection line are substituted into the plane equation to obtain the interface equation: Among them, p(x i ,y i ,z i ) means generating a three-dimensional point and assigning its coordinates to x i ,y i ,-(D+Ax i +By i ) / C; a4. Fill the point cloud on the interface according to the horizontal and vertical coordinate boundaries determined by the plane equation to generate the point cloud required for the top of the wall and the bottom of the roof; traverse the three-dimensional plane space in a loop to continuously generate three-dimensional points to form a plane point set, and finally cut the generated plane point cloud with the intersection line as the boundary to obtain the interface point cloud composed of multiple planes to complete the bottom of the roof and the top of the wall; Model the building walls using the Manhattan model: Extract point clouds in three main directions that satisfy the Manhattan hypothesis from the building wall after completing the top; The RANSAC algorithm is used to fit the extracted point cloud into a plane, and the fitted plane is divided into cube frames of several sizes; A set of planes aligned with three axes are obtained from the candidate planes, and the point cloud space is divided into a set of axis-aligned cubic boxes by using the planes, and the optimal cubic boxes are selected as a subset to represent the three-dimensional model of the building; the selection conditions of the optimal cubic boxes are: first, the face of the constructed model is close to the input point cloud, and second, the selected cubic boxes should meet the compactness requirements set after the combination; Model the roof using the intersecting plane selection method: b1. For the multiple planes in the interface point cloud obtained in step 4.1, first calculate the angle set {a1, a2, ..., an} between the two planes; b2. Select a pair of planes with the smallest angle and determine whether they meet the merging condition. If they meet the merging condition, the principal component analysis technology is used to merge the two planes into a new plane; the merging condition is that the plane angle is less than the preset minimum angle threshold a. min , and the number of support points distributed on each plane is greater than the set minimum number threshold; iterate the process until there are no planes that can be merged; b3. Using the bounding box of the building roof point cloud, clip the plane obtained in step b2, and only keep the part within the bounding box; calculate the intersection points between the clipped planes to obtain a series of candidate intersecting planes, and select the closed space formed by the best set of intersecting planes from the candidate intersecting planes as the model representing the roof; b4. Connect the building wall model and roof model together to obtain a complete single building model.

Images