A method for automatically extracting building object frame models based on graph cuts

Abstract

The present invention discloses a method for automatically extracting a building object frame model based on graph cuts. Firstly, a plane is extracted from a point cloud, and light information of densely matched point clouds is used as prior information. Building reconstruction is converted into a graph cut problem for solution, thereby avoiding the huge amount of calculation caused by dynamic programming solution. The problem is then merged and regularized for optimization. Then, the outer bounding box of the point cloud is cut with a plane to obtain candidate polyhedrons. Finally, a graph cut model is constructed using visibility information to perform binary marking on the polyhedrons, thereby distinguishing the polyhedrons inside and outside the building, thereby realizing the extraction of the building body frame model.

Description

Technical Field

[0001] The present invention relates to the fields of three-dimensional digitization technology, point cloud processing technology and surveying and mapping technology, and in particular to a method for automatically extracting a building object frame model based on graph cutting. Background Art

[0002] Three-dimensional building models are widely used in urban planning, three-dimensional navigation, disaster management, water conservancy and hydropower, etc.

[0003] At present, with the development of technologies such as low-altitude oblique photogrammetry and drone photogrammetry, the accuracy of geographic scene model data is getting higher and higher. However, the amount of data in urban-level geographic scene models is extremely large, reaching TB level at a scale of 5 cm, which is not conducive to subsequent analysis and application. In addition, the geographic scene has no monomer attributes in the form of a three-dimensional grid model, and it is impossible to attach attributes and perform subsequent GIS quantitative analysis. On the display side, the three-dimensional grid model has too many useless details, which is not conducive to storage and rendering, so it needs to be lightweight. If the manual collection method is used to collect building monomers, the efficiency is too low; and automated local error simplification methods such as QEM (quadric error metric) and Lindstorm can only simplify the model structure, but cannot guarantee the global regularization rules, which will greatly affect the visualization effect.

[0004] In summary, as an important part of urban real-life 3D information products, 3D building models are widely used in urban planning, 3D navigation, disaster management, water conservancy and hydropower, etc. However, if the building monomers are collected manually, the efficiency is too low; and the traditional automatic local error simplification method can only simplify the model structure, but cannot guarantee the global regularization rules, which will greatly affect the visualization effect. Summary of the invention

[0005] The purpose of the present invention is to provide a method for automatically extracting a building object frame model based on graph cutting, which solves the above technical problems pointed out in the prior art.

[0006] The present invention provides a method for automatically extracting a building object frame model based on graph cutting, comprising the following steps:

[0007] Plane extraction: collecting and acquiring the three-dimensional point cloud data of the current building; performing a plane extraction operation based on the three-dimensional point cloud data to obtain multiple planes corresponding to the building;

[0008] Plane merging and regularization: All the planes are optimized and merged based on global regularization to obtain multiple new planes;

[0009] Polyhedron cutting: constructing a polyhedron based on all the new planes; cutting the polyhedron to obtain multiple candidate planes;

[0010] Polyhedron graph cut binary labeling based on visibility information: construct multiple second polyhedrons based on each of the candidate planes; perform binary labeling on the second polyhedrons through visibility analysis to obtain a polyhedron inside the building surface and a polyhedron outside the building surface;

[0011] Building surface model extraction: The building model is obtained by merging the internal polyhedrons S of the building surface.

[0012] Preferably, the step of optimizing and merging all the planes based on global regularization to obtain multiple new planes comprises the following steps:

[0013] Based on all the planes, multiple plane pairs (s i ,s j ); Calculate and obtain the plane pair (s i ,s j )’s included angle θ;

[0014] Get each of the plane pairs (s i ,s j ) The number of support points N of each plane;

[0015] Traverse each of the plane pairs (s i ,s j ), determine the plane pair (s i ,s j ) is less than or equal to the angle threshold θ t And the plane pair (s i ,s j ) is the number of support points N of each plane greater than or equal to the point threshold N t If so, then the plane pair (s i ,s j ) are merged to obtain a new plane; if not, the plane pair (s i ,s j ) are taken as new planes.

[0016] Preferably, the plane pair (s i ,s j ) represents two adjacent planes.

[0017] Preferably, the cutting of the polyhedron to obtain a plurality of candidate planes comprises the following steps:

[0018] Calculate and obtain the bounding box corresponding to all the current three-dimensional point cloud data;

[0019] Extracting the coverage of the bounding box to obtain the coverage corresponding to each new plane;

[0020] After arranging the respective coverage surfaces in descending order according to their area sizes, the respective coverage surfaces are traversed in order from large to small to determine whether the polyhedron intersects with the coverage surface. If so, the polyhedron is segmented using the coverage surface to obtain a plurality of candidate planes.

[0021] Preferably, the binary marking of the second polyhedron by visibility analysis to obtain an internal polyhedron on the building surface and an external polyhedron on the building surface comprises the following steps:

[0022] Constructing an empty initial directed graph; mapping the second polyhedral primitive corresponding to the second polyhedron to the initial directed graph to obtain a vertex V, and mapping the common faces of every two adjacent polyhedrons to the initial directed graph to obtain an edge E, thereby obtaining a directed graph;

[0023] Add a source point source and a sink point sink to each vertex V in the directed graph; obtain the weight w of each edge E;

[0024] Based on the source point source, the sink point sink and the weight w, all the vertices V in the directed graph are processed by st-cut to obtain a subset S and a subset T; wherein the source point source belongs to the subset S; the sink point sink belongs to the subset T;

[0025] Based on the subset S, the internal polyhedron of the building surface is obtained, and based on the subset T, the external polyhedron of the building surface is obtained.

[0026] Preferably, after "respectively adding a source point and a sink point to each vertex V in the directed graph", the step further includes: connecting the vertex to the source point source and the sink point sink respectively.

[0027] Preferably, the subset S represents the vertices in the polyhedron inside the building surface;

[0028] The subset T represents the vertices in the building surface exterior polyhedron.

[0029] Preferably, the method of obtaining subset S and subset T by performing st-cut processing on all the vertices V in the directed graph based on the source point source, the sink point sink and the weight w includes: first obtaining subset S by distinguishing through global energy minimization, and then obtaining subset T consisting of all vertices except subset S.

[0030] Preferably, the obtaining of the subset S by differentiation through global energy minimization comprises the following steps:

[0031] Traverse each of the three-dimensional point cloud data: obtain the visibility item E of each of the three-dimensional point cloud data vis (S) and obtain the directional item E of each of the three-dimensional point cloud data orient (S);

[0032] Based on the visibility item E vis (S) and directional term E orient (S) Calculate and obtain the energy function E(S);

[0033] Determine whether the energy function E(S) is greater than 0; if so, determine that the three-dimensional point cloud data is the three-dimensional point cloud data of the external polyhedron of the building surface; and construct a subset S based on the three-dimensional point cloud data of all the external polyhedrons of the building surface.

[0034] Preferably, the energy function E(S) is calculated as follows:

[0035] E(S)=E vis (S)+E orient (S).

[0036] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0037] It can be seen from the analysis of the above-mentioned method for automatically extracting the building body frame model based on graph cuts provided by the present invention that, in specific applications, a plane is first extracted from the point cloud, and the light information of the densely matched point cloud is used as prior information to convert the building reconstruction into a graph cut problem to solve, thereby avoiding the huge amount of calculation caused by dynamic programming solution; and it is merged and regularized and optimized; then, the outer bounding box of the point cloud is cut with a plane to obtain a candidate polyhedron; finally, using visibility information, a graph cut model is constructed to perform binary marking on the polyhedron, and the polyhedrons inside and outside the building are distinguished, thereby realizing the extraction of the building body frame model. The experimental results verify that the algorithm of the present application can obtain better results on the noisy point cloud of the building. On the other hand, the present application realizes the GPU parallelization of the visibility weight calculation, thereby achieving the purpose of efficiently generating the building body frame model, making the method suitable for batch building model reconstruction at the city level; automatically generating lightweight single building models, which is conducive to storage, rendering and subsequent GIS quantitative analysis, greatly reducing manual workload. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A schematic diagram of the overall operation steps of a method for automatically extracting a building object frame model based on graph cutting provided by the present invention;

[0040] Figure 2 A schematic diagram of the original point cloud simulation of a method for automatically extracting a building object frame model based on graph cutting provided by the present invention;

[0041] Figure 3 A schematic diagram of a simulation result of a plane extraction method for automatically extracting a building object frame model based on graph cutting provided by the present invention;

[0042] Figure 4 A schematic diagram of the simulation result after plane merging of the automatic extraction method of the building object frame model based on graph cutting provided by the present invention;

[0043] Figure 5 The building reconstruction result of the automatic extraction method of building object frame model based on graph cutting provided by the present invention Figure 1 Simulation schematic diagram;

[0044] Figure 6 The building reconstruction result of the automatic extraction method of building object frame model based on graph cutting provided by the present invention Figure 2 Simulation schematic diagram;

[0045] Figure 7 A schematic diagram of the operation steps of obtaining multiple new planes in a method for automatically extracting a building object frame model based on graph cutting provided by the present invention;

[0046] Figure 8 A schematic diagram of the effect simulation of the plane regularization of the automatic extraction method of the building object frame model based on graph cuts provided by the present invention;

[0047] Fig. 9 The non-regularized rendering of the automatic extraction method of the building object frame model based on graph cut provided by the present invention;

[0048] Fig.10 A schematic diagram of the operation steps of obtaining multiple candidate planes in a method for automatically extracting a building object frame model based on graph cutting provided by the present invention;

[0049] Fig.11A schematic diagram of a bounding box and plane simulation of a method for automatically extracting a building object frame model based on graph cutting provided by the present invention;

[0050] Fig.12 A schematic diagram of plane sorting simulation of a method for automatically extracting a building object frame model based on graph cutting provided by the present invention;

[0051] Fig.13 A schematic diagram of a polyhedron simulation constructed by a method for automatically extracting a building object frame model based on graph cutting provided by the present invention;

[0052] Fig.14 A schematic diagram of the operation steps of obtaining an internal polyhedron and an external polyhedron on a building surface in a method for automatically extracting a building object frame model based on graph cutting provided by the present invention;

[0053] Fig.15 A schematic diagram of the operation steps of obtaining a subset S of a method for automatically extracting a building object frame model based on graph cuts provided by the present invention;

[0054] Fig.16 A schematic diagram of visibility calculation of a method for automatically extracting a building object frame model based on graph cuts provided by the present invention. DETAILED DESCRIPTION

[0055] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0057] like Figure 1 As shown, the present invention proposes a method for automatically extracting a building object frame model based on graph cutting, comprising the following steps:

[0058] Step S10: plane extraction: collecting and acquiring three-dimensional point cloud data of the current building; performing a plane extraction operation based on the three-dimensional point cloud data to obtain multiple planes corresponding to the building;

[0059] It should be noted that the embodiment of the present application shoots low-altitude oblique images and then generates a dense point cloud (i.e., the above-mentioned three-dimensional point cloud data, such as Figure 2As shown); the plane extraction method adopted in the embodiment of the present application is to optimize the discrete distribution of continuous plane parameters and input points (i.e. the above-mentioned three-dimensional point cloud data) by seeking high fidelity, high simplicity and high integrity from the initial configuration; through an exploration mechanism guided by a multi-objective energy function, and using five geometric operations to process the transformation in the solution space; the huge solution space in the exploration mechanism guided by the multi-objective energy function is processed by five geometric operators to create, delete and modify primitives; the multi-objective energy function is as follows:

[0060] U(x)=ω f U f (x)+ω s U s (x)+ω c U c (x);

[0061] Among them U f (x), U s (x) and U c (x) are fidelity, simplicity and completeness terms, respectively, and their ranges are all [0,1]; ω f ,ω s and ω c The corresponding weights are all positive and their sum is 1;

[0062] Through the above multi-objective energy function, these three objectives are balanced to find a high-quality plane primitive configuration. By minimizing this energy function, the algorithm can improve the initial configuration to achieve higher fidelity, simplicity, and completeness.

[0063] The above-mentioned plane extraction method and the above-mentioned multi-objective energy function are both prior arts and will not be described in detail in this application;

[0064] Since the point cloud obtained by dense matching has large noise, the plane extracted based on it inevitably has some errors and irregularities, which may cause errors in subsequent reconstruction. In order to reduce the impact of errors in plane extraction, the embodiment of the present application merges and regularizes the planes in the subsequent operation process to obtain a new plane (such as Figure 3 As shown), thus reducing the point cloud noise;

[0065] Step S20: plane merging and regularization: optimizing and merging all the planes based on global regularization to obtain multiple new planes;

[0066] It should be noted that due to the presence of noise and outliers, plane extraction may generate some unnecessary plane segments; these plane segments may cause narrow and long structures and small protrusions in the final model; in addition, they also lead to additional calculations and reduce efficiency. In order to solve this problem, the embodiment of the present application uses a plane iterative merging optimization method to effectively reduce point cloud noise and improve plane extraction accuracy (such as Figure 4 shown);

[0067] Step S30: polyhedron cutting: constructing a polyhedron based on all the new planes; cutting the polyhedron to obtain multiple candidate planes;

[0068] It should be noted that after the new planes are obtained by plane merging and regularization, a polyhedron of a spatially complex building will be constructed through these new planes. In order to further identify the building model, this polyhedron needs to be further processed to obtain the plane of each face of the polyhedron (i.e., the candidate planes mentioned above);

[0069] When the polyhedron is further processed to obtain candidate planes, the polyhedron primitives cut out by all supporting planes in the entire scene can be enumerated (polyhedron primitives are polyhedron objects, which are the components of the polyhedron and can be points, lines, and surfaces). However, the space complexity of this method is very high, which will reach Θ(n 3 ) level of computation;

[0070] In fact, each additional plane primitive (adding a new plane primitive) will split all current polyhedral units into their supporting planes (the supporting plane means that the polyhedron is composed of multiple planes. After the newly added plane intersects with the polyhedron, the newly added plane will become part of the new polyhedron); in addition, not considering the spatial range of the plane primitives may generate some meaningless polyhedral primitives. For example, a chimney on the roof may interfere with the segmentation of polyhedral primitives at the other end of the building; considering the computational complexity and memory cost, the embodiment of the present application uses a more refined segmentation method, and the specific operations are shown in steps S31-S33.

[0071] Step S40: Binary labeling of polyhedron graph cut based on visibility information: construct multiple second polyhedrons based on each of the candidate planes; perform binary labeling on the second polyhedrons through visibility analysis to obtain a polyhedron S inside the building surface and a polyhedron outside the building surface;

[0072] It should be noted that the three-dimensional point cloud data of the building surface is generated by photographing the building. In these three-dimensional point cloud data, the polyhedron outside the building is reflected based on the visibility of light as a constraint. The polyhedron outside the building is visible, while the polyhedron inside the building is not visible. Therefore, the polyhedron generated on the building surface is divided into two categories (i.e., the exterior of the building and the interior of the building);

[0073] Based on this, the embodiment of the present application transforms the building surface reconstruction problem into a binary classification problem of polyhedrons, performs visibility analysis on the polyhedrons, and marks each polyhedron as belonging to the inside or outside of the building surface, and then merges the internal polyhedrons to obtain a watertight building model without self-intersections;

[0074] The above-mentioned visibility information refers to the fact that in the generation stage of the dense point cloud (i.e., the above-mentioned three-dimensional point cloud data), each three-dimensional point cloud data can obtain image light information, i.e., visibility information; the embodiment of the present application uses this visibility to construct the visible item E vis (S), which increases the weight of the face traversed by the line of sight (the weight is the visibility term E vis (S) value); the principle is that if a point in the point cloud is on the surface of a building, then its corresponding line of sight, that is, the line formed by the point and the photographic center of its visible image, should be unobstructed; therefore, the weight of the edge can be assigned by the intersection of the line of sight with the polyhedron.

[0075] Step S50: Extracting the building surface model: merging the internal polyhedrons S on the building surface to obtain a building model.

[0076] It should be noted that the above embodiment of the present application adopts the maximum flow minimum cut algorithm to solve, and after obtaining the category to which each polyhedron belongs, the polyhedrons S inside the building surface are merged to obtain a watertight building model without self-intersection;

[0077] For example: The embodiments of the present application have been fully verified in actual experiments and applied in the automatic generation of building LOD models. Figure 5 and Figure 6 As shown, the present embodiment uses 4 dense point clouds of buildings of different complexity for testing, with point counts ranging from 200,000 to 1 million. The point clouds of the data set are all from dense matching of oblique images, so there are a large proportion of outliers and noise.

[0078] Figure 5 Reconstruction results for buildings Figure 1 and Figure 6 Reconstruction results for buildings Figure 2The buildings reconstructed by the method of the embodiment of the present application and the Polyfit algorithm are shown. Table 1 shows the time-consuming statistics of Polyfit, CPU and GPU for reconstructing buildings of different complexities. It can be seen that the method of the embodiment of the present application obtains the lightweight reconstruction models of these buildings well, and highlights their main plane features and retains clear edges. Polyfit is suitable for buildings with simple structures, but for buildings with complex structures, bottlenecks will be encountered in calculation. Compared with Polyfit, the algorithm of the embodiment of the present application can handle complex buildings, such as buildings (c) and (d), which have 74 and 91 faces respectively. Polyfit is more efficient on simple buildings (a) and (e), because at the order of magnitude of this candidate primitive, the advantage of graph cuts is not far beyond dynamic programming, and the time required for the embodiment of the present application to cut three-dimensional candidate polyhedra and calculate visibility is longer than the time required for Polyfit to calculate candidate faces. However, for more complex buildings (b), (f) and (g), the time required for Polyfit is more than 5 to 100 times that of the algorithm of the embodiment of the present application, while for (c), (d) and (h), it cannot be solved at all. On these buildings, the speed advantage of graph cuts is very obvious.

[0079] Table 1 Experimental data parameters and Polyfit, CPU and GPU time statistics (unit: seconds)

[0080]

[0081] Table 2 takes buildings (a)-(d) as an example, and counts the four main steps of plane processing, polyhedron construction, visibility calculation and graph cutting. It can be seen that visibility calculation is the most time-consuming step, accounting for two-thirds of the total time, while graph cutting itself takes very little time. It is worth noting that on a data set of millions of points, the maximum calculation time of the method of the embodiment of the present application does not exceed 15 minutes. The use of GPU parallel computing can increase the speed of visibility calculation by 5 to 20 times, and the improvement effect is more obvious as the complexity of the building increases.

[0082] Table 2 Time statistics of main steps (unit: seconds)

[0083]

[0084] Combining the above qualitative and quantitative experimental analysis, it can be concluded that the building reconstruction algorithm based on visibility and graph cuts proposed in the embodiment of the present application can reconstruct urban buildings with high accuracy and efficiency, especially for more complex buildings, with high efficiency;

[0085] In summary, the embodiment of the present application first extracts a plane from the point cloud, uses the light information of the densely matched point cloud as prior information, and converts the reconstruction of the building into a graph cut problem to solve, avoiding the huge amount of calculation caused by dynamic programming solution; and merges and regularizes it; then, the outer bounding box of the point cloud is cut with a plane to obtain a candidate polyhedron; finally, using visibility information, a graph cut model is constructed to binary mark the polyhedron, distinguishing the polyhedrons inside and outside the building, thereby realizing the extraction of the building frame model. The experimental results verify that the algorithm of the present application can obtain better results on the noisy point cloud of the building; on the other hand, the present application realizes the GPU parallelization of the visibility weight calculation, thereby achieving the purpose of efficiently generating the building frame model, making this method suitable for batch building model reconstruction at the city level; automatically generating lightweight single building models, which is conducive to storage, rendering and subsequent GIS quantitative analysis, greatly reducing manual workload.

[0086] Specifically, Figure 7 As shown, in step S20, all the planes are optimized and merged based on global regularization to obtain multiple new planes, including the following operation steps:

[0087] Step S21: Acquire multiple plane pairs (s i ,s j ); Calculate and obtain the plane pair (s i ,s j )’s included angle θ;

[0088] The plane pair (s i ,s j ) represents two adjacent planes; that is, the above s i With the above j Each represents a plane, and the plane s i With plane s j adjacent;

[0089] The above calculation obtains the plane pair (s i ,s j ) is obtained by calculating the angle between the normal vectors of the two planes, which will not be described in detail in this application;

[0090] Step S22: Obtain each of the plane pairs (s i ,s j ) The number of support points N of each plane;

[0091] Step S23: traverse each of the plane pairs (s i ,s j ), determine the plane pair (s i ,s j) is less than or equal to the angle threshold θ t And the plane pair (s i ,s j ) is the number of support points N of each plane greater than or equal to the point threshold N t ; If (that is, the above judgment conditions are met at the same time, that is, the plane pair (s i ,s j ) is less than or equal to the angle threshold θ t And satisfy the plane pair (s i ,s j ) the number of supporting points N of each plane is greater than or equal to the threshold value N t ), then the plane pair (s i ,s j ) are merged to obtain a new plane; if not, the plane pair (s i ,s j ) are taken as new planes (i.e., plane s i As a new plane, s j as a new plane);

[0092] Repeat the above operation until all the plane pairs (s i ,s j ), and obtain multiple new planes;

[0093] It should be noted that the above embodiment of the present application is to first calculate each pair of planes (s i ,s j )(Each pair of planes (s i ,s j ) is the above plane pair (s i ,s j )) of the angle θ; then, starting from the plane pair with the smallest angle, traverse each plane pair (s i ,s j ), test whether the following two conditions are met: (1) The plane pair (s i ,s j ) the angle θ between the two planes is lower than the threshold θ t (That is, the above plane pair (s i ,s j ) is less than or equal to the angle threshold θ t ); (2) exceeds the specified threshold N t The points are on two planes (i.e. the above plane pair (s i ,s j ) the number of supporting points N of each plane is greater than or equal to the threshold value N t );

[0094] If both conditions are met, merge the two planes to get a new plane (merge the point clouds on the two planes into the same plane, that is, the plane pair (s i ,s j ) to obtain a new plane), and use its corresponding points to generate a new plane; repeat this process until there is no plane pair that can be merged;

[0095] The experimental selection angle threshold θ of the embodiment of the present application t is 10 degrees, the point threshold is N t =min(|s i |,|s j |) / 5, where |s i | represents plane s i The number of support points (the number of support points refers to the number of points contained in the plane);

[0096] like Figure 3 He Ru Figure 4 The difference between the planes before and after merging is shown. The number of planes also drops from 46 to 15.

[0097] The embodiment of the present application performs global regularization on the planes, changing the directions and positions of the planes to normalize their geometric relationships, i.e., parallelism, coplanarity, and orthogonality. The main idea is to first organize the geometric relationships between the planes in layers (i.e., the angles between each pair of planes mentioned above), and then normalize the planes using a greedy algorithm (i.e., merging the above planes). After adjustment, the nearly parallel and nearly orthogonal planes are adjusted to be completely coplanar and orthogonal, respectively. The effect of plane regularization is as follows: Figure 8 As shown, Fig. 9 This is the effect diagram without regularization; it can be seen that it significantly improves the structural regularity and reduces the complexity of the reconstructed building structure.

[0098] Specifically, Fig.10 As shown, in step S30, the polyhedron is cut to obtain multiple candidate planes, including the following steps:

[0099] Step S31: Calculate and obtain the bounding box corresponding to all the current three-dimensional point cloud data;

[0100] It should be noted that if Figure 2 As shown in the figure, the above bounding box is used as the starting point for subsequent polyhedron segmentation. This is because theoretically the plane is boundless. When the space is cut by the plane, in order to be closed, a three-dimensional space (outer box) is given for cutting. This outer box is the point cloud bounding box.

[0101] Step S32: extracting the coverage of the bounding box to obtain the coverage corresponding to each new plane;

[0102] It should be noted that the above coverage extraction operation is performed using 2Dα-shape, which is a mathematical method used to extract closed convex polygons from a point set. These polygons can approximate the shape characteristics of the original point cloud data. Here, 2Dα-shape is used to extract the coverage of each plane (or coverage primitive), that is, to extract the boundary or outline of the plane, such as Fig.11 As shown;

[0103] Step S33: after arranging the respective coverage surfaces in descending order according to their area sizes, traverse the respective coverage surfaces in order from large to small areas, determine whether the polyhedron intersects with the coverage surface, and if so, segment the polyhedron using the coverage surface to obtain a plurality of candidate planes;

[0104] It should be noted that, in the above-mentioned embodiment of the present application, each surface primitive (surface primitive is the above-mentioned covering surface primitive or covering surface) is sorted and traversed (such as Fig.12 As shown), if the polyhedron intersects with the face primitive, the polyhedron is segmented and finally obtained Fig.13 The polyhedron shown; it is worth noting that the final segmentation result may be affected by the cutting order of the surface primitives; the embodiment of the present application arranges the surface primitives in descending order of area size because the larger the surface, the more stable it is. Cutting with a stable surface first can make the segmentation result more stable.

[0105] Specifically, Fig.14 As shown, in step S40, binary labeling is performed on the second polyhedron through visibility analysis to obtain a polyhedron inside the building surface and a polyhedron outside the building surface, including the following operation steps:

[0106] Step S41: constructing an empty initial directed graph; mapping the second polyhedral primitive corresponding to the second polyhedron to the initial directed graph to obtain a vertex V, and mapping the common faces of every two adjacent polyhedrons to the initial directed graph to obtain an edge E, thereby obtaining a directed graph;

[0107] Step S42: adding a source point source and a sink point sink to each vertex V in the directed graph; obtaining a weight w of each edge E;

[0108] After “adding a source point and a sink point to each vertex V in the directed graph”, the method further includes: connecting the vertex to the source point source and the sink point sink respectively;

[0109] The weight w≥0;

[0110] It should be noted that the above embodiment of the present application regards this binary classification problem (i.e., the above-mentioned binary classification problem of converting the building surface reconstruction problem into the obtained polyhedron) as an st-cut problem of a directed graph G = (V, E); by first constructing a directed graph: mapping the polyhedron primitives to the vertices (V) of the graph, and mapping the faces shared by two adjacent polyhedrons to the edges (E) of the graph;

[0111] The above embodiment of the present application expands the vertex V by adding a source point source and a sink point sink; each vertex is connected to the source point and the sink point respectively; all edges have non-negative weights w;

[0112] The above-mentioned source point source can be selected in a certain internal area of ​​the building, and usually a representative point inside the building surface is selected as the source point; the above-mentioned sink point selects a point outside the building as the sink point, and usually a point far away from the building (that is, the source point source is a point of the polyhedron inside the building surface; the sink point sink is a point of the polyhedron outside the building surface).

[0113] Step S43: Based on the source point source, the sink point sink and the weight w, all the vertices V in the directed graph are processed by st-cut to obtain a subset S and a subset T; wherein the source point source belongs to the subset S; the sink point sink belongs to the subset T;

[0114] Wherein, the subset S represents the vertices of the polyhedron inside the building surface;

[0115] The subset T represents the vertices in the exterior polyhedron of the building surface;

[0116] The step of performing st-cut processing on all the vertices V in the directed graph based on the source point source, the sink point sink and the weight w to obtain a subset S and a subset T comprises: firstly obtaining the subset S by distinguishing through global energy minimization, and then obtaining a subset T consisting of all vertices except the subset S;

[0117] Step S44: obtaining an internal polyhedron of the building surface based on the subset S, and obtaining an external polyhedron of the building surface based on the subset T;

[0118] In the above embodiment of the present application, after distinguishing the subsets to which the vertices of the second polyhedron belong, the second polyhedron corresponding to the vertices belonging to the subset S is identified as the internal polyhedron of the building surface, and the second polyhedron corresponding to the vertices belonging to the subset T is identified as the external polyhedron of the building surface;

[0119] It should be noted that, after constructing the directed graph, the above embodiment of the present application performs st-cut, that is, divides the vertex V in the directed graph into two disjoint subsets S and T, so that the source point source∈S and the sink point sink∈T;

[0120] The st-cut has low-order polynomial complexity, and the minimum cut can be efficiently found by minimizing the global energy (by calculating the minimum cut, the mathematical problem is converted into a mathematical problem of solving a graph consisting of vertices and edges G = (V, E)); the graph cut (S, T) (the graph cut (S, T) is the above-mentioned st-cut) corresponds to the binary labeling of the second polyhedron - where S corresponds to the polyhedron inside the building surface, and T corresponds to the external polyhedron;

[0121] Specifically, Fig.15 As shown, the subset S is obtained by distinguishing through global energy minimization, including the following steps:

[0122] Step S441: traverse each of the three-dimensional point cloud data: obtain the visibility item E of each of the three-dimensional point cloud data vis (S) and obtain the directional item E of each of the three-dimensional point cloud data orient (S);

[0123] Step S442: Based on the visibility item E vis (S) and directional term E orient (S) Calculate and obtain the energy function E(S);

[0124] The energy function E(S) is calculated as follows:

[0125] E(S)=E vis (S)+E orient (S);

[0126] Step S443: Determine whether the energy function E(S) is greater than 0; if so, determine that the three-dimensional point cloud data is the three-dimensional point cloud data of the external polyhedron of the building surface; construct a subset S based on the three-dimensional point cloud data of all the external polyhedrons of the building surface (correspondingly, in the directed graph, the vertices of the three-dimensional point cloud data belonging to the external polyhedron of the building surface will be included in the subset S).

[0127] It should be noted that the above visibility item E vis (S), in the generation stage of the dense point cloud (i.e., the above-mentioned three-dimensional point cloud data), each three-dimensional point cloud data can obtain image light information, i.e., visibility information; the embodiment of the present application uses this visibility to construct the visible item E vis (S), which increases the weight of the face traversed by the line of sight (the weight is the visibility term E vis(S) value); the principle is that if a point in the point cloud is on the surface of a building, then its corresponding line of sight, that is, the line formed by the point and the photographic center of its visible image, should be unobstructed; therefore, the weight of the edge can be assigned by the intersection of the line of sight and the polyhedron, as shown in the schematic diagram Fig.11 shown.

[0128] like Fig.16 , for each point p in the point cloud i , and compare it with the photographic center c of the corresponding jth visible image ij Connect them to get the line of sight (indicated by the red dotted line in the figure); considering the scale accuracy σ, the line of sight starts from the photography center c ij Go to point p i The direction is extended by a distance of σ, and an end point e is obtained. i ; S1 to S5 are the planes extracted in step S30; Cell1 to Cell9 are the second polyhedrons constructed in step S41;

[0129] From the Photography Center ij The polyhedrons on the building surface should be visible, while the polyhedrons passing through the building surface should be invisible; therefore, the polyhedrons Cell2 and Cell4 on the same side as the normal vector direction of plane S5 are visible, while the polyhedron Cell7 on the opposite side of the normal vector direction of plane S5 is invisible; ij The polyhedron Cell2 is connected to the source point, and its weight is α vis ; Set the end point e i The polyhedron Cell7 is connected to the sink, and its weight is also α vis ; Along the line of sight, all faces in the opposite direction will increase the weight α vis , unless the point belongs to the patch;

[0130] The above direction term E orient (S), since the edges of a directed graph are directed, it is necessary to consider the direction of the faces corresponding to the edges;

[0131] The embodiment of the present application uses the direction item E orient (S) takes a value of zero for faces whose normal vectors are in the same direction as the line of sight (i.e., the angle is less than 90°), and takes an infinite value for faces whose normal vectors are in the opposite direction to the line of sight (i.e., the angle is greater than 90°);

[0132] The dense point cloud (the dense point cloud is the three-dimensional point cloud data mentioned above) of each building generated by low-altitude oblique images can reach millions. If the visibility calculation is not optimized, it is difficult to apply this method to the reconstruction of lightweight models of buildings at the city level; the visibility calculation process of each line of sight is independent of each other, which provides conditions for GPU parallelism; therefore, the embodiment of the present application allocates a GPU thread to each line of sight to perform visibility calculations.

[0133] In summary, the example of the present invention proposes a method for automatically extracting the building body frame model based on graph cuts. First, the plane is extracted from the point cloud, and the light information of the densely matched point cloud is used as prior information to convert the building reconstruction into a graph cut problem to solve, thereby avoiding the huge amount of calculation caused by dynamic programming solution; and then the plane is used to cut the outer bounding box of the point cloud to obtain a candidate polyhedron; finally, the visibility information is used to construct a graph cut model to perform binary labeling on the polyhedron, and the polyhedron inside and outside the building is distinguished, thereby realizing the extraction of the building body frame model. The experimental results verify that the algorithm of the present application can obtain better results on the noisy point cloud of the building. On the other hand, the present application realizes the GPU parallelization of the visibility weight calculation, thereby achieving the purpose of efficiently generating the building body frame model, making the method suitable for batch building model reconstruction at the city level; the lightweight single building model is automatically generated, which is conducive to storage, rendering and subsequent GIS quantitative analysis, greatly reducing the manual workload.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. A person skilled in the art may modify the technical solutions described in the above embodiments, or replace part or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically extracting building object frame models based on graph cutting, characterized in that: The steps are as follows: Acquire three-dimensional point cloud data of the current building; perform a plane extraction operation based on the three-dimensional point cloud data to obtain multiple planes corresponding to the building; Based on global regularization, all the planes are optimized and merged to obtain multiple new planes; constructing a polyhedron based on all of the new planes; Cutting the polyhedron to obtain multiple candidate planes; Constructing a plurality of second polyhedrons based on each of the candidate planes; Binary labeling is performed on the second polyhedron by visibility analysis to obtain an internal polyhedron on the building surface and an external polyhedron on the building surface; Merging the internal polyhedrons of the building surface to obtain a building model; The step of optimizing and merging all the planes based on global regularization to obtain multiple new planes includes the following steps: Based on all the planes, multiple plane pairs (s i ,s j ); Calculate and obtain the plane pair (s i ,s j )’s included angle θ; Get each of the plane pairs (s i ,s j ) The number of support points N of each plane; Traverse each of the plane pairs (s i ,s j ), determine the plane pair (s i ,s j ) is less than or equal to the angle threshold θ t And the plane pair (s i ,s j ) is the number of support points N of each plane greater than or equal to the point threshold N t If so, then the plane pair (s i ,s j ) are merged to obtain a new plane; if not, the plane pair (s i ,s j ) are taken as new planes; The step of cutting the polyhedron to obtain a plurality of candidate planes comprises the following steps: Calculate and obtain the bounding box corresponding to all the current three-dimensional point cloud data; Extracting the coverage of the bounding box to obtain the coverage corresponding to each new plane; After arranging the respective coverage surfaces in descending order according to their area sizes, traversing the respective coverage surfaces in order from large to small areas, determining whether the polyhedron intersects with the coverage surface, and if so, segmenting the polyhedron using the coverage surface to obtain a plurality of candidate planes; The binary marking of the second polyhedron by visibility analysis to obtain an internal polyhedron on the building surface and an external polyhedron on the building surface includes the following steps: Constructing an empty initial directed graph; mapping the second polyhedral primitive corresponding to the second polyhedron to the initial directed graph to obtain a vertex V, and mapping the common faces of every two adjacent polyhedrons to the initial directed graph to obtain an edge E, thereby obtaining a directed graph; Add source points and sink points to each vertex V in the directed graph; obtain the weight w of each edge E; Based on the source point, the sink point and the weight w, all the vertices V in the directed graph are processed by st-cut to obtain subsets S and subsets T; wherein the source point belongs to the subset S; The sink belongs to the subset T; Based on the subset S, the internal polyhedron of the building surface is obtained, and based on the subset T, the external polyhedron of the building surface is obtained.

2. The method for automatically extracting building object frame models based on graph cuts according to claim 1 is characterized in that: The plane pair (s i ,s j ) represents two adjacent planes.

3. The method for automatically extracting building object frame models based on graph cuts according to claim 2 is characterized in that: After "adding a source point and a sink point to each vertex V in the directed graph respectively", it also includes: connecting the vertex to the source point and the sink point respectively.

4. The method for automatically extracting building object frame models based on graph cuts according to claim 3 is characterized in that: The subset S represents the vertices of the polyhedron inside the building surface; The subset T represents the vertices in the building surface exterior polyhedron.

5. The method for automatically extracting building object frame models based on graph cuts according to claim 4 is characterized in that: The step of processing all the vertices V in the directed graph by st-cut based on the source point, the sink point and the weight w to obtain a subset S and a subset T includes: first obtaining the subset S by distinguishing through global energy minimization, and then obtaining a subset T consisting of all vertices except the subset S.

6. The method for automatically extracting building object frame models based on graph cuts according to claim 5 is characterized in that: The step of obtaining the subset S by distinguishing through global energy minimization includes the following steps: Traverse each of the three-dimensional point cloud data to obtain the visibility item E of each of the three-dimensional point cloud data vis (S) and obtain the directional item E of each of the three-dimensional point cloud data orient (S); Based on the visibility item E vis (S) and directional term E orient (S) Calculate and obtain the energy function E(S); Determine whether the energy function E(S) is greater than 0; if so, determine that the three-dimensional point cloud data is the three-dimensional point cloud data of the external polyhedron of the building surface; and construct a subset S based on the three-dimensional point cloud data of all the external polyhedrons of the building surface.

7. The method for automatically extracting building object frame models based on graph cuts according to claim 6 is characterized in that: The energy function E(S) is calculated as follows: E(S)=E vis (S)+E orient (S)。

Images